Method for controlling image photographing while protecting privacy, and device implementing same

Abstract

The present invention relates to a method for controlling image photographing while protecting privacy, and an apparatus implementing same. A method for controlling image photographing while protecting privacy, according to an embodiment of the present invention, comprises the steps of: allowing a sensor unit of an apparatus to identify the occurrence of a privacy protection event; allowing a control unit of the apparatus to limit photographing of a camera of the apparatus; allowing the sensor unit to identify the occurrence of a privacy release event; and allowing the control unit to release the photographing limit of the camera.

Description

Method for controlling video recording while protecting privacy and device implementing the same

[0001] The present invention relates to a method for controlling video recording while protecting privacy and a device for implementing the same.

[0002] Recently, there has been increasing interest in technology that equips home appliances with cameras to record the process of using or operating them. In particular, for cooking appliances such as ovens, there is a growing need to record or capture the cooking process on video.

[0003] However, there is a possibility that the user's appearance or space may be captured during the video recording process. Particularly in the case of cooking appliances, problems may arise where a camera placed on the door or inside the cooking chamber films the user or the space where the appliance is located during situations such as when the door is opened or closed.

[0004] This specification aims to solve the aforementioned problems by implementing a method and apparatus for controlling video recording so that a home appliance does not record video that may infringe on privacy while recording video.

[0005] In addition, this specification aims to resolve the issue of personal information infringement where the user's appearance is captured through a camera attached to the door or placed inside the main body when the door is opened.

[0006] In addition, this specification aims to prevent the use of low-quality images due to changes in brightness when the door is closed.

[0007] The objects of the present invention are not limited to those mentioned above, and other unmentioned objects and advantages of the present invention may be understood from the following description and will be more clearly understood by the embodiments of the present invention. Furthermore, it will be readily apparent that the objects and advantages of the present invention can be realized by the means and combinations thereof set forth in the claims.

[0008] A method for protecting privacy and controlling video recording according to an embodiment of the present invention includes the steps of: a sensor unit of a device confirming the occurrence of a privacy protection event; a control unit of a device restricting the recording of a camera of a device; a sensor unit confirming the occurrence of a privacy release event; and a control unit releasing the restriction on the recording of a camera.

[0009] A device for protecting privacy and controlling image recording according to one embodiment of the present invention includes a camera, a sensor unit for confirming the occurrence of a privacy protection event and a privacy release event, and a control unit for controlling the camera to restrict recording when a privacy protection event occurs and controlling the camera to release the recording restriction when a privacy release event occurs.

[0010] When the present invention is applied, video recording can be controlled so that the home appliance does not record video that may infringe on privacy while recording video.

[0011] When the present invention is applied, the problem of personal information infringement can be resolved, in which the user's appearance is captured through a camera attached to the door or placed inside the main body when the door of a home appliance is opened.

[0012] When the present invention is applied, it is possible to prevent the use of low-quality images due to changes in brightness when the door is closed.

[0013] The effects of the present invention are not limited to the effects described above, and various effects of the present invention can be easily derived from the composition of the present invention.

[0014] FIG. 1 is a conceptual diagram of a cooking appliance including a door according to one embodiment of the present invention.

[0015] FIG. 2 is a drawing showing the configuration of a device according to one embodiment of the present invention.

[0016] FIG. 3 is a diagram showing the process of a device according to an embodiment of the present invention performing privacy protection and restricting camera shooting.

[0017] FIG. 4 is a diagram showing the process of a cooking device according to one embodiment of the present invention providing a streaming service.

[0018] FIG. 5 is a diagram showing the process of a cooking device according to another embodiment of the present invention providing a time-lapse service.

[0019] FIG. 6 is a diagram showing the process of a device according to an embodiment of the present invention capturing the interior and streaming it through a user terminal.

[0020] FIG. 7 is a diagram showing the process of saving an image when a device according to an embodiment of the present invention captures the interior and provides a time-lapse service.

[0021] FIG. 8 is a diagram showing the process of comparing and deleting stored images according to one embodiment of the present invention.

[0022] FIG. 9 is a diagram showing the process of performing a streaming service using a buffer according to an embodiment of the present invention.

[0023] FIG. 10 is a diagram showing the process of selecting an alternative image according to one embodiment of the present invention.

[0024] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings so that those skilled in the art can easily implement the present invention. The present invention may be embodied in various different forms and is not limited to the embodiments described herein.

[0025] To clearly explain the present invention, parts unrelated to the description have been omitted, and the same reference numerals are assigned to identical or similar components throughout the specification. Additionally, some embodiments of the present invention are described in detail with reference to the exemplary drawings. In assigning reference numerals to the components of each drawing, identical components may have the same reference numeral whenever possible, even if they are shown in different drawings. Furthermore, in describing the present invention, if it is determined that a detailed description of related known components or functions could obscure the essence of the invention, such detailed description may be omitted.

[0026] In describing the components of the present invention, terms such as first, second, A, B, (a), (b), etc., may be used. These terms are intended only to distinguish the components from other components, and the nature, order, sequence, or number of the components are not limited by these terms. Where it is stated that a component is "connected," "combined," or "connected" to another component, it should be understood that the component may be directly connected or connected to the other component, but that other components may be "interposed" between each component, or that each component may be "connected," "combined," or "connected" through other components.

[0027] In addition, for convenience of explanation in implementing the present invention, the components may be described in detail; however, these components may be implemented within a single device or module, or a single component may be divided and implemented across multiple devices or modules.

[0028] The device described in this specification is a home appliance that includes an internal space for placing food, such as a refrigerator, oven, or microwave. For convenience of explanation, this specification focuses on cooking appliances such as ovens or microwaves, but the invention is not limited thereto. Embodiments of the invention are applicable to home appliances that are placed outside or inside the appliance and have the potential to change the shooting area or to photograph people or spaces depending on specific situations.

[0029] The cooking appliance described in this specification can cook food using light waves, microwaves, and / or heater heat, which are types of electromagnetic waves. The cooking appliance may generally be equipped with a cavity, which is a space where food is placed and cooked, and a door that opens and closes the cavity.

[0030] The camera can be placed in the door or inside the cavity.

[0031] FIG. 1 is a conceptual diagram of a cooking appliance including a door according to an embodiment of the present invention. FIG. 1 is a simplified diagram showing the case where the device is a cooking appliance.

[0032] The cooking appliance (1000a) includes a door (80) on the front. The door (80) can open and close the interior area (90) of the cooking appliance (1000a). The method of opening and closing the door is the same as that of 1000b, 1000c, or 1000d.

[0033] 1000b shows an embodiment in which the door (80) opens the right side of the cooking appliance (1000b) to open the interior of the cooking appliance (1000b). 1000c shows an embodiment in which the door (80) opens the lower side of the cooking appliance (1000c) to open the interior of the cooking appliance (1000c). 1000d shows an embodiment in which the door (80) opens the upper side of the cooking appliance (1000d) to open the interior of the cooking appliance (1000d).

[0034] Although not shown in the drawing, buttons for controlling the cooking appliance (1000a) may be placed on the door (80) or on one side of the cooking appliance.

[0035] Although not shown in the drawing, the door (80) may include a screen (e.g., an LCD screen) and may provide an interface through which a user can display certain information and input information.

[0036] Although not shown in the drawing, a camera may be positioned inside the door (80) or on one side of the cooking area (90) of the cooking device (1000a) to film the cooking process.

[0037] A camera positioned inside the door (80) or inside the heating space (90) can capture the process of the incoming food being heated. However, when the door (80) is opened, the camera can capture other objects or spaces other than the food in the interior area (90).

[0038] For example, in the case of a camera placed on the door (80), when the door (80) is opened, it may face another space rather than the interior area (90). Alternatively, a camera placed inside the heating space (90) may also photograph the exterior space on the door (80) side when the door (80) is opened.

[0039] Since the privacy of the user of the cooking device may be infringed during such a process, this specification examines a technology for controlling camera shooting when an event that infringes on privacy occurs while the camera is in operation.

[0040] FIG. 2 is a diagram showing the configuration of a device according to an embodiment of the present invention. The device is an embodiment of a home appliance, and the home appliance provides certain functions, such as cooking, refrigeration, freezing, heating, defrosting, and dishwashing. Although the present specification describes a cooking appliance, the device of the present invention is not limited thereto.

[0041] FIG. 2 is a diagram showing the operation process of a device according to an embodiment of the present invention. The device is described with reference to a cooking appliance, but the present invention is not limited thereto.

[0042] The device (1000) includes a control unit (200), a camera (110), a communication unit (120), a speaker / microphone unit (130), a sensor unit (140), a door (80), and a function unit (500).

[0043] The function unit (500) performs the unique functions of the corresponding device (1000), such as heating, refrigeration, freezing, and dishwashing, according to the control of the control unit (200). For example, if the device (1000) is a cooking appliance, the function unit (500) may provide a cooking function. The function unit (500) may include various components necessary for heating for cooking, and the present invention is not limited to a specific cooking method.

[0044] The control unit (200) controls various components of the device (100). Additionally, the control unit (200) controls the function unit (500) to enable the device (1000) to perform specific functions. The control unit (200) may operate based on a specific OS (e.g., Android) or may operate by a separate operating system, program, firmware, etc. The control unit (200) may store images captured by the camera (110). To this end, the control unit (200) may include memory.

[0045] The camera (110) is positioned on the inside of the door (80) and can photograph the space inside the device (1000). For example, if the device (1000) is a cooking appliance, the camera (110) photographs the inside of the cooking appliance so that the user can check the cooking status of the food placed inside.

[0046] Alternatively, the camera (110) may be placed on one side of the internal area (90) as seen in FIG. 1.

[0047] The communication unit (120) can perform various types of wired or wireless communication functions. The communication unit can communicate with other devices (e.g., an external server, a hub placed in the home, or other home appliances) using communication protocols such as Wi-Fi or Bluetooth.

[0048] The speaker / microphone unit (130) can generate voice, alarm sounds, etc., necessary for the operation of the device (1000), and can receive a predetermined external voice command or receive an external sound. The speaker / microphone unit (130) can be arranged as a single unit or can be arranged at different locations.

[0049] The sensor unit (140) senses the external or internal environment of the device (1000). That is, the sensor unit (140) can detect the occurrence of privacy protection events and privacy release events. To this end, the sensor unit (140) checks the open / closed status of the door (80), and if the door (80) is open, it confirms that a privacy protection event has occurred, and if the door (80) is closed, it confirms that a privacy release event has occurred.

[0050] In addition, the sensor unit (140) may include a temperature sensor, an illuminance sensor, a human sensor, a humidity sensor, etc.

[0051] When the device (1000) is a cooking device, the control unit (200) controls the function of the cooking device (1000a), and the function unit (500) generates microwave or heater heat to cook food placed in the internal area (90). The function unit (500) may include various components for cooking (lamp, fan, heating unit).

[0052] The communication unit (120) can transmit the video captured by the camera (110) in real time to an external server or a connected user terminal.

[0053] In an embodiment of the present invention, the control unit (200) controls the camera (110) to restrict shooting by the camera (110) when a privacy protection event occurs. Additionally, the control unit (220) controls the camera (110) to release the restriction on shooting by the camera (110) when a privacy release event occurs.

[0054] FIG. 3 is a diagram showing the process of a device according to an embodiment of the present invention performing privacy protection and restricting camera shooting. For convenience of explanation, the description focuses on the cooking appliance.

[0055] The sensor unit (140) of the device (1000) confirms the occurrence of a privacy protection event (S1). A privacy protection event refers to the occurrence of a state requiring privacy protection. For example, the case where the door (80) is opened is taken as an example. When the event of S1 occurs, the control unit (200) of the device (1000) restricts the shooting of the camera (110) of the device (1000) (S2).

[0056] Restricting the shooting of the camera (110) includes an embodiment in which the camera (110) is turned off, the camera (110) is stopped from shooting video, or the video (image or video) shot by the camera (110) is deleted.

[0057] Afterward, the privacy protection state (camera shooting restricted) is maintained, and then the sensor unit (140) confirms the occurrence of a privacy release event (S3). A privacy release event means the occurrence of a state where privacy protection is not required. For example, the case where the door (80) is closed is taken as an example.

[0058] When the event of S3 occurs, the control unit (200) of the device (1000) releases the shooting restriction of the camera (110) of the device (1000) (S4). Releasing the shooting restriction of the camera (110) includes embodiments of turning on the camera (110), starting video recording of the camera (110), or saving the video (image or video) captured by the camera (110).

[0059] Prior to S1, the device (1000) can store an image captured by the camera (110). An image refers to one or more images. That is, a single image corresponds to an image of the present invention, and a video composed of two or more images also corresponds to an image of the present invention.

[0060] In addition to the embodiment of FIG. 3, if the speaker / microphone unit (130) is in the process of voice recording, the control unit (200) may stop the voice recording of the speaker / microphone unit (130) in step S2. Also, in step S4, the control unit (200) may start the voice recording of the speaker / microphone unit (130).

[0061] When the embodiment of FIG. 3 is applied to a cooking appliance (1000a), the camera (110) can be controlled according to the door opening / closing state of the cooking appliance (1000a) and the image captured by the camera (110) can be processed.

[0062] That is, when applying the embodiment of FIG. 3, a camera attached to the door or the interior area (90) to photograph the door and the main body of the cooking appliance (1000a) and the interior of the main body can be controlled and the image obtained from the camera (110) can be processed.

[0063] In the event that a privacy protection event occurs, for example, when the door is opened to minimize the exposure of personal information and when the door is closed to prevent the use of low-quality images due to changes in brightness, the control unit (200) can control the shooting of the camera (110) to process the image.

[0064] In one embodiment of the case where a privacy protection event occurs, when the door (80) is opened, the control unit (200) provides an alternative image instead of capturing a video, or discards an image stored for a certain period of time so that an image of the user is not used. As a result, the problem of infringement of personal information, such as the user's appearance or other areas of the kitchen being captured, can be resolved.

[0065] In the case of a privacy release event, when the door (80) is closed, the control unit (200) prevents the use of low-quality images by not using the images captured during the time the camera (110) is adjusted by a change in brightness.

[0066] In summary, if the device is a cooking appliance, the privacy protection event includes an opening event of the door that opens and closes the internal space of the cooking appliance. Additionally, the privacy release event includes a closing event of the door that opens and closes the internal space of the cooking appliance. To this end, the sensor unit (140) can notify the control unit (200) of the open / closed state of the door (80).

[0067] A cooking device (1000a), which is an embodiment of the present invention, can provide a streaming service that transmits the process of cooking food in an internal area (90) to the outside. In addition, the cooking device (1000a) can provide a time-lapse service that records the process of cooking food. Furthermore, the cooking device (1000a) can provide a service that saves the cooking process as a video so that a user can check the cooking process later.

[0068] In the embodiment of FIG. 3, when shooting is restricted (S2), the communication unit (120) can transmit a replacement image that replaces the image captured by the camera (110) to a server or user terminal.

[0069] FIG. 4 is a diagram showing the process of a cooking device according to an embodiment of the present invention providing a streaming service. In FIG. 4, the power of the camera is controlled or the interruption / resumption of shooting is controlled to implement the restriction and release of shooting.

[0070] That is, the step (S2) of limiting shooting in Fig. 3 can be implemented as a step in which the control unit (200) turns off the power of the camera (110) or stops shooting of the camera (110) in the embodiment of Fig. 4.

[0071] Additionally, the step (S4) of releasing the shooting restriction of Fig. 3 can be implemented as a step in which the control unit (200) turns on the power of the camera (110) or resumes shooting of the camera (110).

[0072] The user starts using the streaming service (S11). The user selects to start the streaming service provided by the cooking device (1000a) or selects to start the streaming service using a user terminal connected to the cooking device (1000a).

[0073] Under user control, the cooking appliance (1000a) turns on the power of the camera (110). The sensor unit (140) checks whether the door (80) is closed (S13). If no privacy protection event occurs, for example, if the door (80) is closed, the power of the camera (110) remains on.

[0074] Since the camera (110) is powered on, the camera (110) captures an image, and the communication unit (120) transmits the captured image to another device under the control of the control unit (200) (S15). The device receiving the image may be a server providing a streaming service, or a server of a social network service (SNS) providing various social media. Alternatively, the device receiving the image may be a user terminal linked with a cooking device (1000a).

[0075] The control unit (200) checks whether streaming has ended (S18), and if it has ended, turns off the camera power (S19). Meanwhile, if streaming has not ended, it checks again whether the door is closed (S13).

[0076] If a privacy protection event occurs, for example, when the door (80) is open, the control unit (200) turns off the power to the camera (110) and keeps it in the Off state (S16).

[0077] And the control unit (200) controls the communication unit (120) to transmit the previously stored replacement image to another device (user terminal or server, etc.) (S17).

[0078] And the control unit (200) operates by checking whether the streaming of S18 has ended as described above.

[0079] If a privacy release event occurs after a privacy protection event, for example, if the door (80) is opened and then closed again, the control unit (200) can turn on the camera power and transmit the image taken by the camera to another device (user terminal or server, etc.) (S14, S15).

[0080] An additional embodiment of the process of FIG. 4 is as follows. When a user starts using the streaming service, the control unit (200) turns on the power of the camera (110) and controls the sensor unit (140) to check the open / closed status of the door (80).

[0081] If the camera boot speed is sufficiently fast, the control unit (200) can turn the power of the camera (110) ON / OFF depending on the open / closed state of the door (80). Otherwise, S15 or S17 can be performed while the camera power is on.

[0082] For example, the control unit (200) can transmit an image of the inside of the main body to another device (user terminal or server, etc.) when the door is closed, and transmit a previously stored alternative image to another device (user terminal or server, etc.) when the door is open.

[0083] In summary, depending on the camera's on / off speed, S14 and S15 of FIG. 4 may be executed selectively, or both S14 and S15 of FIG. 4 may be executed. Likewise, depending on the camera's on / off speed, S16 and S17 of FIG. 4 may be executed selectively, or both S16 and S17 of FIG. 4 may be executed.

[0084] The embodiment of FIG. 4 can provide a privacy protection function by transmitting an alternative image instead of an image obtained from a camera so that the user's appearance is not captured when the door is opened while using a streaming service.

[0085] FIG. 5 is a diagram showing the process of a cooking device according to another embodiment of the present invention providing a time-lapse service. The time-lapse service is a service that stores images taken while cooking is in progress, and then collects the stored images to create a video and provides it to the user when cooking is finished.

[0086] When the time-lapse service starts, the camera (110) continues shooting, and the images generated accordingly are stored in the memory of the control unit (200). And if a shooting restriction is started, the images stored in the waiting buffer of the memory are deleted. That is, when the embodiment of FIG. 5 is applied, images taken by the camera (110) are stored in the memory of the control unit (200). And when a door opening situation is detected, some of the stored images are deleted.

[0087] When the user starts cooking (S21), the time-lapse service starts, and the control unit (200) turns on the camera (110) and initializes Frame_Skip_current (S22). Frame_Skip_current is the number of frames remaining before being stored in the waiting buffer, and is information set to delete footage taken initially or immediately after the door is closed in order to ensure image quality. This can be increased, decreased, or initialized during the shooting process.

[0088] The control unit (200) can determine whether to save the image by comparing Frame_Skip_current with Skip_frame_count.

[0089] Skip_frame_count is a preset number representing the number of frames to be deleted within a certain period of time for the sake of video quality or privacy. Skip_frame_count can be fixed in advance and can be changed depending on the operating conditions of the device (1000). Skip_frame_count can be determined based on the operation of the door (80), particularly the opening and closing sensing time of the door (80).

[0090] For example, if the sensing time for detecting the opening and closing of the door (80) is short, Skip_frame_count can be set to a small value (e.g., 10). If the sensing time for detecting the opening and closing of the door (80) is long, Skip_frame_count can be set to a small value (e.g., 30).

[0091] In S22, the control unit (200) can initialize Frame_Skip_current to 0. Alternatively, if shooting is possible immediately, Frame_Skip_current can be set to the Skip_frame_count value. That is, the control unit (200) can set Frame_Skip_current to any one of a value greater than or equal to Skip_frame_count, depending on the shooting speed of the camera (110) or the opening / closing sensing time of the door (80).

[0092] The control unit (200) checks whether the door (80) is closed (S23). If the door is closed, the control unit (200) compares Frame_Skip_current with Skip_frame_count (S24).

[0093] If Frame_Skip_current is set to 0 in S22, the control unit (200) increases Frame_Skip_current by 1 (S25) and proceeds to step S23. If the door remains closed, steps S23 to S25 are repeated, and the image captured by the camera is not stored in the waiting buffer until Frame_Skip_current reaches Skip_frame_count.

[0094] Subsequently, when Frame_Skip_current reaches Skip_frame_count in S24, the control unit (200) stores the image taken by the camera in the waiting buffer (S26). The waiting buffer is a buffer that temporarily stores the image before saving the image as a time-lapse.

[0095] If the waiting buffer is full of images (S27), the control unit (200) moves the images in the waiting buffer to the main buffer (S28). Then, if cooking is not finished (S29-No), proceed to step S23, and if cooking is finished (S29-Yes), proceed to step S33. The control unit (220) controls the communication unit (120) to produce a time-lapse video using the images in the main buffer and transmit it to another device (user terminal or server, etc.) (S33).

[0096] Meanwhile, if the door is not closed in S23, that is, if a privacy protection event occurs, the control unit (200) deletes the image stored in the standby buffer (S31). This deletes the image captured by the camera (110) from the time the door is opened until it is detected, so that unnecessary images are not included in the time-lapse.

[0097] Then, the control unit (200) initializes Frame_Skip_current (S32). As a result, even after the door is closed, the captured images are not stored in the waiting buffer until the number of captured images (Frame_Skip_current) reaches Skip_frame_count for a certain period of time.

[0098] Depending on whether cooking is finished after S32, step S23 or S33 is performed, and if the door remains closed in S23, the control unit (200) repeats S31~S32.

[0099] And when the door is closed, the control unit (200) repeats the process of S24 to S28.

[0100] In the embodiment of FIG. 5, the captured image is first stored in a waiting buffer, and when the waiting buffer is full, the control unit (200) moves the image from the waiting buffer to the main buffer. The time-lapse video is produced using the image stored in the main buffer.

[0101] If the door is opened during cooking, the image stored in the waiting buffer is deleted to avoid using an image that may contain the user's appearance (see S26, S31, S32).

[0102] When the door is closed, a certain number of frames are not saved and are skipped to prevent the use of low-quality images captured while the camera settings are being adjusted due to a difference in brightness. To this end, the control unit (200) initializes Frame_Skip_Current and increases it by 1 for every shot. The control unit (200) discards the captured images without saving them to the waiting buffer until Frame_Skip_Current reaches Skip_frame_count.

[0103] As seen in Figure 5, when the door is opened while using the time-lapse service, the image acquired from the camera is not saved. Additionally, due to the delay of the sensor detecting the opening of the door, there is a possibility that the image may be saved after the door is opened. To prevent this, after the door opening is detected, the image in the waiting buffer is deleted to remove the image that may have been saved for a certain period of time before detection.

[0104] Additionally, when the door is closed while the time-lapse service is in use, an excessively dark or bright image is captured while the camera settings are adjusted due to the difference in brightness between the outside and the inside of the cooking appliance (1000a). To prevent the images captured during that period from being used, the saving of a certain number of images is skipped even after the door is closed. To do this, Frame_Skip_current and Skip_frame_count are used.

[0105] The embodiment of Fig. 5 is summarized as follows.

[0106] To implement the step (2) of restricting shooting as seen in FIG. 3, the control unit (200) can delete images that the camera has captured and stored during a preset time interval based on the time when a privacy protection event occurs.

[0107] More specifically, the control unit (200) includes a buffer that stores images captured by the camera (110) frame by frame. This buffer can be distinguished into a standby buffer and a main buffer.

[0108] And the control unit (200) can delete frames of video captured within a preset time interval based on the time when the privacy protection event occurred (e.g., N frames or frames stored in a waiting buffer) from the buffer (S31). And the control unit (200) does not store video in the buffer after the time when the privacy protection event occurred.

[0109] Then, the control unit (200) stores the video captured after a preset time interval based on the time when the privacy release event occurred in a buffer (S26). For example, as shown in S24 and S25, the control unit (200) can delete the number of captured image frames by Skip_frame_count even after the door (80) is closed.

[0110] As seen in FIGS. 4 and 5, a device (1000) such as a cooking appliance (1000a) can provide a streaming service that provides a video to a user terminal, such as a smartphone or mobile device, to check the internal operating status.

[0111] In addition, a device (1000) such as a cooking device (1000a) can provide a time-lapse service that combines images taken internally to generate a video.

[0112] By applying embodiments of the present invention to a device (1000) capable of providing a streaming service or a time-lapse service, the shooting of a camera or the storage of video can be restricted or stopped depending on the opening and closing of a door. When the door is opened, personal information can be protected by preventing the situation in which the user's appearance or space is captured. In addition, the quality of the video result can be improved by preventing the use of low-quality images that occur due to changes in light after the door is closed.

[0113] In the embodiment of FIG. 5, the Skip_frame_count and the size of the waiting buffer, which define the temporal range for the camera's shooting restriction according to the opening and closing of the door, may be changed.

[0114] For example, if the speed at which the door is opened or closed is fast, the control unit (200) can reduce the size of the waiting buffer. Conversely, if the speed at which the door is opened or closed is slow, the control unit (200) can increase the size of the waiting buffer. The speed at which the door is opened or closed and the size of the waiting buffer may have an inverse relationship.

[0115] Additionally, the control unit (200) can increase or decrease Skip_frame_count depending on the extent to which light afterimages or the like affect the image quality after closing the door. If the quality of the image result is similar to that before opening the door after a short period of time has elapsed after closing the door, the control unit (200) can decrease Skip_frame_count.

[0116] Conversely, if the quality of the video result is similar to that before the door was opened after a long time has elapsed since the door was closed, the control unit (200) may increase Skip_frame_count. The time it takes for the quality of the video result to recover after the door is closed and Skip_frame_count may have a proportional relationship.

[0117] FIG. 6 is a diagram showing the process of a device according to an embodiment of the present invention capturing the interior and streaming it through a user terminal.

[0118] When a cup is placed in the cooking device (1000a) and is being cooked, the camera (110) of the cooking device (1000a) captures it. The captured image is the same as 301 and is linked with the user terminal (300a), so the user terminal (300a) can check the captured image.

[0119] During this process, when the door (80) of the cooking device (1000a) is opened, the camera (110) of the cooking device (1000a) performs shooting restriction measures, such as stopping shooting or turning off the camera. As a result, the cooking device (1000a) provides a predetermined alternative image (302) to the user terminal (300b). Afterward, when the door (80) is closed, the camera (110) of the cooking device (1000a) releases the shooting restriction and starts shooting again. As a result, as indicated in 303, the user terminal (300c) outputs the image captured by the cooking device (1000a).

[0120] If the door (80) remains open, the cooking device (1000a) can transmit an animation image of the oven door opening from an alternative image of 302 to the user terminal (300b).

[0121] FIG. 7 is a diagram showing the process of saving an image when a device according to an embodiment of the present invention captures the interior and provides a time-lapse service.

[0122] A control unit (200) having a processor as one embodiment includes two buffers (waiting buffer, main buffer), and these buffers are located within a predetermined storage area (memory). A camera (110) takes 30 shots per second and transmits the captured images to the control unit (200). The control unit (200) stores the transmitted images in the waiting buffer (201). When the waiting buffer (201) is completely filled with images, the images in the waiting buffer (201) are moved to the main buffer (202). When a door is opened during shooting, the control unit (200) deletes the images stored in the waiting buffer (201) so that the stored images are not used for a certain period of time.

[0123] If the sensing speed of the sensor unit (140) for the opening and closing of the door (80) is fast, the number of images stored in the waiting buffer (201) can be set to be small.

[0124] Meanwhile, as an example of shooting restriction and shooting restriction release, the control unit may perform a process of comparing the captured images.

[0125] FIG. 8 is a diagram showing the process of comparing and deleting stored images according to an embodiment of the present invention. S31 of FIG. 5 is implemented in detail.

[0126] When the door (80) is opened (S41), the control unit (200) performs the task of deleting images in the waiting buffer. During this process, the control unit (200) compares images stored in the waiting buffer (201) with images recently stored in the main buffer (202) (S42). Referring to FIG. 7, the control unit (200) can compare images 51 to 54 in the waiting buffer (201) with image 50 in the main buffer (202).

[0127] As a result of the comparison, the control unit (200) confirms that images 51 and 52 of the waiting buffer (201) are almost identical to images 50 of the main buffer (202).

[0128] Additionally, the control unit (200) confirms that images 53 and 54 are different from image 50 in the main buffer (202). In this case, the control unit (200) deletes only images 53 and 54.

[0129] To summarize, the process is as follows. When the door (80) is opened, the control unit (200) compares a first image (e.g., an image stored in a standby buffer) captured during a preset time interval with a second image (e.g., the most recently stored image in a main buffer) captured before the time interval. If the difference between the first image and the second image is greater than or equal to a preset threshold, the control unit (200) deletes the first image.

[0130] Next is a diagram showing the process of deleting some images using a buffer in a streaming service.

[0131] FIG. 9 is a diagram showing the process of performing a streaming service using a buffer according to an embodiment of the present invention. It is partially identical to the process of FIG. 5, and the parts that differ significantly are S22a, S25a, S28a, and S32a.

[0132] When the streaming service starts, the camera (110) continues shooting, and the images generated accordingly are stored in the memory of the control unit (200). And if a shooting restriction is started, the images stored in the waiting buffer of the memory are deleted. That is, when the embodiment of FIG. 9 is applied, images shot by the camera (110) are stored in the memory of the control unit (200). And when a door opening situation is detected, some of the stored images are deleted.

[0133] When the user starts cooking (S21), the streaming service starts and the control unit (200) turns on the camera (110) and initializes Frame_Skip_current (S22a). Frame_Skip_current is the number of frames remaining before being stored in the waiting buffer, and is information set to delete footage captured initially or immediately after the door is closed in order to ensure image quality. This can be increased, decreased, or initialized during the shooting process.

[0134] The control unit (200) can determine whether to save the image by comparing Frame_Skip_current with Skip_frame_count.

[0135] Skip_frame_count is a preset number representing the number of frames to be deleted within a certain period of time for the sake of video quality or privacy. Skip_frame_count can be fixed in advance and can be changed depending on the operating conditions of the device (1000). Skip_frame_count can be determined based on the operation of the door (80), particularly the opening and closing sensing time of the door (80).

[0136] For example, if the sensing time for detecting the opening and closing of the door (80) is short, Skip_frame_count can be set to a small value (e.g., 10). If the sensing time for detecting the opening and closing of the door (80) is long, Skip_frame_count can be set to a small value (e.g., 30).

[0137] In S22, the control unit (200) can initialize Frame_Skip_current to 0. Alternatively, if shooting is possible immediately, Frame_Skip_current can be set to the Skip_frame_count value. That is, the control unit (200) can set Frame_Skip_current to any one of a value greater than or equal to Skip_frame_count, depending on the shooting speed of the camera (110) or the opening / closing sensing time of the door (80).

[0138] The control unit (200) checks whether the door (80) is closed (S23). If the door is closed, the control unit (200) compares Frame_Skip_current with Skip_frame_count (S24).

[0139] If Frame_Skip_current is set to 0 in S22, the control unit (200) increments Frame_Skip_current by 1, transmits a replacement image to the outside (S25a), and proceeds to step S23. If the door remains closed, steps S23 to S25a are repeated, and the image captured by the camera is not stored in the waiting buffer until Frame_Skip_current reaches Skip_frame_count.

[0140] Subsequently, when Frame_Skip_current reaches Skip_frame_count in S24, the control unit (200) stores the image captured by the camera in the waiting buffer (S26). The waiting buffer is a buffer that temporarily stores the image before transmitting it externally via streaming.

[0141] If the waiting buffer is full of images (S27), the control unit (200) transmits the images in the waiting buffer to the outside (S28a). Then, if cooking has not finished (S29-No), proceed to step S23, and if cooking has finished, end streaming.

[0142] Meanwhile, if the door is not closed in S23, that is, if a privacy protection event occurs, the control unit (200) deletes the image stored in the waiting buffer (S31). This deletes the image captured by the camera (110) after the door is opened and before it is detected, thereby preventing unnecessary images from being transmitted externally through the streaming service.

[0143] Then, the control unit (200) initializes Frame_Skip_current and transmits a replacement image to the outside (S32a). As a result, even after the door is closed, the captured image is not transmitted to the outside until the number of captured images (Frame_Skip_current) reaches Skip_frame_count for a certain period of time.

[0144] Depending on whether cooking is finished after S32a, the S23 or termination step is performed, and if the door remains closed in S23, the control unit (200) repeats S31 to S32a.

[0145] And when the door is closed, the control unit (200) repeats the process of S24 to S28a.

[0146] Images captured for the streaming service are transmitted externally in real time. However, before being transmitted externally, images captured are first stored in a waiting buffer, and when the waiting buffer is full, the control unit (200) transmits the images in the waiting buffer externally.

[0147] If the door is opened during cooking, the image stored in the waiting buffer is deleted and a replacement image is sent externally so that the image that may contain the user's appearance is not used (see S26, S31, S32a).

[0148] When the door is closed, a certain number of frames are not saved and are skipped to prevent the use of low-quality images captured while the camera settings are being adjusted due to a difference in brightness. To this end, the control unit (200) initializes Frame_Skip_Current and increases it by 1 for every shot. The control unit (200) discards the captured images without saving them to the waiting buffer until Frame_Skip_Current reaches Skip_frame_count.

[0149] As shown in Fig. 9, when the door is opened while using the streaming service, the image acquired from the camera is not saved, and a replacement image is transmitted externally. Additionally, due to the delay of the sensor detecting the opening of the door, there is a possibility that the image may be saved after the door is opened. To prevent this, after the door opening is detected, the image in the waiting buffer is deleted to remove the image that may have been saved for a certain period of time before detection.

[0150] Additionally, when the door is closed while the time-lapse service is in use, an excessively dark or bright image is captured while the camera settings are adjusted due to the difference in brightness between the outside and the inside of the cooking appliance (1000a). To prevent the images captured during that period from being used, the saving of a certain number of images is skipped even after the door is closed. To do this, Frame_Skip_current and Skip_frame_count are used.

[0151] FIG. 10 is a diagram showing the process of selecting an alternative image according to one embodiment of the present invention.

[0152] The streaming service is a service that transmits video to the outside, and when the door is open (when a privacy protection event occurs), the device (1000) can transmit an alternative image.

[0153] To select a replacement image, the control unit (200) selects one or more images from among N frames of images taken before the privacy protection event occurs that have the lowest degree of change from the previous frame image (S41). An image with the lowest degree of change means an image that has not undergone a sudden change due to being taken after the door is opened.

[0154] Then, the control unit (200) performs alternative image display processing for the selected image (S42). For example, the control unit (200) can include an hourglass image in the selected image.

[0155] And the control unit (200) transmits the image of S42 until a privacy release event occurs (S43).

[0156] As a result, devices watching the streaming service display an alternative image, and viewers can confirm that an alternative image is being displayed during streaming for some reason. Since an image similar to the previously viewed image is displayed as the alternative, viewers may not feel any aversion to the alternative image.

[0157] An oven according to an embodiment of the present invention may have a door positioned on the front to open the interior. A camera (110) attached to the door or positioned at a specific location inside may film the cooking process.

[0158] A control unit (200) of a device (1000) having an oven as one embodiment can control a camera, a door, and a sensor unit (140) that detects the opening and closing of the door, etc., to control the operation of the device and process an image obtained from the camera (110).

[0159] Additionally, the control unit (200) can control the operation of the device (1000) and protect the privacy of the space where the device (1000) is placed by using an alternative image when the door opens while the camera (110) is taking pictures.

[0160] Additionally, the control unit (200) can control the camera (110) to delete images stored for a certain period of time when the door is opened while the camera is taking pictures. And when the door is closed, images are not stored for a certain period of time, thereby protecting privacy and maintaining image quality.

[0161] Although it has been described that all components constituting an embodiment of the present invention are combined or operate as a single unit, the present invention is not necessarily limited to such an embodiment, and within the scope of the purpose of the present invention, all components may be selectively combined in one or more ways to operate. Furthermore, while all components may each be implemented as a single independent piece of hardware, some or all of the components may be selectively combined to be implemented as a computer program having a program module that performs some or all of the combined functions on one or more pieces of hardware. The codes and code segments constituting the computer program can be easily inferred by those skilled in the art of the present invention. An embodiment of the present invention may be implemented by storing such a computer-readable storage medium and reading and executing it by a computer. The storage medium for the computer program includes a magnetic recording medium, an optical recording medium, and a storage medium including a semiconductor recording element. Additionally, a computer program implementing an embodiment of the present invention includes a program module that is transmitted in real time through an external device.

[0162] Although the present invention has been described above with reference to embodiments, various changes and modifications can be made by those skilled in the art. Therefore, it should be understood that such changes and modifications are included within the scope of the present invention as long as they do not depart from the scope of the invention.

Claims

1. A step in which the sensor part of the device confirms the occurrence of a privacy protection event; A step in which the control unit of the above device restricts shooting by the camera of the above device; The step of the sensor unit confirming the occurrence of a privacy release event; and A method for controlling video recording while protecting privacy, comprising the step of the above-mentioned control unit releasing the shooting restriction of the camera.

2. In Paragraph 1, The above device is a cooking appliance, and The above privacy protection event includes an opening event of a door that opens and closes the internal space of the cooking appliance, and A method for protecting privacy and controlling video recording, wherein the privacy release event includes a closing event of a door that opens and closes the internal space of the cooking appliance.

3. In Paragraph 1, The step of restricting the above shooting includes the step of the control unit turning off the power of the camera or stopping the camera from shooting, and A method for controlling video recording while protecting privacy, wherein the step of releasing the above-mentioned shooting restriction includes the step of the control unit turning on the power of the camera or resuming shooting of the camera.

4. In Paragraph 1, The above device further includes a communication unit that transmits the video captured by the camera in real time to an external server or a linked user terminal. The step of restricting the above shooting A method for controlling video recording while protecting privacy, further comprising the step of the communication unit transmitting a replacement image that replaces the video captured by the camera to the server or the user terminal.

5. In Paragraph 1, The step of restricting the above shooting A method for controlling video recording while protecting privacy, comprising the step of the control unit deleting an image captured and stored by the camera within a preset time interval based on the time when the privacy protection event occurs.

6. In Paragraph 5, The above control unit includes a buffer that stores the video captured by the camera frame by frame, and The step of restricting the above shooting A step of deleting frames of video captured within a preset time interval based on the time when the above privacy protection event occurred from the buffer; and A method for controlling video recording while protecting privacy, comprising the step of not storing the video in a buffer after the time when the above privacy protection event occurs.

7. In Paragraph 6, The step of releasing the above shooting restriction is A method for controlling video recording while protecting privacy, comprising the step of storing in a buffer a video recorded after a preset time interval based on the time when the above privacy release event occurs.

8. In Paragraph 5, The above deletion step A step of comparing a first image captured in the above-mentioned preset time interval with a second image captured before the above-mentioned time interval; and A method for controlling video recording while protecting privacy, comprising the step of deleting the first image when the difference between the first image and the second image is greater than or equal to a preset standard.

9. Camera; A sensor unit that detects the occurrence of privacy protection events and privacy release events; A device for protecting privacy and controlling video recording, comprising a control unit that controls the camera to restrict shooting by the camera when the privacy protection event occurs, and controls the camera to release the shooting restriction by the camera when the privacy release event occurs.

10. In Paragraph 9, The above device is a cooking appliance, and The above privacy protection event includes an opening event of a door that opens and closes the internal space of the cooking appliance, and A device that protects privacy and controls video recording, wherein the privacy release event includes a closing event of a door that opens and closes the internal space of the cooking appliance.

11. In Paragraph 9, The control unit limits the shooting of the camera by turning off the power of the camera or stopping the shooting of the camera, and The above control unit is a device that controls video recording while protecting privacy, which turns on the power of the camera or resumes shooting of the camera to release the shooting restriction of the camera.

12. In Paragraph 9, The above device further includes a communication unit that transmits the video captured by the camera in real time to an external server or a linked user terminal. A device that controls video recording while protecting privacy, wherein the communication unit transmits a replacement image that replaces the video recorded by the camera to the server or the user terminal in order to restrict the recording of the camera.

13. In Paragraph 9, The above control unit is a device that controls video recording while protecting privacy, which deletes images captured and stored by the camera within a preset time interval based on the time when the privacy protection event occurs.

14. In Paragraph 13, The above control unit includes a buffer that stores the video captured by the camera frame by frame, and The above control unit Frames of video captured within a preset time interval based on the time when the above privacy protection event occurred are deleted from the above buffer, and A device that controls video recording while protecting privacy, which does not store video in a buffer after the point in time when the above privacy protection event occurs.

15. In Paragraph 14, The above control unit is a device that controls video recording while protecting privacy, which stores video captured after a preset time interval based on the time when the privacy release event occurs in a buffer.

16. In Paragraph 13, The above control unit, in order to delete the frame of the image from the buffer, A device for controlling video recording while protecting privacy, which compares a first image captured in a preset time interval with a second image captured before the preset time interval, and deletes the first image if the difference between the first image and the second image is greater than or equal to a preset standard.

Images