System for determining placement position of temperature sensor in refrigerator and method for determining placement position of temperature sensor using same

Abstract

A system for determining a placement position of a temperature sensor, according to an embodiment, is characterized by comprising: a stored item identification unit that identifies stored items by capturing images of the inside of a refrigerator; a temperature sensor that is disposed on the inside of the refrigerator and measures the internal temperature of the refrigerator; and a placement position determination unit that determines and provides a placement position of the temperature sensor depending on the stored items stored in the refrigerator by using a placement position determination model for determining the placement position of the temperature sensor.

Description

A system for determining the placement position of a temperature sensor in a refrigerator and a method for determining the placement position of a temperature sensor using the same

[0001] The present invention relates to a system for determining a temperature sensor placement position in a refrigerator and a method for determining a temperature sensor placement position using the same, and more specifically, to a system for determining a temperature sensor placement position in a refrigerator and a method for determining a temperature sensor placement position using the same, which determines an optimal placement position of a temperature sensor in a refrigerator based on at least one of a type, volume, placement position, and allowable storage temperature of a stored item based on a pre-trained deep learning model, and then moves the temperature sensor to the placement position thus determined.

[0002] Food must be stored at the proper temperature for a variety of reasons. For example, proper temperature can prevent the growth and reproduction of microorganisms within the food. Otherwise, bacteria, mold, or fungi can grow within the food, leading to spoilage and the production of toxins.

[0003] Additionally, proper temperature helps preserve nutrients in food. For example, excessive temperature fluctuations can damage nutrients in food, as some vitamins and minerals can decompose or be lost at excessively high temperatures.

[0004] Additionally, proper temperature helps maintain the flavor and quality of food. Excessively high or low temperatures can compromise food freshness or flavor.

[0005] For these various reasons, refrigerators, a type of food storage facility, are found not only in homes but also in restaurants, hospitals, and hotels. These refrigerators incorporate the latest technology. For example, they not only maintain an appropriate temperature but also monitor the temperature inside the refrigerator and alert the user if an abnormality occurs.

[0006] Meanwhile, the refrigerated and frozen areas have distinct temperature settings. For example, the refrigerated area is typically set to 2 to 4 degrees Celsius, while the frozen area is typically set to -15 to -20 degrees Celsius.

[0007] At this time, the refrigerated area is set so that the entire area of ​​the internal space is usually maintained at 2 to 4 degrees Celsius, but there are often cases where such temperature is not maintained due to accidents such as opening and closing the refrigerator door or a power outage.

[0008] Additionally, most refrigerators have separate freshener compartments for storing vegetables, fruits, and other items. These freshener compartments are designed to minimize temperature fluctuations, often with separate drawer-type structures or covers. However, due to various reasons, such as those mentioned above, there may be instances where the temperature is not maintained.

[0009] The present invention is intended to solve the above-mentioned problem, and provides a method for determining the placement position of a temperature sensor in a refrigerator, which determines the optimal placement position of a temperature sensor in a refrigerator based on at least one of the type, volume, placement position, and allowable storage temperature of the stored item based on a pre-trained deep learning model, and then moves the temperature sensor to the determined position.

[0010] A system for determining a temperature sensor placement position in a refrigerator according to one embodiment includes: a storage item identification unit for photographing the inside of a refrigerator to identify a stored item; a temperature sensor placed in the refrigerator and for measuring the temperature inside the refrigerator; and a placement position determination unit for determining a placement position of the temperature sensor according to a stored item stored inside the refrigerator and providing the determined placement position using a placement position determination model for determining the placement position of the temperature sensor.

[0011] In addition, the storage item identification unit may include a photographing device that photographs the inside of the refrigerator; and a modeling unit that identifies the storage item based on the photographing results of the photographing device and generates three-dimensional modeling data for the storage item; and the generated three-dimensional modeling data may be input as input data to the placement location determination model.

[0012] In addition, the placement location determination model may be characterized in that it is learned using learning input data including at least one of the type of stored item, the volume of stored item, the placement location within the refrigerator, and the allowable storage temperature, and learning output data including the placement location of the temperature sensor within the refrigerator according to the learning input data.

[0013] In addition, the placement position determining unit may be characterized by redetermining the placement position of the temperature sensor based on the opening and closing operation of the refrigerator door and the temperature change inside the refrigerator according to the opening and closing operation.

[0014] Additionally, the point in time at which the position of the temperature sensor is re-determined may be determined dependent on the number of opening and closing operations of the refrigerator door.

[0015] A method for determining a location for a temperature sensor in a refrigerator according to a second embodiment includes the steps of: photographing the inside of the refrigerator to identify stored items; measuring the temperature inside the refrigerator using a temperature sensor placed inside the refrigerator; and determining and providing a location for the temperature sensor according to stored items stored inside the refrigerator using a location determination model for determining the location for the temperature sensor.

[0016] According to one embodiment of the present invention, the optimal placement of a temperature sensor within a refrigerator can be determined using a pre-trained deep learning model. Then, depending on the embodiment, the temperature sensor can be moved to the determined placement. This allows for more efficient maintenance of the appropriate temperature for food products sensitive to temperature changes, thereby maximizing freshness and flavor retention.

[0017] FIG. 1 is a drawing showing the configuration of a system for determining the placement position of a temperature sensor in a refrigerator according to one embodiment of the present invention.

[0018] Figure 2 is a flowchart showing the process of identifying a storage item in order through the storage item identification unit.

[0019] Figure 3 is a flowchart showing the entire process of adjusting the placement position of a temperature sensor through a moving means under the control of a placement position determination unit.

[0020] According to an embodiment of the present invention, there is provided a storage object identification unit that photographs the inside of a refrigerator to identify the type and volume of stored objects, and then models the arrangement shape of the stored objects in three dimensions based on the bottom surface on which the stored objects are placed and the door of the refrigerator by reflecting the identified type and volume; a temperature sensor that measures the temperature inside the refrigerator; And a placement position determination unit that inputs the result of the three-dimensional modeling into a predetermined placement position determination model learned to determine the placement position of the temperature sensor, thereby obtaining the placement position of the temperature sensor inside the refrigerator, wherein the placement position determination model is learned using a plurality of input data for learning and a plurality of correct data for learning, wherein each of the plurality of input data for learning includes modeling information that is modeled in three dimensions by reflecting the type and volume information of the predetermined storage object, in which the placement shape in which the predetermined storage object is arranged based on the bottom surface and the door of the refrigerator is reflected, and wherein each of the plurality of correct data for learning includes coordinate information on the position at which the temperature sensor should be arranged inside the refrigerator in order to measure the temperature of the predetermined storage object, wherein the placement position of the temperature sensor acquired from the placement position determination model is acquired at a different position if the placement shape of the stored object placed based on the bottom surface and the door inside the refrigerator is different even if the type and size are the same.

[0021] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings so that those skilled in the art can easily implement the present invention. The present invention may be implemented in various different forms and is not limited to the embodiments described herein.

[0022] In order to clearly explain the present invention, parts that are not related to the description are omitted, and the same reference numerals are used for identical or similar components throughout the specification.

[0023] In addition, in various embodiments, components having the same configuration are described only in representative embodiments using the same symbols, and in other embodiments, only configurations different from the representative embodiments are described.

[0024] Throughout the specification, when a part is said to be "connected" to another part, this includes not only "directly connected" but also "indirectly connected" with other elements intervening. Furthermore, when a part is said to "include" a component, this may mean that the other component is included, rather than excluded, unless otherwise specifically stated.

[0025] FIG. 1 is a drawing showing the configuration of a system (100) for determining the placement position of a temperature sensor in a refrigerator according to one embodiment of the present invention.

[0026] Referring to FIG. 1, a system (100) for determining the location of a temperature sensor in a refrigerator according to one embodiment of the present invention can be largely configured to include a storage item identification unit (110), a temperature sensor (120), a moving means (130), and a location determination unit (140).

[0027] First, the storage item identification unit (110) serves to identify the stored items by photographing the interior of the refrigerator. More specifically, the refrigerator of the present invention may include a cabinet forming a storage space (or storage area) and a door for opening and closing the storage space of the cabinet. In one embodiment, the cabinet may form a storage space divided into upper and lower sections, with a refrigerator compartment formed at the top and a freezer compartment formed at the bottom. The refrigerator compartment may be referred to as the upper storage space, and the freezer compartment may be referred to as the lower storage space.

[0028] The door may be configured to open and close the refrigerator and freezer compartments, respectively. In one embodiment, the door may be rotatably mounted on the cabinet and configured to open and close the refrigerator and freezer compartments, respectively, by rotating. Of course, the door may be configured to open and close the refrigerator or freezer compartments by opening and closing them.

[0029] The door may include a refrigerator door that opens and closes the refrigerator compartment, and a freezer door that opens and closes the freezer compartment. The refrigerator door may be referred to as an upper door, and the freezer door may be referred to as a lower door.

[0030] The refrigerator doors can be arranged in pairs, one on the left side and one on the right side. The left and right refrigerator doors can rotate independently to open and close the refrigerator. The left and right refrigerator doors can be arranged adjacent to each other and be of the same size.

[0031] The freezer door can also be arranged in pairs, with a left and right freezer door positioned side by side. The left and right freezer doors can rotate independently to open and close the freezer. The left and right freezer doors can be arranged adjacent to each other and be of the same size.

[0032] Of course, in this embodiment, for the convenience of explanation and understanding, a refrigerator having a structure in which a refrigerator compartment is placed above a freezer compartment is described as an example, but the present invention is not limited to the shape of the refrigerator and can be applied to all types of refrigerators equipped with a door.

[0033] Meanwhile, the door forms the front appearance of the refrigerator when closed, and can form the appearance of the refrigerator when the refrigerator is installed and seen from the front.

[0034] The door may have a structure that allows the front surface to be selectively illuminated and may be configured to illuminate in a set color or brightness. Thus, the user can change the color or brightness of the front surface of the door without disassembling or disassembling the door, thereby altering the overall appearance of the refrigerator.

[0035] In the present invention, the refrigerator door may be referred to as a panel door or a first door. The right refrigerator door may have a structure different from other doors and may have a structure that allows the rear space, i.e., the space inside the refrigerator, to be seen through. Therefore, the right refrigerator door may be referred to as a transparent door, a see-through door, or a second door.

[0036] Additionally, in one embodiment, the right refrigerator door may be equipped with an Android-based display. In this case, the right refrigerator door may be both a transparent door and a display door. In this case, the right refrigerator door may be referred to as a display door.

[0037] In addition, in one embodiment, a guide rail (not shown) may be provided inside the refrigerator as a movement path for the temperature sensor (120) to be described later to be moved. A plurality of guide rails may be provided in the horizontal and vertical directions on the inner wall of the refrigerator. A movement means (130) connected to the temperature sensor (120) to be described later is provided on the guide rail. The movement means (130) may refer to a physical device for physically moving the temperature sensor (120) so that the temperature sensor (120) can be moved to a placement position determined by a placement position determining unit (140) to be described later. For example, the movement means (130) may be a linear motor including a wheel that rotates along the guide rail. This will be described later.

[0038] The storage identification unit (110) may include a photographing device (111) that photographs the inside of a refrigerator, and a modeling unit (112) that identifies the storage within the photographed image captured by the photographing device (111) and generates three-dimensional modeling data for the storage.

[0039] A photographing device (111) is installed inside a refrigerator and can photograph the inside of the refrigerator and transmit the photographed image to a modeling unit (112). The modeling unit (112) can then use this to create a three-dimensional arrangement image based on not only three-dimensional modeling data but also the arrangement of each stored item. This photographing device (111) can be designed to operate without problems even at low temperatures inside a refrigerator.

[0040] The modeling unit (112) can identify the stored object based on the photographed image transmitted from the photographing device (111). For this purpose, the modeling unit (112) includes an object identification model. The object identification model of the modeling unit (112) is a model trained to identify the stored object by learning a plurality of photographed images of the stored object. The modeling unit (112) uses this to identify what kind of food or food the stored object is stored in the refrigerator, and the size and type of the food or food, and generates 3D modeling data for the stored object based on this. Here, the 3D modeling data may mean a virtual 3D image of the stored object implemented based on the stored object. Here, since the method for generating the 3D modeling data itself is a known technology, further description thereof will be omitted.

[0041] The 3D modeling data generated through the modeling unit (112) can be provided as input data for the placement location determination unit (140) described below. The process of identifying a storage item through the storage item identification unit (110) is as follows in order.

[0042] Figure 2 is a flowchart showing the process of identifying a storage item in order through the storage item identification unit (110).

[0043] Referring to Fig. 2, first, the photographing device (111) of the storage object identification unit (110) photographs the inside of the refrigerator to create a photographed image (S201), and then transmits the created photographed image to the modeling unit (120) (S202). The modeling unit (120) identifies the type of the stored object in the photographed image (S203), and then creates 3D modeling data corresponding to the stored object based on this (S204). The created 3D modeling data can be input as input data to the arrangement position determination unit (140) described below.

[0044] Meanwhile, in one embodiment, the storage item identification unit (110) can generate a three-dimensional arrangement image for each storage item in the process of acquiring an image by photographing the inside of the refrigerator using the photographing device (111). More specifically, the storage item identification unit (110) can divide the inside of the refrigerator into a plurality of virtual spaces based on the photographed image taken of the inside of the refrigerator, and the virtual spaces thus divided can be used to determine the arrangement position of the temperature sensor (120) in the arrangement position determination unit (140) described below.

[0045] A temperature sensor (120) is placed inside a refrigerator and serves to measure the temperature inside the refrigerator. More specifically, the temperature sensor (120) is designed to operate without problems even at low temperatures in the refrigerated or frozen areas inside the refrigerator, and its location can be freely moved inside the refrigerator using a moving means (130) described below. A plurality of temperature sensors (120) may be provided inside the refrigerator. This is to place a temperature sensor (120) for each stored item when there are multiple items requiring separate temperature management inside the refrigerator, or to place the temperature sensor (120) in two or more locations for a single stored item. In this case, the temperature value measured and collected by each temperature sensor (120) can be transmitted to a management terminal or a user terminal via a separate wired or wireless network communication means provided in the refrigerator.

[0046] The moving means (130) serves to move the placement position of the temperature sensor (120) so that it can move freely inside the refrigerator. More specifically, the moving means (130) serves to physically change the placement position of the temperature sensor (120) according to the placement position determined through the placement position determining unit (140) described below. As described above, the moving means (130) can be moved along a plurality of guide rails (not shown) installed along the inner wall of the refrigerator, and the temperature sensor (120) can be electrically connected to the moving means (130).

[0047] Meanwhile, a separate support for attaching a temperature sensor (120) may be provided on the moving means (130). One side of the support may be attached to the temperature sensor (120), and the other side may be attached to the moving means (130). The support may be formed to be detachable from the temperature sensor (120) or the moving means (130). Accordingly, when a specific temperature sensor (120) breaks down, the support may be detached from the moving means (130), or the temperature sensor (120) itself may be detached from the support.

[0048] At this time, the support member can be electrically connected by making contact with the moving means (130) at the same time as the connection, and the temperature sensor (120) can also be electrically connected by making contact with the support member at the same time as the connection.

[0049] The moving means (130) can be supplied with power from a power supply (not shown) mounted on the refrigerator, and the supplied power can be applied to the temperature sensor (120) through an electrically contacted support.

[0050] The placement position determination unit (140) trains a placement position determination model for determining the placement position of the temperature sensor (120), determines the placement position of the temperature sensor (120) according to the stored items stored inside the refrigerator, and then moves the temperature sensor (120) to the predetermined placement position using the previously described moving means (130). This will be examined in more detail as follows.

[0051] Figure 3 is a flowchart showing the entire process of adjusting the placement position of a temperature sensor (120) through a moving means (130) under the control of a placement position determination unit (140).

[0052] Referring to Figure 3, a placement location determination model capable of determining the placement location of a temperature sensor (120) is prepared (S301). Here, "preparation" refers to acquiring a model whose learning has already been completed. This placement location determination model may be learned using a deep learning method, which will be described in more detail later.

[0053] Thereafter, the 3D modeling data previously generated through the modeling unit (112) is input as input data to the placement location determination model (S303). As previously explained, the generation of this 3D modeling data utilizes images of the refrigerator interior captured by the camera as well as information on stored items.

[0054] The placement location determining unit (140) determines the placement location of the temperature sensor (120) among the locations inside the refrigerator based on 3D modeling data for each of the items stored inside the refrigerator using a placement location determining model (S304). Once the placement location is determined, the placement location determining unit (140) causes the temperature sensor (120) to be moved to the corresponding placement location using a moving means (130) (S305).

[0055] An example of this process is as follows:

[0056] For example, assuming that the current internal temperature of the refrigerated area is set to 2 to 4 degrees Celsius, and that food (storage items) that must be stored within this temperature range, that is, within 2 to 4 degrees Celsius, is stored in the refrigerator, the following is true.

[0057] When the food is stored inside the refrigerator, the storage identification unit (110) photographs the food and the inside of the refrigerator using a photographing device (111), and the modeling unit (112) uses an object identification model to identify the type of the food, as well as its shape, size, volume, and the state in which it is loaded inside the refrigerator.

[0058] Based on this, the modeling department (112) creates 3D modeling data for the food and then provides it to the placement location determination department (140).

[0059] The placement location determination unit (140) inputs the provided 3D modeling data into the placement location determination model to obtain the location where the temperature sensor (120) should be placed for the corresponding food (storage item).

[0060] For example, depending on the type of food, as well as its shape, size, volume, or loading state, the part of the food that requires temperature control may be specified. For example, if the identified food is beef that is long and vertically stacked, the parts that require temperature control may be the topmost part in the vertically stacked state, the part closest to the refrigerator door, and the bottommost part in the vertically stacked state. This is because the topmost part may have the highest average temperature, the part closest to the refrigerator door may experience the greatest temperature change depending on the opening and closing of the door, and the bottommost part in the vertically stacked state may have the lowest temperature.

[0061] Alternatively, if the identified food is ice cream, the temperature may be measured at only one or more locations expected to be the hottest, such as, but not limited to, the highest point relative to the floor when the ice cream is loaded or the location closest to the refrigerator door.

[0062] Next, the temperature sensor (120) is moved to the placement position of the temperature sensor (120) determined by the placement position determination unit (140) under the control of the moving means (130).

[0063] Meanwhile, there may be more than one of these foods (storage items) in the refrigerator, and in this case, the placement location determining unit (140) may place a temperature sensor (120) for each food (storage item).

[0064] In addition, in one embodiment, the arrangement position determination unit (140) can predict the temperature and humidity of each area where each stored item is located inside the refrigerator using a temperature and humidity simulation model. More specifically, the temperature and humidity simulation model is a model for predicting the temperature and humidity of each area inside the refrigerator. The arrangement position determination unit (140) receives a three-dimensional arrangement image of the inside of the refrigerator from the previously described storage item identification unit (110). Here, the three-dimensional arrangement image is an image having vector values ​​for identifying the structure inside the refrigerator (including the refrigerated area and the frozen area) in terms of the x-axis, y-axis, and z-axis.

[0065] In addition, the photographed image captured by the photographing device (111) of the storage identification unit (110) may include location and height information for each floor of the internal space of the refrigerator, and the placement position of the temperature sensor (120) may be calculated as a three-dimensional coordinate value that includes both location information and height information on the horizontal plane.

[0066] The placement location determination unit (140) can implement a temperature and humidity simulation model by training a deep learning neural network with input data, output data, and information about the internal and external environment of the refrigerator as learning data. In an embodiment, the deep learning neural network includes at least one of a DNN (Deep Neural Network), a CNN (Convolutional Neural Network), an RNN (Recurrent Neural Network), and a BRDNN (Bidirectional Recurrent Deep Neural Network), but is not limited thereto.

[0067] The interior of a refrigerator has a multi-layered structure to store various foods. Each layer contains compartments for storing various foods. Therefore, a 3D layout image can reflect the vector values ​​for each compartment, as well as detailed vector values ​​for the items stored within each compartment.

[0068] The arrangement position determination unit (140) analyzes the three-dimensional arrangement image to determine the protrusion and depth information of each stored item or food inside the refrigerator due to the arrangement of the stored items. Thereafter, the arrangement position determination unit (140) can output the temperature and humidity of each area for each input compartment space. The temperature or humidity may also be affected by the arrangement of the stored items inside the refrigerator and the distance between the stored items and the inner wall of the refrigerator. Therefore, the arrangement position determination unit (140) may also receive information on the position and shape of the inner wall of the refrigerator from the stored item identification unit (110).

[0069] In addition, in one embodiment, the placement position determining unit (140) may re-determine the placement position of the temperature sensor (120) based on the opening and closing operation of the refrigerator door and the temperature change inside the refrigerator according to the opening and closing operation. More specifically, the placement position determining unit (140) may monitor the opening and closing operation of the refrigerator door and the temperature change inside the refrigerator according to the opening and closing operation in real time. For example, the opening and closing operation of the refrigerator door may be monitored in real time through a motion detection sensor or an opening and closing detection sensor connected to the refrigerator door, and the temperature change inside the refrigerator according to the opening and closing operation may be monitored in real time through the temperature sensor (120).

[0070] Accordingly, if the placement position determining unit (140) determines that a change in temperature inside the refrigerator has occurred due to the opening and closing operation of the refrigerator door, or that the temperature around the stored items managed by each temperature sensor (120) has changed within a significant range, the placement position determining unit (140) can redetermine the placement position of each temperature sensor (120) placed inside the refrigerator. For example, the placement position determining unit (140) can redetermine the placement position of the temperature sensor (120) every time the refrigerator door is opened and closed.

[0071] Meanwhile, in this process, the placement position determination unit (140) counts the number of opening and closing operations of the refrigerator door, and can also adjust the timing for re-determining the placement position of the temperature sensor (120) based on the count result.

[0072] More specifically, the arrangement position determining unit (140) can count the number of times the refrigerator door is opened and closed and learn the change in temperature inside the refrigerator according to the count result. For example, if it is determined that the temperature inside the refrigerator has significantly changed 3 to 4 times or more as a result of counting the number of times the refrigerator door is opened and closed and learning the change in temperature inside the refrigerator according to the count result, the arrangement position determining unit (140) can maintain the arrangement position of each temperature sensor (120) as it is when the refrigerator door is opened and closed less than 3 times, and can re-determine the arrangement position of each temperature sensor (10) when it is determined that the refrigerator door has been opened and closed 3 times or more.

[0073] Meanwhile, the artificial network used in the position determination unit (140) of the present invention is as follows.

[0074] First, the placement location determination model of the placement location determination unit (140) may be a deep learning or machine learning model, trained using a supervised learning method. In this case, the input data for training may be the aforementioned 3D modeling data. Here, the 3D modeling data may include modeling of the shape, type, size, and volume of the stored items, as well as the shape of the refrigerator or freezer where such items are stored, i.e., the shape of the housing, and the door of the refrigerator or freezer.

[0075] In addition, the training correct answer data corresponding to such training input data may include as correct answers the locations where temperature sensors should be placed when various types of stored items are stored in a refrigerator or freezer. For example, even for the same beef, the location where the temperature sensor should be placed may differ in the correct answers depending on its shape. More specifically, a protruding part may correspond to a location where the temperature sensor should be placed compared to a non-protruding part. In addition, even for the same type of beef, the location of the temperature sensor may be different depending on whether the beef is vertically or horizontally loaded in the refrigerator. Specifically, in the case of vertical loading, the required location of the temperature sensor may be the highest location relative to the bottom of the refrigerator, whereas in the case of horizontal loading, the required location of the temperature sensor may be the location closest to the refrigerator door.

[0076] That is, according to one embodiment, depending on the characteristics of each stored item, such as food of various types, shapes, and volumes, the location of the temperature sensor within the refrigerator or freezer can be determined based on data using machine learning or deep learning. Accordingly, refrigerator managers or those monitoring the safety of food within the refrigerator can reliably manage or monitor temperature changes detected by temperature sensors placed at these determined locations.

[0077] Meanwhile, the learning method of the aforementioned placement location determination model will be described below. In this specification, a model may refer to any form of computer program that operates based on a network function, an artificial neural network, or a neural network. Throughout this specification, the terms "model," "neural network," "network function," and "neural network" may be used interchangeably.

[0078] A neural network is a network in which one or more nodes are interconnected through one or more links, forming input and output node relationships within the network. The characteristics of a neural network can be determined by the number of nodes and links, the relationships between nodes and links, and the weights assigned to each link. A neural network can be composed of a set of one or more nodes. A subset of the nodes constituting a neural network can form a layer.

[0079] Among neural networks, a deep neural network (DNN) can refer to a neural network that includes multiple hidden layers in addition to an input and output layer. As illustrated in Figure 3, a deep neural network can have one or more, and preferably two or more, hidden layers in the middle.

[0080] These deep neural networks may include convolutional neural networks (CNNs), recurrent neural networks (RNNs), long short-term memory (LSTM) networks, generative pre-trained transformers (GPTs), autoencoders, generative adversarial networks (GANs), restricted boltzmann machines (RBMs), deep belief networks (DBNs), Q networks, U networks, Siamese networks, generative adversarial networks (GANs), transformers, etc. The aforementioned deep neural networks may be trained using a transfer learning method. Transfer learning involves a pre-training process and a fine-tuning process.

[0081] Here, in the pre-training process, a large amount of unlabeled training data is used for training suitable for the first task. As a result, a pre-trained model (or base model) is obtained. Furthermore, in the fine-tuning process, labeled training data is used for training using supervised learning to suit the second task. As a result, the final model desired through transfer learning is obtained.

[0082] One of the models trained using this transfer learning method is BERT (Bidirectional Encoder Representations from Transformers), but is not limited to this.

[0083] Neural networks, including the aforementioned deep neural networks, can be trained to minimize output errors. Training a neural network involves repeatedly inputting training data into the neural network, calculating the neural network output and target error for the training data, and backpropagating the neural network error from the output layer to the input layer to update the weights of each node in the neural network to reduce the error.

[0084] Meanwhile, a model according to one embodiment may be implemented to borrow at least a portion of a transformer. Here, the transformer may be composed of an encoder that encodes embedded data and a decoder that decodes the encoded data. The transformer may have a structure that receives a series of data, performs encoding and decoding steps, and outputs a series of data of different types. In one embodiment, the series of data may be processed into a form operable by the transformer. The process of processing the series of data into a form operable by the transformer may include an embedding process. Expressions such as data tokens, embedding vectors, and embedding tokens may refer to data embedded in a form that the transformer can process.

[0085] To enable a Transformer to encode and decode a series of data, the encoders and decoders within the Transformer can utilize an attention algorithm. Here, the attention algorithm can refer to an algorithm that, for a given query, calculates the similarity for one or more keys, reflects this similarity in the values ​​corresponding to each key, and then weights and adds the values ​​to calculate an attention value.

[0086] At this point, various types of attention algorithms can be categorized depending on how the query, key, and value are set. For example, attention can be obtained by setting the query, key, and value to be identical, which may indicate a self-attention algorithm. Alternatively, attention can be obtained by reducing the dimensionality of the embedding vector and obtaining individual attention heads for each segmented embedding vector to process a series of input data in parallel, which may indicate a multi-head attention algorithm.

[0087] In one embodiment, a transformer may be composed of modules that perform multiple multi-head self-attention algorithms or multi-head encoder-decoder algorithms. In one embodiment, the transformer may also include additional components that are not attention algorithms, such as embedding, normalization, or softmax. Methods for constructing a transformer using attention algorithms may include methods disclosed in Vaswani et al., Attention Is All You Need, 2017 NIPS, which are incorporated herein by reference.

[0088] Transformers can be applied to various data domains, such as embedded natural language, segmented image data, or audio waveforms. As a result, a transformer can transform a series of input data into a series of output data. Data from various data domains can be transformed to be processed by a transformer, a process called embedding.

[0089] Additionally, the transformer may process additional data representing the relative positional relationship or phase relationship between a series of input data. Alternatively, vectors representing the relative positional relationship or phase relationship between the input data may be additionally reflected in the series of input data to embed the series of input data. In one example, the relative positional relationship between the series of input data may include, but is not limited to, word order within a natural language sentence, the relative positional relationship between each segmented image, the time order of segmented audio waveforms, etc. The process of adding information representing the relative positional relationship or phase relationship between a series of input data may be referred to as positional encoding.

[0090] Although the present invention has been described above with reference to preferred embodiments thereof, it will be understood by those skilled in the art that various modifications and changes may be made to the present invention without departing from the spirit and scope of the present invention as set forth in the claims below.

[0091] The present invention utilizes a deep learning model to determine the optimal placement of a temperature sensor within a refrigerator and implements movement of the temperature sensor to the determined location, thereby efficiently maintaining an appropriate temperature corresponding to the refrigerator's diverse structures and usage environments. Therefore, the present invention is expected to enhance consumer satisfaction and stimulate product production.

Claims

1. A storage item identification unit that photographs the inside of a refrigerator to identify the type and volume of stored items, and then models the arrangement shape of the stored items in three dimensions based on the floor surface where the stored items are placed and the door of the refrigerator, reflecting the identified type and volume; A temperature sensor that measures the temperature inside the refrigerator; and The above three-dimensional modeled result is input into a predetermined placement location determination model learned to determine the placement location of the temperature sensor, and a placement location determination unit is included to obtain the placement location of the temperature sensor inside the refrigerator. The above placement location decision model is learned using multiple learning input data and multiple learning correct data. Each of the above multiple learning input data, It includes modeling information that models the arrangement shape in which the specified storage items are arranged based on the floor surface and the door of the refrigerator in three dimensions by reflecting the type and volume information of the specified storage items. Each of the above multiple learning correct answer data is In order to measure the temperature of the above-mentioned stored material, coordinate information is included regarding the location where the temperature sensor should be placed inside the refrigerator. The placement location of the temperature sensor obtained from the above placement location determination model is, A system for determining the placement position of a temperature sensor inside a refrigerator, characterized in that when the arrangement shape of stored items based on the floor surface and door inside the refrigerator is different even if the type and size are the same, the temperature sensor placement position is acquired at a different position.

2. In paragraph 1, The above storage identification section is, A camera for taking pictures inside a refrigerator; and A system for determining the placement location of a temperature sensor in a refrigerator, comprising: a modeling unit that identifies the type and volume of a stored object based on the shooting results of the above shooting device and generates three-dimensional modeling data for the stored object.

3. In paragraph 1, The above placement location determining unit is, A system for determining the arrangement position of a temperature sensor inside a refrigerator, characterized in that the arrangement position of the temperature sensor is re-determined based on the opening and closing operation of the refrigerator door and the temperature change inside the refrigerator according to the opening and closing operation.

4. In paragraph 3, The timing for re-determining the placement of the above temperature sensor is: A system for determining the placement position of a temperature sensor inside a refrigerator, the location of which is determined dependent on the number of opening and closing operations of the refrigerator door.

5. A method for determining the placement location of a temperature sensor in a refrigerator, performed by a system for determining the placement location of a temperature sensor in a refrigerator, A step of photographing the inside of a refrigerator to identify the type and volume of stored items, and then modeling the arrangement shape of the stored items based on the floor surface where the stored items are placed and the door of the refrigerator in three dimensions to reflect the identified type and volume; Step for measuring the temperature inside the refrigerator; and A step of inputting the result of the three-dimensional modeling into a predetermined placement location determination model learned to determine the placement location of the temperature sensor, thereby obtaining the placement location of the temperature sensor inside the refrigerator, The above placement location decision model is learned using multiple learning input data and multiple learning correct data. Each of the above multiple learning input data, It includes modeling information that models the arrangement shape in which the specified storage items are arranged based on the floor surface and the door of the refrigerator in three dimensions by reflecting the type and volume information of the specified storage items. Each of the above multiple learning correct answer data is In order to measure the temperature of the above-mentioned stored material, coordinate information is included regarding the location where the temperature sensor should be placed inside the refrigerator. The placement location of the temperature sensor obtained from the above placement location determination model is, A method for determining the placement position of a temperature sensor inside a refrigerator, characterized in that when the arrangement shape of stored items based on the floor surface and door inside the refrigerator is different even if the type and size are the same, the temperature sensor is acquired at a different position.

Images