Electronic device for driving self-regulating ventilation system using prediction model and method for driving same

Abstract

According to various embodiments, an electronic device for driving a self-regulating ventilation system using a prediction model may comprise: at least one camera module installed inside a building to which a ventilation system is applied; at least one ventilation control module; and a processor that acquires at least one piece of image data obtained by the at least one camera module capturing images of at least one identified person present inside the building and / or in a room and regulates the at least one ventilation control module, wherein the processor is configured to: output detection prediction data for the number of identified people present inside the building; when the detection prediction data is determined to be greater than or equal to a preset first threshold value, regulate and drive the at least one ventilation control module to a first level of ventilation intensity; and when the detection prediction data is determined to be greater than or equal to a second threshold value, which is set to be greater than the first threshold value, regulate and drive the at least one ventilation control module to a second level of ventilation intensity.

Description

Electronic device for driving an autonomous ventilation system using a prediction model and a method for driving the same

[0001] Various embodiments of the present invention relate to an electronic device and a method for driving an autonomous ventilation system utilizing a prediction model. More specifically, the invention relates to an electronic device and a method for driving an autonomous ventilation system utilizing a prediction model, which can easily verify the presence of an identifier based on output result data obtained by inputting infrared image data preprocessed from an infrared camera module into a prediction model and automatically drive the ventilation system accordingly.

[0002] Recently, air pollution, including fine dust, has become increasingly severe, necessitating comfortable indoor and bathroom environments to prepare for it. In particular, there is a need to develop smart, consumer-customized optimal ventilation systems, as well as systems capable of identifying the number of people indoors and adjusting the ventilation volume accordingly.

[0003] However, conventionally, there was no such device or system as described above, and there was a problem in that the user simply had to directly operate an air purifier equipped with a ventilation system that could be installed inside the building's ventilation ducts.

[0004] Meanwhile, to automate such ventilation systems, artificial intelligence and AI capable of directly identifying identifiers based on it are being implemented.

[0005] However, these conventional artificial intelligence systems had a problem in that they were simply focused on accuracy and lacked separate preprocessing, making it difficult to accurately distinguish images while also making it difficult to easily identify identifiers.

[0006] Accordingly, the present embodiment relates to an electronic device and a method for driving an autonomous ventilation system utilizing a prediction model, wherein data acquired from an infrared camera module is preprocessed and input into a prediction model, and the ventilation system is driven based on the number of identifiers output from the data output from the prediction model.

[0007] According to various embodiments, an electronic device for driving an autonomous ventilation system utilizing a prediction model comprises: at least one camera module installed inside a building where the ventilation system is applied; at least one ventilation control module; and a processor that acquires at least one image data of at least one identifier existing inside the building and / or indoor space through the at least one camera module and controls the at least one ventilation control module, wherein the processor outputs detection prediction data regarding the number of identifiers existing inside the building, and if the detection prediction data is determined to be greater than or equal to a preset first threshold, the at least one ventilation control module is driven by controlling the ventilation intensity to a level 1, and if the detection prediction data is determined to be greater than or equal to a second threshold set to be greater than the first threshold, the at least one ventilation control module is driven by controlling the ventilation intensity to a level 2.

[0008] According to the present embodiment, by inputting corrected image data generated by correcting based on a pre-processed infrared camera image from an infrared camera module into a prediction model, identifiers for buildings and / or indoors included in the output detection prediction data can be easily extracted, and the ventilation control module can be driven according to the number of extracted identifiers identified in the buildings and / or indoors to automatically adjust the intensity of ventilation.

[0009] According to another embodiment, infrared image data captured using a single infrared camera module is corrected and input into a prediction model, and by verifying the movement of an identifier identified in the detection prediction data output from the prediction model and the time value set for external movement, the intensity of the ventilation control module can be maintained even if the identifier actually present inside the building is in a blind spot, and the intensity of the ventilation control module can be changed by identifying an identifier that has clearly moved outside the building. Since this can be performed automatically, there is an advantage that there is no need for a separate manager to verify it.

[0010] FIG. 1 illustrates a block diagram of an electronic device and network according to various embodiments of the present invention.

[0011] FIG. 2 is a flowchart illustrating how an electronic device operates according to various embodiments.

[0012] FIG. 3 is a flowchart illustrating different ways in which an electronic device operates according to various embodiments.

[0013] FIG. 4 is an illustrative diagram for explaining the basic operation of a camera module according to various embodiments.

[0014] FIG. 5 is an illustrative diagram for explaining the basic structure of a prediction model according to various embodiments.

[0015] FIGS. 6 and 7 are example diagrams of a prediction model extracting an identifier captured by a camera module according to various embodiments.

[0016] FIG. 8 is an example diagram for determining the case of an identifier that is not captured by a camera module by a prediction model according to various embodiments.

[0017] Hereinafter, various embodiments of this document are described with reference to the accompanying drawings. The embodiments and the terms used therein are not intended to limit the technology described in this document to specific embodiments and should be understood to include various modifications, equivalents, and / or substitutions of said embodiments. In relation to the description of the drawings, similar reference numerals may be used for similar components. A singular expression may include a plural expression unless the context clearly indicates otherwise. In this document, expressions such as "A or B" or "at least one of A and / or B" may include all possible combinations of items listed together. Expressions such as "first," "second," "first," or "second" may modify said components regardless of order or importance and are used only to distinguish one component from another and do not limit said components. When it is mentioned that a certain (e.g., 1st) component is "(functionally or telecommunicationally) connected" or "connected" to another (e.g., 2nd) component, said certain component may be directly connected to said other component or connected through another component (e.g., 3rd component).

[0018] In this document, "configured to" may be used interchangeably with, depending on the context, for example, hardware- or software-wise, "suitable for," "capable of," "modified to," "made to," "capable of," or "designed to." In some cases, the expression "device configured to" may mean that the device is "capable of" in conjunction with other devices or components. For example, the phrase "processor configured to perform A, B, and C" may mean a dedicated processor for performing the corresponding operations (e.g., an embedded processor), or a general-purpose processor capable of performing the corresponding operations by executing one or more software programs stored in a memory device (e.g., a CPU or application processor).

[0019] An electronic device according to various embodiments of the present document may include, for example, at least one of a smartphone, a tablet PC, a desktop PC, a laptop PC, a netbook computer, a workstation, and a server.

[0020] Referring to FIG. 1, an electronic device (101) within a network environment (100) in various embodiments is described. The electronic device (101) may include a bus (110), a processor (120), a memory (130), an input / output interface (140), a display (150), a communication interface (160), and at least one ventilation control module (170). In some embodiments, the electronic device (101) may omit at least one of the components or additionally include other components. The bus (110) may include a circuit that connects the components (110-170) to each other and transmits communication (e.g., control messages or data) between the components. The processor (120) may include one or more of a central processing unit, an application processor, or a communication processor (CP). The processor (120) may, for example, perform operations or data processing regarding the control and / or communication of at least one other component of the electronic device (101).

[0021] The memory (130) may include volatile and / or non-volatile memory. The memory (130) may store instructions or data related to at least one other component of the electronic device (101), for example. According to one embodiment, the memory (130) may store software and / or programs.

[0022] The input / output interface (140) can, for example, transmit commands or data input from a patient or other external device to other component(s) of the electronic device (101), or output commands or data received from other component(s) of the electronic device (101) to the patient or other external device. Additionally, the input / output interface (140) may include an infrared camera module (141) installed externally. As an example, the infrared camera module (141) can obtain complete data regarding the thermal state of an object or facility by using a technology that detects infrared radiation emitted from an object with a special sensor and images the temperature value. Additionally, the infrared camera module (141) may have a highly sophisticated algorithm built in and an external process installed internally. In addition, the infrared camera module (141) has the advantage of being able to easily and quickly identify problems that are not visible to the naked eye, thereby preventing accidents and reducing maintenance costs by identifying problems in facilities such as power plants, steel mills, and power companies where continuous operation is important, as well as airports, oil refineries, and public facilities.

[0023] The display (150) may include, for example, a liquid crystal display (LCD), a light-emitting diode (LED) display, an organic light-emitting diode (OLED) display, a microelectromechanical system (MEMS) display, or an electronic paper display. The display (150) may display various content (e.g., text, images, videos, icons, and / or symbols, etc.) to a patient, for example. The display (150) may include a touch screen and may receive touch, gesture, proximity, or hovering input using, for example, an electronic pen or a part of the patient's body. The communication interface (160) may establish communication between, for example, the electronic device (101) and an external device (e.g., a first external electronic device (102), a second external electronic device (104), or a server (108)). For example, the communication interface (160) can be connected to a network (161) via wireless or wired communication to communicate with an external device (e.g., a second external electronic device (104) or a server (108)).

[0024] Wireless communication may include cellular communication using at least one of, for example, LTE, LTE-A (LTE Advance), CDMA (code division multiple access), WCDMA (wideband CDMA), UMTS (universal mobile telecommunications system), WiBro (Wireless Broadband), or GSM (Global System for Mobile Communications). According to one embodiment, wireless communication may include at least one of, for example, WiFi (wireless fidelity), Bluetooth, Bluetooth Low Energy (BLE), Zigbee, NFC (near field communication), Magnetic Secure Transmission, Radio Frequency (RF), or Body Area Network (BAN). According to one embodiment, wireless communication may include GNSS. GNSS may be, for example, GPS (Global Positioning System), Glonass (Global Navigation Satellite System), Beidou Navigation Satellite System (hereinafter "Beidou"), or Galileo, the European global satellite-based navigation system. Hereinafter, in this document, "GPS" may be used interchangeably with "GNSS".Wired communication may include, for example, at least one of USB (Universal Serial Bus), HDMI (High Definition Multimedia Interface), RS-232 (Recommended Standard 232), power line communication, POTS (Plain Old Telephone Service), USART (Universal Synchronous / Asynchronous Receiver / Transmitter), I2C (Inter-Integrated Circuit), SPI (Serial Peripheral Interface), CAN (Controller Area Network), UART (Universal Asynchronous Receiver / Transmitter), Ethernet, DSI (Display Serial Interface), CSI (Camera Serial Interface), GPIO (General Purpose Input / Output), etc. These communication protocols cover wired communication methods widely used in microcontrollers and other embedded systems. The network (161) may include at least one of a telecommunication network, for example, a computer network (e.g., LAN or WAN), the Internet, or a telephone network.

[0025] At least one ventilation control module (170) may be composed of a duct that draws in indoor air and discharges it outside, and a duct that draws in outside air and supplies it to the indoor space. Additionally, at least one ventilation control module (170) may be composed of a heat exchanger (a waste heat recovery type ventilation device, a ventilation unit), and the heat exchanger may be largely composed of a blower consisting of an exhaust fan and an intake fan plus a motor, a filter, a heat exchanger called a heat element, and a rectangular enclosure covering them. Furthermore, at least one ventilation control module (170) may be equipped with a blower that discharges or draws in air, a heat exchanger including a heat element, and a filtration device, and a filter that cleanly filters outside air as an essential device for controlling ventilation. In this embodiment, at least one ventilation control module (170) may be installed inside a building or as a separate device inside a building, but is not limited thereto and may be used in various ways.

[0026] Each of the first and second external electronic devices (102, 104, 106) may be of the same or different type as the electronic device (101). According to various embodiments, all or part of the operations performed on the electronic device (101) may be performed on one or more other electronic devices (e.g., electronic devices (102, 104, 106), or a server (108)). According to one embodiment, when the electronic device (101) needs to perform a function or service automatically or upon request, the electronic device (101) may request at least some associated function from another device (e.g., electronic devices (102, 104, 106), or a server (108)) instead of performing the function or service itself or additionally. The other electronic device (e.g., electronic devices (102, 104, 106), or a server (108)) may perform the requested function or additional function and transmit the result to the electronic device (101). The electronic device (101) can provide the requested function or service by processing the received result as is or additionally. For this purpose, for example, cloud computing, distributed computing, or client-server computing technology may be used.

[0027]

[0028] FIG. 2 is a flowchart illustrating how an electronic device operates according to various embodiments.

[0029] FIG. 4 is an illustrative diagram for explaining the basic operation of a camera module according to various embodiments.

[0030] FIG. 5 is an illustrative diagram for explaining the basic structure of a prediction model according to various embodiments.

[0031] FIGS. 6 and 7 are example diagrams of a prediction model extracting an identifier captured by a camera module according to various embodiments.

[0032]

[0033] In operation 201, an electronic device (101) (e.g., the processor (120) of FIG. 1) can acquire at least one image data of at least one identifier existing inside a building and / or indoors through at least one camera module. According to one embodiment, the at least one image data may include infrared image data acquired from an infrared camera module (141) as shown in FIG. 4. Specifically, the electronic device (101) can effectively determine the number of identifiers by simultaneously utilizing infrared image data that has been pre-processed in the infrared camera module (141) in which an external processor (not shown) is installed on the inside. This has the advantage of reducing the load on the main processor of the electronic device (101) and improving the overall image processing speed by removing noise through the infrared image captured by the sensor via the external processor connected to the infrared camera module (141).

[0034] In operation 203, the electronic device (101) (e.g., the processor (120) of FIG. 1) can input at least one image data into a denoise block system to obtain at least one corrected image data from which noise has been removed. According to one embodiment, the electronic device (101) can input infrared image data that has been preprocessed first from an infrared camera module (141) into a denoise block system to perform secondary preprocessing of the infrared image data. Here, the denoise block system may include a function to remove noise and highlight objects, and a function to preprocess and remove the infrared image data before the infrared camera module (141) combines it into a single image or video. In this embodiment, the electronic device (101) can use infrared image data that has been preprocessed first by an external processor (not shown) connected to the infrared camera module (141) and secondarily preprocessed by a processor (the processor (120) of FIG. 1). Subsequently, the electronic device (101) can input the preprocessed infrared image data (see FIG. 4) into a denoising block system to obtain corrected image data composed of the second preprocessed infrared image data (see FIG. 4). Additionally, the electronic device (101) can obtain corrected image data by combining the second preprocessed infrared image data.

[0035] According to another embodiment, the denoise block system can divide image data into a first image portion data and a second image portion data, respectively, and can reduce noise by selecting either the first image portion data or the second image portion data and performing a max pooling operation. This allows the electronic device (101) to easily combine a converted image with noise removal and an image with the original by distinguishing between the first image portion data to be amplified and the second image portion data to be preserved among the image data. In this embodiment, the partial image data containing an identifier can be set as the first image portion data, the partial image data containing a background can be set as the second image portion data, the first image portion data can be selected as the image data to be amplified, and the same can be set and selected in reverse. In addition, the denoise block system of the present embodiment can enhance spatial data by sequentially performing convolution operations and depth-wise convolution operations on either the first image portion data or the second image portion data with reduced noise, and can output corrected image data with removed noise by combining the portion data of either the first image portion data or the second image portion data with enhanced spatial processing with the other portion data of the unprocessed first image portion data or the second image portion data. This allows the electronic device (101) to combine spatial data similarly to the original data by performing convolution operations and depth-wise convolution operations so that the first portion image data selected as the image to be amplified can be naturally combined, and the second portion image data maintained as the original data can be combined with the first portion image data with reduced noise, thereby enabling easy noise removal while maintaining the original image data, so that the training data of the prediction model (400) can be clearly set.

[0036] In operation 205, an electronic device (101) (e.g., processor (120) of FIG. 1) can input at least one image data into a data augmentation system to obtain at least one augmented image data. According to one embodiment, the electronic device (101) can input infrared image data obtained from an infrared camera module (141) into a data augmentation system, create identification image data by inserting an object into image data selected as a class recognized by at least one identifier, include surrounding information of the identification image data by applying padding to the identification image data, and obtain augmented image data by maintaining the identification image data and the infrared image data before augmentation according to probability values ​​set for the identification image data and the infrared image data before augmentation. Specifically, the data augmentation system can augment infrared image data by utilizing various techniques. At this time, the electronic device (101) can increase the diversity of the data by adding new variations while maintaining the characteristics of the existing image, and the following methods described below may be applied.

[0037] The first advantage applied to the data augmentation system according to the present embodiment is geometric transformation, which can improve object recognition capabilities at various angles and distances by applying geometric transformations such as rotation, tilting, and scaling to infrared image data.

[0038] A second advantage applied to the data augmentation system according to the present embodiment is class balancing, which allows for generating or modifying more images of the deficient class to resolve imbalances between classes within the dataset in infrared image data.

[0039] According to the embodiments, the electronic device (101) can effectively compensate for the difficulty of object recognition that may occur in low-resolution thermal images by applying such a data augmentation system. In particular, the electronic device (101) of the embodiments can maintain high recognition performance even in situations where the boundaries of objects are ambiguous or it is difficult to detect small objects.

[0040] As a result, the electronic device (101) including the data augmentation process according to the present embodiment can enrich the training data of the prediction model (400), improve the generalization ability of the model, and contribute to significantly improving the accuracy of object detection in various real environments.

[0041] Next, the electronic device (101) inputs the augmented image data into a prediction model (400) to output detection prediction data regarding the number of identifiers present inside the building. The prediction model (400) according to the present embodiment is continuously improved using a composite loss function, thereby enabling it to output more accurate detection prediction data for the input image data.

[0042] Based on the detection prediction data, the electronic device (101) controls the operation of the ventilation control module. For example, if the detection prediction data is above a preset first threshold value, it can be operated by controlling the ventilation intensity to level 1, and if it is above a second threshold value, it can be operated by controlling the ventilation intensity to level 2. A detailed explanation of this can be specifically described in the following operation.

[0043] In this way, the electronic device (101) can accurately determine the real-time conditions of the indoor environment and perform appropriate ventilation control accordingly. This can greatly improve the efficiency of the smart ventilation system and provide significant assistance in managing indoor air quality.

[0044] In operation 207, an electronic device (101) (e.g., the processor (120) of FIG. 1) can input at least one corrected image data and at least one augmented image data into a prediction model (400) to output detection prediction data regarding the number of identifiers present inside a building. According to one embodiment, the prediction model (400) can be trained based on a plurality of image data obtained from a plurality of camera modules installed inside a plurality of buildings, a plurality of corrected image data output by inputting the plurality of image data into a denoise block system, a plurality of augmented image data output by inputting the plurality of image data into a data augmentation system, a plurality of detection prediction data, and discrimination data in which at least one identifier is identified from the plurality of detection prediction data. Additionally, during the training process of the prediction model, the diversity of the training data is increased by using an ObjectBlend Data Augmentation technique. Specifically, the prediction model (400) may be used as a regression model. Specifically, the regression model may use algorithms of the following types: linear regression, regression tree, support vector regression, kernel regression, etc. This is merely an example and is not limited thereto. As an example, a prediction model (400) for predicting that there is no identifier inside a building and / or indoors, or that there are 1, 3, 5, 7, or more people, may include a convolutional neural network (CNN). As a CNN structure, at least one of AlexNet, LENET, NIN, VGGNet, ResNet, WideResNet, GoogleNet, FractaNet, DenseNet, FitNet, RitResNet, HighwayNet, MobileNet, and DeeplySupervisedNet may be used.According to one embodiment, the prediction model (400) may be implemented using a plurality of CNN structures. For example, the prediction model (400) may use a 1D CNN and a 2D CNN together, and may be implemented using either a GRU neural network or an LSTM neural network. In one embodiment, the 1D CNN and the 2D CNN may include a block in which a CNN layer having a plurality of filters of a specific size, a ReLu layer, a pooling layer, and a dropout layer are sequentially combined for feature extraction, and may include a fully connected (FC) layer and an activation layer (e.g., sigmoid, softmax, etc.). According to one embodiment, the convolution layer may extract features from input data. Additionally, the convolution layer may consist of a filter for extracting features and an activation function that changes the value of the filter for extracting features into a non-linear value. The filter may include a function that detects whether the input data contains the feature.

[0045] As an example, the electronic device (101) may perform a learning process of a prediction model (400) for predicting whether there are no identifiers inside a building and / or indoors, or whether there are 1, 3, 5, 7, or more people, by using a prediction model (400) to which arbitrary weights are assigned to obtain a result value (output data), comparing the obtained result value with the labeled data of the training data, and performing backpropagation according to the error to optimize the weights. Specifically, the learning of the prediction model (400) means a process of training the prediction model (400) based on training data and labeled data or unlabeled data so that the prediction model (400) can determine output data for the input data. That is, the prediction model (400) forms rules and makes judgments regarding the data. According to one embodiment, the electronic device (101) may use a plurality of learning algorithms among a plurality of learning algorithms that calculate the predicted value. For example, an ensemble method may be used in the prediction model (400), and better prediction performance can be obtained compared to using learning algorithms separately. Training the prediction model (400) may mean adjusting the weights of the model. According to one embodiment, as a learning method, various methods such as supervised learning, unsupervised learning, reinforcement learning, imitation learning, and federated learning may be used.

[0046] Although not illustrated, the electronic device (101) may include an evaluation step for evaluating the performance of the prediction model (400) during the learning process of the prediction model (400). In the evaluation step, the prediction model (400) may be evaluated using an evaluation data set. The evaluation of the prediction model (400) may be a step of evaluating the prediction model (400) learned by the learning step and making predictions for new data using the prediction model (400). Specifically, the evaluation step may be a step of measuring whether the learned prediction model (400) is capable of generalization for new data. For example, the electronic device (101) may include having no identifier inside the building and / or indoor space, or having no identifier inside the building and / or indoor space, or having one, three, five, seven, or more, based on the determination of a plurality of sample techniques described below to increase the number of identifiers inside the building and / or indoor space to one, three, five, seven, or more.

[0047] Additionally, the prediction model (400) is composed of a lightweight object detection model optimized for low-resolution thermal images (hereinafter referred to as the "proposed model") as illustrated in FIG. 5. The proposed model is specially designed to overcome the low resolution of the infrared camera module (141) without using RGB image data. This model divides the image into two parts, performs a max pooling operation on one part to reduce noise, and then sequentially performs a convolution operation and a depth-wise convolution operation to enhance spatial information.

[0048] Accordingly, the prediction model (400), as illustrated in FIGS. 6 and 7, shows high efficiency in recognizing identifiers and determining the number of people in an image at long and short distances compared to previous models. In particular, the proposed model exhibits excellent performance in detecting overlapping or partially obscured objects.

[0049] In addition, the prediction model (400) showed the advantage of significantly improved accuracy (mAP), greatly improved inference speed compared to existing models, and significantly reduced memory usage by using the proposed model. As a result of testing in a low-spec MCU environment, the proposed model showed significant improvements in accuracy and efficiency compared to existing lightweight object detection models. This performance improvement is attributed to the unique structure and optimization techniques of the model.

[0050] The results of this performance comparison can be summarized as shown in [Table 1] below.

[0051] Model Flash Usage RAM Usage Total MAC Inference Time mAP Improvement (%P) MobileNet v2 - SSD Baseline Baseline Baseline Baseline YOLO v1 Approx. 41% decrease Approx. 11% decrease Approx. 54% increase Approx. 19% increase Approx. 41%P improvement Proposed Model (FLARE) Approx. 66% decrease Approx. 38% decrease Approx. 39% decrease Approx. 35% decrease Approx. 44%P improvement

[0052] Thus, the present embodiment can clearly determine the number of identifiers by operating a prediction model (400) that improves accuracy compared to the conventional model and increases the efficiency of identifier identification.

[0053] According to another embodiment, the prediction model (400) calculates a loss function and minimizes it during the process of inputting augmented image data and outputting detection prediction data, thereby improving the model and outputting detection prediction data with increased accuracy.

[0054] Specifically, the prediction model (400) can implement more AI models by utilizing data due to the recent development of various learning algorithms, but it may not be easy to secure the best data available for AI learning. This is because when training a complex AI model with insufficient training data, an overfitting phenomenon occurs where the test error increases as the small amount of training data itself is memorized as a whole and general representations cannot be learned. However, as the size of the training data set increases, the test error decreases, and overfitting can be avoided. As such, the prediction model (400) can use a data augmentation technique as one method to resolve the overfitting problem by increasing the amount of training data. Data augmentation is a technique that increases the number of data through various algorithms such as modification, substitution, and synthesis based on a small amount of data, and through this, the imbalance present in the training data can also be resolved. However, as mentioned above, the imbalance in training data can be pointed out as the biggest disadvantage of data augmentation in object detection in low-resolution thermal images.

[0055] Conventional methods for data augmentation result in an imbalance in training data because the collected minority class data is significantly insufficient compared to the majority class. However, the data augmentation of the present embodiment can augment data in various ways, such as by using a specialized algorithm to augment low-resolution thermal image data and adjusting the ratio between the majority and minority classes, in order to solve this problem.

[0056] The data augmentation of the prediction model (400) may have a cropping structure that creates new data by cropping a portion of the core part of the data in various ways, a flipping structure that changes the data by adding a flip, and a mix-up structure that combines or mixes two or more data. Additionally, a method of inserting objects of a minority class into images of a majority class may be adopted, and in this process, padding may be applied around the inserted objects to include surrounding information of the objects. To prevent duplication between newly inserted objects and existing objects, an Intersection over Union (IoU) threshold is set and verified, and a function may be included to insert new objects into the image background or replace existing objects with objects of the minority class according to the set probability. Here, the newly inserted objects may be augmented partial image data, and the existing objects may be the original image data that has not been augmented. The Intersection over Union (IoU) threshold plays an important role in the data augmentation process. In the present invention, the IoU threshold is set to 0.1, but generally, this value can be adjusted between 0.0 and 0.5. Such a low IoU threshold can be set to enable various object placements while minimizing overlap between newly inserted objects and existing objects. In this embodiment, the same IoU threshold is applied to all object classes, but different thresholds can be set for each object class as needed. For example, a lower threshold can be applied to small object classes, and a higher IoU threshold can be applied to large object classes. The IoU calculation method is the value obtained by dividing the intersection area of ​​two bounding boxes by the union area of ​​the two bounding boxes, and can be expressed as a formula as [Equation 1] below.

[0057] [Mathematical Formula 1]

[0058] IoU = (A ∩ B) / (A ∪ B)

[0059] Here, A and B represent the regions of the two bounding boxes, respectively.

[0060] In addition, in this embodiment, as an exception to the application of the IoU threshold during the data augmentation process, the IoU threshold may be temporarily raised or lowered in specific situations (e.g., very dense object clusters). Also, when inserting objects of a sparse class to resolve imbalances in the dataset, the IoU threshold may be applied more leniently. This embodiment helps to generate various object placement scenarios through a flexible approach and can improve the robustness of the model, particularly when considering situations where the boundaries between objects may be ambiguous in thermal images. Furthermore, based on this IoU threshold, the prediction model (400) exhibits stable performance in various object placement situations and can consistently process augmented images and original images without distinction.

[0061] The prediction model (400) in this embodiment adopts this data augmentation structure and can be trained by augmenting image data through methods such as cutting and newly augmenting some data, mixing with the original, and inserting and replacing objects. This allows the prediction model (400) to use data augmentation to compensate for cases where the identification of identifiers is unclear due to combinations between identifiers and some objects.

[0062] According to another embodiment, the prediction model (400) minimizes the loss function when outputting the detection prediction data using augmented image data, and can output the detection prediction data by receiving at least one augmented image data input with the loss function minimized. Specifically, the prediction model (400) can expand the loss function by including additional elements in the process of minimizing the conventional loss function. The prediction model (400) in this embodiment uses a box loss, which is fundamentally used among loss functions, and a focal loss, which refers to a cross-entropy loss multiplied by weights proportional to various learned values, as a basis, and can additionally combine a confidence loss to verify that an object is detected in the predicted bounding box. This can be configured to include a hard positive mining function that determines an identifier as an identifier, unlike a hard negative mining function in which the prediction model (400) determines that it is not the correct answer.

[0063] In operation 209, the electronic device (101) (e.g., the processor (120) of FIG. 1) can determine whether the detection prediction data has exceeded a first threshold value. According to one embodiment, the first threshold value may be set to one identifier, or it may be a reference point where the identifier is set to two identifiers. In this embodiment, the first threshold value may be set to one identifier. This may be set by determining that ventilation is required when even one person enters the building and / or indoor space, and the method of setting this may be adjusted by changing the first threshold value by an administrator. Additionally, if the electronic device (101) determines that the detection prediction data has not exceeded the first threshold value, it may return to operation 203 without separately operating at least one ventilation control module (170) and preprocess the newly input infrared image data until it exceeds the first threshold value.

[0064] Meanwhile, in operation 209, if the electronic device (101) (e.g., the processor (120) of FIG. 1) determines that the detection prediction data has exceeded a first threshold value, in operation 211, the electronic device (101) (e.g., the processor (120) of FIG. 1) can determine whether the detection prediction data has exceeded a second threshold value. According to one embodiment, the electronic device (101) can determine that the first threshold value has been exceeded when the identifier output in the detection prediction data is one or two people, as shown in FIG. 5 and FIG. 6. Subsequently, the electronic device (101) can determine whether the identifier output in the detection prediction data has exceeded a second threshold value that is set to be greater than the first threshold value. According to one embodiment, the second threshold value may be set to three people, may be a reference point for setting four people, or may be set to two people. This can be set when it is determined that ventilation is needed more than for one person when two to four people among multiple people enter the building and / or indoor space, and the setting method can be adjusted by changing the second threshold value by the administrator.

[0065] In operation 213, if the electronic device (101) (e.g., the processor (120) of FIG. 1) determines that the detection prediction data has not exceeded a second threshold value, it may drive at least one ventilation control module (170) by adjusting the ventilation intensity to a level 1. According to one embodiment, the electronic device (101) may determine that the first threshold value has been exceeded but the second threshold value has not been exceeded when the identifier output in the detection prediction data is one person. Subsequently, the electronic device (101) may drive at least one ventilation control module (170) by adjusting the ventilation intensity to a level 1 until it detects that the identifier extracted in the detection prediction data has not exceeded the first threshold value or has exceeded the second threshold value.

[0066] Meanwhile, in operation 211, if the electronic device (101) (e.g., the processor (120) of FIG. 1) determines that the detection prediction data has exceeded a second threshold value, in operation 215, the electronic device (101) (e.g., the processor (120) of FIG. 1) can determine whether the detection prediction data has exceeded a third threshold value. According to one embodiment, the electronic device (101) can determine that the second threshold value has been exceeded when the identifier output in the detection prediction data is 3 or 4 people, as shown in FIG. 5 and FIG. 6. Subsequently, the electronic device (101) can determine whether the identifier output in the detection prediction data has exceeded a third threshold value that is set to be greater than the second threshold value. According to one embodiment, the third threshold value may be set to 5 people, may be a reference point for setting the identifier to 6 people, or may be set to 4 people. This can be set when it is determined that ventilation is more necessary when 4 to 6 people enter a building and / or indoor space compared to when 2 to 3 people enter, and the setting method can be adjusted by changing the third threshold value by the manager.

[0067] In operation 217, if the electronic device (101) (e.g., the processor (120) of FIG. 1) determines that the detection prediction data has not exceeded a third threshold value, it can drive at least one ventilation control module (170) by adjusting the ventilation intensity to two levels. According to one embodiment, the electronic device (101) can determine that the second threshold value has been exceeded but the third threshold value has not been exceeded when the identifier output in the detection prediction data is three people. Subsequently, the electronic device (101) can drive at least one ventilation control module (170) by adjusting the ventilation intensity to two levels until it detects that the identifier extracted in the detection prediction data has not exceeded the second threshold value or has exceeded the third threshold value.

[0068] Meanwhile, in operation 215, if the electronic device (101) (e.g., the processor (120) of FIG. 1) determines that the detection prediction data has exceeded a third threshold value, in operation 219, the electronic device (101) (e.g., the processor (120) of FIG. 1) can determine whether the detection prediction data has exceeded a fourth threshold value. According to one embodiment, the electronic device (101) can determine that the third threshold value has been exceeded when the identifier output in the detection prediction data is 5 or 6 people, as shown in FIG. 5 and FIG. 6. Subsequently, the electronic device (101) can determine whether the identifier output in the detection prediction data has exceeded a fourth threshold value that is set to be greater than the third threshold value. According to one embodiment, the fourth threshold value may be set to 7 people, may be a reference point for setting the identifier to 6 people, or may be set to 8 people or more. This can be set by determining that when 6 to 7 people or more among multiple people enter a building and / or indoor space, ventilation needs to be driven most significantly compared to 5 to 6 people, and the setting method can be adjusted by changing the fourth threshold value by the manager.

[0069] In operation 221, if the electronic device (101) (e.g., the processor (120) of FIG. 1) determines that the detection prediction data has not exceeded a fourth threshold value, it can drive at least one ventilation control module (170) by adjusting the ventilation intensity to three levels. According to one embodiment, the electronic device (101) can determine that the third threshold value has been exceeded but the fourth threshold value has not been exceeded when the identifier output in the detection prediction data is six people. Subsequently, the electronic device (101) can drive at least one ventilation control module (170) by adjusting the ventilation intensity to three levels until it detects that the identifier extracted in the detection prediction data has not exceeded the third threshold value or has exceeded the fourth threshold value.

[0070] Meanwhile, in operation 219, if the electronic device (101) (e.g., the processor (120) of FIG. 1) determines that the detection prediction data has exceeded a fourth threshold value, in operation 223, the electronic device (101) (e.g., the processor (120) of FIG. 1) can drive at least one ventilation control module (170) by adjusting the ventilation intensity to four levels. According to one embodiment, the electronic device (101) can determine that the fourth threshold value has been exceeded when the identifier output in the detection prediction data is seven people. Subsequently, the electronic device (101) can drive at least one ventilation control module (170) by adjusting the ventilation intensity to four levels, which is the maximum level, until it detects that the identifier extracted in the detection prediction data has not exceeded the fourth threshold value.

[0071]

[0072] According to the present embodiment, by inputting correction image data generated by correcting based on a pre-processed infrared camera image from an infrared camera module (141) into a prediction model (400), identifiers for buildings and / or indoors included in the output detection prediction data can be easily extracted, and the ventilation control module can be driven according to the number of extracted identifiers identified in the buildings and / or indoors to automatically adjust the intensity of ventilation.

[0073] In addition, according to the present embodiment, the augmented image data generated by inputting an infrared camera image obtained from an infrared camera module (141) into a data augmentation system can clearly identify the number of identifiers, the distance of the identifiers, and the identifiers, thereby enabling clear learning and prediction of the prediction model (400). Furthermore, identifiers in buildings and / or indoors included in the detection prediction data output by inputting into the prediction model (400) can be easily extracted, and the ventilation control module can be driven according to the number of extracted identifiers identified in buildings and / or indoors to automatically adjust the intensity of ventilation.

[0074]

[0075] FIG. 3 is a flowchart illustrating different ways in which an electronic device operates according to various embodiments.

[0076] FIG. 8 is an example diagram for determining the case of an identifier that is not captured by a camera module by a prediction model (400) according to various embodiments.

[0077]

[0078] In operation 301, the electronic device (101) (e.g., the processor (120) of FIG. 1) can acquire detection prediction data from the prediction model (400) at regular intervals at preset times. According to one embodiment, the electronic device (101) can output detection prediction data by inputting correction image data and augmentation image data, which consist of secondary preprocessed infrared image data, into the prediction model (400) in real time and / or at preset times. This allows the electronic device (101) to acquire data in real time, or it can be configured by the manager's settings to reduce electrical waste resulting from the operation of the prediction model (400) relative to power waste for the ventilation control module (170). Subsequently, the electronic device (101) can extract an identifier from the extracted detection prediction data, and a detailed explanation thereof may be omitted as it is described in detail above.

[0079] In operation 303, the electronic device (101) (e.g., the processor (120) of FIG. 1) can store the identifier in memory (e.g., the memory (130) of FIG. 1) while driving at least one ventilation control module (170) when the detection prediction data identifies at least one identifier. According to one embodiment, the electronic device (101) can drive at least one ventilation control module (170) to one of the selected values ​​among operations 209 to 213 when there is at least one identifier extracted from the detection prediction data, and can store the number of identifiers in memory (130). This can be configured so that there may be a blind spot in the range of the building and / or interior captured by the infrared camera module (141), and to determine if the identifier goes into the blind spot or if the identifier has actually left the building and / or interior.

[0080] In operation 305, the electronic device (101) (e.g., the processor (120) of FIG. 1) can determine whether at least one identifier has moved to the outside of the building. According to one embodiment, the electronic device (101) can determine the direction in which the identifier of the detection prediction data moves, as shown in FIG. 8, and then, even if the identifier is not identified in the detection prediction data, it can determine the direction in which the identifier goes to a blind spot (see FIG. 8 (a)) and the direction in which the identifier goes outside the building and / or interior (see FIG. 8 (b)) by distinguishing the identifier in the previously stored memory (130), thereby determining whether the identifier remains inside the actual building and / or interior or has gone outside the building and / or interior even if the identifier is not identified in the detection prediction data.

[0081] In operation 305, if the electronic device (101) (e.g., the processor (120) of FIG. 1) determines that at least one identifier has moved outside the building, in operation 307, the electronic device (101) (e.g., the processor (120) of FIG. 1) can determine whether a preset time has been exceeded based on the point in time when it is determined that at least one identifier has moved outside the building. According to one embodiment, the electronic device (101) can clearly confirm in the detection prediction data that the identifier has gone outside the building and / or the indoor space, as shown in FIG. 8b, and if it confirms in the detection prediction data that the identifier has clearly gone outside the building and / or the indoor space, it can confirm that the identifier has moved outside the building and / or the indoor space. Here, the electronic device (101) can determine whether a preset time has been exceeded in order to examine whether the identifier has temporarily gone out or whether it is clearly confirmed that it has gone out. According to one embodiment, the preset time can be set to 5 seconds, 3 seconds, or 10 seconds. This allows the preset time to be differentiated and verified according to the structure and type of the building and / or indoor space, and can be set by maintaining the intensity of at least one ventilation control module (170).

[0082] In operation 305, the electronic device (101) (e.g., the processor (120) of FIG. 1) determines that at least one identifier has not moved outside the building, or in operation 307, the electronic device (101) (e.g., the processor (120) of FIG. 1) determines that at least one identifier has not exceeded a preset time based on the point in time when it was determined that the identifier has moved outside the building and / or indoor area, and in operation 309, the electronic device (101) (e.g., the processor (120) of FIG. 1) may determine that at least one identifier is inside the building and / or indoor area even if it is not identified in the detection prediction data. According to one embodiment, the electronic device (101) can determine that the identifier is inside the building and / or indoors even if it is not identified in the detection prediction data, if it can confirm that the identifier has moved to a blind spot of the infrared camera module (141) as shown in FIG. 8 (a), or if it confirms that the identifier has moved briefly outside the building and / or indoors but is identified again in the detection prediction data within a preset time.

[0083] In operation 311, the electronic device (101) (e.g., the processor (120) of FIG. 1) can maintain the intensity of at least one ventilation control module (170). According to one embodiment, the electronic device (101) can determine that an identifier is inside a building and / or indoor space even if the identifier is not identified in the detection prediction data, so that the total number of identifiers is maintained, and thus the intensity of at least one ventilation control module (170) can be maintained according to intensity control based on a threshold value between 213 and 223 intensity levels. Afterward, the electronic device (101) can return to operation 301 and repeat the operation until a preset time is exceeded while determining that at least one identifier has moved outside the building.

[0084] Meanwhile, in operation 307, if the electronic device (101) (e.g., the processor (120) of FIG. 1) determines that a preset time has been exceeded based on the point in time when it is determined that at least one identifier has moved outside the building, then in operation 313, the electronic device (101) (e.g., the processor (120) of FIG. 1) may determine that at least one identifier has moved outside the building and / or indoor area in the detection prediction data. According to one embodiment, if the electronic device (101) confirms that the identifier has moved outside the building and / or indoor area based on the infrared camera module (141) as shown in FIG. 8 (b), and confirms that the identifier has not been identified in the detection prediction data for more than a preset time, then it may determine that the identifier is outside the building and / or indoor area while not being identified in the detection prediction data. This allows the electronic device (101) to determine that the identifier has indeed gone outside the building and / or indoor area.

[0085] In operation 315, the electronic device (101) (e.g., the processor (120) of FIG. 1) can drive at least one ventilation control module (170) by adjusting its intensity and delete identifiers that have moved outside from memory (e.g., the memory (130) of FIG. 1). According to one embodiment, the electronic device (101) can determine that an identifier is outside the building and / or indoors without being identified in the detection prediction data, and thus the total number of identifiers has changed, so the intensity of at least one ventilation control module (170) can be adjusted or terminated according to the intensity control based on a threshold value among 213 to 223 intensity. Subsequently, the electronic device (101) can clearly change the number of identifiers present in the building and / or indoors by deleting the identifier from memory (130) since there is no longer a need to record at least one identifier that has been determined to have moved outside the building.

[0086]

[0087] According to the present embodiment, infrared image data captured using a single infrared camera module is corrected and input into a prediction model (400), and by checking the time value set for the movement of an identifier identified in the detection prediction data output from the prediction model (400) and the movement of the identifier outside, the intensity of the ventilation control module can be maintained even if the identifier existing inside the actual building is in a blind spot, and the intensity of the ventilation control module can be changed by confirming the identifier that has clearly moved outside the building, and this can be performed automatically, thus having the advantage of not requiring a separate manager to check.

[0088]

[0089] According to various embodiments, an electronic device for driving an autonomous ventilation system utilizing a prediction model comprises: at least one camera module installed inside a building where the ventilation system is applied; at least one ventilation control module; and a processor that acquires at least one image data of at least one identifier existing inside the building and / or indoor space through the at least one camera module and controls the at least one ventilation control module, wherein the processor outputs detection prediction data regarding the number of identifiers existing inside the building, and if the detection prediction data is determined to be greater than or equal to a preset first threshold, the at least one ventilation control module is driven by controlling the ventilation intensity to a level 1, and if the detection prediction data is determined to be greater than or equal to a second threshold set to be greater than the first threshold, the at least one ventilation control module is driven by controlling the ventilation intensity to a level 2.

[0090] According to various embodiments, the processor inputs the at least one image data into a data augmentation system to obtain at least one augmented image data, and the detection prediction data is output by inputting the at least one augmented image data into the prediction model.

[0091] According to various embodiments, the prediction model is learned based on a plurality of image data obtained from a plurality of camera modules installed inside a plurality of buildings, a plurality of augmented image data output by inputting the plurality of image data into the data augmentation system, a plurality of detection prediction data, and discrimination data in which at least one identifier is identified from the plurality of detection prediction data.

[0092] According to various embodiments, the processor inputs the at least one image data into a denoise block system to obtain at least one corrected image data with noise removed, and the detection prediction data is characterized by inputting the at least one corrected image data into the prediction model and outputting it.

[0093] According to various embodiments, the prediction model is learned based on a plurality of image data obtained from a plurality of camera modules installed inside a plurality of buildings, a plurality of corrected image data output by inputting the plurality of image data into the denoise block system, a plurality of detection prediction data, and discrimination data in which at least one identifier is identified from the plurality of detection prediction data.

[0094] According to various embodiments, the at least one image data includes infrared image data obtained from an infrared camera module.

[0095] According to various embodiments, the processor inputs the infrared image data into the data augmentation system, generates identification image data by inserting an object into the image data selected as a class recognized by at least one identifier, applies padding to the identification image data to include surrounding information of the identification image data, and maintains the identification image data and the infrared image data before augmentation according to a probability value pre-set for the identification image data and the infrared image data before augmentation, thereby acquiring the augmented image data.

[0096] According to various embodiments, the prediction model minimizes a loss function when outputting the detection prediction data using the augmented image data, and receives at least one augmented image data with the minimized loss function and outputs the detection prediction data.

[0097] According to various embodiments, the processor is configured to drive the at least one ventilation control module by adjusting the ventilation intensity to 3 levels when the detection prediction data is determined to be greater than or equal to a third threshold value set to be greater than the second threshold value, and to drive the at least one ventilation control module by adjusting the ventilation intensity to 4 levels when the detection prediction data is determined to be greater than or equal to a fourth threshold value set to be greater than the third threshold value.

[0098] According to various embodiments, the processor is configured to acquire the infrared image data preprocessed from an external processor through a communication interface, and to input the preprocessed infrared image data into the denoise block system to acquire the corrected image data composed of the infrared image data that has been secondarily preprocessed.

[0099] According to various embodiments, the denoise block system is configured to divide the image data into a first image portion data and a second image portion data, reduce noise by performing a max pooling operation on either the first image portion data or the second image portion data, enhance spatial data by sequentially performing a convolution operation and a depth-wise convolution operation on either the noise-reduced first image portion data or the second image portion data, and output the noise-removed corrected image data by combining the portion data of either the first image portion data or the second image portion data with enhanced spatial processing with the other portion data of the unprocessed first image portion data or the second image portion data.

[0100] According to various embodiments, the processor is configured to drive the at least one ventilation control module by adjusting the ventilation intensity to 3 levels when the detection prediction data is determined to be greater than or equal to a third threshold value set to be greater than the second threshold value, and to drive the at least one ventilation control module by adjusting the ventilation intensity to 4 levels when the detection prediction data is determined to be greater than or equal to a fourth threshold value set to be greater than the third threshold value.

[0101] According to various embodiments, the processor further includes a memory, wherein the processor acquires the detection prediction data from the prediction model at predetermined intervals of time, and when the detection prediction data identifies the at least one identifier, the processor is configured to store the identifier in the memory while driving the at least one ventilation control module.

[0102] According to various embodiments, the processor is configured to maintain the intensity of the at least one ventilation control module when it is determined that the identifier is inside the building and / or indoor space even if it is not identified in the detection prediction data, when it is confirmed that the at least one identifier has moved in the direction of the building and / or indoor space in the detection prediction data; and when it is determined that the at least one identifier has moved in the direction of the building and / or indoor space out of the detection prediction data and is determined to be over a preset time value while not being identified in the detection prediction data, when it is determined that the identifier has gone outside the building and / or indoor space, the processor is configured to adjust the intensity of the at least one ventilation control module and delete the identifier from the memory.

[0103]

[0104] The terms “module” or “part” as used in this document include a unit composed of hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. “Module” or “part” may be a component formed integrally or a minimum unit or part thereof that performs one or more functions. “Module” or “part” may be implemented mechanically or electronically and may include, for example, an application-specific integrated circuit (ASIC) chip, field-programmable gate arrays (FPGAs), or programmable logic device known or to be developed that performs certain operations, and may be executed by a processor (120). At least part of the device (e.g., modules or functions thereof) or method (e.g., operations) according to various embodiments may be implemented as instructions stored in a computer-readable storage medium (e.g., memory (130)) in the form of a program module. When the above instruction is executed by a processor (e.g., processor (120)), the processor may perform a function corresponding to the above instruction. Computer-readable recording media may include a hard disk, a floppy disk, a magnetic medium (e.g., magnetic tape), an optical recording medium (e.g., CD-ROM, DVD, magneto-optical medium (e.g., floptical disk), built-in memory, etc. Instructions may include code generated by a compiler or code that can be executed by an interpreter. A module or program module according to various embodiments may include at least one of the aforementioned components, some of which may be omitted, or additionally include other components. Operations performed by a module, program module, or other components according to various embodiments may be executed sequentially, in parallel, iteratively, or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

[0105] Furthermore, the embodiments disclosed in this document are presented for the purpose of explaining and understanding the disclosed technical content and are not intended to limit the scope of this disclosure. Accordingly, the scope of this disclosure should be interpreted to include all modifications or various other embodiments based on the technical concept of this disclosure.

Claims

1. An electronic device for driving an autonomous ventilation system utilizing a prediction model, At least one camera module installed on the interior side of a building equipped with a ventilation system; At least one ventilation control module; and The apparatus includes a processor that acquires at least one image data of at least one identifier existing inside a building and / or indoors through the at least one camera module and controls the at least one ventilation control module. The processor is configured to output detection prediction data regarding the number of identifiers existing inside the building, and if the detection prediction data is determined to be greater than or equal to a preset first threshold, to drive the at least one ventilation control module by adjusting the ventilation intensity to level 1, and if the detection prediction data is determined to be greater than or equal to a second threshold set to be greater than the first threshold, to drive the at least one ventilation control module by adjusting the ventilation intensity to level 2. Electronic device.

2. In Paragraph 1, The processor inputs the at least one image data into a data augmentation system to obtain at least one augmented image data, and The above detection prediction data is characterized by being output by inputting the at least one augmented image data into the prediction model. Electronic device.

3. In Paragraph 2, The above prediction model is learned based on a plurality of image data obtained from a plurality of camera modules installed inside a plurality of buildings, a plurality of augmented image data output by inputting the plurality of image data into the data augmentation system, a plurality of detection prediction data, and discrimination data in which at least one identifier is identified from the plurality of detection prediction data. Electronic device.

4. In Paragraph 1, The processor inputs the at least one image data into a denoise block system to obtain at least one corrected image data from which noise has been removed, and The above detection prediction data is characterized by being output by inputting the above at least one correction image data into the prediction model. Electronic device.

5. In Paragraph 4, The above prediction model is, Learning based on multiple image data acquired from multiple camera modules installed inside multiple buildings, multiple corrected image data output by inputting the multiple image data into the denoise block system, multiple detection prediction data, and discrimination data in which at least one identifier is identified from the multiple detection prediction data. Electronic device.

6. In either Paragraph 3 or Paragraph 5, The above at least one image data includes infrared image data obtained from an infrared camera module, Electronic device.

7. In Paragraph 6, The above processor is, The infrared image data is input into the data augmentation system, and an object is inserted into the image data selected as a class recognized by at least one identifier to generate identification image data. Applying padding to the above-mentioned identification image data to include surrounding information of the above-mentioned identification image data, and A set to acquire the augmented image data by maintaining the identification image data and the infrared image data before augmentation according to a probability value pre-set for the identification image data and the infrared image data before augmentation, Electronic device.

8. In Paragraph 7, The above prediction model minimizes a loss function when outputting the detection prediction data using the augmented image data, and receives at least one augmented image data as input and outputs the detection prediction data with the loss function minimized. Electronic device.

9. In Paragraph 7, The above processor is, If the above detection prediction data is determined to be greater than or equal to a third threshold value set to be greater than the second threshold value, the at least one ventilation control module is driven by adjusting the ventilation intensity to three levels, and If the above detection prediction data is determined to be greater than or equal to a fourth threshold value which is set to be greater than the third threshold value, the at least one ventilation control module is configured to drive by adjusting the ventilation intensity to four levels. Electronic device.

10. In Paragraph 6, The above processor is, Acquire the infrared image data preprocessed from an external processor through a communication interface, and The above-mentioned preprocessed infrared image data is input into the denoise block system to obtain the correction image data composed of the above-mentioned secondarily preprocessed infrared image data, Electronic device.

11. In Paragraph 10, The above denoise block system is, The above image data is divided into a first image portion data and a second image portion data, respectively, and Noise is reduced by selecting either the first image portion data or the second image portion data and performing a max pooling operation, and The spatial data is enhanced by sequentially performing a convolution operation and a depth-wise convolution operation on either the first image portion data or the second image portion data with reduced noise, and A method configured to output noise-removed corrected image data by combining one of the first image portion data or the second image portion data with enhanced spatial processing and the other of the unprocessed first image portion data or the second image portion data. Electronic device.

12. In either Paragraph 3 or Paragraph 5, The above processor is, If the above detection prediction data is determined to be greater than or equal to a third threshold value set to be greater than the second threshold value, the at least one ventilation control module is driven by adjusting the ventilation intensity to three levels, and If the above detection prediction data is determined to be greater than or equal to a fourth threshold value which is set to be greater than the third threshold value, the at least one ventilation control module is configured to drive by adjusting the ventilation intensity to four levels. Electronic device.

13. In Paragraph 12, Including memory; further, The above processor is, The detection prediction data from the above prediction model is obtained at regular intervals at preset times, and When the above detection prediction data identifies the at least one identifier, the at least one ventilation control module is driven and the identifier is stored in the memory. Electronic device.

14. In Paragraph 13, The above processor is, If it is confirmed in the above detection prediction data that the at least one identifier has moved in the direction of the building and / or the interior, even if the identifier is not identified in the above detection prediction data, it is determined that the identifier is inside the building and / or the interior, and the intensity of the at least one ventilation control module is maintained, and If it is confirmed in the above detection prediction data that the at least one identifier has moved in the direction of the building and / or the indoor outside, and if it is determined that the identifier has moved beyond a preset time value while not being identified in the above detection prediction data, it is determined that the identifier has gone out of the building and / or the indoor outside, the intensity of the at least one ventilation control module is adjusted, and the identifier is configured to be deleted from the memory. Electronic device.

Images