Image search-based IoT device control system

Abstract

Disclosed is an IOT device control system. The disclosed IOT device control system, in which an IOT server, a local computer, and an IOT device form a network through a router, comprises: the local computer in which an operating system and one or more application programs are installed; the IOT server that receives a predetermined mode command defining driving situations of the local computer and the IOT device, and transmits a control command to the local computer and the IOT device so that the local computer and the IOT device are executed as defined in the received mode command; and one or more IOT devices driven by the control command received from the IOT server.

Description

Image Search-based IoT Device Control System

[0001] The present invention relates to an image search-based IoT device control system, and more specifically, to an image search-based IoT device control system in which an IoT server, which is networked with a local computer and an IoT device through a router, enables situation recognition and situation-specific control of the local computer by searching for screen images of the local computer, thereby preventing local information from being leaked to a separate external cloud server, making it safe in terms of security, and enabling more stable operation by operating independently from an external cloud server.

[0002] Generally, the Internet of Things (IoT) refers to intelligent information and communication technology or services that enable communication between people and objects, and between objects themselves, by providing network connectivity to various objects based on the Internet.

[0003] With the development of such IoT technology, it has become possible to comprehensively control surrounding networked IoT devices along with the execution of specific applications installed on a local computer.

[0004] Figure 1 is a diagram illustrating an IoT device control system linked with a conventional external cloud server.

[0005] This will be explained with reference to Fig. 1.

[0006] An operating system and multiple applications are installed on the local computer.

[0007] When an application is executed, it is necessary to control various IoT devices according to the execution stage.

[0008] For example, when a golf simulation program is running, lighting, heating and cooling equipment, beam projectors, etc., are controlled in accordance with the program's progress.

[0009] To control IoT devices, the IoT server needs information about the current status of the local computer and the execution status of the application.

[0010] Typically, the company that developed the application program and the company that developed the IoT server that integrally controls local computers and IoT devices are different companies.

[0011] Therefore, the company that developed the IoT server obtains context information from the application program through the cooperation of the company that developed the application program, and recognizes the status of the local computer and the application program.

[0012] Conventionally, as shown in FIG. 1, a local computer and an IoT device are connected to an external cloud server via the Internet, and current status information of the local computer and the application program is transmitted to the external cloud server.

[0013] In other words, the status of applications on the local computer and status information of the IoT device are disclosed externally in real time.

[0014] Furthermore, the IoT server connects to an external cloud server to receive current status information from the local computer and applications, recognizes the current situation, and then controls the local computer and IoT devices.

[0015] If a network is formed in this way, the status information of local computers and IoT devices is exposed to the outside, which creates security vulnerabilities.

[0016] Furthermore, if a problem occurs with the external cloud server, local computers and IoT devices may become uncontrollable, potentially leading to an unstable situation.

[0017] Meanwhile, if the application development company and the IoT server development company do not cooperate, it becomes impossible to build a system for local computers and IoT devices.

[0018] Therefore, there is an urgent need to develop a new IoT device control system capable of controlling IoT devices independently of external cloud servers, in accordance with the progress of the local computer.

[0019] The present invention is proposed to solve the above-mentioned problems and aims to control an IoT device in accordance with the progress of a local computer independently of an external cloud server.

[0020] In addition, the present invention aims to enable an IoT server to recognize the current status of a local computer and an application program through image search.

[0021] In addition, the present invention aims to recognize the situation of a local computer by examining the location and RGB values ​​of each pixel of the local computer.

[0022] The present invention relates to an IoT device control system in which an IoT server, a local computer, and an IoT device form a network through a router, wherein

[0023] A local computer having an operating system and one or more applications installed;

[0024] An IoT server that receives a predetermined mode command defining the operating status of a local computer and an IoT device, and transmits a control command to the local computer and the IoT device so that the local computer and the IoT device are executed according to the definition in the received mode command;

[0025] An IoT device control system is provided, characterized by including one or more IoT devices driven by the control command received from the above IoT server.

[0026] In addition, the present invention is characterized by further including an administrator computer that forms a network through the IoT server, local computer, and IoT device and a router, and transmits a predetermined mode command to the IoT server.

[0027] In addition, the IoT server of the present invention is characterized by searching for a screen image of the local computer to recognize the current status of the local computer, and transmitting a control signal corresponding to the recognized current status to the local computer and the IoT device to control them in order to execute a received mode command.

[0028] In addition, the situation recognition through screen image search of the local computer in the present invention is characterized by being determined by comparing the position and RGB value of each pixel of the local computer with the pre-set pixel position and RGB value for each situation.

[0029] In addition, the application program of the present invention is a golf simulation program, and the IoT device is characterized as being an environmental facility necessary for the operation of the golf simulation program.

[0030] The image search-based IoT device control system according to the present invention has the effect of preventing information from being exposed outside the local network.

[0031] In addition, the present invention operates independently of an external cloud server, which has the effect of increasing the stability of the control system.

[0032] In addition, the present invention has the effect of enabling the construction of a control system for controlling an IoT device without the assistance of an application program developer.

[0033] In addition to the effects described above, the specific effects of the present invention are described together with the specific details for implementing the invention below.

[0034] Figure 1 is a diagram illustrating an IoT device control system linked with a conventional external cloud server.

[0035] FIG. 2 is a diagram illustrating an image search-based IoT device control system according to the present invention.

[0036] Hereinafter, various embodiments of this document are described with reference to the accompanying drawings. However, this is not intended to limit the technology described in this document to specific embodiments and should be understood to include various modifications, equivalents, and / or alternatives to the embodiments of this document. In relation to the description of the drawings, similar reference numerals may be used for similar components.

[0037] Furthermore, expressions such as "First," "Second," etc., used in this document may modify various components regardless of order and / or importance, and are used merely to distinguish one component from another without limiting such components. For example, "Part 1" and "Part 2" may refer to different parts regardless of order or importance. For instance, without departing from the scope of rights described in this document, Part 1 may be renamed Part 2, and similarly, Part 2 may be renamed Part 1.

[0038] Furthermore, the terms used in this document are used merely to describe specific embodiments and are not intended to limit the scope of other embodiments. Singular expressions may include plural expressions unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as generally understood by those skilled in the art described in this document. Terms used in this document that are defined in general dictionaries may be interpreted as having the same or similar meaning as they have in the context of the relevant technology, and are not to be interpreted in an ideal or overly formal sense unless explicitly defined in this document. In some cases, even terms defined in this document may not be interpreted to exclude the embodiments of this document.

[0039] FIG. 2 is a diagram illustrating an image search-based IoT device control system according to the present invention.

[0040] This will be explained with reference to Fig. 2.

[0041] The IoT device control system according to the present invention includes an administrator computer, an IoT server, a local computer, and an IoT device, and the IoT device is controlled in accordance with the execution status of an application program installed on the local computer.

[0042] The administrator computer, IoT server, local computer, and IoT device form a network through a router.

[0043] An operating system and one or more applications are installed on the local computer.

[0044] The control system controls the local computer, application program, and IoT device in a predetermined manner for each situation, and in this specification, each situation defined in the control system is defined as a "mode."

[0045] The administrator computer is configured to send a specific mode command to the IoT server.

[0046] The IoT server is configured to receive a specific mode command that defines the operating status of a local computer, an application program, and an IoT device, and to transmit a control command to the local computer and the IoT device so that the local computer and the IoT device are executed according to the definition in the received mode command.

[0047] Depending on the environment in which the system is built, the administrator may be able to directly input mode commands into the IoT server without going through the administrator's computer.

[0048] In addition, depending on the system construction environment, there may be one IoT server in the control system or one IoT server for each local computer, and the IoT server may be physically installed externally independently of the local computer or installed inside the local computer.

[0049] The IoT device is driven by the control command received from the IoT server and there may be one or more devices.

[0050] In order for an IoT server to control an IoT device, it must recognize the current status of the local computer and the application program so that it can send corresponding control commands to the IoT device.

[0051] In order to improve security issues and system instability resulting from the conventional integration of an external cloud server, a local computer, and an IoT device, the present invention devises a method to recognize the current status of a local computer and an application program independently of an external cloud server.

[0052] To this end, the present invention adopts a screen image recognition method.

[0053] The IoT server searches for a screen image of the local computer to recognize the current status of the local computer, and controls the local computer and IoT device by transmitting a control signal corresponding to the recognized current status to execute the received mode command.

[0054] Context recognition through screen image retrieval on the local computer is determined by comparing the location and RGB values ​​of each pixel on the local computer with the pre-configured pixel locations and RGB values ​​for each situation.

[0055] In other words, after saving images for each situation, the pixel locations and RGB values ​​of the currently retrieved image are compared with the previously saved images for that situation to accurately recognize the current situation.

[0056] The IoT device control system according to the present invention can be applied in various fields.

[0057] For the sake of understanding, the case in which the IoT device control system according to the present invention is applied to a screen golf practice range is briefly described, but it is clearly stated that it is not limited thereto and can be applied to various fields.

[0058] Referring to Figure 2, a network is formed in a screen golf facility by a router between an administrator computer, an IoT server, and two local computers.

[0059] A golf simulation program is installed as an application on the local computer, and IoT devices such as a beam projector, heating and cooling equipment, and lighting are installed near the local computer and connected via a network.

[0060] Mode commands can be defined, for example, as work mode, practice range mode, match mode, exit mode, and departure mode.

[0061] "Arrival mode" is the situation where an administrator arrives at work, turns on the computer, and turns on the lights.

[0062] When the IoT server receives a clock-in mode command from the administrator computer, it recognizes that the local computer screen is off, turns on the computer, and sends control commands to the local computer and the lighting device to turn on the lighting device.

[0063] Practice Range Mode is a mode for practicing your swing before the Match Mode, where scores are calculated.

[0064] When the IoT server receives a practice mode command from the administrator computer, it searches for the screen image of the local computer.

[0065] If the screen image is confirmed to be the operating system screen, the golf simulation program is launched because it has not yet been executed. Then, if the screen image indicates that the golf simulation program is in a waiting state, the golf simulation program is controlled to enter the practice range stage.

[0066] In addition, the IoT server operates the heating and cooling devices defined in the practice mode at a preset temperature.

[0067] Accordingly, the IoT device control system according to the present invention is an invention that, when it is necessary to control an IoT device in accordance with the execution situation of an application program, searches for a screen image of a local computer to recognize the situation, and then controls the local computer and the IoT device in accordance with the situation.

[0068] By enabling situational awareness through screen image search on the local computer, there is no need to connect with a separate external cloud server, which further enhances security and improves system stability.

[0069] Furthermore, from the perspective of IoT server developers, there is the advantage of being able to build a control system independently as collaboration with application developers is not required.

[0070]

[0071] Although preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above. It is understood that various modifications can be made by those skilled in the art without departing from the essence of the invention as claimed in the claims, and such modified embodiments should not be understood individually from the technical spirit or perspective of the present invention.

Claims

1. In an IoT device control system in which an IoT server, a local computer, and an IoT device form a network through a router, A local computer having an operating system and one or more applications installed; An IoT server that receives a predetermined mode command defining the operating status of a local computer and an IoT device, and transmits a control command to the local computer and the IoT device so that the local computer and the IoT device are executed according to the definition in the received mode command; An IoT device control system characterized by including one or more IoT devices driven by the control command received from the above IoT server.

2. In Paragraph 1, An IoT device control system characterized by further including: an administrator computer that forms a network through the above-mentioned IoT server, local computer, and IoT device and a router, and transmits a predetermined mode command to the above-mentioned IoT server.

3. In Paragraph 1, An IoT device control system characterized in that the above IoT server searches for a screen image of the local computer to recognize the current status of the local computer, and transmits a control signal corresponding to the recognized current status to the local computer and the IoT device to execute a received mode command.

4. In Paragraph 3, An IoT device control system characterized by determining situation recognition through screen image search of the local computer by comparing the location and RGB values ​​of each pixel of the local computer with the pre-set pixel location and RGB values ​​for each situation.

5. In Paragraph 4, An IoT device control system characterized in that the above-mentioned application is a golf simulation program, and the above-mentioned IoT device is an environmental facility necessary for the operation of the golf simulation program.

Images