Image search-based IoT device control system

Abstract

We provide an image search-based IoT device control system. [Solution] The present invention provides an IoT device control system in which an IoT server, a local computer, and IoT devices form a network via a router, and is characterized by including: a local computer on which an operating system and one or more applications are installed; an IoT server that receives a predetermined mode instruction that defines the operating status of the local computer and IoT devices, and sends control instructions to the local computer and IoT devices so that the local computer and IoT devices are executed as defined in the received mode instruction; and one or more IoT devices that are driven by the control instructions received from the IoT server.

Description

Technical Field

[0001] The present invention relates to an image search-based IoT device control system. More specifically, an IoT server in which a network is formed via a local computer, an IoT device, and a router enables recognition of the situation of the local computer and control according to the situation through search of the screen image of the local computer, so that local information does not leak to a separate external cloud server, which is secure in terms of security, is independently operated from the external cloud server, and relates to an image search-based IoT device control system that enables more stable operation.

Background Art

[0002] Generally, IoT (Internet of Things) refers to an intelligent information communication technology or service that enables communication between humans and things and between things by providing network connections to various things based on the Internet. With the development of such IoT technology, it has become possible to integrally control peripheral IoT devices connected by a network along with the execution of a specific application installed in a local computer.

[0003] FIG. 1 is a diagram showing a conventional IoT device control system linked with an external cloud server. This will be described with reference to FIG. 1. An operating system and a plurality of applications are installed in the local computer. When a certain application is executed, it is necessary to control various IoT devices according to the execution stage.

[0004] For example, when a golf simulation program is executed, lighting, air conditioning equipment, a beam projector, etc. are controlled according to the progress of the program. To control IoT devices, the IoT server needs information about the current status of the local computer and the execution status of applications.

[0005] Typically, the company that develops the application and the company that develops the IoT server that provides integrated control of the local computer and IoT devices are different companies. Therefore, the company that developed the IoT server receives application status information from the company that developed the application, and recognizes the status of the local computer and the application.

[0006] Conventionally, as shown in Figure 1, a local computer and an IoT device are connected to an external cloud server via the internet, and current status information of the local computer and applications is transmitted to the external cloud server. In other words, the status of applications on the local computer and the status information of IoT devices are exposed to the outside world in real time. Furthermore, the IoT server connects to an external cloud server, receives current status information from the local computer and applications, recognizes the current status, and then controls the local computer and IoT devices. When a network is formed in this manner, the status information of local computers and IoT devices is exposed to the outside, which presents security vulnerabilities. Furthermore, if a problem occurs with the external cloud server, local computers and IoT devices may become uncontrollable and unstable.

[0007] On the other hand, without cooperation between application development companies and IoT server development companies, it is impossible to build a system for local computers and IoT devices. Therefore, there is a strong need for the development of a new IoT device control system that can control IoT devices independently of external cloud servers, in accordance with the progress of the local computer. [Prior art documents] [Patent Documents]

[0008] [Patent Document 1] Korean Published Patent Gazette No. 10-2020-0075080 [Overview of the project] [Problems that the invention aims to solve]

[0009] This invention aims to solve these problems, and its objective is to control IoT devices in accordance with the progress of a local computer, independently of an external cloud server. Furthermore, the present invention aims to enable an IoT server to recognize the current status of a local computer and applications through image retrieval. Furthermore, the present invention aims to recognize the status of a local computer by examining the position and RGB values ​​of each pixel on the local computer. [Means for solving the problem]

[0010] The present invention relates to an IoT device control system in which an IoT server, a local computer, and IoT devices form a network via a router, A local computer on which an operating system and one or more applications are installed, An IoT server receives a predetermined mode instruction that defines the operating status of a local computer and an IoT device, and sends control instructions to the local computer and the IoT device so that they are executed according to the definition in the received mode instruction. The present invention provides an IoT device control system that includes one or more IoT devices driven by control commands received from the IoT server.

[0011] Furthermore, the present invention is characterized by further including an administrator computer that forms a network with the IoT server, local computer, and IoT devices via a router and transmits predetermined mode commands to the IoT server.

[0012] Furthermore, the IoT server of the present invention is characterized by searching the screen image of the local computer to recognize the current status of the local computer, and transmitting control signals corresponding to the recognized current status to the local computer and IoT devices in order to execute the received mode command.

[0013] Furthermore, in the present invention, the recognition of the situation by searching the screen image of the local computer is characterized in that the position and RGB value of each pixel of the local computer are determined by comparing them with the pre-set pixel positions and RGB values ​​for each situation.

[0014] Furthermore, the application of the present invention is a golf simulation program, and the IoT device is an environmental facility necessary for operating the golf simulation program. [Effects of the Invention]

[0015] The image search-based IoT device control system according to the present invention has the effect of preventing information from being exposed outside the local network. Furthermore, this invention operates independently of external cloud servers, which has the effect of improving the stability of the control system. Furthermore, the present invention has the effect of enabling the construction of a control system for controlling IoT devices without the need for application developers. In addition to the effects described above, the specific effects of the present invention will be described together with the following specific details for carrying out the invention. [Brief explanation of the drawing]

[0016] [Figure 1] FIG. is a diagram showing an IoT device control system that interacts with a conventional external cloud server. [Figure 2] FIG. is a diagram showing an image search-based IoT device control system according to the present invention.

Embodiments for Carrying Out the Invention

[0017] Hereinafter, various embodiments of this document will be described with reference to the accompanying drawings. However, this is not intended to limit the technology described in this document to specific embodiments, and should be understood to include various modifications, equivalents, and / or alternatives of the embodiments of this document. In connection with the description of the drawings, the same reference numerals can be used for similar components.

[0018] Also, expressions such as "first", "second", etc. used in this document can modify various components regardless of order and / or importance, and are only used to distinguish one component from another, and do not limit the component. For example, "the first part" and "the second part" can indicate different parts regardless of order or importance. For example, without departing from the scope of the rights described in this document, the first component can be named the second component, and similarly, the second component can also be named the first component.

[0019] Furthermore, the terminology used in this document is used solely to describe specific embodiments and does not limit the scope of other embodiments. Singular expressions may include plural expressions unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as those generally understood by those with ordinary skill in the art described herein. Terms used herein that are defined in general dictionaries may be interpreted in the same or similar sense as they have in the context of the relevant technology, and not in an ideal or overly formal sense unless explicitly defined herein. In some cases, even terms defined herein may not be interpreted in a way that excludes the embodiments described herein.

[0020] Figure 2 shows an image search-based IoT device control system according to the present invention. Refer to Figure 2 for explanation. The IoT device control system according to the present invention includes an administrator computer, an IoT server, a local computer, and IoT devices, and the IoT devices are controlled according to the execution status of applications installed on the local computer.

[0021] The administrator computer, IoT server, local computer, and IoT devices form a network via a router. The local computer will have an operating system and one or more applications installed. The control system controls local computers, applications, and IoT devices in a predetermined manner depending on the situation, but in this specification, each situation defined by the control system is defined as a "mode".

[0022] The administrator computer is configured to send predetermined mode commands to the IoT server. The IoT server is configured to receive a predetermined mode command that defines the operating status of the local computer, application, and IoT device, and to send control commands to the local computer and IoT device so that they are executed according to the definition in the received mode command.

[0023] Depending on the environment in which the system is built, the administrator can also directly input mode commands to the IoT server without going through the administrator's computer. Furthermore, depending on the system configuration environment, there may be one IoT server in the control system, or one IoT server for each local computer. The IoT server may be physically located externally, independently of the local computer, or it may be located inside the local computer. There are one or more IoT devices, which are driven by the control commands received from the IoT server.

[0024] For an IoT server to control IoT devices, it needs to recognize the current status of the local computer and its applications, and then send corresponding control commands to the IoT devices. Conventionally, to improve security issues and system instability caused by the interaction between external cloud servers, local computers, and IoT devices, the present invention provides a method for recognizing the current status of local computers and applications independently of external cloud servers. For this reason, the present invention employs a screen image recognition method.

[0025] The IoT server searches the screen image of the local computer to recognize the current status of the local computer, and transmits control signals corresponding to the recognized current status to the local computer and IoT devices to execute the received mode command. Situation recognition through computer screen image search is determined by comparing the position and RGB values ​​of each pixel on the local computer with the pixel positions and RGB values ​​for each situation that have already been set.

[0026] In other words, after saving images for each situation, the pixel positions and RGB values ​​of the currently searched image are compared with the already saved situation-specific images to accurately recognize the current situation. The IoT device control system according to the present invention can be applied to various fields. For the sake of understanding, we will briefly describe how the IoT device control system according to the present invention can be applied to a screen golf practice range, but we will clarify that it is not limited to this and can be applied to various fields.

[0027] Referring to Figure 2, the screen golf practice area has an administrator computer, an IoT server, and two local computers that form a network via a router. The local computer has a golf simulation program installed as an application, and IoT devices such as a beam projector, heating and cooling equipment, and lighting are installed near the local computer and connected to the network.

[0028] Mode commands can define, for example, a work attendance mode, a practice field mode, a match mode, a departure mode, and a clock-out mode. "Attendance mode" refers to a situation where the manager arrives at the office, turns on the computer, and switches on the lights. When the IoT server receives a clock-in command from the administrator computer, it recognizes that the local computer's screen is off and sends control commands to the local computer and the lighting device to power on the computer and turn on the lighting device. The practice range mode is a mode for practicing your swing before the match mode, where scores are calculated.

[0029] When the IoT server receives a practice range mode command from the administrator computer, it searches for a screen image on the local computer. When the screen image is confirmed as the operating system screen, the golf simulation program is executed because it has not yet run. Furthermore, if the screen image indicates that the golf simulation program is in a standby state, the program is controlled to proceed to the practice range step. Furthermore, the IoT server drives the heating and cooling system defined in practice range mode to a predetermined temperature.

[0030] Thus, the IoT device control system according to the present invention is an invention that, when it is necessary to control an IoT device according to the execution status of an application, searches the screen image of the local computer to recognize the situation, and then controls the local computer and the IoT device according to that situation. By enabling situational awareness through screen image searches on the local computer, there is no need to link with a separate external cloud server, resulting in enhanced security and improved system stability. Furthermore, from the perspective of an IoT server developer, there is the advantage of being able to build a control system independently, as collaboration with application developers is not required.

[0031] Although preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and various modifications can be made by persons with ordinary skill in the art to which the invention pertains without departing from the gist of the invention as claimed in the claims. Moreover, these modified embodiments should not be understood individually from the technical idea or prospects of the present invention.

Claims

1. An IoT device control system in which an IoT server, a local computer, and IoT devices form a network via a router, A local computer on which an operating system and one or more applications are installed, An IoT server receives a predetermined mode instruction that defines the operating status of a local computer and an IoT device, and sends control instructions to the local computer and the IoT device so that the local computer and the IoT device are executed according to the definition in the received mode instruction. An IoT device control system characterized by including one or more IoT devices driven by control commands received from the IoT server.

2. The IoT device control system according to claim 1, further comprising an administrator computer that forms a network with the IoT server, local computer, and IoT device via a router and transmits a predetermined mode command to the IoT server.

3. The IoT device control system according to claim 1, characterized in that the IoT server searches the screen image of the local computer to recognize the current status of the local computer, and transmits control signals corresponding to the recognized current status to the local computer and the IoT device to execute the received mode command.

4. The IoT device control system according to claim 3, characterized in that the situation recognition by searching the screen image of the local computer is determined by comparing the position and RGB value of each pixel of the local computer with the preset pixel position and RGB value for each situation.

5. The IoT device control system according to claim 4, characterized in that the application is a golf simulation program, and the IoT device is environmental equipment necessary for operating the golf simulation program.

Images