Registration system, air conditioning system and registration program

Abstract

A registration system, an air conditioning system, and a registration program are provided for registering a specific area to be controlled in real space. [Solution] A registration system having a control unit and a memory unit, in which the control unit registers in the memory unit a specific range within the real space determined based on the location information of a user terminal in the real space and the operation content on the user terminal.

Description

[Technical Field]

[0001] The present disclosure relates to a registration system, an air conditioning system, and a registration program. [Background technology]

[0002] In systems that improve the comfort of real spaces, such as air conditioning systems, there are cases where air conditioning control is performed differently for a specific area where a user is present than for other areas. As an example, Patent Document 1 listed below discloses an air conditioning system that controls the air volume and direction so that the air does not blow into a specific area in a real space where a user holding a portable transmitter is present. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-211149 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the case of the above air conditioning system, the size and shape of the specific area to be controlled by the air conditioning control are registered in advance. As in this example, the size and shape of the specific area to be controlled are However, there is a problem that users cannot freely change the settings.

[0005] The present disclosure provides a registration system, an air conditioning system, and a registration program for registering a specific area to be controlled in real space. [Means for solving the problem]

[0006] A first aspect of the present disclosure is A registration system having a control unit and a storage unit, The control unit A specific range in the real space determined based on position information of the user terminal in the real space and the operation content on the user terminal is registered in the storage unit.

[0007] According to the first aspect of the present disclosure, it is possible to provide a registration system that registers a specific range to be controlled in real space.

[0008] A second aspect of the present disclosure is a registration system according to the first aspect, The control unit A specific range within the real space is determined based on a trajectory of position information of the user terminal when a movement operation is performed on the user terminal, and is registered in the storage unit.

[0009] A third aspect of the present disclosure is the registration system according to the first aspect, the user terminal is capable of accepting an instruction operation; The control unit A surrounding area based on the position information of the user terminal at the time when the user terminal receives the instruction operation is determined as the specific area and registered in the storage unit.

[0010] A fourth aspect of the present disclosure is the registration system according to the first aspect, the user terminal is capable of accepting an instruction operation; The control unit The user terminal determines a surrounding range based on the position information in the real space of the object identified by receiving the instruction operation as a specific range within the real space, and registers the range in the storage unit.

[0011] A fifth aspect of the present disclosure is a registration system according to the fourth aspect, the user terminal is capable of accepting a selection instruction operation; The control unit The user terminal determines a surrounding range based on the position information in the real space of the part of the object identified by receiving a selection instruction operation as a specific range within the real space, and registers it in the memory unit.

[0012] A sixth aspect of the present disclosure is a registration system according to the first aspect, the user terminal is capable of accepting a setting operation; The control unit The size or shape of the specific range is determined based on a setting operation received by the user terminal.

[0013] A seventh aspect of the present disclosure is a registration system according to the sixth aspect, the user terminal is capable of accepting a setting operation; The control unit The size of the specific range is determined based on the location information of the user terminal when the user terminal receives the setting operation for the starting point, or the location information used as a reference when determining the specific range, and the location information of the user terminal when the user terminal receives the setting operation for the end point.

[0014] An eighth aspect of the present disclosure is a registration system according to the fourth aspect, The user terminal An imaging device; a display device that displays an image captured by the imaging device, The control unit of the user terminal An object included in the image is identified as the target object.

[0015] A ninth aspect of the present disclosure is a registration system according to the fourth aspect, The user terminal An imaging device; a display device that displays an image captured by the imaging device on a screen and allows a user to point to a position on the screen where the image is displayed; The control unit of the user terminal Of the plurality of objects included in the image, the object at the pointed position is identified as the target object.

[0016] A tenth aspect of the present disclosure is a registration system according to the first aspect, The user terminal a display device for displaying an image; The control unit of the user terminal An image showing the specific range is superimposed on the image and displayed on the display device.

[0017] An eleventh aspect of the present disclosure is a registration system according to the tenth aspect, The user terminal An imaging device is included, The control unit of the user terminal A three-dimensional image showing the specific range, which is generated based on position information and orientation information of the user terminal in real space, is superimposed on the image captured by the imaging device and displayed.

[0018] A twelfth aspect of the present disclosure is a registration system according to the first aspect, comprising: the user terminal has an output device; When the user terminal is located within a specific range in the real space, the output device notifies the user of a stimulus.

[0019] A thirteenth aspect of the present disclosure is a registration system according to the first aspect, the user terminal has an output device; The output device notifies a user of a different stimulus depending on a positional relationship between the specific range in the real space and the user terminal.

[0020] A fourteenth aspect of the present disclosure is a registration system according to the fourth aspect, the user terminal has an output device; When the user terminal is located in a specific range in the real space, the output device: Notifying a different stimulus depending on whether the specific range is determined by position information of the object in the real space, or When the specific range is determined by position information of the object in the real space, different stimuli are notified depending on whether the object is a person or not.

[0021] A fifteenth aspect of the present disclosure is a registration system according to the first aspect, the user terminal has a sensor; The control unit of the user terminal The shape data of the real space is compared with the data measured by the sensor to calculate the position information of the user terminal in the real space.

[0022] A sixteenth aspect of the present disclosure is a registration system according to the fifteenth aspect, the sensor is an imaging device; The control unit of the user terminal The position information of the user terminal in the real space is calculated by comparing the three-dimensional data in the real space with a map generated from the image captured by the imaging device.

[0023] A seventeenth aspect of the present disclosure is a registration system according to the first aspect, The control unit Position information of the user terminal in the real space is calculated based on data measured by a three-dimensional position measurement sensor attached in the real space.

[0024] An eighteenth aspect of the present disclosure is a registration system according to the seventeenth aspect, the three-dimensional position measurement sensor is configured with a plurality of imaging devices attached at different positions; The control unit By matching the images captured by each of the plurality of imaging devices, position information of the user terminal in real space is calculated.

[0025] A nineteenth aspect of the present disclosure is a registration system according to the first aspect, The control unit By comparing data measured by a sensor possessed by the user terminal with data measured by a sensor attached in the real space, position information of the user terminal in the real space is calculated.

[0026] A twentieth aspect of the present disclosure is a registration system according to the first aspect, the user terminal is capable of accepting an air conditioning instruction operation, The control unit When the user terminal receives an air conditioning instruction operation, air conditioning control is performed for a specific range in the real space registered in the storage unit.

[0027] A twenty-first aspect of the present disclosure is a registration system according to the twentieth aspect, The control unit By using a plurality of air conditioning devices installed in the real space, different air conditioning controls are performed in a specific range in the real space registered in the memory unit and in a non-specific range other than the specific range.

[0028] In addition, an air conditioning system according to a 22nd aspect of the present disclosure includes: The registration system according to any one of the first to twenty-first aspects is included.

[0029] According to the twenty-second aspect of the present disclosure, it is possible to provide an air conditioning system that registers a specific range to be controlled in real space.

[0030] Furthermore, a registration program according to a twenty-third aspect of the present disclosure includes: a registration step of registering, in a storage unit, a specific range in the real space determined based on position information of the user terminal in the real space and operation content on the user terminal; The control unit executes the above.

[0031] According to the 23rd aspect of the present disclosure, it is possible to provide a registration program for registering a specific range to be controlled in real space. [Brief explanation of the drawings]

[0032] [Figure 1] FIG. 1 is a diagram illustrating an example of a system configuration of an air conditioning system. [Figure 2] FIG. 1 is a diagram illustrating an example of a real space to which an air conditioning system is applied. [Figure 3] FIG. 2 is a diagram illustrating an example of the hardware configuration of a control device and a user terminal. [Figure 4] 3A and 3B are diagrams showing operation modes of the air conditioning system and an example of an operation screen. [Figure 5] FIG. 2 is a diagram illustrating an example of a functional configuration of a control device in a specific operation mode. [Figure 6] 10 is a first diagram showing a functional configuration of a specific range input unit and a specific example of a specific range determination process. FIG. [Figure 7] FIG. 10 is a second diagram showing a specific example of the functional configuration of the specific range input unit and the specific range determination process. [Figure 8] FIG. 10 is a third diagram showing a specific example of the functional configuration of the specific range input unit and the specific range determination process. [Figure 9] FIG. 4 is a fourth diagram showing a specific example of the functional configuration of the specific range input unit and the specific range determination process. [Figure 10] FIG. 5 is a fifth diagram showing a specific example of the functional configuration of the specific range input unit and the specific range determination process. [Figure 11] FIG. 6 is a sixth diagram showing a specific example of the functional configuration of the specific range input unit and the specific range determination process. [Figure 12] 10A and 10B are diagrams illustrating a functional configuration of a specific range input unit and a specific example of a specific range deletion process. [Figure 13] 10A and 10B are diagrams illustrating a functional configuration of a specific range input unit and a specific example of specific range movement processing. [Figure 14] 10A and 10B are diagrams illustrating a functional configuration of a specific range input unit and a specific example of a driving information input process. [Figure 15]10A and 10B are diagrams illustrating a functional configuration of a specific range input unit and a specific example of driving information inversion processing. [Figure 16] 10 is a first diagram showing a functional configuration of a specific range output unit and a specific example of specific range output processing. FIG. [Figure 17] 10 is a first flowchart showing the flow of a specific range registration process in a specific operation mode. [Figure 18] 10 is a second flowchart showing the flow of the specific range registration process in the specific operation mode. [Figure 19] FIG. 1 is a diagram illustrating an example of air conditioning control by an air conditioning system. [Figure 20] FIG. 2 is a diagram illustrating an example of the functional configuration of a control device and a user terminal in a specific operation mode. [Figure 21] 10A and 10B are diagrams illustrating functional configurations of a transmission unit and a specific range output unit, and a specific example of a specific range display process. [Figure 22] FIG. 2 is a second diagram showing a functional configuration of the specific range output unit and a specific example of the specific range output process. [Figure 23] FIG. 10 is a diagram illustrating an example of calculation of coordinate information indicating the position of a user terminal in real space. [Figure 24] FIG. 2 is a diagram illustrating an example of a functional configuration of a user terminal. [Figure 25] FIG. 10 illustrates an example of a functional configuration of a specific range input unit. [Figure 26] FIG. 10 is a first diagram showing a specific example of the center position determination process. [Figure 27] FIG. 10 is a second diagram showing a specific example of the center position determination process. [Figure 28] FIG. 10 is a third diagram showing a specific example of the center position determination process. [Figure 29] FIG. 4 is a fourth diagram showing a specific example of the center position determination process. DETAILED DESCRIPTION OF THE INVENTION

[0033] Hereinafter, each embodiment will be described with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configuration are designated by the same reference numerals, and redundant description will be omitted.

[0034] [First embodiment] <System configuration of air conditioning system> First, a description will be given of the system configuration of an air conditioning system according to Embodiment 1. Fig. 1 is a diagram showing an example of the system configuration of an air conditioning system.

[0035] 1, the air conditioning system 100 includes an air conditioning device 110, an imaging device 120, a display device 130, and a user terminal 140. The air conditioning device 110 and the user terminal 140 are connected via wireless communication.

[0036] The air conditioner 110 includes, for example, an indoor unit 111 of a home air conditioner, an indoor unit 112a of a commercial air conditioner, an edge device 112b (control device), and an operation panel 112c.

[0037] The indoor unit 111 controls the air conditioning of the real space. Specifically, the indoor unit 111 Various instructions and information transmitted from the user terminal 140 via wireless communication; A captured image (e.g., an RGB image) transmitted from the imaging device 120, Data measured by a sensor included in the indoor unit 111; Based on this, a built-in control device operates to control the air conditioning of the real space. The indoor unit 111 may also have a built-in imaging device 120. The indoor unit 111 also transmits various types of information generated by the operation of the built-in control device to the user terminal 140 or the display device 130 via wireless communication.

[0038] Similarly, the indoor unit 112a controls the air conditioning of the real space. Various instructions and information transmitted from the operation panel 112c or transmitted via wireless communication from the user terminal 140, A captured image transmitted from the imaging device 120, Data measured by a sensor included in the indoor unit 112a, Based on this, the edge device 112b (control device) operates to control the air conditioning of the real space.

[0039] Furthermore, the indoor unit 112a transmits various types of information generated by the operation of the edge device 112b to the user terminal 140 via wireless communication, or transmits the information to the display device 130. The operation panel 112c may also function as the display device 130.

[0040] In the first embodiment, A control device built into the indoor unit 111, and Edge device 112b connected to indoor unit 112a, It is possible to perform air conditioning control for a specific range in the real space differently from that for a non-specific range (range other than the specific range).

[0041] The imaging device 120 includes, for example, an indoor security camera 121, a monitoring camera 122, etc. The security camera 121, the monitoring camera 122, etc. capture images of the real space in which the indoor unit 111 of a domestic air conditioner is installed or the real space in which the indoor unit 112a of a commercial air conditioner is installed, and transmits the captured images to the air conditioner 110.

[0042] The display device 130 includes, for example, a monitor 131. The monitor 131 is installed in a real space where the indoor unit 111 of a home air conditioner is installed, or in a real space where the indoor unit 112a of a commercial air conditioner is installed. The monitor 131 displays a display image generated by a control device built into the indoor unit 111 or an edge device 112b connected to the indoor unit 112a.

[0043] The user terminal 140 includes, for example, a smartphone 141, a remote control 142 for a home air conditioner, a wearable device (not shown), etc. The smartphone 141, remote control 142, wearable device (not shown), etc. are held (or worn) by a user in real space, and transmit various instructions and information input by the user to the air conditioner 110. This allows the user to register a specific range in real space via the user terminal 140.

[0044] Furthermore, the smartphone 141, remote control 142, wearable device (not shown), etc. notify the user of the registered specific range based on various information transmitted from the air conditioning device 110. This allows the user to know, via the user terminal 140, which range in the real space has been registered as the specific range.

[0045] <Example of real space> Next, a description will be given of a specific example of a real space to which the air conditioning system 100 is applied. Fig. 2 is a diagram showing an example of a real space to which the air conditioning system is applied.

[0046] 1, real space 210 is the room of a user 211. An air conditioning system 100 applied to real space 210 includes an indoor unit 111 with a built-in imaging device 120, and a smartphone 141. A control device (not shown) of the indoor unit 111 identifies each position in real space 210 using coordinate information based on coordinate axes 212.

[0047] On the other hand, real space 220 is the office of users 221 and 222. The air conditioning system 100 applied to real space 220 has an indoor unit 112a, an edge device 112b (not shown), an operation panel 112c, a security camera 121, a monitor 131, and a smartphone 141. Note that edge device 112b (not shown) identifies each position in real space 220 by coordinate information based on coordinate axes 223.

[0048] <Hardware configuration of the control device and user terminal> Next, the hardware configuration of the control device built into the indoor unit 111 of the air conditioning device 110 or the edge device 112b (control device) connected to the indoor unit 112a of the air conditioning device 110, and the hardware configuration of the user terminal 140 will be explained using Figure 3.

[0049] FIG. 3 is a diagram illustrating an example of the hardware configuration of the control device and the hardware configuration of the user terminal.

[0050] (1) Hardware configuration of the control device Of these, 3a in Figure 3 shows an example of the hardware configuration of the control device of the air conditioning device 110. As shown in 3a in Figure 3, the control device of the air conditioning device 110 has a processor 301, memory 302, and auxiliary storage device 303. The control device of the air conditioning device 110 also has an I / F (Interface) device 304 and a communication device 305. The hardware components of the control device of the air conditioning device 110 are connected to each other via a bus 306.

[0051] The processor 301 has various arithmetic devices such as a CPU (Central Processing Unit), etc. The processor 301 reads various programs (for example, a registered program, which will be described later) onto the memory 302 and executes them.

[0052] The memory 302 has a main storage device such as a ROM (Read Only Memory) and a RAM (Random Access Memory). The processor 301 and the memory 302 form a so-called computer (a "control unit" as hardware), and the processor 301 executes various programs read onto the memory 302, causing the control device of the air conditioning device 110 to realize various functions.

[0053] The auxiliary storage device 303 is an example of a storage unit, and stores various programs and various data used when the processor 301 executes the various programs.

[0054] The I / F device 304 is a connection device that connects the control device of the air conditioning apparatus 110 with other devices. In this embodiment, the other devices include the imaging device 120 and the display device 130. In this embodiment, the other devices also include a radio wave transmitting / receiving device 310, which is an example of a sensor included in the indoor unit 111 or the indoor unit 112a of the air conditioning apparatus 110. The radio wave transmitting / receiving device 310 is, for example, a TOF depth sensor (an example of a three-dimensional position measurement sensor) that acquires radio wave information (a reflected signal of laser light) by scanning a real space with a laser light. By using the radio wave information acquired by the radio wave transmitting / receiving device 310 in any direction within the real space 210 or 220, it is possible to calculate coordinate information (a so-called depth map) that indicates the positions of multiple objects within the real space 210 or 220.

[0055] The communication apparatus 305 is a communication device that transmits and receives information between the control apparatus of the air conditioning apparatus 110 and the user terminal 140.

[0056] (2) Hardware configuration of the user terminal 3b in Fig. 3 shows an example of the hardware configuration of the user terminal 140. As shown in 3b in Fig. 3, the user terminal 140 has a processor 321, a memory 322, an auxiliary storage device 323, a communication device 324, an operation device 325, and a display device 326. The user terminal 140 also has an imaging device 327, an acceleration sensor 328, an audio input device 329, an audio output device 330, and a vibration device 331. The hardware components of the user terminal 140 are connected to each other via a bus 332.

[0057] The processor 321 has various arithmetic devices such as a CPU (Central Processing Unit), etc. The processor 321 reads various programs (for example, an air conditioning operation program described below) into the memory 322 and executes them.

[0058] The memory 322 has a main storage device such as a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The processor 321 and the memory 322 form a so-called computer (a "control unit" as hardware), and the processor 321 executes various programs read onto the memory 322, thereby enabling the user terminal 140 to realize various functions.

[0059] The auxiliary storage device 323 stores various programs and various data used when the processor 321 executes the various programs.

[0060] The communication device 324 is a communication device that transmits and receives information between the user terminal 140 and the control device of the air conditioning apparatus 110.

[0061] The operation device 325 receives various instructions from the users 211, 221, 222, etc. to the user terminal 140. The display device 326 displays a display image based on various information transmitted from the control device of the air conditioner 110.

[0062] The imaging device 327 captures images of the real spaces 210 and 220, for example. The acceleration sensor 328 measures the acceleration of the user terminal 140 in three axial directions. The audio input device 329 inputs the audio of the users 211, 221, 222, etc. The audio output device 330 outputs sounds based on various information transmitted from the control device of the air conditioning device 110, for example. The vibration device 331 outputs vibrations based on various information transmitted from the control device of the air conditioning device 110, for example.

[0063] <Operation mode and operation screen> Next, we will explain the operation modes that are realized by executing the air conditioning operation program on a user terminal when the air conditioning system 100 is in operation, and the operation screens that are displayed on the user terminal 140. Figure 4 is a diagram showing an example of the operation modes of the air conditioning system and the operation screens.

[0064] Of these, 4a in Fig. 4 shows the operation mode when the air conditioning system 100 is operating. As shown in 4a in Fig. 4, the air conditioning system 100 transitions to either a "normal operation mode" or a "specific operation mode" during operation.

[0065] The "normal operation mode" is a mode in which the air conditioning system 100 is operated based on conventional operation information to control the air conditioning of the real spaces 210 and 220. On the other hand, the "specific operation mode" is a mode in which, when the air conditioning system 100 is operated, air conditioning control is performed based on different operation information for the specific range and the non-specific range.

[0066] 4b in Fig. 4 shows an example of an operation screen displayed on the user terminal 140 when the air conditioning system 100 is operated in a specific operation mode. As shown in 4b in Fig. 4, pressing the "specific operation" button 412 transitions to an operation screen 400 for the specific operation mode. The operation screen 400 includes an operation information display field 411, a specific range input field 413, a "specific range display" button 414, an "operation information input" button 415, a "start" button 416, and a "stop" button 417.

[0067] The operating information display field 411 displays the operating information set for the specific range. If there are multiple specific ranges, the operating information is displayed for each specific range. The example in FIG. 4 shows that the operating information set for "specific range 1" is temperature = "XX°C", wind direction = "not blowing", and air volume = "0".

[0068] The specific range input field 413 includes instruction buttons used when the users 211, 221, 222, etc. input a specific range. The "input range" button is pressed to instruct the start of inputting a specific range.

[0069] The "increase / decrease" button is pressed when inputting the size of the specific range. Note that instead of pressing the "increase / decrease" button to input the size of the specific range in the "radius information" input field ("Rmm" in the figure), you can perform a setting operation and input the numerical value indicating the size of the specific range directly into the "radius information" input field.

[0070] The "Target Photograph" button is a button that is pressed when photographing a target object to capture the target area in a specific range. Pressing the "Target Photograph" button starts up the imaging device 327 and starts photographing.

[0071] The "Delete Range" button is a button that is pressed when deleting a specific range that has already been registered. The "Move Range" button is a button that is pressed when moving a specific range that has already been registered.

[0072] The "reverse" button is a button that is pressed to swap the operating information used when performing air conditioning control for a specific range that has already been registered with the operating information used when performing air conditioning control for a non-specific range that is different from the specific range.

[0073] When the "Display specific range" button 414 is pressed, the imaging device 327 is activated, and an image captured by the imaging device 327 is displayed on the display screen, along with a superimposed display image based on various information transmitted from the control device of the air conditioning device 110.

[0074] The "input driving information" button 415 is a button that is pressed by the users 211, 221, 222, etc. when they input driving information. When the "input driving information" button 415 is pressed, items that can be input in a specific driving mode are displayed, and the users 211, 221, 222, etc. input driving information corresponding to each item.

[0075] The driving information may be configured so that the same content can be registered for each specific range, or so that different content can be registered for each location within the same specific range. Also, the history of previously registered driving information may be read out and registered for another specific range. Furthermore, the driving information associated with each currently registered specific range may be swapped and registered between specific ranges.

[0076] The "Start" button 416 is a button for issuing an air conditioning instruction operation, used when instructing the air conditioner 110 to start air conditioning control in a specific operation mode. The "Stop" button 417 is a button for issuing an air conditioning instruction operation, used when instructing the air conditioner 110 to stop air conditioning control in a specific operation mode.

[0077] <Controller functional configuration> Next, the functional configuration of the control device of the air conditioning apparatus 110 in the specific operation mode will be described. Fig. 5 is a diagram showing an example of the functional configuration of the control device in the specific operation mode. As described above, a registration program is installed in the control device of the air conditioning apparatus 110, and by executing this program, the control device of the air conditioning apparatus 110 functions as a specific range input unit 501, an air conditioning control unit 502, and a specific range output unit 503.

[0078] The specific range input unit 501 determines the specific range and operation information based on various instructions and information transmitted from the user terminal 140, captured images transmitted from the imaging device 120, data measured by sensors in the indoor unit 111, etc. The specific range input unit 501 also registers the determined specific range and operation information in the air conditioning control unit 502. Specifically, the specific range input unit 501 notifies the air conditioning control unit 502 of the determined specific range and operation information. In response, the air conditioning control unit 502 stores the determined specific range and operation information in a specific range and operation information storage unit, which is a storage area available to the air conditioning control unit 502 within the auxiliary storage device 303 of the control device of the air conditioning apparatus 110.

[0079] The air conditioning control unit 502 performs air conditioning control based on the specific range and operation information registered by the specific range input unit 501.

[0080] The specific range output unit 503 transmits to the display device 130 an image (specific range image) for notifying the users 211, 221, 222, etc. of the specific range registered in the air conditioning control unit 502.

[0081] <Specific examples of functional configuration of specific range input unit and specific range determination process> Next, the details of the functional configuration of the specific range input unit 501 (here, the functional configuration related to determining the specific range) and specific examples of the specific range determination process will be described for patterns (1) to (6).

[0082] (1) Pattern 1 6 is a first diagram showing a functional configuration of the specific range input unit and a specific example of the specific range determination process. As shown in FIG. 6, the specific range input unit 501 includes a captured image acquisition unit 601, a radio wave information acquisition unit 602, a position information acquisition unit 603, an operation content acquisition unit 604, a radius information determination unit 605, and a specific range determination unit 607.

[0083] The captured image acquisition unit 601 acquires the captured image captured by the imaging device 120 and notifies the location information acquisition unit 603 of the image.

[0084] The radio wave information acquisition unit 602 acquires radio wave information transmitted and received by the radio wave transmitting and receiving device 310 and notifies the position information acquisition unit 603 of the information.

[0085] Based on the notified captured image and radio wave information, the position information acquisition unit 603 calculates coordinate information (x, y, z) indicating the position of the user terminal 140 in the real space 210 or 220. For example, the radio wave information acquisition unit 602 calculates coordinate information indicating the positions of multiple objects in the real space based on the radio wave information, and extracts coordinate information indicating the position of the user terminal 140 that has been image-recognized based on the captured image from the calculated coordinate information indicating the positions of the multiple objects.

[0086] The operation content acquisition unit 604 acquires the operation content transmitted from the user terminal 140. The example in Fig. 6 shows that the "input range" button and the "increase / decrease" button are pressed as operation content.

[0087] When the operation content acquisition unit 604 acquires a range input instruction as a result of pressing the "range input" button, it notifies the specific range determination unit 607 of the range input instruction. When the specific range determination unit 607 receives a range input instruction from the operation content acquisition unit 604, it stores the coordinate information notified by the position information acquisition unit 603 as center position information (x0, y0, z0) that indicates the position of the center point of the specific range.

[0088] Furthermore, when the operation content acquiring unit 604 acquires tap information due to the "increase / decrease" button being tapped n times (due to repeated tapping operations), it notifies the radius information identifying unit 605 of the tap information. Alternatively, when the operation content acquiring unit 604 acquires tap information due to the "increase / decrease" button being pressed and held for t seconds (due to the pressing operation continuing for a predetermined time), it notifies the radius information identifying unit 605 of the tap information.

[0089] The radius information identification unit 605 calculates radius information (R) based on the tap information notified by the operation content acquisition unit 604, and notifies the specific range determination unit 607. The radius information identification unit 605 calculates radius information (R) proportional to the number of taps (number of times repeated) or the long press time (duration).

[0090] The specific range determination unit 607 acquires shape information of a predetermined shape of the specific range by referring to the shape memory unit 610. The example in Fig. 6 shows a state in which spherical shape information is acquired by referring to the shape memory unit 610. However, the shape information acquired by referring to the shape memory unit 610 is not limited to spherical shape information, and may be any other three-dimensional shape information, such as conical shape information.

[0091] Furthermore, the specific range determination unit 607 determines a specific range 621 based on the central position information (x0, y0, z0) of the specific range, the radius information (R) notified by the radius information determination unit 605, and the spherical shape information acquired from the shape storage unit 610. In other words, the specific range determination unit 607 determines the surrounding range based on the central position information as the specific range 621.

[0092] The example in Fig. 6 shows how specific range 620 is generated based on intermediate tap information. In this way, according to the example in Fig. 6, user 211 can adjust the size of the specific range by adjusting the number of taps or the long press time.

[0093] The user terminal 140 The user 211 taps the "increase / decrease" button, or By pressing and holding Each time the radius information (R) changes by a certain amount, a predetermined stimulus (sound, light, or vibration) may be notified to the user 211. This allows the user 211 to know that the radius information (R) has changed by a certain amount.

[0094] (2) Pattern 2 7 is a second diagram showing a specific example of the functional configuration of the specific range input unit and the specific range determination process. As shown in FIG. 7, the specific range input unit 501 includes a captured image acquisition unit 601, a radio wave information acquisition unit 602, a position information acquisition unit 603, an operation content acquisition unit 701, and a specific range determination unit 607.

[0095] Of these, the captured image acquisition unit 601 to the position information acquisition unit 603 have already been explained using FIG. 6, and therefore explanations thereof will be omitted here.

[0096] The operation content acquisition unit 701 acquires the operation content transmitted from the user terminal 140. The example in Fig. 7 shows that the "input range" button is pressed and radius information (R) is directly input into the "radius information" input field as the operation content.

[0097] When the operation content acquisition unit 701 acquires a range input instruction as a result of pressing the "range input" button, it notifies the specific range determination unit 607 of the range input instruction. When the specific range determination unit 607 receives a range input instruction from the operation content acquisition unit 604, it stores the coordinate information notified by the position information acquisition unit 603 as center position information (x0, y0, z0) of the specific range.

[0098] In addition, when the operation content acquisition unit 701 acquires the radius information (R) by directly inputting the radius information (R) into the "radius information" input field, it notifies the specific range determination unit 607 of the radius information (R).

[0099] The method of determining the specific range 621 by the specific range determination unit 607 is the same as that shown in FIG. 6, and therefore the description thereof will be omitted here.

[0100] (3) Pattern 3 8 is a third diagram showing a specific example of the functional configuration of the specific range input unit and the specific range determination process. As shown in FIG. 8, the specific range input unit 501 includes a captured image acquisition unit 601, a radio wave information acquisition unit 602, a position information acquisition unit 603, an operation content acquisition unit 801, a center position and radius information calculation unit 802, and a specific range determination unit 803.

[0101] Of these, the photographed image acquisition unit 601 to the position information acquisition unit 603 have already been explained using Fig. 6, so explanations will be omitted here. However, in the case of Fig. 8, the position information acquisition unit 603 notifies the calculated coordinate information to the center position and radius information calculation unit 802.

[0102] The operation content acquisition unit 801 acquires the operation content transmitted from the user terminal 140 and notifies the center position and radius information calculation unit 802. The example in Fig. 8 shows that the "input range" button is pressed as the operation content, and an instruction to input a range is notified.

[0103] As shown in FIG. 8, in the case of pattern 3, the user 211 presses the “input range” button, then performs a movement operation on the user terminal 140, and specifies the position and size of a specific range by drawing an arc trajectory in the real space 210.

[0104] Specifically, after the operation content acquisition unit 801 notifies the center position and radius information calculation unit 802 of the range input instruction, the center position and radius information calculation unit 802 calculates the coordinate information (coordinate information (x1, y1, z1) to (x m ,y m ,z m )) to get the

[0105] Furthermore, the center position and radius information calculation unit 802 calculates the acquired coordinate information (coordinate information (x1, y1, z1) to (x m ,y m ,z m )), the central position information (x0, y0, z0) indicating the position of the central point of the arc is calculated and notified to the specific range determination unit 803.

[0106] Furthermore, the center position and radius information calculation unit 802 calculates the acquired coordinate information (coordinate information (x1, y1, z1) to (x m ,y m ,z m )), the maximum value of the distance from the central position information (x0, y0, z0). Furthermore, the central position and radius information calculation unit 802 notifies the specific range determination unit 803 of the calculated maximum value as radius information (R).

[0107] The specific range determination unit 803 acquires shape information of a predetermined shape of the specific range by referring to the shape memory unit 610. The example of Fig. 8 shows a state in which spherical shape information is acquired by referring to the shape memory unit 610. However, the shape information acquired by referring to the shape memory unit 610 is not limited to spherical shape information, and may be any other three-dimensional shape information such as conical shape information.

[0108] Furthermore, the specific range determination unit 803 determines the specific range 621 based on the central position information (x0, y0, z0) and radius information (R) notified by the central position and radius information calculation unit 802, and the spherical shape information acquired from the shape storage unit 610. In other words, the specific range determination unit 803 determines the peripheral range based on the central position information as the specific range 621.

[0109] (4) Pattern 4 9 is a fourth diagram showing a specific example of the functional configuration of the specific range input unit and the specific range determination process. As shown in FIG. 9, the specific range input unit 501 includes a captured image acquisition unit 601, a radio wave information acquisition unit 602, a position information acquisition unit 603, an operation content acquisition unit 801, a center position and radius information calculation unit 802, a projection direction determination unit 901, and a specific range determination unit 902.

[0110] Of these, the captured image acquisition unit 601 to the position information acquisition unit 603 have already been explained using Fig. 6, and therefore their explanation will be omitted here. Also, the operation content acquisition unit 801 to the center position and radius information calculation unit 802 have already been explained using Fig. 8, and therefore their explanation will be omitted here.

[0111] The projection direction determination unit 901 determines the imaging direction when the imaging device 120 captures the image of the user terminal 140 as the projection direction and notifies the specific range determination unit 902 of this.

[0112] The specific range determination unit 902 acquires shape information of the predetermined shape of the specific range by referring to the shape storage unit 610. The example of Fig. 9 shows how spherical shape information is acquired by referring to the shape storage unit 610.

[0113] Furthermore, the specific range determination unit 902 calculates a specific range based on the center position information (x0, y0, z0) and radius information (R) notified by the center position and radius information calculation unit 802, and the spherical shape information acquired from the shape memory unit 610. Furthermore, based on the projection direction notified by the projection direction determination unit 901, the specific range determination unit 902 shifts the position of the center point by a predetermined amount in the projection direction while maintaining the size and shape of the calculated specific range, and calculates center position information (x'0, y'0, z'0). In this way, the specific range determination unit 902 determines the projected region as the specific range 621. In other words, the specific range determination unit 902 determines the peripheral range based on the center position information as the specific range 621.

[0114] In the example of FIG. 9, the specific range is calculated and then projected. However, the specific range may be calculated after projecting a trajectory based on a movement operation on the user terminal 140. Specifically, first, coordinate information (coordinate information (x1, y1, z1) to (x m ,y m ,z m )) is projected in the shooting direction based on the captured image captured by the imaging device 120. Then, the projection area specified by the coordinate information after the projection (the area specified based on the center position information, radius information (R), and spherical shape information) is determined as the specific range.

[0115] (5) Pattern 5 10 is a fifth diagram showing a specific example of the functional configuration of the specific range input unit and the specific range determination process. As shown in Fig. 10, the specific range input unit 501 includes a captured image acquisition unit 601, a radio wave information acquisition unit 602, a position information acquisition unit 603, an operation content acquisition unit 801, a center position and radius information calculation unit 802, a projection direction determination unit 1001, and a specific range determination unit 1002.

[0116] Of these, the captured image acquisition unit 601 to the position information acquisition unit 603 have already been explained using Fig. 6, and therefore their explanation will be omitted here. Also, the operation content acquisition unit 801 to the center position and radius information calculation unit 802 have already been explained using Fig. 8, and therefore their explanation will be omitted here.

[0117] The projection direction determination unit 1001 determines a predetermined direction according to the positional relationship between the user terminal 140 and the user 211 as the projection direction, and notifies the specific range determination unit 1002 of the determined projection direction.

[0118] The specific range determination unit 1002 acquires shape information of the predetermined shape of the specific range by referring to the shape storage unit 610. The example of Fig. 10 shows how spherical shape information is acquired by referring to the shape storage unit 610.

[0119] Furthermore, the specific range determination unit 1002 calculates a specific range based on the center position information (x0, y0, z0) and radius information (R) notified by the center position and radius information calculation unit 802, and the spherical shape information acquired from the shape storage unit 610. Furthermore, based on the projection direction notified by the projection direction determination unit 1001, the specific range determination unit 1002 shifts the position of the center point by a predetermined amount in the projection direction while maintaining the size and shape of the calculated specific range, and calculates center position information (x'0, y'0, z'0). In this way, the specific range determination unit 1002 determines the projected region as the specific range 621. In other words, the specific range determination unit 1002 determines the peripheral range based on the center position information as the specific range 621.

[0120] In the example of FIG. 10, the specific range is calculated and then projected. However, the specific range may be calculated after projecting a trajectory based on a movement operation on the user terminal 140.

[0121] Specifically, first, coordinate information (coordinate information (x1, y1, z1) to (x m ,y m ,z m )) is projected based on the positional relationship between the user 211 and the user terminal 140, which is identified from the image captured by the imaging device 120. Then, the projection area identified by the coordinate information after the projection (the area identified based on the center position information, radius information (R), and spherical shape information) is determined as the specific range.

[0122] (6) Pattern 6 11 is a sixth diagram showing a specific example of the functional configuration of the specific area input unit and the specific area determination process. As shown in FIG. 11, the specific area input unit 501 includes a captured image acquisition unit 601, a radio wave information acquisition unit 602, a position information acquisition unit 603, an operation content acquisition unit 1101, a target area calculation unit 1102, and a specific area determination unit 1103.

[0123] Of these, the captured image acquisition unit 601 to the position information acquisition unit 603 have already been explained using FIG. 6, and therefore explanations thereof will be omitted here.

[0124] The operation content acquisition unit 1101 acquires the operation content transmitted from the user terminal 140. The example in Fig. 11 shows that, as the operation content, the "input range" button is pressed first, and then the "photograph target" button is pressed.

[0125] The operation content acquisition unit 1101 acquires a range input instruction when the "input range" button is pressed. Furthermore, the operation content acquisition unit 1101 acquires a shooting instruction when the shutter button (not shown) is pressed after shooting has started on the user terminal 140 when the "photograph target" button is pressed, and notifies the target area calculation unit 1102. Note that when the shooting instruction is notified by the operation content acquisition unit 1101, the target area calculation unit 1102 holds the coordinate information notified by the position information acquisition unit 603. The position coordinates held at this time are reference position information (x b ,y b ,z b )

[0126] Furthermore, when the operation content acquisition unit 1101 acquires a captured image from the user terminal 140 as a result of pressing a shutter button (not shown), it notifies the target area calculation unit 1102 of the captured image.

[0127] The target area calculation unit 1102 calculates the reference position information (x b ,y b ,z b), the object included in the captured image is identified, and coordinate information of the area of ​​the object (object area) in the real space 210 is calculated.

[0128] In addition, the target area calculation unit 1102 calculates the reference position information (x b ,y b ,z b ), the layout information storage unit 1104 is referenced to identify the object included in the captured image. The layout information storage unit 1104 is assumed to have stored therein in advance layout information in which the object name of each object (for example, a foliage plant, a chair, etc.) arranged in the real space 210 is associated with coordinate information (shape data such as three-dimensional data) of the object area.

[0129] Therefore, the target area calculation unit 1102 calculates the object name corresponding to the object included in the captured image, and the reference position information (x b ,y b ,z b ) from the layout information, the object name associated with the coordinate information of the vicinity. Then, the target area calculation unit 1102 notifies the specific range determination unit 1103 of the coordinate information of the object area associated with the identified object name as the coordinate information of the area of ​​the photographed object (target area).

[0130] The specific range determination unit 1103 determines the coordinate information of the target area notified by the target area calculation unit 1102 as the specific range 621 .

[0131] <Specific example of functional configuration of specific range input unit and specific range deletion process> Next, the details of the functional configuration of the specific range input unit 501 (here, the functional configuration related to deleting a specific range) and a specific example of the specific range deletion process will be described.

[0132] 12 is a diagram showing a functional configuration of the specific area input unit and a specific example of specific area deletion processing. As shown in FIG. 12, the specific area input unit 501 includes a captured image acquisition unit 601, a radio wave information acquisition unit 602, a location information acquisition unit 603, an operation content acquisition unit 1201, a deletion target determination unit 1202, and a specific area deletion unit 1203.

[0133] Of these, the captured image acquisition unit 601 to the position information acquisition unit 603 have already been explained using Fig. 6, so explanations will be omitted here. However, in the case of Fig. 12, the position information acquisition unit 603 notifies the deletion target determination unit 1202 of the calculated coordinate information.

[0134] The operation content acquisition unit 1201 acquires the operation content transmitted from the user terminal 140 and notifies the deletion target determination unit 1202. The example in Fig. 12 shows that the "Delete Range" button has been pressed as the operation content, resulting in notification of an instruction to delete a range.

[0135] When the operation content acquisition unit 1201 notifies the deletion instruction of the area, the deletion target determination unit 1202 converts the coordinate information notified by the position information acquisition unit 603 into deletion position information (x d ,y d , z d ) and keep it as

[0136] In addition, the deletion target determination unit 1202 reads out the specific range information already registered in the air conditioning control unit 502, and d ,y d , z d ) is included in the specified range. Furthermore, the deletion target determination unit 1202 notifies the specified range deletion unit 1203 of the determined specific range as a specific range to be deleted.

[0137] The specific range deletion unit 1203 deletes the specific range notified by the deletion target determination unit 1202 from the specific range information already registered in the air conditioning control unit 502 .

[0138] In this way, when the "delete range" button is pressed while the user terminal 140 is located within the specific range to be deleted, the specific range deletion unit 1203 deletes the specific range to be deleted.

[0139] <Specific examples of functional configuration of specific range input unit and specific range movement processing> Next, the details of the functional configuration of the specific range input unit 501 (here, the functional configuration related to the movement of the specific range) and a specific example of the specific range movement process will be described.

[0140] 13 is a diagram showing a functional configuration of the specific area input unit and a specific example of specific area movement processing. As shown in FIG. 13, the specific area input unit 501 includes a captured image acquisition unit 601, a radio wave information acquisition unit 602, a position information acquisition unit 603, an operation content acquisition unit 1301, a movement target determination unit 1302, and a specific area update unit 1303.

[0141] Of these, the captured image acquisition unit 601 to the position information acquisition unit 603 have already been explained using Fig. 6, so explanations will be omitted here. However, in the case of Fig. 13, the position information acquisition unit 603 notifies the movement target determination unit 1302 of the calculated coordinate information.

[0142] The operation content acquisition unit 1301 acquires the operation content transmitted from the user terminal 140 and notifies the movement target determination unit 1302. The example in Fig. 13 shows that the "range movement" button is pressed as the operation content, resulting in the notification of a range movement start instruction and a range movement end instruction.

[0143] When the operation content acquisition unit 1301 notifies the movement target determination unit 1302 of the range movement start instruction, the movement target determination unit 1302 converts the coordinate information notified by the position information acquisition unit 603 into the pre-movement position information (x t ,y t ,z t ) and keep it as

[0144] In addition, the movement target determination unit 1302 reads out specific range information already registered in the air conditioning control unit 502, and calculates the pre-movement position information (x t ,y t ,z t Furthermore, the movement target determination unit 1302 acquires pre-movement center position information (x0, y0, z0) which is center position information of the determined specific range.

[0145] Furthermore, after a movement operation on the user terminal 140, when a range movement end instruction is notified from the operation content acquisition unit 1301, the movement target determination unit 1302 converts the coordinate information notified from the position information acquisition unit 603 into post-movement position information (x' t ,y' t ,z' t ) and keep it as

[0146] In addition, the movement target determination unit 1302 uses the pre-movement position information (x t ,y t ,z t ) and the position information after movement (x' t ,y' t ,z' t ) is added to the pre-movement center position information (x0, y0, z0). In this way, the movement target determination unit 1302 calculates post-movement center position information (x'0, y'0, z'0).

[0147] Furthermore, the movement target determination unit 1302 notifies the specific range update unit 1303 of the calculated post-movement center position information (x'0, y'0, z'0).

[0148] The specific range update unit 1303 updates the corresponding specific range information already registered in the air conditioning control unit 502 based on the post-movement center position information (x′0, y′0, z′0) notified by the movement target determination unit 1302.

[0149] In this way, when the user terminal 140 is located within the specific range of the movement target and the "range movement" button is pressed and a movement operation is performed, the specific range update unit 1303 moves the specific range of the movement target.

[0150] <Specific examples of functional configuration of specific range input unit and driving information input processing> Next, the details of the functional configuration of the specific range input unit 501 (here, the functional configuration related to input of driving information) and a specific example of the driving information input process will be described.

[0151] 14 is a diagram showing a specific example of the functional configuration of the specific range input unit and the driving information input process. As shown in FIG. 14, the specific range input unit 501 includes a captured image acquisition unit 601, a radio wave information acquisition unit 602, a position information acquisition unit 603, an operation content acquisition unit 1401, a driving information acquisition unit 1402, and a driving information setting unit 1403.

[0152] Of these, the captured image acquisition unit 601 to the position information acquisition unit 603 have already been explained using Fig. 6, so explanations will be omitted here. However, in the case of Fig. 14, the position information acquisition unit 603 notifies the driving information acquisition unit 1402 of the calculated coordinate information.

[0153] The operation content acquisition unit 1401 acquires the operation content transmitted from the user terminal 140 and notifies the driving information acquisition unit 1402. The example of Fig. 14 shows that the operation content is that the "input driving information" button 415 is pressed, thereby notifying an instruction to input driving information. Furthermore, the example of Fig. 14 shows that the operation content is that the driving information is input, thereby notifying the driving information.

[0154] When the operation content acquisition unit 1401 notifies the driving information input instruction, the driving information acquisition unit 1402 converts the coordinate information notified by the position information acquisition unit 603 into input position information (x i ,y i ,z i ) and keep it as

[0155] In addition, the operating information acquisition unit 1402 reads out the specific range information already registered in the air conditioning control unit 502, and acquires the input position information (x i ,y i ,z i Further, the driving information acquisition unit 1402 notifies the driving information setting unit 1403 of the determined specific range together with the notified driving information.

[0156] The operating information setting unit 1403 registers the operating information notified by the operating information acquisition unit 1402 as operating information for the notified specific range in association with the specific range.

[0157] In this way, when the "input driving information" button is pressed and driving information is input while the user terminal 140 is located within the specific range for which driving information is to be registered, the driving information setting unit 1403 registers the driving information in association with the specific range.

[0158] <Specific examples of functional configuration of specific range input unit and driving information inversion processing> Next, details of the functional configuration of the specific range input unit 501 (here, the functional configuration related to the inversion of driving information) and a specific example of driving information inversion processing will be described. Fig. 15 is a diagram showing the functional configuration of the specific range input unit and a specific example of driving information inversion processing. As shown in Fig. 15, the specific range input unit 501 has a captured image acquisition unit 601, a radio wave information acquisition unit 602, a position information acquisition unit 603, an operation content acquisition unit 1501, an inversion instruction acquisition unit 1502, and a driving information update unit 1503.

[0159] Of these, the captured image acquisition unit 601 to the position information acquisition unit 603 have already been explained using Fig. 6, so explanations will be omitted here. However, in the case of Fig. 15, the position information acquisition unit 603 notifies the calculated coordinate information to the inversion instruction acquisition unit 1502.

[0160] The operation content acquisition unit 1501 acquires the operation content transmitted from the user terminal 140 and notifies the inversion instruction acquisition unit 1502. The example in Fig. 15 shows that the inversion instruction is notified by pressing the "invert" button as the operation content.

[0161] When the operation content acquisition unit 1501 notifies the inversion instruction, the inversion instruction acquisition unit 1502 converts the coordinate information notified by the position information acquisition unit 603 into inversion position information (x r ,y r ,z r ) and keep it as

[0162] In addition, the reversal instruction acquisition unit 1502 reads out the specific range information already registered in the air conditioning control unit 502, and acquires the reversal position information (x r ,y r ,z rFurthermore, the reversal instruction acquisition unit 1502 reads out the driving information associated with the determined specific range and the driving information associated with the non-specific range, and notifies the driving information update unit 1503 of them.

[0163] The operating information update unit 1503 replaces the operating information associated with the specific range notified by the inversion instruction acquisition unit 1502 with the operating information associated with the non-specific range, and registers the replaced operating information in the air conditioning control unit 502.

[0164] In this way, when the "reverse" button is pressed while the user terminal 140 is located within the specific range where the driving information is to be reversed, the driving information update unit 1503 replaces the driving information of the specific range with the driving information of the non-specific range.

[0165] <Specific examples of functional configuration of specific range output unit and specific range display process> Next, the functional configuration of the specific range output unit 503 (here, the functional configuration related to the display of the specific range) and a specific example of the specific range display process will be described in detail. Fig. 16 is a first diagram showing the functional configuration of the specific range output unit and a specific example of the specific range output process. As shown in Fig. 16, the specific range output unit 503 has a specific range information acquisition unit 1601, a captured image acquisition unit 1602, and a specific range image generation unit 1603.

[0166] The specific range information acquisition unit 1601 reads out the specific range information registered in the air conditioning control unit 502 and notifies the specific range image generation unit 1603 of the information.

[0167] The captured image acquisition unit 1602 acquires the captured image captured by the image capturing device 120 and notifies the specific range image generation unit 1603 of the image.

[0168] The specific range image generation unit 1603 generates an image showing the specific range based on the specific range information notified by the specific range information acquisition unit 1601. Furthermore, the specific range image generation unit 1603 generates the specific range image by superimposing the generated image showing the specific range on the captured image notified by the captured image acquisition unit 1602 at a position according to the specific range information.

[0169] Furthermore, the specific range image generating unit 1603 transmits the captured image or the generated specific range image to the monitor 131, thereby displaying the captured image or the specific range image on the monitor 131.

[0170] The example of FIG. 16 shows a captured image 1611 displayed on the monitor 131, and a specific range image 1612 displayed on the monitor 131.

[0171] <Flow of specific range registration process in specific operation mode> Next, we will explain the flow of the specific range registration process in the specific operation mode of the air conditioning system 100. Figures 17 and 18 are first and second flowcharts showing the flow of the specific range registration process in the specific operation mode.

[0172] In step S 1701 , the specific range input unit 501 calculates the coordinate information of the user terminal 140 .

[0173] In step S1702, specific range input unit 501 determines whether or not a range input instruction has been acquired. If it is determined in step S1702 that a range input instruction has not been acquired (NO in step S1702), the process proceeds to step S1706.

[0174] On the other hand, if it is determined in step S1702 that a range input instruction has been acquired (YES in step S1702), the specific range input unit 501 proceeds to step S1703.

[0175] In step S1703, the specific range input unit 501 acquires the operation content for the user terminal 140.

[0176] In step S1704, the specific range input unit 501 determines a specific range based on the coordinate information of the user terminal 140 and the operation content on the user terminal 140, and generates specific range information.

[0177] In step S1705, the specific range input unit 501 registers the generated specific range information in the air conditioning control unit 502.

[0178] In step S1706, specific range input unit 501 determines whether or not a range deletion instruction has been acquired. If it is determined in step S1706 that a range deletion instruction has not been acquired (NO in step S1706), the process proceeds to step S1709.

[0179] On the other hand, if it is determined in step S1706 that a range deletion instruction has been acquired (YES in step S1706), specific range input unit 501 proceeds to step S1707.

[0180] In step S1707, the specific range input unit 501 determines the specific range to be deleted based on the coordinate information of the user terminal 140.

[0181] In step S1708, the specific range input unit 501 deletes the specific range information of the determined specific range.

[0182] In step S1709, the specific range input unit 501 determines whether or not a range movement start instruction and a range movement end instruction have been acquired. If it is determined in step S1709 that a range movement start instruction and a range movement end instruction have not been acquired (NO in step S1709), the process proceeds to step S1801 in FIG.

[0183] On the other hand, if it is determined in step S1709 that the range movement start instruction and range movement end instruction have been acquired (YES in step S1709), the process proceeds to step S1710.

[0184] In step S1710, the specific range input unit 501 determines the specific range of the movement target based on the coordinate information of the user terminal 140.

[0185] In step S1711, the specific range input unit 501 moves the determined specific range and registers specific range information of the specific range after the movement.

[0186] In step S1801, the specific range input unit 501 determines whether or not a driving information input instruction has been acquired. If it is determined in step S1801 that a driving information input instruction has not been acquired (NO in step S1801), the process proceeds to step S1804.

[0187] On the other hand, if it is determined in step S1801 that a driving information input instruction has been acquired (YES in step S1801), the process proceeds to step S1802.

[0188] In step S1802, the specific range input unit 501 determines the specific range for which driving information is to be registered based on the coordinate information of the user terminal 140.

[0189] In step S1803, the specific range input unit 501 registers the driving information in association with the determined specific range.

[0190] In step S1804, the specific range input unit 501 determines whether or not a reversal instruction has been acquired. If it is determined in step S1804 that a reversal instruction has not been acquired (NO in step S1804), the process proceeds to step S1807.

[0191] On the other hand, if it is determined in step S1804 that a reversal instruction has been acquired (YES in step S1804), the process proceeds to step S1805.

[0192] In step S1805, the specific range input unit 501 determines the specific range for which the driving information is to be inverted based on the coordinate information of the user terminal 140.

[0193] In step S1806, the specific range input unit 501 inverts the driving information associated with the determined specific range and the driving information associated with the non-specific range, and registers the inverted driving information.

[0194] In step S1807, the specific range input unit 501 determines whether or not to end the specific operation mode. If it is determined in step S1807 that the specific operation mode is to be continued (NO in step S1807), the process returns to step S1701 in FIG.

[0195] On the other hand, if it is determined in step S1807 that the specific operation mode is to be ended (YES in step S1807), the specific range registration process in the specific operation mode is ended.

[0196] <Example of air conditioning system operation> Next, an example of the operation of the air conditioning system 100 will be described. Fig. 19 is a diagram showing an example of air conditioning control by the air conditioning system. The example in Fig. 19 shows a state in which operation information is registered so that the blown air hits specific range 621, and operation information is registered so that the blown air does not hit specific ranges 1901 and 1902.

[0197] In this way, according to the air conditioning system 100 of the first embodiment, the user 211 can freely change the size and shape of the specific range to be controlled, and multiple specific ranges can be registered for one user 211.

[0198] <Summary> As is clear from the above description, the air conditioning system 100 according to the first embodiment: The system has an air conditioning device that can control air conditioning for a specific area in the real space, and a user terminal. A specific range determined based on the location information of the user device in real space and the operation content on the user device is registered.

[0199] As a result, according to the first embodiment, it is possible to provide an air conditioning system and a registration program that registers a specific range to be controlled in real space.

[0200] [Second embodiment] In the above first embodiment, the specific range output unit was described as being realized in the control device of the air conditioning apparatus 110. However, the specific range output unit may also be realized in, for example, the user terminal 140. Below, the second embodiment will be described, focusing on the differences from the above first embodiment.

[0201] <Functional configuration of the control device and user terminal> First, the functional configuration of the control device of the air conditioning apparatus 110 and the functional configuration of the user terminal 140 in the air conditioning system according to the second embodiment will be described.

[0202] Fig. 20 is a diagram showing an example of the functional configuration of a control device and a user terminal in a specific operation mode. As shown in Fig. 20, in the second embodiment, the control device of the air conditioning device 110 executes a registration program to function as a specific range input unit 501, an air conditioning control unit 502, and a transmission unit 2001.

[0203] Of these, the specific range input unit 501 and the air conditioning control unit 502 have already been explained using FIG. 5 in the first embodiment, and therefore explanations thereof will be omitted here.

[0204] The transmission unit 2001 transmits to the user terminal 140 various types of information for notifying the users 211, 221, 222, etc. of the specific range registered by the air conditioning control unit 502.

[0205] On the other hand, in the second embodiment, the user terminal 140 functions as a specific range output unit 2002 by executing an air conditioning operation program.

[0206] The specific range output unit 2002 performs processing to notify the users 211, 221, 222, etc. of the specific range based on the various information transmitted from the transmission unit 2001.

[0207] <Functional configuration of the transmission unit and the specific range output unit, and specific example of the specific range output process> Next, the functional configurations of the transmission unit 2001 and the specific range output unit 2002 (here, the functional configurations related to the specific range output process) and a specific example of the specific range output process will be described in detail. Fig. 21 is a diagram showing the functional configurations of the transmission unit and the specific range output unit, and a specific example of the specific range output process.

[0208] As shown in FIG. 21, the transmission unit 2001 includes a captured image acquisition unit 2101 , a radio wave information acquisition unit 2102 , a location information acquisition unit 2103 , a location information transmission unit 2104 , a specific range information acquisition unit 2105 , and a specific range transmission unit 2106 .

[0209] The captured image acquisition unit 2101 acquires the captured image captured by the image capturing device 120 and notifies the location information acquisition unit 2103 of the image.

[0210] The radio wave information acquisition unit 2102 acquires radio wave information received by the radio wave transmitting / receiving device 310 and notifies the location information acquisition unit 2103 of the information.

[0211] The position information acquisition unit 2103 calculates coordinate information (x, y, z) indicating the position of the user terminal 140 in the real space 210 or 220 based on the notified captured image and radio wave information.

[0212] The position information transmitting unit 2104 transmits the coordinate information (x, y, z) calculated by the position information acquiring unit 2103 to the specific range output unit 2002 of the user terminal 140.

[0213] The specific range information acquisition unit 2105 reads out the specific range information registered in the air conditioning control unit 502 and notifies the specific range transmission unit 2106 of the information.

[0214] The specific range transmitting unit 2106 transmits the specific range information notified by the specific range information acquiring unit 2105 to the specific range output unit 2002 .

[0215] On the other hand, the specific range output unit 2002 includes a position information acquisition unit 2111 , a specific range information acquisition unit 2112 , a specific range image generation unit 2113 , a captured image acquisition unit 2114 , and a display control unit 2115 .

[0216] The position information acquisition unit 2111 acquires the coordinate information (x, y, z) transmitted from the position information transmission unit 2104 and notifies the specific range image generation unit 2113 of the coordinate information.

[0217] The specific range information acquisition unit 2112 acquires the specific range information transmitted from the specific range transmission unit 2106 and notifies the specific range image generation unit 2113 of the information.

[0218] The specific range image generation unit 2113 identifies where the user terminal 140 is currently located in real space, based on the coordinate information (x, y, z) notified by the position information acquisition unit 2111. Furthermore, the specific range image generation unit 2113 identifies in which direction the user terminal 140 is currently facing (the attitude of the user terminal 140), based on the output of the acceleration sensor 328. Furthermore, the specific range image generation unit 2113 calculates, from the identified position and direction (coordinate information and attitude information), coordinate information of each position within the imaging range when captured by the imaging device 327 of the user terminal 140. Furthermore, the specific range image generation unit 2113 applies the specific range information notified by the specific range information acquisition unit 2112 to the coordinate information of each position within the imaging range.

[0219] This allows the specific range information acquisition unit 2112 to generate a specific range image at a position within the captured image when the imaging device 327 of the user terminal 140 starts capturing images, with the orientation corresponding to the orientation of the user terminal 140. Note that the specific range image generation unit 2113 notifies the display control unit 2115 of the generated specific range image (three-dimensional image).

[0220] When the "Display specific range" button 414 is pressed on the operation screen 400 of the user terminal 140 to issue an instruction to display a specific range, and thus imaging by the imaging device 327 is started, the captured image acquisition unit 2114 acquires the captured image from the imaging device 327. In addition, the captured image acquisition unit 2114 notifies the display control unit 2115 of the acquired captured image.

[0221] The display control unit 2115 generates a display image by superimposing the specific range image notified by the specific range image generation unit 2113 on the captured image notified by the captured image acquisition unit 2114. The display control unit 2115 also displays the generated display image on the display screen of the display device 326 of the user terminal 140.

[0222] The example in Figure 21 shows how user 211 issues a command to display a specific range by pressing the "Display specific range" button 414 on the operation screen 400 of user terminal 140 at a position specified by coordinate information (x, y, z), thereby starting shooting.

[0223] 21 shows a case where specific range information is registered for a foliage plant 2120 placed in real space 210. In this case, a display image 2122 in which the specific range image is superimposed on a captured image 2121 of the foliage plant 2120 taken from the side is displayed on the display screen of display device 326 of user terminal 140, for example.

[0224] Also, on the display screen of the display device 326 of the user terminal 140, for example, a display image 2132 in which the specific range image is superimposed on a photographed image 2131 of the foliage plant 2120 photographed from directly above is displayed.

[0225] In this way, by visualizing the registered specific range within the real space 210 using the augmented reality function of the user terminal 140, the user 211 can understand, via the user terminal 140, which position within the real space 210 is registered as the specific range.

[0226] <Summary> As is clear from the above description, the air conditioning system 100 according to the second embodiment has the following functions in addition to those described in the first embodiment: -It has the function of superimposing a 3D image showing a specific range, which is generated based on the position information and orientation information of the user device in real space, onto the image taken by performing a shooting operation on the user device.

[0227] As a result, according to the second embodiment, it is possible to enjoy the same effects as in the first embodiment, and the user can grasp the registered specific range via the user terminal 140.

[0228] [Third embodiment] In the above second embodiment, a case was described in which the registered specific range information is notified to the user by displaying a display image on the display screen of the user terminal 140, in which a specific range image is superimposed on a captured image taken by the imaging device 327 of the user terminal 140.

[0229] However, the method of notifying the user of the specific range information is not limited to this. For example, the user may be notified of whether the user terminal 140 is located within the specific range or outside the specific range (within the non-specific range) in the real space by outputting a predetermined stimulus (sound, light, or vibration).

[0230] The third embodiment will be described below, focusing on the differences from the first and second embodiments.

[0231] <Functional configuration of the specific range output unit in the user terminal, and specific examples of specific range output processing> First, we will explain the details of the functional configuration of the specific range output unit 2002 in the user terminal 140 (here, the functional configuration related to the specific range output processing) and a specific example of the specific range output processing. Fig. 22 is a second diagram showing the functional configuration of the specific range output unit and a specific example of the specific range output processing.

[0232] As shown in FIG. 22, the specific range output unit 2002 includes a position information acquisition unit 2111 , a specific range information acquisition unit 2112 , a positional relationship determination unit 2201 , and an output device control unit 2202 .

[0233] The position information acquisition unit 2111 has the same function as the position information acquisition unit 2111 described in Figure 21, and acquires the coordinate information (x, y, z) transmitted from the position information transmission unit 2104 and notifies the positional relationship determination unit 2201.

[0234] The specific range information acquisition unit 2112 has the same function as the specific range information acquisition unit 2112 described in FIG. 21, and acquires the specific range information transmitted from the specific range transmission unit 2106 and notifies the positional relationship determination unit 2201.

[0235] The positional relationship determination unit 2201 identifies where the user terminal 140 is currently located in real space, based on the coordinate information (x, y, z) notified by the positional information acquisition unit 2111. Furthermore, the positional relationship determination unit 2201 determines the positional relationship between the specific range and the user terminal 140, based on the specific range information notified by the specific range information acquisition unit 2112. Furthermore, the positional relationship determination unit 2201 notifies the output device control unit 2202 of the determined positional relationship.

[0236] Based on the positional relationship notified by the positional relationship determination unit 2201, the output device control unit 2202 controls the output device (display device 326, audio output device 330, or vibration device 331) to notify the user by outputting light, sound, or vibration. This allows the user to understand the positional relationship between the specific range and the user terminal 140. Note that the output device control unit 2202 may control the display device 326 to output light, or may control an LED (not shown) to output light.

[0237] The example of Fig. 22 shows a state in which a user 211 is holding a user terminal 140 at a position specified by coordinate information (x, y, z). The example of Fig. 22 also shows a state in which a specific range 621 is registered.

[0238] Under such circumstances, the output device control unit 2202 controls the user terminal 140 so that, for example, when the user terminal 140 is located within the specific range 621, it outputs either light, sound, or vibration, and when the user terminal 140 is located outside the specific range 621, it does not output any of light, sound, or vibration.

[0239] Alternatively, the output device control unit 2202 may control the output so that, for example, the magnitude, volume, image content, color or amount of light of the vibration changes depending on whether the user terminal 140 is located within the specific range 621 or outside the specific range 621.

[0240] Alternatively, the output device control unit 2202 may control the user terminal 140 to switch stimuli, for example, by outputting light when the user terminal 140 is located within the specific range 621 and outputting sound or vibration when the user terminal 140 is located outside the specific range 621.

[0241] In either case, the output device control unit 2202 controls the output when a range corresponding to the coordinate information (x, y, z) changes from within the specific range 621 to outside the specific range 621 (inside a non-specific range) when a movement operation is performed on the user terminal 140. Alternatively, the output device control unit 2202 controls the output when a range corresponding to the coordinate information (x, y, z) changes from outside the specific range 621 (inside a non-specific range) to inside the specific range 621 when a movement operation is performed on the user terminal 140.

[0242] Furthermore, the output device control unit 2202 may perform control so as to increase the output of any one of sound, light, or vibration when the user terminal 140 approaches the center position of the specific range 621, for example. Furthermore, the output device control unit 2202 may perform control so as to decrease the output of any one of sound, light, or vibration when the user terminal 140 approaches the boundary surface of the specific range 621, for example.

[0243] Furthermore, the output device control unit 2202 may control the output in accordance with the corresponding driving information when, for example, the user terminal 140 outputs light, sound, or vibration because the user terminal 140 is located within the specific range 621. In this case, for example, different driving information is registered for each of a plurality of specific ranges, and when the user terminal 140 moves across the plurality of specific ranges, any of the magnitude, volume, image content, and color or amount of light of the vibration will change.

[0244] In addition, if different driving information is registered at each location within a specific range, the magnitude, volume, image content, color or amount of light of the vibration will change while the user terminal 140 moves within the specific range.

[0245] In this way, by controlling the output using the output function of the user terminal 140 so that the user can understand the registered specific range, The user 211 can determine which range in the real space is registered as a specific range, and What driving information is registered? can be grasped via the user terminal 140.

[0246] <Summary> As is clear from the above description, the air conditioning system 100 according to the third embodiment has the following functions in addition to those described in the first embodiment: -It has the function of controlling output according to the movement operation on the user terminal.

[0247] As a result, according to the third embodiment, it is possible to enjoy the same effects as in the first embodiment, and the user can grasp the registered specific range via the user terminal 140.

[0248] [Fourth embodiment] In the first embodiment, when calculating coordinate information indicating the position of the user terminal 140 in real space, Radio wave information (laser light reflection signal) acquired by a radio wave transmitting / receiving device 310 (an example of a three-dimensional position measuring sensor), A captured image (RGB image) captured by the imaging device 120; The case where the above formula is used has been described.

[0249] However, the method of calculating coordinate information indicating the position in real space of the user terminal 140 is not limited to this. For example, a three-dimensional position measurement sensor may be configured by a plurality of image capture devices 120 attached at different positions in the real space 210 or 220, A photographed image taken by the first imaging device 120, A photographed image taken by the second imaging device 120, ... The image captured by the α-th imaging device 120, By matching these, coordinate information indicating the position of the user terminal 140 in real space may be calculated.

[0250] Alternatively, SLAM (Simultaneous Localization and Mapping) technology may be used to calculate coordinate information indicating the position of the user terminal 140 in real space. Specifically, the three-dimensional position measurement sensor is configured by an image capture device 120 attached via an arm in the real space 210 or 220, and a sensor that emits high-intensity light that can be received by the image capture device 120 is attached to the user terminal 140, Three-dimensional spatial data acquired from images captured while swinging the imaging device 120; Relative position data between the user terminal 140 and the imaging device 120, obtained by the imaging device 120 receiving high-intensity light emitted by the user terminal 140; and By comparing these, coordinate information indicating the position of the user terminal 140 in real space may be calculated.

[0251] Furthermore, in the above first embodiment and this embodiment, it has been described that the coordinate information indicating the position of the user terminal 140 in real space is calculated by a device other than the user terminal 140 (a control device for an air conditioning device) using data acquired by a device other than the user terminal 140.

[0252] However, the coordinate information indicating the position of the user terminal 140 in real space may be calculated by the user terminal 140 using data acquired by the user terminal 140. Specifically, an API (Application Programming Interface) capable of generating a depth map based on an image captured by the imaging device 327 of the user terminal 140 may be provided in the user terminal 140, A depth map generated using an API from an image captured by an imaging device 327; Layout information (shape data such as three-dimensional data) within the real space 210 or 220, By comparing these, coordinate information indicating the position of the user terminal 140 in real space may be calculated.

[0253] Fig. 23 is a diagram showing an example of calculation of coordinate information indicating the position of the user terminal in real space. The calculation example in Fig. 23 shows how the user terminal 140 constantly captures images of the real space 210 or 220 while the user terminal 140 is being moved.

[0254] 23, a photographed image 2301 and a photographed image 2302 are photographed at different times. As shown in Fig. 23, by using the photographed image 2301 and the photographed image 2302 photographed from different positions, the user terminal 140 can calculate depth distances l1 and l2 from feature point X of an object placed in the real space 210 or 220 based on epipolar geometry.

[0255] Here, it is assumed that coordinate information (a, b, c) of the object 2303 has been acquired in advance as layout information of the object. In this case, the user terminal 140 can calculate the current coordinate information (x, y, z) of the user terminal 140 based on the coordinate information (a, b, c) of the object and the calculated depth distances l1 and l2.

[0256] In this way, the configuration, number, and arrangement of sensors for calculating coordinate information indicating the position of user terminal 140 in real space are arbitrary, and a combination suitable for the real space environment is selected.

[0257] [Fifth embodiment] In the first embodiment, a case where a predetermined area in real space (an area where no object exists) is registered as a specific range has been mainly described. Alternatively, in the first embodiment, when an area where an object exists is registered as a specific range, it has been described that coordinate information of the object area where the object exists is stored in advance.

[0258] In contrast to this, in the fifth embodiment, a case will be described in which an area in which an object or person, or a part of them, in real space exists is registered as a specific range. Also, in the fifth embodiment, a case will be described in which, when registering an area in which an object (or part of it) exists as a specific range, coordinate information of the object area of ​​the object is not stored in advance and is calculated each time.

[0259] <User terminal functional configuration> First, the functional configuration of the user terminal 140 according to the fifth embodiment (here, the functional configuration related to the reference position calculation process) will be described in detail. Fig. 24 is a diagram showing an example of the functional configuration of the user terminal.

[0260] In the fifth embodiment, the user terminal 140 functions as a captured image acquisition unit 2401, a depth map generation unit 2402, an object detection unit 2403, and a center position determination unit 2404 by executing an air conditioning operation program.

[0261] The captured image acquisition unit 2401 acquires a captured image captured by the imaging device 327 and notifies the depth map generation unit 2402 and the object detection unit 2403 of the image.

[0262] The object detection unit 2403 detects objects contained in the captured image acquired by the captured image acquisition unit 2401, extracts objects of the type specified by the user from the detected objects, and notifies the depth map generation unit 2402.

[0263] The depth map generation unit 2402 is an API that generates a depth map based on a captured image. The depth map generation unit 2402 generates a depth map in the real space 210 or 220 based on the captured image notified by the captured image acquisition unit 2401.

[0264] Furthermore, the depth map generating unit 1422 identifies an area in the generated depth map that corresponds to the object notified by the object detecting unit 2403, and notifies the center position determining unit 2404 of the area.

[0265] The center position determination unit 2404 calculates the center position information of the object notified by the object detection unit 2403 based on the depth map of the area specified by the depth map generation unit 2402 .

[0266] Furthermore, the central position determination unit 2404 transmits the calculated central position information to the control device of the air conditioner 110.

[0267] <Functional configuration of specific range input section> Next, a description will be given of the functional configuration of a specific range input unit 501 (here, the functional configuration related to determining a specific range) that is realized by executing a registration program in a control device of an air conditioning apparatus 110 according to a fifth embodiment. Fig. 25 is a diagram showing an example of the functional configuration of the specific range input unit. The differences from the specific range input unit shown in Fig. 6 are that Fig. 25 has a central position information acquisition unit 2501, and that the function of the specific range determination unit 2502 differs from the function of the specific range determination unit 607 in Fig. 6.

[0268] The central position information acquisition unit 2501 acquires central position information from the user terminal 140 and notifies the specific range determination unit 2502 of the central position information.

[0269] The specific range determination unit 2502 acquires shape information (for example, spherical shape information) of the predetermined shape of the specific range with reference to the shape storage unit 610. Furthermore, the specific range determination unit 607 determines the specific range information based on the center position information notified by the center position information acquisition unit 2501, the radius information (R) notified by the radius information identification unit 605, and the spherical shape information acquired from the shape storage unit 610.

[0270] <Specific example of center position determination process> Next, specific examples of the central position determination process by the user terminal 140 will be described for patterns (1) to (4).

[0271] (1) Pattern 1 26 is a first diagram showing a specific example of the center position determination process. As shown in Fig. 26, the imaging device 327 of the user terminal 140 captures an image of the real space 220, whereby the captured image acquisition unit 2401 displays the captured image 2610 on the display screen, and the object detection unit 2403 detects multiple objects included in the captured image 2610. Furthermore, the depth map generation unit 2402 generates a depth map based on the captured image 2610.

[0272] When the captured image 2610 is displayed, the user 211 performs an operation to specify the type of object to be detected. The example of Fig. 26 shows a state in which the user 211 specifies "thing" as the type of object to be detected.

[0273] When "thing" is specified as the type of object to be detected, the object detection unit 2403 identifies the object closest to the user terminal 140 from among the multiple detected objects as the object, and notifies the depth map generation unit 2402 of the identified object. Note that the example in Fig. 26 shows a state in which the "desk" closest to the user terminal 140 is identified as the object.

[0274] When the object is specified, the depth map generator 2402 identifies the region of the specified object from the generated depth map and notifies the center position determiner 2404 of the region.

[0275] The center position determination unit 2404 calculates center position information of the object based on the notified depth map of the object region. In FIG. 26, reference numeral 2620 denotes the center position information calculated by the center position determination unit 2404.

[0276] The central position information indicated by the reference numeral 2620 is input to the specific range input unit 501, and the peripheral range based on the central position information is determined as the specific range 2630.

[0277] (2) Pattern 2 27 is a second diagram showing a specific example of the center position determination process. As shown in Fig. 27, the imaging device 327 of the user terminal 140 captures an image of the real space 220, whereby the captured image acquisition unit 2401 displays the captured image 2710 on the display screen, and the object detection unit 2403 detects multiple objects included in the captured image 2710. Furthermore, the depth map generation unit 2402 generates a depth map based on the captured image 2710.

[0278] When the captured image 2710 is displayed, the user 211 performs an instruction operation regarding the type of object to be detected. In the example of FIG. 27, the user 211 selects the following as the type of object to be detected: This shows how a "person" is being instructed.

[0279] When "person" is specified as the type of object to be detected, the object detection unit 2403 identifies the person closest to the user terminal 140 as the object from among the multiple detected objects, and notifies the depth map generation unit 2402 of the identified object. Note that the example in Fig. 27 shows a state in which the user 222 closest to the user terminal 140 is identified as the object.

[0280] When the object is specified, the depth map generator 2402 identifies the region of the specified object from the generated depth map and notifies the center position determiner 2404 of the region.

[0281] The center position determination unit 2404 calculates center position information of the object based on the notified depth map of the object region. In FIG. 27, reference numeral 2720 denotes the center position information calculated by the center position determination unit 2404.

[0282] The central position information indicated by the reference numeral 2720 is input to the specific range input unit 501 , and the peripheral range based on the central position information is determined as the specific range 2730 .

[0283] (3) Pattern 3 28 is a third diagram showing a specific example of the center position determination process. As shown in Fig. 28, the imaging device 327 of the user terminal 140 captures an image of the real space 220, whereby the captured image acquisition unit 2401 displays the captured image 2810 on the display screen, and the object detection unit 2403 detects multiple objects included in the captured image 2810. Furthermore, the depth map generation unit 2402 generates a depth map based on the captured image 2810.

[0284] When the photographed image 2810 is displayed, the user 211 performs a selection instruction operation for a part of the object to be detected. The example of Fig. 28 shows a state in which the user 211 selects and instructs the "head" of a "person" as the part of the object to be detected.

[0285] When the "head" of a "person" is selected and instructed as the part of the object to be detected, the object detection unit 2403 identifies the head of the person closest to the user terminal 140 as the object among the multiple detected objects, and notifies the depth map generation unit 2402 of the identified object. The example of Fig. 28 shows how the head of the user 222 closest to the user terminal 140 is identified as the object.

[0286] When the object is specified, the depth map generator 2402 identifies the region of the specified object from the generated depth map and notifies the center position determiner 2404 of the region.

[0287] The center position determination unit 2404 calculates center position information of the object based on the notified depth map of the object region. In FIG. 28, reference numeral 2820 denotes the center position information calculated by the center position determination unit 2404.

[0288] The central position information indicated by the reference numeral 2820 is input to the specific range input unit 501, and the peripheral range based on the central position information is determined as the specific range 2830.

[0289] (4) Pattern 4 Fig. 29 is a fourth diagram showing a specific example of the center position determination process. As shown in Fig. 29, the imaging device 327 of the user terminal 140 captures an image of the real space 220, whereby the captured image acquisition unit 2401 displays the captured image 2910 on the display screen, and the object detection unit 2403 detects multiple objects included in the captured image 2910. Furthermore, the depth map generation unit 2402 generates a depth map based on the captured image 2910.

[0290] When the captured image 2910 is displayed, the user 211 performs a pointing operation on the object to be detected. Note that the display screen on which the captured image 2910 is displayed is pointable. The example in Fig. 29 shows how the user 211 points to the position of "notebook PC" on the display screen, thereby accepting a pointing operation on the object to be detected.

[0291] When a "notebook PC" is designated as the object to be detected, the object detection unit 2403 identifies the designated "notebook PC" as the object from among the multiple detected objects, and notifies the depth map generation unit 2402 of the identified object.

[0292] When the object is specified, the depth map generator 2402 identifies the region of the specified object from the generated depth map and notifies the center position determiner 2404 of the region.

[0293] 29, reference numeral 2920 denotes the center position information calculated by the center position determination unit 2404.

[0294] The central position information indicated by the reference numeral 2920 is input to the specific range input unit 501, and the peripheral range based on the central position information is determined as the specific range 2930.

[0295] <Summary> As is clear from the above description, the air conditioning system 100 according to the fifth embodiment: The user device can accept a pointing operation on the object, and the specific range is determined to be a peripheral range based on the central position information of the object in real space accepted by the user device. The user terminal is capable of receiving an instruction to select a part of the object, and the specific range is determined to be a peripheral range based on the central position information in real space of the part of the object received by the user terminal. The display screen on which the captured image captured by the imaging device of the user terminal is displayed is configured to be pointable, and the user terminal identifies the object at the pointed position among multiple objects included in the captured image as the target object.

[0296] As a result, according to the fifth embodiment, it becomes possible to register the area in which the object exists as a specific range, and it becomes possible to identify the object via the captured image displayed on the user terminal.

[0297] [Other embodiments] In each of the above embodiments, the temperature, wind direction, and wind volume are set as the operating information associated with the specific range, but the items that can be registered as the operating information associated with the specific range are not limited to these.

[0298] Furthermore, in the above embodiments, the specific range input unit 501 has been described as being realized in a control device within the air conditioning apparatus 110, but the specific range input unit 501 may be realized in a device other than the control device within the air conditioning apparatus 110. Alternatively, the specific range input unit 501 may be realized in a device outside the air conditioning apparatus 110 (any device within the air conditioning system 100).

[0299] Similarly, in the above embodiments, the specific range output unit 503 has been described as being realized in a control device within the air conditioning device 110 or in the user terminal 140, but the specific range output unit 503 may be realized in a device other than the control device within the air conditioning device 110. Alternatively, the specific range output unit 503 may be realized in a device other than the air conditioning device 110 and the user terminal 140 (any device within the air conditioning system 100).

[0300] Furthermore, in the above embodiments, the air conditioning system 100 has been described as including an air conditioning apparatus 110, an imaging device 120, a display device 130, and a user terminal 140. However, the configuration of the air conditioning system 100 is not limited to this, and may include, for example, a server device that controls multiple air conditioning apparatuses 110. Furthermore, in cases where a server device is included, the specific range input unit 501 and the specific range output unit 503 may be realized in the server device.

[0301] Furthermore, in each of the above embodiments, the size of the specific range is described as being determined by a predetermined setting operation, but it may also be configured so that the shape of the specific range is determined by a predetermined setting operation.

[0302] In the first embodiment, n tapping operations and a continuous pressing operation for a predetermined time are exemplified as setting operations for determining the size of the specific range, but the setting operations for determining the size of the specific range are not limited to these. For example, the size of the specific range may be determined based on the coordinate information of the user terminal 140 when the setting operation of the start point is performed on the user terminal 140 and the coordinate information of the user terminal 140 when the setting operation of the end point is performed on the user terminal 140.

[0303] Alternatively, in cases where a peripheral range based on central position information is determined as a specific range, the size of the specific range may be determined based on the coordinate information of the user terminal when the end point setting operation is performed and the central position information.

[0304] Furthermore, in the second embodiment, a case has been described in which a specific range image (three-dimensional image) is superimposed on a captured image captured by the imaging device 327 of the user terminal 140, but the target on which the specific range image is superimposed is not limited to a captured image. For example, the specific range image may be superimposed on an image (still image) of the real space 210 or 220 to which layout information within the real space 210 or 220 is linked. This is because the specific range image can be appropriately superimposed if coordinate information of the real space 210 or 220 is linked to each position on the still image.

[0305] In addition, in the above third embodiment, variations in the method of notifying the user of a predetermined stimulus (sound, light, or vibration) by the output device control unit 2202 have been described, but the variations in the method of notifying the user are not limited to this.

[0306] For example, as in the above-described fifth embodiment, when a pointing operation regarding an object can be accepted, a different stimulus may be notified depending on whether the specific range is determined based on information about the center position of the object in real space. Specifically, a different stimulus may be given when the user terminal 140 is located within the specific range depending on whether the specific range is determined to be an area where no object exists or an area where an object exists.

[0307] Furthermore, as in the fifth embodiment, when a pointing operation for a "person" as the object can be accepted, a different stimulus may be notified depending on whether the object is a person or not. Specifically, when the area in which the object exists is determined as the specific range, a different stimulus may be given when the user terminal 140 is located within the specific range depending on whether the object is a person or an object other than a person.

[0308] Furthermore, in the above embodiments, the smartphone 141, the remote control 142, a wearable device, and the like have been exemplified as the user terminal 140, but the user terminal 140 does not have to be an electronic device. For example, as in patterns 3 to 5 described in the first embodiment, in a case where the specific range is determined by a movement operation on the user terminal 140, the user terminal 140 may be a simple housing without a built-in electronic device. In this case, the shape of the housing is arbitrary, and may be, for example, a rod-shaped member.

[0309] Furthermore, although the above embodiments have been described as being realized in an air conditioning system, the target system is not limited to an air conditioning system, and may be any other system as long as it is a registration system that determines a specific range in a real space and registers the determined specific range. In other words, the air conditioning system described in the above embodiments is one application example of a registration system.

[0310] Although the embodiments have been described above, it will be understood that various changes in form and details can be made without departing from the spirit and scope of the claims. [Explanation of symbols]

[0311] 100: Air conditioning system 110: Air conditioning equipment 120: Imaging device 130:Display device 140: User terminal 210, 220: Real space 326:Display device 327: Imaging device 330: Audio output device 331: Vibration device 400: Operation screen 501: Specific range input section 503: Specific range output section 601: Image capture unit 602: Radio wave information acquisition section 603: Location information acquisition unit 604: Operation content acquisition section 605: Radius information specification section 607: Specific range determination unit 701: Operation content acquisition section 801: Operation content acquisition section 802: Center position and radius information calculation unit 803: Specific range determination unit 902: Specific range determination unit 1001: Projection direction determination unit 1002: Specific range determination unit 1101: Operation content acquisition section 1102: Target area calculation unit 1103: Specific range determination unit 1111: Operation content acquisition section 1112: Reference position calculation section 1113: Target area calculation unit 1114: Specific range determination unit 1201: Operation content acquisition section 1202: Deletion target determination section 1203: Specific range deletion part 1301: Operation content acquisition section 1302: Movement target determination unit 1303: Specific range update part 1401: Operation content acquisition section 1402: Operation information acquisition unit 1403: Operation information setting section 1501: Operation content acquisition section 1502: Reverse instruction acquisition unit 1503: Operation information update unit 1601: Specific range information acquisition unit 1602: Image capture unit 1603: Specific range image generation unit 1611: Photographed image 1612: Specific range image 1901, 1902: Specific range 2001: Transmitting unit 2002: Specific range output section 2101: Image acquisition unit 2102: Radio wave information acquisition section 2103: Location information acquisition unit 2104: Location information transmission unit 2105: Specific range information acquisition unit 2106: Specific range transmission unit 2111: Location information acquisition unit 2112: Specific range information acquisition unit 2113: Specific range image generation unit 2114: Image acquisition unit 2115: Display control unit 2121, 2131: Images 2122, 2132: Display image 2201: Positional relationship determination unit 2202: Output device control unit 2401: Image acquisition unit 2402: Depth map generator 2403: Object detection unit 2404: Center position determination section 2501: Center position information acquisition unit 2502: Specific range determination unit

Claims

1. A registration system having a control unit and a storage unit, The control unit a specific range in the real space determined based on position information of the user terminal in the real space and operation content on the user terminal (excluding an operation of specifying a virtual boundary surface that is set in advance to identify a boundary surface of a range in the real space on an image captured of the real space), the specific range being surrounded by a three-dimensional shape, and is registered in the storage unit; Registration system.

2. The control unit The registration system according to claim 1 , wherein a specific range within the real space is determined based on a trajectory of position information of the user terminal when a movement operation is performed on the user terminal, and is registered in the storage unit.

3. the user terminal is capable of accepting an instruction operation; The control unit The registration system according to claim 1 , wherein a surrounding area based on position information of the user terminal at the time the user terminal accepts the instruction operation is determined as the specific area and registered in the storage unit.

4. the user terminal is capable of accepting an instruction operation; The control unit The registration system according to claim 1, wherein a surrounding area based on the position information in the real space of an object identified by the user terminal receiving an instruction operation is determined as a specific area within the real space and registered in the memory unit.

5. the user terminal is capable of accepting a selection instruction operation; The control unit The registration system according to claim 4, wherein a surrounding area based on position information in the real space of the part of the object identified by the user terminal receiving a selection instruction operation is determined as a specific area within the real space and registered in the memory unit.

6. the user terminal is capable of accepting a setting operation; The control unit The registration system according to claim 1 , wherein the size or shape of the specific area is determined based on a setting operation received by the user terminal.

7. the user terminal is capable of accepting a setting operation; The control unit The registration system of claim 6, wherein the size of the specific range is determined based on the location information of the user terminal when the user terminal accepts the setting operation of the starting point, or the location information used as a reference when determining the specific range, and the location information of the user terminal when the user terminal accepts the setting operation of the end point.

8. The user terminal An imaging device; a display device that displays an image captured by the imaging device, The control unit of the user terminal The registration system according to claim 4 , wherein an object included in the image is identified as the target object.

9. The user terminal An imaging device; a display device that displays an image captured by the imaging device on a screen and allows a user to point to a position on the screen where the image is displayed; The control unit of the user terminal The registration system according to claim 4 , wherein an object at a pointed position among a plurality of objects included in the image is identified as the target object.

10. The user terminal a display device for displaying an image; The control unit of the user terminal The registration system according to claim 1 , wherein an image showing the specific range is superimposed on the image and displayed on the display device.

11. The user terminal An imaging device is included, The control unit of the user terminal The registration system according to claim 10, wherein a three-dimensional image showing the specific range, generated based on position information and orientation information of the user terminal in real space, is superimposed on the image captured by the imaging device and displayed.

12. the user terminal has an output device; The registration system according to claim 1 , wherein the output device notifies a user of a stimulus when the user terminal is located within a specific range in the real space.

13. the user terminal has an output device; The registration system according to claim 1 , wherein the output device notifies different stimuli depending on a positional relationship between the specific range in the real space and the user terminal.

14. the user terminal has an output device; When the user terminal is located in a specific range in the real space, the output device: Notifying a different stimulus depending on whether the specific range is determined by position information of the object in the real space, or When the specific range is determined based on position information of the object in the real space, notifying the user of a different stimulus depending on whether the object is a person or not. The registration system of claim 4.

15. the user terminal has a sensor; The control unit of the user terminal The registration system according to claim 1 , wherein position information of the user terminal in the real space is calculated by comparing the shape data of the real space with data measured by the sensor.

16. the sensor is an imaging device; The control unit of the user terminal The registration system according to claim 15, wherein the position information of the user terminal in the real space is calculated by comparing the three-dimensional data in the real space with a map generated from an image captured by the imaging device.

17. The control unit The registration system according to claim 1 , wherein position information of the user terminal in the real space is calculated based on data measured by a three-dimensional position measurement sensor attached in the real space.

18. the three-dimensional position measurement sensor is configured with a plurality of imaging devices attached at different positions; The control unit The registration system according to claim 17 , wherein the position information of the user terminal in real space is calculated by matching images taken by the plurality of image capture devices.

19. The control unit 2. The registration system of claim 1, wherein the position information of the user terminal in the real space is calculated by comparing relative position data between the user terminal and a three-dimensional position measurement sensor attached in the real space with three-dimensional spatial data measured by the three-dimensional position measurement sensor attached in the real space.

20. the user terminal is capable of receiving a start instruction operation for operating a device that improves comfort in a real space; The registration system of claim 1, wherein when the user terminal receives a start instruction operation, the control unit instructs a device that improves the comfort of the real space to perform control that improves the comfort of the real space for a specific range within the real space registered in the memory unit.

21. the user terminal is capable of accepting an air conditioning instruction operation, The registration system of claim 1, wherein when the user terminal receives an air conditioning instruction operation, the control unit instructs an air conditioning device installed in the real space to perform air conditioning control for a specific range within the real space registered in the memory unit.

22. The control unit The registration system of claim 21, wherein the system instructs a plurality of air conditioning devices installed in the real space to perform different air conditioning control in a specific range within the real space registered in the memory unit and in a non-specific range other than the specific range.

23. 23. An air conditioning system including a registration system according to any one of claims 1 to 22.

24. a registration step of registering in a storage unit a specific range in the real space that is determined based on position information of the user terminal in the real space and operation content on the user terminal (excluding an operation of specifying a virtual boundary surface that is set in advance to identify a boundary surface of a range in the real space on an image captured of the real space), the specific range being surrounded by a three-dimensional shape; A registered program for causing the control unit to execute the above.

Images