Electronic devices and programs

Abstract

To facilitate appropriate selection of a cellular communication network to be connected in a situation where a public cellular communication network and a local cellular communication network are mixed.SOLUTION: An electronic apparatus includes a display unit and at least one processor. The at least one processor acquires, via a communication interface, an identifier assigned to each of a plurality of slices from a communication network that is logically divided into the plurality of slices according to service types. The at least one processor causes the display unit to display an image showing predetermined information related to the communication network, on the basis of the acquired identifier.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present disclosure relates to an electronic device and Program Mu Regarding. [Background technology]

[0002] In recent years, cellular communication systems conforming to the fifth-generation (5G) cellular communication standard have been attracting attention. It is expected that such cellular communication systems will be built not only as public cellular communication networks operated by telecommunications carriers, but also as local cellular communication networks operated by private companies, organizations, or individuals. Note that local cellular communication networks are sometimes called private cellular communication networks or non-public cellular communication networks.

[0003] Local cellular networks can be flexibly constructed and used by various entities according to the needs of the region or the individual needs of the industry. Local cellular networks are also less congested than public cellular networks and have a better propagation environment, making them more likely to provide good services to electronic devices.

[0004] In a situation where public and local cellular networks coexist, electronic devices have the option of selecting one of these two types of cellular networks to connect to. A common approach is for electronic devices to preferentially select the home network, which is the cellular network with which the device has a contract, or to select the cellular network that provides the highest reception power at the device. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] 3GPP(R) Technical Report “3GPP TR 23.734 V16.2.0:Study on enhancement of 5G System(5GS) for vertical and Local Area Network(LAN) services (Release 16)”, June 2019 Summary of the Invention

[0006] The present disclosure One aspect The electronic device includes a display unit and A camera that captures the space, and at least one processor, which acquires, via a communication interface, an identifier assigned to each of the plurality of slices from a communication network that is logically divided into a plurality of slices according to service types. Based on the acquired identifier, the at least one processor performs the following: An image of the space taken by the camera and a indicating predetermined information about the communication network No. 1 The image is displayed on the display unit. The at least one processor causes the display unit to display a second image, which is different from the first image and is positioned in the space. The at least one processor causes the display unit to display the first image in a fixed state at a predetermined position so that the user and the first image always face each other. The at least one processor causes the second image to be displayed in a position corresponding to the angle at which the user views it. Another aspect of the present disclosure provides an electronic device comprising a display unit having a light-transmitting display member and at least one processor. The at least one processor acquires, via a communication interface, an identifier assigned to each of a plurality of slices from a communication network logically divided into the plurality of slices according to service type. Based on the acquired identifier, the at least one processor displays a first image showing predetermined information about the communication network on the display surface of the display member so that the first image is visible in a space visible through the display member. The at least one processor displays an object image having a first surface and a second surface located in the space on the display surface of the display member. The at least one processor displays the first image on the first surface. The at least one processor displays a third image different from the first image on the second surface. Another aspect of the present disclosure provides an electronic device comprising a display unit, a camera for capturing an image of a space, and at least one processor. The at least one processor acquires, via a communication interface, an identifier assigned to each of a plurality of slices from a communication network that is logically divided into slices according to service types. Based on the acquired identifier, the at least one processor causes the display unit to display an image of the space captured by the camera and a first image showing predetermined information about the communication network superimposed on the image of the space. The at least one processor causes the display unit to display an object image having a first surface and a second surface located in the space. The at least one processor displays the first image on the first surface. The at least one processor displays a third image different from the first image on the second surface. Another aspect of the present disclosure provides an electronic device comprising a display unit having a light-transmitting display member and at least one processor. The at least one processor acquires, via a communication interface, an identifier assigned to each of a plurality of slices from a communication network logically divided into slices according to service types. Based on the acquired identifier, the at least one processor displays a first image on the display surface of the display member so that the first image showing predetermined information about the communication network is visible in a space visible through the display member. The at least one processor displays an object image having a first surface and a second surface located in the space on the display surface of the display member. The at least one processor displays the first image on the first surface. The at least one processor displays a fifth image showing the service type on the second surface. Another aspect of the present disclosure provides an electronic device comprising a display unit, a camera for capturing an image of a space, and at least one processor. The at least one processor acquires, via a communication interface, an identifier assigned to each of a plurality of slices from a communication network logically divided into slices according to service types. Based on the acquired identifier, the at least one processor causes the display unit to display an image of the space captured by the camera and a first image showing predetermined information about the communication network superimposed on the image of the space. The at least one processor causes the display unit to display an object image having a first surface and a second surface located in the space. The at least one processor displays the first image on the first surface. The at least one processor displays a fifth image showing the service type on the second surface.

[0007] In addition, the present disclosure One aspect The program is displayed and a camera that captures the space. The program causes a computer provided in an electronic device to execute a process of acquiring, via a communication interface, an identifier assigned to each of a plurality of slices from a communication network that is logically divided into a plurality of slices according to service types. An image of the space taken by the camera and a indicating predetermined information about the communication network No. 1 A process for displaying an image on the display unit is executed. The program causes the computer to execute a process of displaying a second image, which is different from the first image and is positioned in the space, on the display unit. The program causes the computer to execute a process of displaying the first image in a state where it is fixed at a predetermined position on the display unit so that the user and the first image always face each other. The program causes the computer to execute a process of displaying the second image in a position corresponding to the angle at which the user views it. Another aspect of the present disclosure provides a program that causes a computer provided in an electronic device having a display unit with a light-transmitting display member to execute a process of acquiring, via a communication interface, an identifier assigned to each of a plurality of slices from a communication network that is logically divided into a plurality of slices according to service type. The program causes the computer to execute a process of displaying, based on the acquired identifier, a first image showing predetermined information about the communication network on a display surface of the display member so that the first image is visible in a space visible through the display member. The program causes the computer to execute a process of displaying an object image having a first surface and a second surface located in the space on the display surface of the display member. The program causes the computer to execute a process of displaying the first image on the first surface. The program causes the computer to execute a process of displaying a third image different from the first image on the second surface. Another aspect of the present disclosure provides a program that causes a computer installed in an electronic device having a display unit and a camera for capturing images of a space to execute a process of acquiring, via a communication interface, an identifier assigned to each of a plurality of slices from a communication network that is logically divided into a plurality of slices according to service types. The program causes the computer to execute a process of displaying, on the display unit, an image of the space captured by the camera and a first image showing predetermined information about the communication network superimposed on the image of the space, based on the acquired identifier. The program also causes the computer to execute a process of displaying, on the display unit, an object image having a first surface and a second surface located in the space. The program also causes the computer to execute a process of displaying the first image on the first surface. The program also causes the computer to execute a process of displaying a third image different from the first image on the second surface. Another aspect of the present disclosure provides a program that causes a computer provided in an electronic device having a display unit with a light-transmitting display member to execute a process of acquiring, via a communication interface, an identifier assigned to each of a plurality of slices from a communication network that is logically divided into a plurality of slices according to service type. The program causes the computer to execute a process of displaying, based on the acquired identifier, a first image on the display surface of the display member, the first image indicating predetermined information about the communication network, so that the first image is visible in a space visible through the display member. The program also causes the computer to execute a process of displaying an object image having a first surface and a second surface located in the space on the display surface of the display member. The program also causes the computer to execute a process of displaying the first image on the first surface. The program also causes the computer to execute a process of displaying a fifth image indicating the service type on the second surface. Another aspect of the present disclosure provides a program that causes a computer installed in an electronic device having a display unit and a camera for capturing images of a space to execute a process of acquiring, via a communication interface, an identifier assigned to each of a plurality of slices from a communication network that is logically divided into a plurality of slices according to service type. The program causes the computer to execute a process of displaying, on the display unit, an image of the space captured by the camera and a first image showing predetermined information about the communication network superimposed on the image of the space, based on the acquired identifier. The program also causes the computer to execute a process of displaying, on the display unit, an object image having a first surface and a second surface located in the space. The program also causes the computer to execute a process of displaying the first image on the first surface. The program also causes the computer to execute a process of displaying a fifth image showing the service type on the second surface. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram illustrating a configuration of a communication system according to an embodiment. [Figure 2] FIG. 1 is a diagram illustrating a configuration of an electronic device according to an embodiment. [Figure 3] 10 is a flowchart showing a control procedure for a network-related image display process. [Figure 4] FIG. 10 is a diagram illustrating a specifying operation of a type of provided service according to an embodiment. [Figure 5] FIG. 10 is a diagram showing an example of a network-related image displayed on a display. [Figure 6] FIG. 10 is a diagram showing an example of a network-related image displayed on a display. [Figure 7] FIG. 10 is a diagram showing an example of a network-related image displayed on a display. [Figure 8] FIG. 10 is a diagram showing an example of a network-related image displayed on a display. [Figure 9] FIG. 10 is a diagram showing an example of a network-related image displayed on a display. [Figure 10] FIG. 10 is a diagram showing an example of a network-related image displayed on a display. [Figure 11] FIG. 10 is a diagram showing an example of a network-related image displayed on a display. [Figure 12] FIG. 10 is a diagram showing an example of a network-related image displayed on a display. [Figure 13] FIG. 10 is a diagram showing the movement of a user from Area 1 to Area 2. [Figure 14] FIG. 10 is a diagram showing an example of a network-related image displayed on a display. [Figure 15] FIG. 10 is a diagram showing an example of a network-related image displayed on a display. [Figure 16] FIG. 10 is a diagram showing an example of a network-related image displayed on a display. [Figure 17] FIG. 10 is a diagram showing an example of a network-related image displayed on a display. [Figure 18]FIG. 10 is a diagram showing an example of a network-related image displayed on a display. [Figure 19] FIG. 10 is a diagram showing an example of a network-related image displayed on a display. [Figure 20] FIG. 10 is a diagram showing an example of a network-related image displayed on a display. [Figure 21] FIG. 10 is a diagram showing an example of a network-related image displayed on a display. [Figure 22] FIG. 10 is a diagram showing an example of a network-related image displayed on a display. [Figure 23] FIG. 10 is a diagram showing an example of a network-related image displayed on a display. [Figure 24] FIG. 10 is a diagram showing an example of a network-related image displayed on a display. [Figure 25] FIG. 10 is a diagram showing an example of a guide image displayed on a display. [Figure 26] FIG. 10 is a diagram showing an example of a guide image displayed on a display. [Figure 27] FIG. 2 is a diagram showing an example of a map image displayed on a display. [Figure 28] FIG. 10 is a schematic perspective view showing the configuration of an electronic device according to a first modified example. [Figure 29] 10A and 10B are diagrams showing examples of a visual recognition area and a virtual image visually recognized by a user wearing an electronic device of Modification 1. [Figure 30] FIG. 1 is a diagram illustrating a visual recognition area in space. [Figure 31] FIG. 10 is a diagram showing the configuration of an electronic device according to a first modified example. [Figure 32] 10 is a flowchart showing a control procedure for a network-related image display process. [Figure 33] FIG. 10 is a diagram showing an example of a network-related image displayed on a display. [Figure 34] FIG. 10 is a diagram showing an example of a network-related image displayed on a display. [Figure 35] FIG. 10 is a schematic diagram showing the configuration of a display system according to Modification 2. [Figure 36] FIG. 2 is a block diagram showing a main functional configuration of the information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0010] When a local cellular communication network is constructed for the purpose of providing a specific service, the types of services provided by the local cellular communication network may be limited. On the other hand, a public cellular communication network may provide general-purpose services but may not provide specialized services tailored to the individual needs of a region or industry.

[0011] Therefore, in a situation where public cellular communication networks and local cellular communication networks coexist, if the general connection destination selection method described above is applied, a situation may arise in which the service that the user of the electronic device wants to use is not provided in the cellular communication network to which the electronic device is connected.

[0012] Therefore, an object of the present disclosure is to facilitate appropriate selection of a cellular communication network to connect to in a situation where public cellular communication networks and local cellular communication networks coexist.

[0013] Hereinafter, embodiments will be described with reference to the drawings. In the description of the drawings, the same or similar parts are denoted by the same or similar reference numerals.

[0014] [Configuration of communication system] First, the configuration of a communication system according to an embodiment will be described. Fig. 1 is a diagram showing the configuration of a communication system according to an embodiment. The communication system according to an embodiment is a cellular communication system that complies with the 3GPP standard for fifth generation (5G).

[0015] As shown in FIG. 1, the communication system according to one embodiment includes an electronic device 100, a public cellular communication network 210, a local cellular communication network 220, and a server 300.

[0016] The electronic device 100 is a device having a function of performing cellular communication with a cellular communication network. The electronic device 100 may be any device that has a function of performing cellular communication and a UI (User Interface). For example, the electronic device 100 is a smartphone, a tablet terminal, a notebook PC (Personal Computer), a wearable terminal device, or a mobile router. In the 3GPP standard, the electronic device 100 is called UE (User Equipment).

[0017] The public cellular communication network 210 is a cellular communication network operated by a telecommunications carrier. The public cellular communication network 210 is sometimes called public 5G. A license is issued nationwide to the telecommunications carrier that operates the public cellular communication network 210.

[0018] The local cellular communication network 220 is a cellular communication network that can be flexibly constructed and used by various entities according to local needs and individual needs of industrial fields. The local cellular communication network 220 is sometimes called local 5G. For example, a general company, organization, or individual can receive frequency allocation and operate the local cellular communication network 220 themselves. A license for the local cellular communication network 220 is issued only for a local area such as within the facilities of a general company.

[0019] The public cellular communication network 210 and the local cellular communication network 220 comply with the 5G 3GPP standard. The public cellular communication network 210 includes a base station 211, and the public cellular communication network 210 includes a base station 221. In FIG. 1, each cellular communication network includes only one base station, but in reality, each cellular communication network may include multiple base stations. In the 5G 3GPP standard, the base station is called a gNB. The base station constitutes an NG-RAN (Next Generation Radio Access Network), which is a 5G radio access network.

[0020] Each of the public cellular communication network 210 and the local cellular communication network 220 further includes a 5G core network (5GC). 5G assumes that a variety of electronic devices will be connected to the cellular communication network, and must support a variety of services with different requirements, such as high speed, large capacity, high reliability, and low latency. For this reason, the 5GC is theoretically divided into multiple slices according to different services (service requirements).

[0021] Here, each slice is assigned an identifier called S-NSSAI (Single-Network Slice Selection Assistance Information). Each slice is associated with one service type (SST). The service types specified in the standard are eMBB (high speed, large capacity), mIoT (multiple connections, low power consumption, low cost), and URLLC (low latency, high reliability), but service types not specified in the standard can also be used.

[0022] When the local cellular communication network 220 is constructed for the purpose of providing a specific service, the types of services (SST) provided by the local cellular communication network 220 may be limited. On the other hand, the public cellular communication network 210 may provide general-purpose services but may not provide specialized services that meet the individual needs of a region or an industry. Note that the "services provided by the cellular communication network" may also be considered as "functions supported by the cellular communication network."

[0023] The server 300 communicates with the electronic device 100 via the public cellular communication network 210 or the local cellular communication network 220. The server 300 may be a server for managing the local cellular communication network 220.

[0024] At least one electronic device 500 may be connected to the electronic device 100 by wire or wirelessly. The electronic device 500 establishes, for example, a WLAN (Wireless Local Area Network) connection with the electronic device 100, and communicates with the cellular communication network to which the electronic device 100 is connected via the electronic device 100. This allows the electronic device 500 to communicate with the cellular communication network via the electronic device 100 even if the electronic device 500 does not have a cellular communication function.

[0025] The extension device 400 may be connected to the electronic device 100 by wire or wirelessly. The extension device 400 establishes, for example, a USB (Universal Serial Bus) connection with the electronic device 100. The extension device 400 may have a function of supplying power to the electronic device 100. The extension device 400 may be a cradle.

[0026] At least one electronic device 600 may be connected to the extension device 400 by wire or wirelessly. The electronic device 600 establishes, for example, a WLAN connection with the extension device 400, and communicates with the cellular communication network to which the electronic device 100 is connected via the extension device 400 and the electronic device 100. This allows the electronic device 600 to communicate with the cellular communication network via the extension device 400 and the electronic device 100 even if the electronic device 600 does not have a cellular communication function.

[0027] [Configuration of electronic device] Next, a description will be given of the configuration of an electronic device 100 according to an embodiment. As shown in Fig. 1, the electronic device 100 includes a touch panel display 110, at least one physical button 113, a microphone 121a, and a speaker 122a. However, the configurations indicated by dashed lines in Figs. 1 and 2 are not essential.

[0028] Touch panel display 110 is provided with its display surface exposed from housing 101 of electronic device 100. Touch panel display 110 has a touch panel 111 and a display 112.

[0029] The touch panel 111 accepts operation input (touch input) to the electronic device 100. Methods for detecting a touch include, for example, a resistive film method or a capacitive method, but any method may be used. The display 112 outputs video.

[0030] Display 112 displays objects such as characters (including symbols), images, and figures on a screen. Display 112 is, for example, a liquid crystal display or an organic EL (Electro Luminescence) display. In touch panel display 110, display 112 is provided so as to overlap touch panel 111, and the display area of ​​display 112 overlaps with touch panel 111.

[0031] The physical button 113 accepts an operation input (press) to the electronic device 100. The physical button 113 is, for example, a home button, a power button, a volume adjustment button, or the like.

[0032] The microphone 121a receives audio input to the electronic device 100. The microphone 121a also collects surrounding sounds.

[0033] The speaker 122a outputs audio, such as telephone calls and information about various programs.

[0034] FIG. 2 is a diagram showing the configuration of the electronic device 100 according to an embodiment.

[0035] As shown in FIG. 2, the electronic device 100 has a touch panel 111, a display 112, physical buttons 113, an audio input unit 121, an audio output unit 122, a sensor 130, storage 140, a communication interface 150, a SIM (Subscriber Identity Module) interface 160, a connection interface 170, a battery 180, and a controller 190.

[0036] Touch panel 111 inputs a signal corresponding to a touch operation to controller 190. Display 112 displays objects such as characters, images, and figures on the screen based on the signal input from controller 190.

[0037] The audio input unit 121 inputs a signal corresponding to the received audio to the controller 190. The audio input unit 121 may be the microphone 121a shown in Fig. 1 or an input interface to which an external microphone can be connected. The external microphone may be provided in the extension device 400.

[0038] The audio output unit 122 outputs audio based on a signal input from the controller 190. The audio output unit 122 may be the speaker 122a shown in Fig. 1 or an output interface to which an external speaker can be connected. The external speaker may be provided in the extension device 400.

[0039] The sensor 130 detects various physical quantities and data and outputs the detection results to the controller 190. For example, the sensor 130 includes a position sensor, an acceleration sensor, and a temperature sensor. The position sensor detects the position of the device itself and outputs the position data to the controller 190. The position sensor may include a Global Navigation Satellite System (GNSS) receiver. The GNSS receiver performs positioning based on GNSS satellite signals and outputs GNSS position data indicating the geographical position (latitude and longitude) of the device itself to the controller 190. The acceleration sensor detects acceleration applied to the device itself and outputs the acceleration data to the controller 190. The acceleration sensor may be a multi-axis acceleration sensor including multiple acceleration sensors. The temperature sensor detects temperature and outputs the temperature data to the controller 190. At least one of these sensors may be provided in an external device (e.g., the extension device 400).

[0040] The storage 140 includes at least one memory that stores programs and data. The storage 140 is also used as a working area that temporarily stores processing results of the controller 190. The storage 140 may include any non-transitory storage medium, such as a semiconductor storage medium or a magnetic storage medium. The storage 140 may include multiple types of storage media. The storage 140 may include a combination of a portable storage medium, such as a memory card, an optical disk, or a magneto-optical disk, and a storage medium reader. The storage 140 may include a storage device used as a temporary storage area, such as a random access memory (RAM).

[0041] The communication interface 150 communicates wirelessly. The communication interface 150 includes a cellular interface 151 and a WLAN interface 152. The cellular interface 151 is configured in accordance with, for example, the 5G cellular communication standard. The WLAN interface 152 is configured in accordance with, for example, the IEEE 802.11 standard.

[0042] A SIM (or UIM (User Identity Module)) is attached to the SIM interface 160. The SIM interface 160 may be configured to be able to load and remove a SIM. When the SIM interface 160 receives a command to read or write information from the controller 190, it reads information recorded in the SIM and writes the information to the SIM. The SIM may be an embedded eSIM (Embedded SIM). The SIM may be provided by a telecommunications carrier or obtained by other means.

[0043] In one embodiment, a SIM for the public cellular communication network 210 and a SIM for the local cellular communication network 220 may be separately prepared. When using a SIM for the public cellular communication network 210, the controller 190 uses the public cellular communication network 210 as a home network and the local cellular communication network 220 as a roaming network. On the other hand, when using a SIM for the local cellular communication network 220, the controller 190 uses the local cellular communication network 220 as a home network and the public cellular communication network 210 as a roaming network. Alternatively, the SIM interface 160 may be capable of accepting both a SIM for the public cellular communication network 210 and a SIM for the local cellular communication network 220.

[0044] The connection interface 170 is an interface that is electrically connected to the extension device 400. The connection interface 170 may be any interface that can be electrically connected to the extension device 400, and is, for example, a USB interface.

[0045] Battery 180 stores power for driving the device itself. Battery 180 may be any type of secondary battery, and is, for example, a lithium ion battery.

[0046] The controller 190 is an arithmetic processing device. Examples of arithmetic processing devices include, but are not limited to, a central processing unit (CPU), a system-on-chip (SoC), a micro control unit (MCU), a field-programmable gate array (FPGA), and a coprocessor. The controller 190 also includes a graphics processing unit (GPU), a video RAM (VRAM), etc., and draws various images on the display 112. The controller 190 may be configured with multiple arithmetic processing devices, and various controls may be performed by the cooperation of the multiple arithmetic processing devices.

[0047] The controller 190 comprehensively controls the operation of the electronic device 100 to realize various functions. The controller 190 executes various controls based on operation input detected by the touch panel 111 and / or the physical button 113. The controller 190 may output, in response to the input operation signal, via the audio output unit 122, the display 112, etc.

[0048] [Electronic device operation] Next, a description will be given of the operation of the electronic device 100 according to an embodiment of the present disclosure. Fig. 3 is a flowchart showing a control procedure by the controller 190 for a network-related image display process according to an aspect of the present disclosure.

[0049] When the network-related image display process shown in FIG. 3 starts, the controller 190 controls the cellular interface 151 to perform a network search (step S101). A network search refers to an operation of detecting a connectable cellular communication network. For example, the cellular interface 151 attempts to receive radio signals (e.g., synchronization signals and reference signals) transmitted by base stations of each cellular communication network (i.e., the public cellular communication network 210 and the local cellular communication network 220). The controller 190 measures the received power of the radio signal, and when the received power exceeds a threshold, determines that the cellular communication network corresponding to this received power is connectable. Hereinafter, the cellular communication network detected by the network search will be referred to as the "detected cellular communication network."

[0050] There are three patterns of detected cellular communication networks: (1) only the public cellular communication network 210, (2) only the local cellular communication network 220, or (3) both the public cellular communication network 210 and the local cellular communication network 220. In the following, the pattern in which the detected cellular communication network is only the public cellular communication network 210 or only the local cellular communication network 220 will be mainly described, but the detected cellular communication network may be both the public cellular communication network 210 and the local cellular communication network 220.

[0051] Next, the controller 190 acquires network-related information from the detected cellular communication network via the cellular interface 151 (step S102). The network-related information includes, for example, the following information (a) to (c):

[0052] (a) Network Identifier The network identifier is an identifier that identifies the detected cellular communication network.

[0053] (b) Information indicating the type of service provided The provided service type is the type of service provided by the detected cellular communication network to the electronic device 100. As described above, each of the public cellular communication network 210 and the local cellular communication network 220 is logically divided into a plurality of slices corresponding to different service types (see FIG. 4). An identifier called S-NSSAI is assigned to each slice. The controller 190 acquires the S-NSSAI of each slice of the detected cellular communication network as information indicating the type of service provided by the detected cellular communication network. The controller 190 may also acquire information other than the S-NSSAI as information indicating the type of service provided.

[0054] (c) Information indicating the network type The network type indicates whether the detected cellular communication network is the public cellular communication network 210 or the local cellular communication network 220 .

[0055] Furthermore, the controller 190 may acquire the received power measured during the network search in step S101 as the network-related information.

[0056] Next, the controller 190 causes the display 112 to display a network-related image showing predetermined information related to the detected cellular communication network based on the network-related information acquired in step S102 (step S103).

[0057] Next, the controller 190 determines whether or not an instruction to end the display operation by the electronic device 100 has been given (step S104).

[0058] In step S104, if it is determined that an instruction to end the display operation by electronic device 100 has not been issued (step S104; NO), controller 190 returns the process to step S101 and repeats the subsequent processes.

[0059] Furthermore, in step S104, if it is determined that an instruction to end the display operation by electronic device 100 has been given (step S104; YES), controller 190 ends the network-related image display process.

[0060] A specific display mode when the display control of the net-related image is performed in step S103 will be described below.

[0061] As one display mode, for example, as shown in FIG. 5, the controller 190 displays an image I1 (e.g., an image of a circled number) indicating the number of slices (e.g., 4) of the detected cellular communication network as a network-related image in the upper part 112A (so-called pictogram area) of the display 112.

[0062] 6, the controller 190 may display an image I1 indicating the number of slices of the detected cellular communication network together with an image I2 indicating the communication quality with the detected cellular communication network. In this case, for example, when a predetermined operation is performed to switch the communication mode between 4G communication and 5G communication, the network-related image is switched to either a network-related image I3 corresponding to 4G (slice-incompatible communication network) or network-related images I1 and I2 corresponding to 5G (slice-compatible communication network) and displayed as shown in FIG. Note that the controller 190 may display image I1 so that it is superimposed on image I2 (for example, FIG. 6), or may display images I1 and I2 so that they are spaced apart, or may display image I2 so that it is superimposed on image I1.

[0063] 8, controller 190 may normally display only image I2 indicating the communication quality with the detected cellular communication network, and when a predetermined operation is performed via touch panel 111, may display image I1 indicating the number of slices of the detected cellular communication network together with image I2. Note that the predetermined operation is not limited to a predetermined operation via touch panel 111, and may also be a predetermined operation via physical button 113 or voice input unit 121.

[0064] 9, the controller 190 may normally display only an image I2 indicating the communication quality with the detected cellular communication network, and when a predetermined operation corresponding to the image I2 is performed, an image I1 indicating the number of slices of the detected cellular communication network may be displayed. Here, the predetermined operation corresponding to the image I2 may be, for example, a touch operation on the image I2 via the touch panel 111, a cursor designation operation on the image I2 via the physical button 113, or a predetermined voice operation via the voice input unit 121. After the image I1 indicating the number of slices of the detected cellular communication network is displayed, the display may be returned to the image I2 when a predetermined time has elapsed (or a predetermined operation has been performed). The image normally displayed is not limited to the image I2 indicating the communication quality with the detected cellular communication network, and may be, for example, an app icon that uses slices according to the type of service provided by the detected cellular communication network.

[0065] 10, the controller 190 may display, as a network-related image, an image I4 indicating the number of slices (e.g., 2 / 4) that satisfy a predetermined condition out of the number of slices (e.g., 4) in the detected cellular communication network. The number of slices that satisfy the predetermined condition is, for example, the number of slices that have been used by the electronic device 100 (number of slices). The number of slices that satisfy the predetermined condition may also be, for example, the number of slices that satisfy a predetermined communication volume or communication frequency or the number of slices that satisfy a predetermined communication quality. The image indicating the number of slices that satisfy the predetermined condition out of the number of slices in the detected cellular communication network is not limited to the image I4 shown in FIG. 10. As shown in FIG. 11, the controller 190 may display an image I5 in which the number of slices (e.g., 2) that satisfies the predetermined condition is underlined in order to distinguish it from the number of slices in the detected cellular communication network.

[0066] 12, the controller 190 may display an image I1 indicating the number of slices (e.g., 4) of the detected cellular communication network as a network-related image, and may also display an image I6 indicating the type of service provided to the electronic device 100 by the detected cellular communication network around the image I1. The image I6 indicating the service type displays a number corresponding to the service type (see FIG. 4). Based on a predetermined operation corresponding to the image I6, the use of communication via the slice corresponding to the corresponding service type can be controlled to be on or off. Note that the display content of the image I6 indicating the service type may be any content that can identify the service type, and may be, for example, an identifier (S-NSSAI) of the corresponding slice or an icon set for the corresponding service type.

[0067] Here, when an image I6 indicating the type of service provided to the electronic device 100 by the detected cellular communication network is displayed around the image I1 as described above, the controller 190 specifies the type of service to be provided from among the following candidate service types No. 1 to No. 8 based on the information indicating the type of service to be provided acquired in step S102 of the network-related image display process (see FIG. 3), as shown in Fig. 4. Note that when there are multiple detected cellular communication networks, the controller 190 specifies the type of service to be provided for each detected cellular communication network.

[0068] No.1: Data communication (Internet communication) The controller 190 determines whether communication with a specific device or server is possible via the detected cellular communication network, or whether a certain number of UL / DL packets are transmitted and received via the detected cellular communication network per unit time. If it is determined that communication with a specific device or server is possible via the detected cellular communication network, the controller 190 identifies data communication as one of the types of services provided by the detected cellular communication network. Alternatively, if it is determined that a certain number of uplink / downlink packets are transmitted and received via the detected cellular communication network per unit time, the controller 190 identifies data communication as one of the types of services provided by the detected cellular communication network.

[0069] No.2: Voice communication The controller 190 determines whether communication with the SIP server is possible via the detected cellular communication network or the line (CS) status of the detected cellular communication network. If it is determined that communication with the SIP server is possible via the detected cellular communication network, the controller 190 specifies voice communication as one of the types of services provided by the detected cellular communication network. Alternatively, if it is determined that the line (CS) of the detected cellular communication network is available, the controller 190 specifies voice communication as one of the types of services provided by the detected cellular communication network.

[0070] No.3: Low Power Consumption The controller 190 determines whether or not a power saving function has been set in the electronic device 100 from the detected cellular communication network. The power saving function is set in the electronic device 100 by, for example, an RRC Reconfiguration message, which is a type of RRC message. When it is determined that the power saving function has been set in the electronic device 100 from the detected cellular communication network, the controller 190 identifies the power saving function as one of the types of services provided by the detected cellular communication network.

[0071] No.4: Temperature abnormality suppression function (overheating assistance) The controller 190 determines whether or not a temperature abnormality suppression function has been set in the electronic device 100 from the detected cellular communication network. The temperature abnormality suppression function is set in the electronic device 100 by, for example, an RRC Reconfiguration message, which is a type of RRC message. If it is determined that the temperature abnormality suppression function has been set in the electronic device 100 from the detected cellular communication network, the controller 190 identifies the temperature abnormality suppression function as one of the service types provided by the detected cellular communication network.

[0072] No.5: High speed and large capacity (eMBB) The controller 190 determines whether the SST (slice / service type) of the S-NSSAI acquired from the detected cellular communication network includes the value "1" assigned to eMBB by the standard. The S-NSSAI is notified to the electronic device 100 by, for example, a Registration Accept message, which is a type of NAS (Non-Access Stratum) message. If the SST of the SNSSAI acquired from the detected cellular communication network includes the value "1" assigned to eMBB by the standard, the controller 190 identifies eMBB as one of the service types provided by the detected cellular communication network.

[0073] No.6: High reliability, low latency (URLLC) The controller 190 determines whether the SST of the S-NSSAI acquired from the detected cellular communication network includes the value "2" assigned to URLLC in the standard. If the SST of the S-NSSAI acquired from the detected cellular communication network includes the value "2" assigned to URLLC in the standard, the controller 190 identifies URLLC as one of the service types provided by the detected cellular communication network.

[0074] No.7: Multi-connectivity (mIoT) The controller 190 determines whether the SST of the S-NSSAI acquired from the detected cellular communication network includes the value "2" assigned to mIoT in the standard. If the SST of the S-NSSAI acquired from the detected cellular communication network includes the value "2" assigned to mIoT in the standard, the controller 190 identifies mIoT as one of the service types provided by the detected cellular communication network.

[0075] No.8:Other services Controller 190 determines whether the SST of the S-NSSAI acquired from the detected cellular communication network includes a value assigned to each service in accordance with the proprietary specifications. For example, SST "x" is assigned to Business, SST "y" is assigned to Factory, SST "z" is assigned to Transport, SST "xx" is assigned to Airport, SST "yy" is assigned to Drone, and SST "zz" is assigned to Train. If the SST of the S-NSSAI acquired from the detected cellular communication network includes a value assigned in accordance with the proprietary specifications, controller 190 identifies the service type corresponding to this value as the service type provided by the detected cellular communication network.

[0076] Furthermore, the controller 190 may normally display an image I2 indicating the communication quality with the detected cellular communication network together with an image I1 indicating the number of slices (for example, 4) of the detected cellular communication network, as shown in Fig. 6, and when a predetermined operation corresponding to the image I1 is performed, an image I6 indicating the type of service that the detected cellular communication network provides to the electronic device 100 may be displayed around the image I1, with the image I1 at the center, as shown in Fig. 12. Here, the predetermined operation corresponding to the image I1 may be, for example, a touch operation on the image I1 via the touch panel 111, a cursor designation operation on the image I1 via the physical button 113, a predetermined voice operation via the voice input unit 121, or the like.

[0077] 13, when the user moves from Area 1 to Area 2, the controller 190 may determine whether or not there has been a change in the number of slices of the detected cellular communication network based on the network-related information acquired in step S102 of the network-related image display process (see FIG. 3), and if there has been a change in the number of slices, may notify the user of that change. Specifically, as shown in FIG. 14, for example, if the number of slices of the detected cellular communication network increases from 4 to 5 as the user moves from Area 1 to Area 2, the controller 190 displays an image I7 (for example, an image with the word "add" circled) indicating that the number of slices of the detected cellular communication network has increased. On the other hand, if the number of slices of the detected cellular communication network decreases from 4 to, for example, 3 as the user moves from Area 1 to Area 2, the controller 190 displays an image I8 (for example, an image with the word "remove" circled) indicating that the number of slices of the detected cellular communication network has decreased.

[0078] When the number of slices of the detected cellular communication network increases, the controller 190 may notify the user that the number of slices of the detected cellular communication network has increased by changing the display color of the image I2 indicating the communication quality with the detected cellular communication network to a preset color (e.g., red) or a preset display mode (e.g., fast blinking). When the number of slices of the detected cellular communication network decreases, the controller 190 may notify the user that the number of slices of the detected cellular communication network has decreased by changing the display color of the image I2 indicating the communication quality with the detected cellular communication network to a preset color (e.g., blue) or a preset display mode (e.g., slow blinking). The notification method regarding the increase or decrease in the number of slices of the detected cellular communication network is not limited to the above-described method. For example, the audio output unit 122 may output a sound or the like indicating that the number of slices of the detected cellular communication network has increased or decreased. The notification may also be made by illuminating a light-emitting unit (not shown) provided in the electronic device 100. Furthermore, a vibration function provided in the electronic device 100 may be activated to notify that the number of slices of the detected cellular communication network has increased or decreased.

[0079] Furthermore, when a predetermined operation is performed corresponding to image I7 (see FIG. 14 ) indicating an increase in the number of slices of the detected cellular communication network, controller 190 may display image I1 indicating the number of slices of the detected cellular communication network (e.g., from 4 to 5) at the center, as shown in FIG. 15 , with image I6 indicating the service types provided to electronic device 100 by the detected cellular communication network at the center. At this time, image I6 indicating the service type corresponding to the newly available slice (e.g., service type No. 5) displays the word “New.” Here, image I6 is displayed to help the user easily notice which service type corresponds to the newly available slice. Controller 190 may, for example, change the display mode of the image indicating the service type corresponding to the newly available slice to make it easier for the user to notice (changing the display mode may, for example, blink the image indicating the service type corresponding to the newly available slice at a predetermined first speed, change the color of the image to a predetermined first color, or move the display position of the image to a predetermined first position, etc.). On the other hand, when a predetermined operation is performed corresponding to image I8 (see FIG. 14) indicating that the number of slices of the detected cellular communication network has decreased, controller 190 may display image I1 indicating the number of slices of the detected cellular communication network (e.g., from 4 to 3) at the center, and image I6 indicating the service types provided to electronic device 100 by the detected cellular communication network, around image I1, as shown in FIG. 16. At this time, image I6 indicating the service type corresponding to the unavailable slice (e.g., service type No. 4) displays a pattern indicating that the service type is unavailable. Here, image I6 is displayed to make it easier for the user to notice which service type corresponds to the unavailable slice.The controller 190 may, for example, make it easier for the user to notice by changing the display mode of the image indicating the service type corresponding to the unavailable slice (changing the display mode may, for example, blink the image indicating the service type corresponding to the unavailable slice at a predetermined second speed, change the color of the image to a predetermined second color, or move the display position of the image to a predetermined second position, etc.).

[0080] Furthermore, the controller 190 may determine whether or not a predetermined service type is included in the detected cellular communication network based on the network-related information acquired in step S102 of the network-related image display process (see FIG. 3), and may notify the detected cellular communication network if the predetermined service type is included in the detected cellular communication network. Specifically, the controller 190 may determine whether or not a service type (e.g., voice communication (No. 2) and high reliability, low latency (No. 6); see FIG. 4) specifically recommended for the electronic device 100 is included in the detected cellular communication network based on the network-related information, and if the service type is included, may notify the detected cellular communication network that the service type specifically recommended for the electronic device 100 is included in the detected cellular communication network by changing the display color of the meter portion of the image I2 indicating the communication quality with the detected cellular communication network to a preset display color (e.g., purple), as shown in FIG. 17. Here, it is assumed that the service type specifically recommended for the electronic device 100 is preset and stored in the storage 140. As shown in FIG. 18, the display color of image I1 indicating the number of slices in the detected cellular communication network may be changed to a preset display color (e.g., yellow) to notify electronic device 100 that the detected cellular communication network includes a service type that is specifically recommended.

[0081] Furthermore, based on the network-related information, controller 190 may determine whether the detected cellular communication network includes a service type (e.g., voice communication (No. 2); see FIG. 4 ) that is specifically recommended for an application (e.g., a calling application) executed in electronic device 100. If the service type is included, controller 190 may notify that the detected cellular communication network includes the service type specifically recommended for the application by displaying, on display 112, an image I1 indicating the number of slices in the detected cellular communication network together with an image I9 that indicates the application while the application is being executed or when an operation to designate an icon related to the application is performed, as shown in FIG. 19 . Here, it is assumed that the service type specifically recommended for the application executed in electronic device 100 is set in advance and stored in storage 140.

[0082] Note that the notification method for notifying that the detected cellular communication network includes a service type specifically recommended for electronic device 100 and the notification method for notifying that the detected cellular communication network includes a service type specifically recommended for an application executed on electronic device 100 are not limited to the above-described methods. For example, a sound indicating that the detected cellular communication network includes a service type specifically recommended for electronic device 100 or a service type specifically recommended for an application executed on electronic device 100 may be output from audio output unit 122. Furthermore, a light-emitting unit (not shown) provided on electronic device 100 may be illuminated or a vibration function provided on electronic device 100 may be activated to notify that the detected cellular communication network includes a service type specifically recommended for electronic device 100 or a service type specifically recommended for an application executed on electronic device 100.

[0083] 20 to 22, the controller 190 may display images I10 to I12 indicating characteristics of the detected cellular communication network as network-related images. Specifically, as shown in FIG. 20, when the service type included in the detected cellular communication network is only multi-connection (mIoT) (No. 7) (or when many service types specialized in multi-connection are included), the controller 190 displays an image I2 indicating the communication quality with the detected cellular communication network, together with an image I10 (an image showing the letter "multi" in a circle) indicating that the service types of the detected cellular communication network include multi-connection (mIoT). Also, as shown in FIG. 21, when the service type included in the detected cellular communication network is only Drone (No. 8) (or when many service types specialized in Drone are included), the controller 190 displays an image I2 indicating the communication quality with the detected cellular communication network, together with an image I11 (an image showing the letter "D" in a circle) indicating that the service types of the detected cellular communication network include Drone. Furthermore, as shown in FIG. 22, when the service type included in the detected cellular communication network is only high reliability, low latency (URLLC) (No. 6), which is ideal for autonomous driving communication, etc. (or when many service types specialized in high reliability and low latency are included), the controller 190 displays an image I12 indicating the communication quality with the detected cellular communication network, as well as an image I2 indicating the communication quality with the detected cellular communication network (an image showing the word Auto in a circle) indicating that the service types of the detected cellular communication network include high reliability and low latency.

[0084] The controller 190 may indicate the characteristics of the detected cellular communication network by, for example, changing the display color or display mode of the image I2. For example, the controller 190 may indicate that the detected cellular communication network includes more service types that are specialized for high reliability and low latency as the blinking speed of the image I2 increases.

[0085] Furthermore, among the service types No.1 to No.8 shown in FIG. 4, for example, if service types No.1 and No.2 are classified as type 1 and service types No.3 to No.8 are classified as type 2 (feature set), and the service types included in the detected cellular communication network are only service types belonging to type 2, as shown in FIG. 23, the controller 190 may display an image I2 indicating the communication quality with the detected cellular communication network, as well as an image I13 (an image showing the character "SPECIAL" in a circle) indicating that the service type of the detected cellular communication network corresponds to type 2 (feature set).

[0086] Also, as shown in FIG. 24, when the detected cellular communication network is a local cellular communication network 220, the controller 190 may acquire, via the cellular interface 151, owner information indicating the owner of the license corresponding to the local cellular communication network 220 (e.g., ABC Co., Ltd.) and communication area information indicating the communication area of ​​the local cellular communication network 220 (e.g., Area 1), and based on the acquired owner information and communication area information, display an image I14 indicating the owner and an image I15 indicating the communication area, together with an image I1 indicating the number of slices of the detected cellular communication network and an image I2 indicating the communication quality with the detected cellular communication network.

[0087] Furthermore, when a predetermined operation (for example, a touch operation) is performed on the image I14 indicating the owner, the controller 190 may cause the display 112 to display a guide image I16 indicating the telephone number and email address of the owner, ABC Corporation, as shown in Fig. 25. Instead of the guide image I16, the controller 190 may cause the display 112 to display a guide image I17 provided with, for example, a Tel button B1 for calling the telephone number of ABC Corporation, a Mail button B2 for creating an email addressed to the email address of ABC Corporation, and a LINK button B3 for accessing the homepage of ABC Corporation, as shown in Fig. 26.

[0088] Furthermore, when a predetermined operation (for example, a touch operation) is performed on the image I15 indicating the communication area, the controller 190 may cause a map image I18 indicating Area 1, which is the communication area, to be displayed on the display 112, as shown in Fig. 27. Furthermore, when displaying the map image I18, the controller 190 may cause the user's current location P to be displayed.

[0089] [Variation 1] Next, an electronic device 100A according to Modification 1 will be described. This electronic device 100A differs from the above embodiment in that it is intended for a wearable terminal device. Below, differences from the above embodiment will be described, and commonalities will be omitted.

[0090] As shown in FIG. 28, the electronic device 100A includes a main body 10a, a visor 114 (display member) attached to the main body 10a, and the like.

[0091] The main body 10a is an annular member whose circumference is adjustable. Various devices such as a depth sensor 133 and a camera 134 are built into the main body 10a. When the main body 10a is worn on the head, the user's field of vision is covered by a visor 114.

[0092] The visor 114 is optically transparent. The user can view the real space through the visor 114. A laser scanner 115 (see FIG. 31 ) built into the main body 10a projects and displays an image such as a virtual image on a display surface of the visor 114 facing the user's eyes. The user views the virtual image through light reflected from the display surface. At this time, the user also views the real space through the visor 114, providing a visual effect as if the virtual image were present in the real space.

[0093] As shown in FIG. 29 , when the virtual image 30 is displayed, the user views the virtual image 30 at a predetermined position in a space 40, facing a predetermined direction. In this modification, the space 40 is a real space viewed by the user through a visor 114. Because the virtual image 30 is projected onto the optically transparent visor 114, it is viewed as a translucent image superimposed on the real space. While FIG. 29 illustrates a planar window screen as an example of the virtual image 30, the present invention is not limited thereto, and the virtual image 30 may be, for example, any of various three-dimensional images. When the virtual image 30 is a window screen, the virtual image 30 has a front and a back surface, and necessary information is displayed on the front surface, while normally no information is displayed on the back surface.

[0094] Electronic device 100A detects user visual recognition area 41 based on the user's position and orientation in space 40 (in other words, the position and orientation of electronic device 100A). As shown in FIG. 30, visual recognition area 41 is an area in space 40 located in front of user U wearing electronic device 100A. For example, visual recognition area 41 is an area within a predetermined angular range in each of the left-right and up-down directions from the front of user U. In this case, when a three-dimensional object corresponding to the shape of visual recognition area 41 is cut by a plane perpendicular to the front direction of user U, the shape of the cut surface is rectangular. Note that the shape of visual recognition area 41 may be determined so that the cut surface has a shape other than a rectangle (for example, a circle or an ellipse). The shape of visual recognition area 41 (for example, the angular range in the left-right and up-down directions from the front) can be identified by, for example, the following method.

[0095] In electronic device 100A, adjustment of the field of view (hereinafter referred to as calibration) is performed according to a predetermined procedure at a predetermined timing, such as when the electronic device is first started up. This calibration specifies the range that can be seen by the user, and thereafter, virtual image 30 is displayed within this range. The shape of the visible range specified by this calibration can be used as the shape of visible area 41.

[0096] Furthermore, calibration is not limited to being performed according to the above-described predetermined procedure, and may be performed automatically during normal operation of electronic device 100A. For example, if the user does not react to a display that should elicit a reaction, the range in which the display is being performed may be considered to be outside the user's field of view, and the field of view (and the shape of visible area 41) may be adjusted. Also, a test display may be performed at a position that is determined to be outside the field of view, and if the user reacts to the display, the range in which the display is being performed may be considered to be within the user's field of view, and the field of view (and the shape of visible area 41) may be adjusted.

[0097] The shape of visible area 41 may be predetermined and fixed at the time of shipment, etc., without being based on the result of adjusting the field of view. For example, the shape of visible area 41 may be determined to be the maximum displayable range in terms of the optical design of display unit 110A.

[0098] Virtual image 30 is generated in a state where the display position and orientation in space 40 are determined in response to a predetermined operation by the user. Electronic device 100A projects and displays, on visor 114, virtual image 30 of the generated virtual images 30 whose display position is determined to be within visible area 41. In FIG. 29, visible area 41 is indicated by a dashed line.

[0099] The display position and orientation of the virtual image 30 on the visor 114 are updated in real time in response to changes in the user's viewing area 41. That is, the display position and orientation of the virtual image 30 change in response to changes in the viewing area 41 so that the user recognizes that "the virtual image 30 is located in the space 40 at a set position and orientation." For example, when the user moves from the front side of the virtual image 30 to the back side, the shape (angle) of the displayed virtual image 30 gradually changes in response to this movement. Furthermore, when the user turns toward the back side of the virtual image 30 after going around to the back side of the virtual image 30, the back side of the virtual image 30 is displayed so that the back side of the virtual image 30 can be seen. Furthermore, in response to changes in the viewing area 41, any virtual image 30 whose display position is outside the viewing area 41 is no longer displayed, and if any virtual image 30 whose display position is within the viewing area 41 is present, that virtual image 30 is newly displayed.

[0100] 29, when a user holds out their hand (or finger) in front, the direction in which the hand is extended is detected by electronic device 100A, and virtual line 51 extending in that direction and pointer 52 are displayed on the display surface of visor 114 for visual confirmation by the user. Pointer 52 is displayed at the intersection of virtual line 51 and virtual image 30. If virtual line 51 does not intersect with virtual image 30, pointer 52 may be displayed at the intersection of virtual line 51 and a wall surface or the like of space 40. If the distance between the user's hand and virtual image 30 is within a predetermined reference distance, the display of virtual line 51 may be omitted, and pointer 52 may be directly displayed at a position corresponding to the position of the user's fingertip.

[0101] By changing the direction in which the user extends his / her hand, the direction of the virtual line 51 and the position of the pointer 52 can be adjusted. By adjusting the pointer 52 so that it is positioned on a predetermined operation object (e.g., the function bar 31, the window shape change button 32, the close button 33, etc.) included in the virtual image 30 and performing a predetermined gesture, the gesture is detected by the electronic device 100A, and a predetermined operation can be performed on the operation object. For example, by performing a gesture to select the operation object (e.g., a gesture of pinching the fingertips) while the pointer 52 is positioned on the close button 33, the virtual image 30 can be closed (deleted). Furthermore, by performing a selection gesture while the pointer 52 is positioned on the function bar 31, and then performing a gesture of moving the hand back and forth or left and right while the selection state is maintained, the virtual image 30 can be moved in the depth direction and left and right directions. Operations on the virtual image 30 are not limited to these.

[0102] In this way, electronic device 100A of this modification realizes a visual effect as if virtual image 30 exists in real space, and can accept user operations on virtual image 30 and reflect them in the display of virtual image 30. In other words, electronic device 100A of this modification provides MR (mixed reality).

[0103] Next, the functional configuration of electronic device 100A will be described with reference to Fig. 31. Electronic device 100A has display unit 110A, sensor unit 130A, storage 140, communication interface 150, SIM interface 160, connection interface 170, battery 180, and controller 190. Of the components shown in Fig. 31, each unit except for visor 114 of display unit 110A is built into main body unit 10a and operates using power supplied from battery 180 also built into main body unit 10a.

[0104] Controller 190 performs various control operations by reading and executing program 141 stored in storage 140. By executing program 141, controller 190 performs, for example, a visual recognition area detection process and a display control process. Of these, the visual recognition area detection process is a process for detecting a visual recognition area 41 of a user within space 40. Furthermore, the display control process is a process for displaying, on display unit 110A, virtual images 30 whose positions in space 40 are determined to be within visual recognition area 41.

[0105] The storage 140 stores a program 141 executed by the controller 190, various setting data, etc. The program 141 is stored in the storage 140 in the form of computer-readable program code.

[0106] The data stored in the storage 140 includes virtual image data 142 related to the virtual image 30. The virtual image data 142 includes data related to the display content of the virtual image 30 (e.g., image data), data on the display position, and data on the orientation.

[0107] The display unit 110A includes a visor 114, a laser scanner 115, and an optical system that guides light output from the laser scanner 115 onto the display surface of the visor 114. The laser scanner 115 irradiates the optical system with pulsed laser light, which is on / off controlled for each pixel, while scanning in a predetermined direction in accordance with a control signal from the controller 190. The laser light incident on the optical system forms a display screen consisting of a two-dimensional pixel matrix on the display surface of the visor 114. The type of the laser scanner 115 is not particularly limited, but for example, a type in which a mirror is operated by a MEMS (microelectromechanical system) to scan the laser light can be used. The laser scanner 115 includes three light-emitting elements that emit, for example, RGB laser light. The display unit 110A can display in color by projecting light from these light-emitting elements onto the visor 114.

[0108] The sensor unit 130A includes an acceleration sensor 131, an angular velocity sensor 132, a depth sensor 133, a camera 134, and an eye tracker 135. Note that the sensor unit 130A may further include sensors not shown in FIG.

[0109] Acceleration sensor 131 detects acceleration and outputs the detection result to controller 190. From the detection result by acceleration sensor 131, translational motion of electronic device 100A in three orthogonal axial directions can be detected.

[0110] Angular velocity sensor 132 (gyro sensor) detects angular velocity and outputs the detection result to controller 190. From the detection result by angular velocity sensor 132, the rotational motion of electronic device 100A can be detected.

[0111] The depth sensor 133 is an infrared camera that detects the distance to a subject using a ToF (Time of Flight) method, and outputs the distance detection result to the controller 190. The depth sensor 133 is provided on the front surface of the main body 10a so that it can capture an image of the visible area 41. By repeatedly performing measurements using the depth sensor 133 each time the position and orientation of the user changes in the space 40 and combining the results, it is possible to perform three-dimensional mapping of the entire space 40 (i.e., to obtain the three-dimensional structure).

[0112] Camera 134 captures an image of space 40 with a group of RGB imaging elements, obtains color image data as the captured image, and outputs the color image data to controller 190. Camera 134 is provided on the front surface of main body 10a so as to be able to capture an image of visible area 41. The image output from camera 134 is used to detect the position and orientation of electronic device 100A, and is also transmitted from communication interface 150 to an external device and used to display visible area 41 of the user of electronic device 100A on the external device.

[0113] The eye tracker 135 detects the user's line of sight and outputs the detection result to the controller 190. The method for detecting the line of sight is not particularly limited, but for example, a method can be used in which an eye tracking camera captures an image of the reflection point of near-infrared light on the user's eye, and the image captured by the camera 134 is analyzed to identify the object the user is looking at. A part of the configuration of the eye tracker 135 may be provided on the periphery of the visor 114, for example.

[0114] In the electronic device 100A configured as described above, the controller 190 performs the following control operations.

[0115] The controller 190 performs three-dimensional mapping of the space 40 based on the distance data to the subject input from the depth sensor 133. The controller 190 repeats this three-dimensional mapping every time the user's position and orientation change, updating the results each time. The controller 190 also performs three-dimensional mapping for each continuous space 40. Therefore, when the user moves between multiple rooms separated by walls or the like, the controller 190 recognizes each room as a single space 40 and performs three-dimensional mapping for each room separately.

[0116] Controller 190 detects user's visual recognition area 41 within space 40. Specifically, controller 190 identifies the position and orientation of user (electronic device 100A) in space 40 based on the detection results of acceleration sensor 131, angular velocity sensor 132, depth sensor 133, camera 134, and eye tracker 135, and the accumulated results of 3D mapping. Then, controller 190 detects (identifies) visual recognition area 41 based on the identified position and orientation and the predetermined shape of visual recognition area 41. Controller 190 also continuously detects the user's position and orientation in real time, and updates visual recognition area 41 in response to changes in the user's position and orientation. Note that detection of visual recognition area 41 may be performed using the detection results of some of acceleration sensor 131, angular velocity sensor 132, depth sensor 133, camera 134, and eye tracker 135.

[0117] The controller 190 generates virtual image data 142 related to the virtual image 30 in response to a user operation. That is, when the controller 190 detects a predetermined operation (gesture) instructing the generation of the virtual image 30, the controller 190 specifies the display content (e.g., image data), display position, and orientation of the virtual image, and generates virtual image data 142 including data representing the results of these specifications.

[0118] Controller 190 causes display unit 110A to display virtual image 30, the display position of which is determined within visible area 41. Controller 190 identifies virtual image 30, the display position of which is determined within visible area 41, based on display position information included in virtual image data 142, and generates image data for a display screen to be displayed on display unit 110A based on the positional relationship between visible area 41 and the display position of the identified virtual image 30 at that time. Controller 190 causes laser scanner 115 to perform a scanning operation based on this image data, and forms a display screen including the identified virtual image 30 on the display surface of visor 114. In other words, controller 190 displays virtual image 30 on the display surface of visor 114 so that virtual image 30 is visible in space 40 viewed through visor 114. By continuously performing this display control process, controller 190 updates the display content on display unit 110A in real time in accordance with the user's movement (changes in visible area 41). If the electronic device 100A is set to retain the virtual image data 142 even when the power is turned off, the next time the electronic device 100A is started up, the existing virtual image data 142 is read, and if there is a virtual image 30 located within the visible area 41, it is displayed on the display unit 110A.

[0119] Note that virtual image data 142 may be generated based on instruction data acquired from an external device via communication interface 150, and virtual image 30 may be displayed based on the virtual image data 142. Alternatively, virtual image data 142 itself may be acquired from an external device via communication interface 150, and virtual image 30 may be displayed based on the virtual image data 142. For example, an external device operated by a remote instructor may display an image captured by camera 134 of electronic device 100A, and may receive an instruction to display virtual image 30 from the external device, and display the instructed virtual image 30 on display unit 110A of electronic device 100A. This enables, for example, an operation in which virtual image 30 showing the content of a task is displayed near a task target, and the remote instructor can instruct the user of electronic device 100A to perform the task.

[0120] Controller 190 detects the position and orientation of the user's hand (and / or finger) based on the images captured by depth sensor 133 and camera 134, and displays on display unit 110A virtual line 51 extending in the detected direction and pointer 52. Controller 190 also detects a gesture of the user's hand (and / or finger) based on the images captured by depth sensor 133 and camera 134, and executes processing according to the content of the detected gesture and the position of pointer 52 at that time.

[0121] Next, a description will be given of the operation of electronic device 100A. Fig. 32 is a flowchart showing a control procedure for network-related image display processing by electronic device 100A.

[0122] When the network-related image display process shown in FIG. 32 is started, the controller 190 controls the cellular interface 151 to perform a network search (step S201).

[0123] Next, the controller 190 acquires network-related information from the detected cellular communication network via the cellular interface 151 (step S202).

[0124] Next, controller 190 causes display 112 to display a network-related image showing predetermined information related to the detected cellular communication network based on the network-related information acquired in step S202 (step S203).

[0125] Next, the controller 190 detects the visible region 41 based on the position and orientation of the user (step S204).

[0126] Next, the controller 190 determines whether or not there is a virtual image 30 whose display position is determined within the detected visible region 41 (step S205).

[0127] In step S205, if it is determined that there is no virtual image 30 whose display position is determined within the detected visible region 41 (step S205; NO), the controller 190 advances the process to step S207.

[0128] Also, in step S205, if it is determined that there is a virtual image 30 whose display position is determined within the detected visible area 41 (step S205; YES), the controller 190 causes the virtual image 30 to be displayed on the display unit 110A (step S206).

[0129] Next, controller 190 determines whether an instruction to end the display operation by electronic device 100A has been issued (step S207).

[0130] In step S207, if it is determined that an instruction to end the display operation by electronic device 100A has not been issued (step S207; NO), controller 190 returns the process to step S201 and repeats the subsequent processes.

[0131] Furthermore, in step S207, if it is determined that an instruction to end the display operation by electronic device 100A has been given (step S207; YES), controller 190 ends the network-related image display process.

[0132] The following describes a specific display mode when the display control of the net-related image is performed in step S203 and the display control of the virtual image 30 is performed in step S206.

[0133] 33, for example, controller 190 displays virtual image 30 on display unit 110A and also displays image I1 indicating the number of slices of the detected cellular communication network as a network-related image in the upper left corner of display unit 110A. Specifically, controller 190 displays image I1 in a fixed state in the upper left corner of display unit 110A so that the user and image I1 always face each other. In other words, even if the user goes around to the back side of virtual image 30 displayed on display unit 110A, image I1 is displayed in the upper left corner of display unit 110A.

[0134] Furthermore, similar to the electronic device 100 described above, the controller 190 may display an image I1 indicating the number of slices of the detected cellular communication network together with an image I2 indicating the communication quality with the detected cellular communication network (see FIG. 6).

[0135] Note that the controller 190 may display an object displaying an image I1 indicating the number of slices of the detected cellular communication network, like the virtual image 30, in a state where the display position and orientation in the space 40 are determined in accordance with, for example, a predetermined operation by the user. In such a case, the controller 190 displays an image I2 indicating the communication quality with the detected cellular communication network on a surface (second surface) opposite to a surface (first surface) on which the image I1 of the object is displayed. Note that the image displayed on the second surface is not limited to the image I2 indicating the communication quality with the detected cellular communication network, and may be, for example, an application icon that uses a slice according to the type of service provided by the detected cellular communication network, or an image I19 indicating the type of service (slice) that can be provided by the detected cellular communication network, as shown in FIG. 34. Furthermore, the controller 190 may be configured to be able to switch between a first setting in which the object is displayed in a fixed state on the display unit 110A so that the first or second surface always faces the user, and a second setting in which the object is displayed in a position that corresponds to the angle at which the user views it, depending on a predetermined user operation (for example, a gesture operation, an operation of the physical button 113, a voice operation using the voice input unit 121, etc.).

[0136] [Variation 2] Next, a description will be given of the configuration of the display system 1 according to Modification 2. Modification 2 differs from Modification 1 in that an external information processing device 20 executes some of the processing that was executed by the controller 190 of the electronic device 100A in Modification 1. Below, the differences from Modification 1 will be described, and a description of the commonalities will be omitted.

[0137] 35, the display system 1 includes an electronic device 100A and an information processing device 20 (server) communicatively connected to the electronic device 100A. At least a part of the communication path between the electronic device 100A and the information processing device 20 may be wireless. The hardware configuration of the electronic device 100A may be the same as that of Modification 1, but a processor for performing the same processing as that performed by the information processing device 20 may be omitted.

[0138] As shown in FIG. 36, the information processing device 20 includes a CPU 21, a RAM 22, a storage unit 23, an operation display unit 24, a communication unit 25, and the like, and these units are connected by a bus .

[0139] The CPU 21 is a processor that performs various types of arithmetic processing and controls the overall operation of each unit of the information processing device 20. The CPU 21 reads and executes a program 231 stored in the storage unit 23 to perform various control operations.

[0140] The RAM 22 provides a working memory space for the CPU 21 and stores temporary data.

[0141] The storage unit 23 is a non-transitory recording medium readable by the CPU 21 as a computer. The storage unit 23 stores a program 231 executed by the CPU 21, various setting data, and the like. The program 231 is stored in the storage unit 23 in the form of computer-readable program code. The storage unit 23 may be, for example, a non-volatile storage device such as an SSD equipped with a flash memory or an HDD (Hard Disk Drive).

[0142] The operation display unit 24 includes a display device such as a liquid crystal display, and input devices such as a mouse and a keyboard. The operation display unit 24 displays various information such as the operation status and processing results of the display system 1 on the display device. Here, the operation status of the display system 1 may include images captured in real time by the camera 134 of the electronic device 100A. The operation display unit 24 also converts user input operations on the input device into operation signals and outputs the operation signals to the CPU 21.

[0143] The communication unit 25 communicates with the electronic device 100A to transmit and receive data. For example, the communication unit 25 receives data including some or all of the detection results by the sensor unit 130A of the electronic device 100A, information related to user operations (gestures) detected by the electronic device 100A, and the like. The communication unit 25 may also be capable of communicating with devices other than the electronic device 100A.

[0144] In the display system 1 configured as described above, the CPU 21 of the information processing device 20 executes at least a part of the processing executed by the controller 190 of the electronic device 100A in the first modification. For example, the CPU 21 may perform three-dimensional mapping of the space 40 based on the detection results of the depth sensor 133. The CPU 21 may also detect the user's visible area 41 within the space 40 based on the detection results of each unit of the sensor unit 130A. The CPU 21 may also generate virtual image data 142 related to the virtual image 30 in response to the user's operation of the electronic device 100A. The CPU 21 may also detect the position and orientation of the user's hand (and / or fingers) based on the images captured by the depth sensor 133 and the camera 134.

[0145] The processing results by the CPU 21 are transmitted to the electronic device 100A via the communication unit 25. The controller 190 of the electronic device 100A operates each unit (for example, the display unit 110A) of the electronic device 100A based on the received processing results. The CPU 21 may also transmit a control signal to the electronic device 100A to control the display of the display unit 110A of the electronic device 100A.

[0146] In this way, by executing at least a part of the processing in the information processing device 20, the device configuration of the electronic device 100A can be simplified and manufacturing costs can be reduced. Furthermore, by using a higher performance information processing device 20, various processes related to MR can be performed faster and with higher precision. Therefore, it is possible to improve the precision of 3D mapping of the space 40, improve the display quality of the display unit 110A, and increase the response speed of the display unit 110A to user actions.

[0147] 〔others〕 Note that the above embodiment is merely an example, and various modifications are possible. For example, in the above-described first modification, the visor 114 having optical transparency is used to allow the user to view the real space, but this is not limited to this. For example, the visor 114 having light blocking properties may be used to allow the user to view the image of the space 40 captured by the camera 134. That is, the controller 190 may display, on the display unit 110A, the image of the space 40 captured by the camera 134 and the virtual image 30 superimposed on the image of the space 40. Even with such a configuration, MR that blends the virtual image 30 into the real space can be realized.

[0148] Furthermore, VR can be realized that allows the user to feel as if they are in a virtual space by using a pre-generated image of a virtual space instead of an image of a real space captured by camera 134. In this VR, the user's visual recognition area 41 is also specified, and virtual image 30, the display position of which is determined to be within visual recognition area 41 in the virtual space, is displayed.

[0149] The electronic device 100A is not limited to the one having the annular main body 10a illustrated in Fig. 28, and may have any structure as long as it has a display unit that is visible to the user when worn. For example, it may have a structure that covers the entire head like a helmet. It may also have a frame that is hung on the ears like glasses, with various devices built into the frame.

[0150] Although the example in which a user's gesture is detected and accepted as an input operation has been described, the present invention is not limited to this. For example, an input operation may be accepted using a controller that the user holds in their hand or wears on their body.

[0151] Furthermore, in the above variant example 1, the explanation has been mainly focused on network-related images that are specialized for the wearable terminal device that is the target of electronic device 100A, but electronic device 100A may also be configured to display various network-related images that can be displayed by electronic device 100.

[0152] In addition, the specific details of the configurations and controls shown in the above embodiments can be appropriately changed without departing from the spirit of the present disclosure. Furthermore, the configurations and controls shown in the above embodiments can be appropriately combined without departing from the spirit of the present disclosure. [Industrial Applicability]

[0153] The present disclosure can be used in electronic devices, programs, and display methods.

[0154] 1 Display System 100,100A electronic equipment 110 Touch Panel Display 110A display section 111 Touch Panel 112 Display 113 Physical Buttons 114 Visor 115 Laser Scanner 121 Audio input section 122 Audio output section 130 sensors 130A sensor part 131 Accelerometer 132 Angular velocity sensor 133 Depth Sensor 134 Camera 135 Eye Tracker 140 Storage 141 Programs 142 Virtual Image Data 150 Communication Interface 151 Cellular Interface 152 WLAN interfaces 160 SIM interface 170 Connection Interface 180 Battery 190 Controller 20 Information processing equipment 21 CPU 22 RAM 23 Memory section 231 Programs 24 Operation display section 25 Communications Department 26 Bus 30 Virtual Images 31 Feature Bar 32 Window shape change button 33 Close button 40 space 41 Visibility Zone 51 Virtual Line 52 Pointer 210 Public Cellular Network 220 Local Cellular Network 300 servers 400 Expansion Equipment 500 Electronic equipment 600 Electronic equipment U User

Claims

1. A display unit; A camera that captures the space, at least one processor; Equipped with The at least one processor: acquiring, via a communication interface, identifiers assigned to each of a plurality of slices from a communication network that is logically divided into a plurality of slices according to service types; displaying, on the display unit, an image of the space photographed by the camera and a first image showing predetermined information related to the communication network superimposed on the image of the space based on the acquired identifier; displaying a second image, which is different from the first image and is positioned in the space, on the display unit; displaying the first image in a fixed state at a predetermined position on the display unit so that the user and the first image always face each other; The electronic device displays the second image in a position corresponding to the angle at which the user views the second image.

2. A display unit having a light-transmitting display member; at least one processor; Equipped with The at least one processor: acquiring, via a communication interface, identifiers assigned to each of a plurality of slices from a communication network that is logically divided into a plurality of slices according to service types; displaying a first image showing predetermined information about the communication network on a display surface of the display member based on the acquired identifier so that the first image is visible in a space visible through the display member; displaying an object image having a first surface and a second surface located in the space on a display surface of the display member; displaying the first image on the first surface; An electronic device that displays a third image different from the first image on the second surface.

3. A display unit, A camera that captures the space, at least one processor; Equipped with The at least one processor: acquiring, via a communication interface, identifiers assigned to each of a plurality of slices from a communication network that is logically divided into a plurality of slices according to service types; displaying, on the display unit, an image of the space photographed by the camera and a first image showing predetermined information related to the communication network superimposed on the image of the space based on the acquired identifier; displaying an object image having a first surface and a second surface located in the space on the display unit; displaying the first image on the first surface; An electronic device that displays a third image different from the first image on the second surface.

4. A display unit having a light-transmitting display member; at least one processor; Equipped with The at least one processor: acquiring, via a communication interface, identifiers assigned to each of a plurality of slices from a communication network that is logically divided into a plurality of slices according to service types; displaying a first image showing predetermined information about the communication network on a display surface of the display member based on the acquired identifier so that the first image is visible in a space visible through the display member; displaying an object image having a first surface and a second surface located in the space on a display surface of the display member; displaying the first image on the first surface; The electronic device displays a fifth image indicating the service type on the second surface.

5. A display unit, A camera that captures the space, at least one processor; Equipped with The at least one processor: acquiring, via a communication interface, identifiers assigned to each of a plurality of slices from a communication network that is logically divided into a plurality of slices according to service types; displaying, on the display unit, an image of the space photographed by the camera and a first image showing predetermined information related to the communication network superimposed on the image of the space based on the acquired identifier; displaying an object image having a first surface and a second surface located in the space on the display unit; displaying the first image on the first surface; The electronic device displays a fifth image indicating the service type on the second surface.

6. 4. The electronic device of claim 2, wherein the at least one processor switches, based on a predetermined operation, between a first setting in which the object image is displayed in a fixed state on the display unit so that the first surface or the second surface always faces the user, and a second setting in which the object image is displayed in a position corresponding to the angle at which the user views it.

7. A computer provided in an electronic device having a display unit and a camera for photographing a space, a process of acquiring, via a communication interface, identifiers assigned to each of a plurality of slices from a communication network that is logically divided into a plurality of slices according to service types; a process of displaying, on the display unit, an image of the space photographed by the camera and a first image showing predetermined information related to the communication network superimposed on the image of the space, based on the acquired identifier; a process of displaying a second image, which is different from the first image and is located in the space, on the display unit; a process of displaying the first image in a fixed state at a predetermined position on the display unit so that the user and the first image always face each other; a process of displaying the second image in a position corresponding to an angle at which the second image is viewed by the user; A program that executes the following.

8. A computer provided in an electronic device having a display unit with a light-transmitting display member, a process of acquiring, via a communication interface, identifiers assigned to each of a plurality of slices from a communication network that is logically divided into a plurality of slices according to service types; a process of displaying a first image showing predetermined information about the communication network on a display surface of the display member based on the acquired identifier so that the first image showing predetermined information about the communication network is visible in a space visible through the display member; a process of displaying an object image having a first surface and a second surface located in the space on a display surface of the display member; displaying the first image on the first surface; displaying a third image different from the first image on the second surface; A program that executes the following.

9. A computer provided in an electronic device having a display unit and a camera for photographing a space, a process of acquiring, via a communication interface, identifiers assigned to each of a plurality of slices from a communication network that is logically divided into a plurality of slices according to service types; a process of displaying, on the display unit, an image of the space photographed by the camera and a first image showing predetermined information related to the communication network superimposed on the image of the space, based on the acquired identifier; a process of displaying an object image having a first surface and a second surface located in the space on the display unit; displaying the first image on the first surface; displaying a third image different from the first image on the second surface; A program that executes the following.

10. A computer provided in an electronic device having a display unit with a light-transmitting display member, a process of acquiring, via a communication interface, identifiers assigned to each of a plurality of slices from a communication network that is logically divided into a plurality of slices according to service types; a process of displaying a first image showing predetermined information about the communication network on a display surface of the display member based on the acquired identifier so that the first image showing predetermined information about the communication network is visible in a space visible through the display member; a process of displaying an object image having a first surface and a second surface located in the space on a display surface of the display member; displaying the first image on the first surface; a process of displaying a fifth image indicating the service type on the second surface; A program that executes the following.

11. A computer provided in an electronic device having a display unit and a camera for photographing a space, a process of acquiring, via a communication interface, identifiers assigned to each of a plurality of slices from a communication network that is logically divided into a plurality of slices according to service types; a process of displaying, on the display unit, an image of the space photographed by the camera and a first image showing predetermined information related to the communication network superimposed on the image of the space, based on the acquired identifier; a process of displaying an object image having a first surface and a second surface located in the space on the display unit; displaying the first image on the first surface; a process of displaying a fifth image indicating the service type on the second surface; A program that executes the following.

Images