Communication system, information terminal and program

Abstract

To improve accuracy of estimating a position of a device worn on a body.SOLUTION: An information terminal 1 has: a skeleton estimation unit 151 that generates skeleton data indicating the skeleton of a person wearing a plurality of relay devices 2; a model creation unit 152 that creates a plurality of radio wave propagation models between the information terminal 1 and the plurality of relay devices 2 based on the skeleton data and wearing position data indicating an approximate terminal position where the information terminal 1 is worn and a plurality of approximate device positions where the plurality of relay devices 2 are worn; and a second position estimation unit 155 that determines estimation positions of the plurality of relay devices 2 based on a correction direction and correction strength obtained by correcting the arrival direction and strength of the radio waves received from the plurality of relay devices 2 using the radio wave propagation model.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present invention relates to a communication system, an information terminal, and a program. [Background technology]

[0002] 2. Description of the Related Art Conventionally, there is known a technique for correcting a human position estimated by triangulation using Wi-Fi (registered trademark), using information on a position estimated by an inertial sensor (see, for example, Non-Patent Document 1). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] A. Poulose, OS Eyobu and DS Han, "A Combined PDR and Wi-Fi Trilateration Algorithm for Indoor Localization," 2019 International Conference on Artificial Intelligence in Information and Communication Summary of the Invention [Problem to be solved by the invention]

[0004] It is being considered to realize high-speed data communication between a base station and an information terminal by relaying radio waves transmitted from a mobile phone network base station to the information terminal using radio waves in the millimeter wave band or higher by multiple relay devices worn on the body. In order to maintain good communication between the information terminal and the relay device in an environment where the information terminal and the relay device move due to human movements, the information terminal needs to estimate the position of the relay device with high accuracy and receive radio waves in the direction corresponding to the estimated position.

[0005] However, when the relay device is worn on the body, the propagation state of radio waves between the information terminal and the relay device changes depending on the person's movement, resulting in a problem in that the accuracy of detecting the position of the relay device based on the radio waves transmitted by the relay device decreases.

[0006] Therefore, the present invention has been made in consideration of these points, and aims to improve the accuracy of estimating the position of a device worn on the body. [Means for solving the problem]

[0007] A communication system according to a first aspect of the present invention includes an information terminal, and a plurality of relay devices that are attached to a human body and capable of communicating with the information terminal and a base station of a mobile phone network, each of the relay devices having a detection unit that outputs detection data of at least one of an acceleration and an angular velocity of the relay device. the information terminal includes: a data receiving unit that receives the detection data from each of the plurality of relay devices; a skeleton estimation unit that generates skeleton data indicating a skeleton of a person to which the plurality of relay devices are attached, estimated based on a plurality of pieces of the detection data corresponding to the plurality of relay devices; a model creation unit that creates a plurality of radio wave propagation models between the information terminal and each of the plurality of relay devices based on the skeleton data and wearing position data indicating an approximate terminal position where the information terminal is attached and a plurality of approximate device positions where the plurality of relay devices are attached, respectively; a radio wave measurement unit that measures a radio wave state including an arrival direction and intensity of the radio waves received from each of the plurality of relay devices by sweeping radio waves in a predetermined sweep range; and a position estimation unit that determines estimated positions of the plurality of relay devices based on a corrected direction and corrected intensity obtained by correcting the arrival direction and the intensity using the radio wave propagation model, and the radio wave measurement unit sweeps radio waves in the sweep range determined based on the estimated position determined most recently by the position estimation unit.

[0008] The model creation unit may estimate a position that is a distance away from the approximate terminal position in the skeleton data corresponding to the approximate terminal position as a terminal position that is the position of the information terminal, and estimate a position that is a distance away from the approximate device position in the skeleton data corresponding to the approximate device position as a device position that is the position of the relay device, and create the radio wave propagation model based on the terminal position, the device position, and a skeleton position and orientation indicated by the skeleton data within a predetermined range from the terminal position and the device position.

[0009] The model creation unit may create the radio wave propagation model that reflects the influence of reflected waves generated when the radio waves emitted from the device position are reflected by a part of the person's body corresponding to the skeletal data while propagating to the terminal position, or diffracted waves generated when the radio waves emitted from the device position go around behind the part of the person's body corresponding to the skeletal data while propagating to the terminal position.

[0010] The model creation unit may further identify the terminal position and the device position based on a result of identifying a relationship between a body of a person appearing in captured image data of a person wearing the multiple relay devices and the positions of each of the multiple relay devices.

[0011] The model creation unit may create the radio wave propagation model further based on data indicating clothing worn by a person wearing the plurality of relay devices.

[0012] The model creation unit may create the radio wave propagation model further based on shapes and positions of objects captured in captured image data of areas around the plurality of relay devices.

[0013] The skeleton estimation unit may generate the skeleton data by inputting a plurality of pieces of detection data corresponding to the plurality of relay devices to a skeleton estimation model that outputs the skeleton data by inputting the plurality of detection data.

[0014] The skeleton estimation unit may generate the skeleton data using the skeleton estimation model selected from a plurality of skeleton estimation model candidates based on a type or a number of the plurality of relay devices.

[0015] The position estimation unit may further estimate an orientation of each of the plurality of relay devices, and the radio wave measurement unit may determine the sweep range further based on the orientation of each of the plurality of relay devices estimated by the position estimation unit.

[0016] an information terminal according to a second aspect of the present invention is an information terminal capable of communicating with a plurality of relay devices, the information terminal comprising: a data receiving unit that receives detection data indicating at least one of an acceleration and an angular velocity of the relay device from each of the plurality of relay devices; a skeleton estimation unit that generates skeleton data indicating a skeleton of a person wearing the plurality of relay devices, the skeleton data being estimated based on a plurality of the detection data corresponding to the plurality of relay devices; a model creation unit that creates a plurality of radio wave propagation models between the information terminal and each of the plurality of relay devices, based on the skeleton data and wearing position data indicating an approximate terminal position where the information terminal is worn and a plurality of approximate device positions where the plurality of relay devices are worn, respectively; a radio wave measurement unit that measures an arrival direction and an intensity of radio waves received from each of the plurality of relay devices by sweeping radio waves in a predetermined sweep range; and a position estimation unit that determines estimated positions of the plurality of relay devices, based on a corrected direction and a corrected intensity obtained by correcting the arrival direction and the intensity using the radio wave propagation model, and the radio wave measurement unit sweeps the radio waves in the sweep range determined based on the estimated position determined most recently by the position estimation unit.

[0017] A program according to a third aspect of the present invention is a program executed by a processor included in an information terminal capable of communicating with a plurality of relay devices, the program being executed by the processor to cause the processor to include: a data receiving unit that receives detection data indicating at least one of an acceleration and an angular velocity of the relay device from each of the plurality of relay devices; a skeleton estimating unit that generates skeleton data indicating a skeleton of a person wearing the plurality of relay devices, the skeleton data being estimated based on a plurality of pieces of the detection data corresponding to the plurality of relay devices; an approximate terminal position at which the information terminal is worn; and a plurality of approximate device positions at which each of the plurality of relay devices is worn. a model creation unit that creates a plurality of radio wave propagation models between the information terminal and each of the plurality of relay devices based on the skeletal data and wearing position data indicating the position of the information terminal and the plurality of relay devices; a radio wave measurement unit that measures a radio wave state including an arrival direction and intensity of the radio waves received from each of the plurality of relay devices by sweeping radio waves within a predetermined sweep range; and a position estimation unit that determines estimated positions of the plurality of relay devices based on a corrected direction and corrected intensity obtained by correcting the arrival direction and the intensity using the radio wave propagation model, and the radio wave measurement unit sweeps radio waves within the sweep range determined based on the estimated position determined immediately before by the position estimation unit. Effect of the Invention

[0018] The present invention has the advantage of being able to improve the accuracy of estimating the position of a device worn on the body. [Brief description of the drawings]

[0019] [Figure 1] 1 is a diagram for explaining an overview of a communication system S. [Diagram 2] 1 is a diagram for explaining an overview of a communication system S. [Diagram 3] 1 is a diagram for explaining an overview of the processing flow in a communication system S. [Figure 4] 13 is a diagram showing an example of skeleton data generated by the information terminal 1. FIG. [Diagram 5]FIG. 1 is a diagram for explaining a radio wave propagation model. [Figure 6] FIG. 11 is a diagram for explaining a process of estimating a position. [Figure 7] 2 is a diagram showing configurations of an information terminal 1 and a relay device 2. FIG. [Figure 8] 4 is a flowchart showing a process flow in a communication system S. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] [Outline of Communication System S] 1 and 2 are diagrams for explaining an overview of a communication system S. The communication system S includes an information terminal 1 and a plurality of relay devices 2 attached to the body of a user U who uses the information terminal 1. As shown in Fig. 2, the information terminal 1 transmits and receives data to and from a base station 3 of a mobile phone network via the plurality of relay devices 2.

[0021] The multiple relay devices 2 are attached to, for example, the body of a user U. In the example shown in FIG. 1, the multiple relay devices 2 are attached to both wrists, both ankles, and the head of the user U. The relay devices 2 are devices that can communicate with the information terminal 1 and a base station 3 of a mobile phone network while attached to the human body. The multiple relay devices 2 transmit and receive data to and from the base station 3 using radio waves in a first frequency band (e.g., millimeter waves). The multiple relay devices 2 transmit and receive data to and from the information terminal 1 using radio waves in a second frequency band (e.g., terahertz waves).

[0022] Each of the multiple relay devices 2 transmits data received from the base station 3 to the information terminal 1, and transmits data received from the information terminal 1 to the base station 3. Since the information terminal 1 transmits and receives data to and from the base station 3 via the multiple relay devices 2, the information terminal 1 can transmit and receive data to and from the base station 3 faster than when the information terminal 1 transmits and receives data directly to and from the base station 3.

[0023] 2, the information terminal 1 can change the direction of the antenna in a time-division manner to transmit and receive radio waves to and from the multiple relay devices 2 in multiple different directions. However, if the frequency of the radio waves used when the information terminal 1 transmits and receives data to and from the multiple relay devices 2 is high, the radio waves have strong directivity. Therefore, the information terminal 1 needs to estimate the position of the relay device 2 with high accuracy and point the antenna toward the estimated position.

[0024] Therefore, in order to estimate the position of the relay device 2 with high accuracy, the information terminal 1 estimates the positions of the multiple relay devices 2 based on the position estimation radio waves received from the multiple relay devices 2. However, the radio waves used for communication between the information terminal 1 and the relay device 2 may be reflected by the body of the user U before reaching the destination, or may be diffracted by the body of the user U before reaching the destination. The propagation environment near the user U changes dynamically with the change in the posture of the user U. If the propagation environment changes with the change in the posture of the user U, the strength of the radio waves received by the information terminal 1 changes, causing an error in the position estimation.

[0025] In order to reduce the influence of the error, the information terminal 1 creates a radio wave propagation model corresponding to the radio wave propagation environment that changes according to the posture of the user U (i.e., the positions of the information terminal 1 and the multiple relay devices 2). The information terminal 1 corrects the characteristics of the received radio waves based on the radio wave propagation model, and estimates the positions of the multiple relay devices 2 using the corrected results, thereby improving the accuracy of the estimated position. The information terminal 1 controls the orientation of the antenna based on the estimated positions of the multiple relay devices 2, thereby maintaining a state in which communication is possible with the multiple relay devices 2 even if the positional relationship between the information terminal 1 and the multiple relay devices 2 changes due to human movement.

[0026] [Outline of processing flow in communication system S] Fig. 3 is a diagram for explaining an outline of a process flow in the communication system S. The process shown in Fig. 3 is executed by the information terminal 1 at a predetermined time interval. The predetermined time interval is set to be equal to or less than the time it takes for the position of the relay device 2 to move a distance that deteriorates the communication quality between the information terminal 1 and the relay device 2, and is, for example, 100 milliseconds.

[0027] The information terminal 1 estimates the latest skeletal state of the user U based on information (e.g., acceleration or angular velocity) indicating the moving state of the relay device 2 measured by inertial sensors of the information terminal 1 and the multiple relay devices 2, and the positions of the information terminal 1 and the multiple relay devices 2 identified at the immediately previous processing timing. At the first processing timing, the positions of the information terminal 1 and the multiple relay devices 2 at the immediately previous processing timing are assumed to be initial positions corresponding to, for example, a state in which the user U is standing upright.

[0028] The information terminal 1, for example, displays a screen for setting an initial position, and accepts input by the user U of the position where the information terminal 1 is held and the body parts where the multiple relay devices 2 are attached, thereby specifying the initial positions of the information terminal 1 and the multiple relay devices 2. The information terminal 1 accepts input of the name of an approximate position indicating a body part, such as "head," "right wrist," "left wrist," "right ankle," and "left ankle," and specifies the position stored in association with the accepted name as the initial position.

[0029] The information terminal 1 estimates the latest skeletal state of the user U by identifying the distance and direction moved by the information terminal 1 and the multiple relay devices 2 between the previous processing timing and the latest processing timing based on at least one of the acceleration and angular velocity (hereinafter referred to as "acceleration or angular velocity") indicated by the detection data input from the sensors of the information terminal 1 and the multiple relay devices 2 (S1 in FIG. 3). As an example, the information terminal 1 generates the latest skeletal data by inputting the positions of the information terminal 1 and the multiple relay devices 2 at the previous processing timing and the acceleration or angular velocity of the information terminal 1 and the multiple relay devices 2 into a skeleton estimation model that outputs skeleton data indicating the state of the skeleton after the change by inputting the distance and direction moved by multiple positions in the skeleton.

[0030] FIG. 4 is a diagram showing an example of skeletal data generated by the information terminal 1. As shown in FIG. 4, the skeletal data is data showing the positional relationship between the main bones and joints of a human being. The skeletal data may be image data, or may be data showing the position coordinates of feature points of a plurality of bones and joints. The skeletal estimation model updates the skeletal data by changing the positions or orientations of the affected joints and bones by moving a specified part in the previous skeletal data by a specified distance in a specified direction. The skeletal estimation model may be a program that generates skeletal data by being executed by a processor included in the information terminal 1, or may be a program that generates skeletal data by being executed by a processor external to the information terminal 1.

[0031] Returning to Fig. 3, the information terminal 1 creates a radio wave propagation model based on the generated skeleton data (S2 in Fig. 3). Fig. 5 is a diagram for explaining the radio wave propagation model. 5, for example, a direct wave, a reflected wave, and a diffracted wave are generated based on a radio wave transmitted from a relay device 2 to the information terminal 1, and these radio waves reach the information terminal 1. Then, the information terminal 1 creates a radio wave propagation model for each of the multiple relay devices 2, which indicates the intensity and phase of the reflected wave and the diffracted wave relative to the direct wave.

[0032] Furthermore, in parallel with creating the radio wave propagation model, the information terminal 1 may estimate the latest positions and orientations of the information terminal 1 and the multiple relay devices 2 based on the generated skeleton data (S3 in FIG. 3). The latest positions and orientations of the information terminal 1 and the multiple relay devices 2 are, for example, relative positions and relative orientations of each of the multiple relay devices 2 based on the position of the information terminal 1. The information terminal 1 estimates, as the latest position of the relay device 2, a position that is away from an approximate position, which is a skeleton position corresponding to the names of the mounting parts of the information terminal 1 and the multiple relay devices 2 in the latest skeleton data, by a distance between the approximate position and the positions where the information terminal 1 and the multiple relay devices 2 are mounted.

[0033] FIG. 6 is a diagram for explaining the process of estimating a position. The star H in FIG. 6 indicates the position of the head in the skeleton data. The hexagonal figure in FIG. 6 indicates the relay device 2 worn on the left ear. In this way, even if the wearing position is "head", the position of the head in the skeleton data is different from the position where the relay device 2 is actually worn. Therefore, the information terminal 1 calculates the position coordinates of the information terminal 1 and the multiple relay devices 2 by adding the difference amount based on the distance and direction to the wearing position of the information terminal 1 and the multiple relay devices 2 to the position coordinates corresponding to the wearing part in the skeleton data. As a result, the information terminal 1 can estimate the relative positions of the multiple relay devices 2 with respect to the information terminal 1 with high accuracy.

[0034] In parallel with estimating the relative positions of the multiple relay devices 2 based on the skeleton data, the information terminal 1 estimates the positions and orientations of the multiple relay devices 2 based on the created radio wave propagation model and radio waves for position estimation transmitted by each of the multiple relay devices 2 (S4 in FIG. 3). The radio waves for position estimation are radio waves transmitted with a predetermined strength in a predetermined direction of the relay device 2. The radio waves for position estimation may be radio waves including a signal (RSSI signal) indicating the strength of the radio waves received by the relay device 2 from the information terminal 1. The information terminal 1 estimates the orientation of the relay device 2 based on the direction in which the radio waves are received, and estimates the distance to the relay device 2 based on the strength of the received radio waves, thereby identifying the relative positions of the multiple relay devices 2 with respect to the information terminal 1.

[0035] However, when the state of radio waves received by the information terminal 1 from the relay devices 2 is poor, the accuracy of the position based on the radio waves estimated in S4 is low. Also, when the accuracy of the inertial sensors provided in the information terminal 1 and the relay devices 2 is low, the accuracy of the position based on the skeleton data estimated in S3 is low. Therefore, when the information terminal 1 estimates the positions (hereinafter may be referred to as "first estimated positions") and orientations (hereinafter may be referred to as "first estimated orientations") of the relay devices 2 based on the skeleton data, the information terminal 1 may specify the positions and orientations of the relay devices 2 based on the positions (hereinafter may be referred to as "second estimated positions") and orientations (hereinafter may be referred to as "second estimated orientations") of the relay devices 2 estimated based on the radio waves and the first estimated positions and first estimated orientations (S5 in FIG. 3).

[0036] Although details will be described later, the information terminal 1 calculates the positions and orientations of the multiple relay devices 2 based on a first reliability of the first estimated position and the first estimated orientation based on the skeleton data and a second reliability of the second estimated position and the second estimated orientation based on the radio waves. The information terminal 1 calculates the positions of the multiple relay devices 2 by taking a weighted average of the first estimated position and the second estimated position using weights corresponding to the first reliability of the first estimated position estimated based on the skeleton data and the second reliability of the second estimated position estimated based on the radio waves. The information terminal 1 may calculate the orientations of the multiple relay devices 2 by taking a weighted average of the first estimated orientation and the second estimated orientation using weights corresponding to the first reliability and the second reliability.

[0037] [Configuration of Information Terminal 1 and Relay Device 2] 7 is a diagram showing the configuration of the information terminal 1 and the relay device 2. The information terminal 1 has an inertial sensor 11, a receiving antenna 12, a receiving antenna 13, a storage unit 14, and a control unit 15. The control unit 15 has a skeleton estimation unit 151, a model creation unit 152, a first position estimation unit 153, a radio wave measurement unit 154, a second position estimation unit 155, a weight determination unit 156, and a position identification unit 157. The relay device 2 has an inertial sensor 21, a transmitting antenna 22, a data generation unit 23, and a transmitting antenna 24.

[0038] First, a description will be given of each unit of the relay device 2. The inertial sensor 21 is a sensor that can measure the acceleration or angular velocity of the relay device 2. The inertial sensor 21 is a detection unit that outputs detection data indicating the acceleration or angular velocity to the skeleton estimation unit 151. The inertial sensor 21 is, for example, a six-axis sensor, but may be another type of sensor.

[0039] The data generator 23 generates data to be transmitted by the relay device 2 to the information terminal 1. The data generator 23 generates, for example, data for position estimation. The transmitting antenna 24 transmits radio waves for position estimation including the data for position estimation.

[0040] 7, the relay device 2 may further include a communication circuit for transmitting and receiving data to and from the base station 3. The relay device 2 transmits data received from the information terminal 1 to the base station 3 via the communication circuit, and transmits data received from the base station 3 to the information terminal 1 via the communication circuit.

[0041] Next, a description will be given of each unit of the information terminal 1. The inertial sensor 11 is a sensor that can measure the acceleration or angular velocity of the information terminal 1. The inertial sensor 11 is, for example, a six-axis sensor, but may be another type of sensor. The inertial sensor 11 inputs data indicating the detected acceleration or angular velocity to the skeleton estimation unit 151.

[0042] The receiving antenna 12 is an antenna for receiving the acceleration or angular velocity of the relay device 2 from the relay device 2. The receiving antenna 12 is an antenna for receiving data indicating the acceleration or angular velocity transmitted from the relay device 2 by, for example, Bluetooth (registered trademark). The receiving antenna 12 functions as a data receiving unit that receives detection data from each of the multiple relay devices 2. The receiving antenna 12 inputs the received data indicating the acceleration or angular velocity to the skeleton estimation unit 151.

[0043] The receiving antenna 13 is an antenna for receiving data and radio waves for position estimation that the relay device 2 receives from the base station 3. The receiving antenna 13 supports frequencies higher than those that the receiving antenna 12 can receive, and can receive radio waves in the millimeter wave band or terahertz band, for example.

[0044] The storage unit 14 includes a storage medium such as a Read Only Memory (ROM) and a Random Access Memory (RAM). The storage unit 14 stores a program executed by the control unit 15. The storage unit 14 also stores the position and orientation of the relay device 2 in association with a relay device ID for identifying the multiple relay devices 2 and a time.

[0045] The control unit 15 has, for example, a CPU (Central Processing Unit). The control unit 15 executes the programs stored in the storage unit 14 to function as a skeleton estimation unit 151, a model creation unit 152, a first position estimation unit 153, a radio wave measurement unit 154, a second position estimation unit 155, a weight determination unit 156, and a position identification unit 157.

[0046] The skeleton estimation unit 151 generates skeleton data indicating the skeleton of a person wearing the multiple relay devices 2, which is estimated based on the multiple detection data corresponding to the multiple relay devices 2. The skeleton estimation unit 151 generates the skeleton data based on the positions of the multiple relay devices 2 most recently identified by the position identification unit 157 and the multiple detection data.

[0047] Specifically, the skeleton estimation unit 151 generates the latest skeleton data based on the previous positions and orientations of the information terminal 1 and the multiple relay devices 2, the previous skeleton data, the acceleration or angular velocity of the information terminal 1 detected by the inertial sensor 11, and the acceleration or angular velocities of the multiple relay devices 2 indicated by the multiple detection data corresponding to the multiple relay devices 2. The skeleton estimation unit 151 stores the generated skeleton data in the storage unit 14.

[0048] The skeleton estimation unit 151 generates skeleton data by inputting a plurality of detection data corresponding to a plurality of relay devices 2 into a skeleton estimation model that outputs skeleton data by inputting the plurality of detection data. As outlined above, the skeleton estimation unit 151 inputs the positions of the information terminal 1 and the plurality of relay devices 2 and the accelerations or angular velocities of the information terminal 1 and the plurality of relay devices 2 at the immediately previous processing timing into a skeleton estimation model that outputs skeleton data indicating the state of the skeleton after the change by inputting the distances and directions of movement of a plurality of positions in the skeleton. The skeleton estimation unit 151 generates the skeleton data output from the skeleton estimation model as the latest skeleton data.

[0049] It is assumed that an appropriate skeleton estimation model may differ depending on the type or number of the multiple relay devices 2. For example, a skeleton estimation model created by machine learning using detection data output from 10 sensors as training data when 10 relay devices 2 are attached may not be appropriate when only 5 relay devices 2 are attached. Also, a skeleton estimation model created by machine learning using detection data output from a 6-axis inertial sensor as training data may not be appropriate when the relay device 2 has a 3-axis inertial sensor.

[0050] Therefore, the skeleton estimation unit 151 may generate skeleton data using a skeleton estimation model selected from a plurality of skeleton estimation model candidates based on the type or the number of a plurality of relay devices 2. By the skeleton estimation unit 151 operating in this manner, it is possible to improve the accuracy of the latest skeleton data generated by the skeleton estimation unit 151.

[0051] The model creation unit 152 creates a plurality of radio wave propagation models between the information terminal 1 and each of the plurality of relay devices 2, based on the wearing position data indicating the terminal outline position where the information terminal 1 is worn and the plurality of device outline positions where each of the plurality of relay devices 2 is worn, and the skeleton data generated by the skeleton estimation unit 151. The terminal outline position and the device outline position are represented by names of body parts such as "right hand", "right wrist", "right ear", etc., and the model creation unit 152 receives the terminal outline position and the device outline position from, for example, the user U. The model creation unit 152 may acquire the positions designated by the user U in the body image as shown in FIG. 1 displayed on the screen of the information terminal 1 as the terminal outline position and the device outline position.

[0052] The model creation unit 152 estimates a position away from the approximate terminal position in the skeleton data by a distance corresponding to the approximate terminal position as the terminal position, which is the position of the information terminal 1, by, for example, the method described with reference to FIG. 6. The model creation unit 152 also estimates a position away from the approximate device position in the skeleton data by a distance corresponding to the approximate device position as the device position, which is the position of the relay device 2. The model creation unit 152 creates a radio wave propagation model based on the terminal position, the device position, and the terminal position and the position and orientation of the skeleton indicated by the skeleton data within a predetermined range from the device position. The model creation unit 152 creates a radio wave propagation model that reflects the influence of a reflected wave generated by a part of the body of a person corresponding to the skeleton data while a radio wave emitted from the position of the relay device 2 is propagated to the terminal position, or a diffracted wave generated by a part of the body of a person corresponding to the skeleton data while a radio wave emitted from the position of the relay device 2 is propagated to the terminal position.

[0053] The model creation unit 152 may create a model by machine learning using the terminal position, device position, and skeletal position and orientation, and a known propagation model as teacher data. In this case, the model creation unit 152 outputs a radio wave propagation model when it receives the terminal position, device position, and skeletal data specified at the immediately preceding processing timing.

[0054] The model creation unit 152 may specify the terminal position and the device position further based on a result of specifying a relationship between a body of a person appearing in captured image data of a person wearing multiple relay devices 2 and the positions of each of the multiple relay devices 2. The model creation unit 152 may acquire captured image data of a person wearing multiple relay devices 2 from an external device, and correct the estimated terminal position and device position based on a result of specifying a relationship between a body of the person appearing in the captured image data of the person wearing multiple relay devices 2 and the positions of each of the multiple relay devices 2.

[0055] For example, suppose that the user U sets the wearing position of the relay device 2 to "right wrist", but the relay device 2 is actually worn on the right elbow. In this case, when the model creation unit 152 determines that the approximate position is "right elbow" based on the captured image data, it corrects the device position corresponding to the right wrist to the position corresponding to the right elbow in the skeleton data. By operating in this manner, the model creation unit 152 can create an appropriate radio wave propagation model even if the user U makes a mistake in setting the wearing position or the wearing position of the relay device 2.

[0056] Incidentally, the intensities of the reflected waves and diffracted waves change based on the material and shape of the clothes worn by the user U. Therefore, the model creation unit 152 may create the radio wave propagation model based further on data indicating the clothes worn by a person wearing the multiple relay devices 2. The data indicating the clothes is set in advance by the user U and stored in the storage unit 14, for example. By configuring the model creation unit 152 in this way, it is possible to prevent the accuracy of the radio wave propagation model from being reduced due to the clothes of the user U.

[0057] Furthermore, the generation state of reflected waves and diffracted waves also changes depending on the presence or absence of objects around the user U and the shapes of the objects. Therefore, the model creation unit 152 may create a radio wave propagation model further based on the shapes and positions of objects captured in captured image data of the surroundings of the multiple relay devices 2. By configuring the model creation unit 152 in this way, it is possible to suppress differences in the accuracy of the radio wave propagation model depending on the surrounding conditions of the user U.

[0058] The first position estimation unit 153 determines a first estimated position, which is the position of each of the multiple relay devices 2, based on a plurality of detection data output by the plurality of inertial sensors 21. As an example, the first position estimation unit 153 determines a first estimated position, which is the position of each of the multiple relay devices 2, based on mounting position data indicating an approximate position where each of the multiple relay devices 2 is mounted and skeleton data generated by the skeleton estimation unit 151 based on the plurality of detection data. As described with reference to FIG. 6, for example, the first position estimation unit 153 estimates a position that is away from a position corresponding to the approximate position in the skeleton data by a distance corresponding to the approximate position as the first estimated position.

[0059] Specifically, the first position estimation unit 153 first acquires wearing part data indicating approximate positions of the wearing parts of the multiple relay devices 2 set by the user U via, for example, an operation device included in the information terminal 1. The wearing part data is data including information for identifying the wearing part, such as "right wrist", "left wrist", etc.

[0060] For example, as described above, the first position estimation unit 153 calculates the position coordinates of the information terminal 1 and the multiple relay devices 2 by adding a difference amount based on the distance and direction to the mounting positions of the information terminal 1 and the multiple relay devices 2 to the coordinates of the approximate positions in the skeleton data corresponding to the mounting positions of the information terminal 1 and the multiple relay devices 2. The first position estimation unit 153 notifies the position identification unit 157 of the determined first estimated position.

[0061] Since the skeleton data is generated by estimating the most probable skeleton state based on the accelerations or angular velocities measured in the multiple relay devices 2, even if there is an error in the accelerations or angular velocities measured in some of the relay devices 2, the influence of the error is unlikely to appear in the skeleton data. Therefore, the first position estimation unit 153 estimates the position of the relay device 2 using the skeleton data, thereby improving the accuracy of the first estimated position.

[0062] The first position estimation unit 153 may specify the orientation of each of the multiple relay devices 2 based on the skeleton data and the wearing position data indicating the approximate positions where each of the multiple relay devices 2 is worn. The first position estimation unit 153 specifies the orientation of each of the multiple relay devices 2 at the first estimated position based on, for example, the orientation of a bone near the first estimated position determined based on the wearing position data in the skeleton data created by the skeleton estimation unit 151.

[0063] The first position estimation unit 153 may obtain captured image data of a person wearing a plurality of relay devices 2 from an external device, and correct the first estimated position based on the result of identifying the relationship between the body of the person shown in the captured image data and the positions of the plurality of relay devices 2. For example, assume that the user U sets the wearing part of the relay device 2 to "right wrist", but the relay device 2 is actually worn on the right elbow. In such a case, when the first position estimation unit 153 determines that the approximate position is "right elbow" based on the captured image data, it corrects the first estimated position corresponding to the right wrist to the position corresponding to the right elbow in the skeleton data. By operating in this manner, the first position estimation unit 153 can correctly estimate the position of the relay device 2 even if the user U makes a mistake in setting the wearing position or makes a mistake in the wearing position of the relay device 2.

[0064] The first position estimation unit 153 may estimate the position of the relay device 2 without using the skeleton data. For example, the first position estimation unit 153 may determine the first estimated position by specifying a movement amount and a movement direction from the previous position of the relay device 2 specified by the position specifying unit 157 based on the acceleration or angular velocity of the relay device 2 output by the inertial sensor 21, and adding the specified movement amount in the specified movement direction to the previous position of the relay device 2.

[0065] The radio wave measurement unit 154 measures the radio wave state including the direction of arrival and strength of the radio waves received from each of the multiple relay devices 2, for example, by sweeping radio waves within a sweep range determined based on the positions of the multiple relay devices 2 identified by the second position estimation unit 155 or the position identification unit 157 at the previous processing timing.

[0066] Specifically, the radio wave measurement unit 154 controls the receiving antenna 13 to sequentially point the receiving antenna 13 to a plurality of directions within a sweep range corresponding to each of the plurality of second estimated positions among a plurality of directions to which the receiving antenna 13 can be directed. While sequentially pointing the receiving antenna 13 to a plurality of directions, the radio wave measurement unit 154 measures the intensity of the radio waves received by the receiving antenna 13 in each direction, and determines the direction with the greatest intensity as the arrival direction of the radio waves. The radio wave measurement unit 154 identifies the arrival direction and intensity of the radio waves transmitted by each of the plurality of relay devices 2 based on the relay device ID included in the position estimation signal transmitted by each of the plurality of relay devices 2.

[0067] The radio wave measurement unit 154 may determine the sweep range further based on the orientation of each of the multiple relay devices 2 identified at the immediately preceding processing timing by the second position estimation unit 155 or the position identification unit 157. When the orientation of the relay device 2 is such that a radio wave transmitted by the relay device 2 may be reflected by any part of the body and reach the information terminal 1, the radio wave measurement unit 154 determines the sweep range to include the direction from which the reflected wave arrives.

[0068] Based on the radio wave state, the second position estimation unit 155 determines second estimated positions, which are positions of the multiple relay devices 2. The second position estimation unit 155 determines that the relay device 2 is located at a distance corresponding to the strength of the radio wave measured by the radio wave measurement unit 154, in the arrival direction of the radio wave measured by the radio wave measurement unit 154, based on the position of the information terminal 1. The second position estimation unit 155 determines the second estimated positions of the multiple relay devices 2 based on a corrected direction and corrected strength obtained by correcting the arrival direction and strength of the radio wave for position estimation using a radio wave propagation model.

[0069] The second position estimation unit 155 determines the second estimated position, for example, by inputting the arrival direction and strength of the radio wave for position estimation into a radio wave propagation model and acquiring the corrected direction and corrected strength output from the radio wave propagation model. The second position estimation unit 155 may acquire the corrected direction and corrected strength by inputting the arrival direction and strength of the radio wave for position estimation into an arithmetic expression indicated by the radio wave propagation model. The second position estimation unit 155 specifies, for each of the multiple relay devices 2, a second estimated position of the relay device 2 based on the position of the information terminal 1. The second position estimation unit 155 notifies the weight determination unit 156 of the second estimated position.

[0070] The weight determination unit 156 determines a first weight for a first estimated position and a second weight for a second estimated position in a calculation executed by the position determination unit 157 to identify the position of each of the multiple relay devices 2, based on the first reliability of the detection data output by the inertial sensor 11 and the inertial sensor 21 or the second reliability of the radio wave state received by the receiving antenna 13. For example, the weight determination unit 156 determines that the second reliability is lower as the strength of the radio wave received by the receiving antenna 13 is weaker or the variance of the arrival direction of the radio wave is greater, and reduces the ratio of the second weight to the first weight.

[0071] On the other hand, the weight determination unit 156 may determine that the first reliability is lower as the difference between the temperature or humidity around the relay device 2 and a predetermined temperature or humidity suitable for the operation of the relay device 2 is larger, and may increase the ratio of the second weight to the first weight. In order to identify the temperatures or humidity around the multiple relay devices 2, the weight determination unit 156 may obtain environmental data indicating the temperature or humidity from a temperature sensor or humidity sensor (not shown), or may obtain environmental data input by the user U.

[0072] The weight determination unit 156 may increase the ratio of the second weight to the first weight when the acceleration or angular velocity output from the inertial sensor 11 and the inertial sensor 21 indicates an abnormal value outside a preset normal range. The weight determination unit 156 may decrease the ratio of the second weight to the first weight as the intensity of the radio wave received by the receiving antenna 12 decreases.

[0073] Note that the temperature may differ depending on the positions of the multiple relay devices 2, and the state of the propagation path of the radio waves from the relay device 2 to the information terminal 1 may differ. Therefore, the weight determination unit 156 may determine the first weight and the second weight for each position where the multiple relay devices 2 are attached. The weight determination unit 156 may determine the first weight and the second weight for each of the multiple relay devices 2.

[0074] The position identifying unit 157 performs a predetermined calculation including the first estimated position and the second estimated position as variables to identify the position of each of the multiple relay devices 2. For example, the position identifying unit 157 determines the average value or median value of the first estimated position and the second estimated position as the position of each of the multiple relay devices 2.

[0075] However, as described above, the first reliability of the first estimated position based on the detection data output by the inertial sensor 11 and the inertial sensor 21 may be low depending on the surrounding environment of the inertial sensor 11 and the inertial sensor 21. Also, the second reliability of the second estimated position based on the radio wave state received by the receiving antenna 13 may be low depending on the surrounding environment of the information terminal 1 and the relay device 2.

[0076] Therefore, the position identifying unit 157 functions as a Kalman filter in cooperation with the weight determining unit 156, for example, and identifies the positions of the multiple relay devices 2 with higher accuracy than the first estimated position and the second estimated position. The position identifying unit 157 identifies the position of each of the multiple relay devices 2 by calculating a weighted average of the first estimated position and the second estimated position based on the first weight and the second weight.

[0077] Specifically, the position identifying unit 157 identifies the position of each of the multiple relay devices 2 by, for example, the following arithmetic expression 1. When the weight determining unit 156 determines different first weights and second weights for each position at which the multiple relay devices 2 are attached, the position identifying unit 157 identifies the position of each of the positions at which the multiple relay devices 2 are attached by arithmetic expression 1. When the weight determining unit 156 determines different first weights and second weights for each of the multiple relay devices 2, the position identifying unit 157 identifies the position of each of the multiple relay devices 2 by arithmetic expression 1. [Equation 1] Position of relay device 2 = first estimated position x first weight + second estimated position x second weight However, the first weight + the second weight = 1

[0078] The position determination unit 157 thus performs a weighted average based on the reliability of the first estimated position based on the detection data of the inertial sensors 11 and 21, and the second estimated position based on radio waves, thereby improving the accuracy of the determined position of each of the multiple relay devices 2.

[0079] [Processing flow in communication system S] Fig. 8 is a flowchart showing the flow of processing in the communication system S. The flowchart shown in Fig. 8 starts from the point in time when the information terminal 1 and the relay device 2 are powered on.

[0080] The relay device 2 transmits detection data indicating the acceleration or angular velocity detected by the inertial sensor 21 (S21), and transmits radio waves for position estimation (S22). The information terminal 1 receives the detection data transmitted by the relay device 2 through the receiving antenna 12 (S11). The skeleton estimation unit 151 estimates a current skeleton state based on the previous skeleton state, the previous positions of the multiple relay devices 2, and the acceleration or angular velocity of the multiple relay devices 2 indicated by the detection data, and generates skeleton data (S12). Next, the first position estimation unit 153 estimates the positions of the multiple relay devices 2 (first estimated positions) based on the generated skeleton data (S13).

[0081] In parallel with the operation of the first position estimation unit 153, the model creation unit 152 creates a radio wave propagation model (S14). The radio wave measurement unit 154 receives radio waves for position estimation transmitted by the relay device 2 by sweeping radio waves in a direction based on the estimated positions of the multiple relay devices 2, and the second position estimation unit 155 estimates the position (second estimated position) of each of the multiple relay devices 2 based on the radio wave propagation model and the radio wave state indicated by the received radio waves for position estimation (S15).

[0082] The position identifying unit 157 identifies the position of each of the multiple relay devices 2 by weighted averaging using the first weight and the second weight (S16). The latest position identified by the position identifying unit 157 is used when the model creating unit 152 creates a radio wave propagation model and when the skeleton estimating unit 151 generates skeleton data at the next processing timing.

[0083] [Effects of communication system S] As described above, the information terminal 1 creates a plurality of radio wave propagation models between the information terminal 1 and each of the plurality of relay devices 2 based on the attachment position data indicating the terminal approximate position where the information terminal 1 is attached and the plurality of device approximate positions where each of the plurality of relay devices 2 is attached, and the skeleton data. Then, the second position estimation unit 155 determines the estimated positions of the plurality of relay devices 2 based on the corrected direction and corrected strength obtained by correcting the arrival direction and strength of the position estimation radio wave using the radio wave propagation model. By configuring the information terminal 1 in this way, even if the posture of the user U changes, the positions of the plurality of relay devices 2 are estimated taking into account the influence of the reflected wave and the diffracted wave according to the posture, thereby improving the estimation accuracy.

[0084] In the above description, the control unit 15 has the first position estimation unit 153, and the position identification unit 157 performs a weighted average of the first estimated position and the second estimated position. However, the control unit 15 may not have the first position estimation unit 153, and the position identification unit 157 may determine the second estimated position to be the position of the relay device 2.

[0085] Furthermore, this invention will make it possible to contribute to Goal 9 of the United Nations' Sustainable Development Goals (SDGs), which is to "build resilient infrastructure, promote industry, innovation and infrastructure."

[0086] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. For example, all or part of the device can be configured by distributing or integrating functionally or physically in any unit. In addition, new embodiments resulting from any combination of multiple embodiments are also included in the embodiments of the present invention. The effect of the new embodiment resulting from the combination combines the effect of the original embodiment. [Explanation of symbols]

[0087] 1 Information terminal 2. Relay Device 3 base station 11 Inertial Sensor 12 Receiving antenna 13 Receiving antenna 14 Storage section 15 Control section 21 Inertial Sensor 22 Transmitting Antenna 23 Data Generation Division 24 Transmitting Antenna 151 Skeleton Estimation Section 152 Model Creation Department 153 1st position estimation section 154 Radio wave measuring section 155 Second position estimation section 156 Decision Section 157 Location identification part

Claims

1. An information terminal and a plurality of relay devices capable of communicating with the information terminal and a base station of a mobile phone network while being worn on a person's body, each of the plurality of relay devices includes a detection unit that outputs detection data of at least one of an acceleration and an angular velocity of the relay device; The information terminal includes: a data receiving unit that receives the detection data from each of the plurality of relay devices; a skeleton estimation unit that generates skeleton data indicating skeletons of a person wearing the plurality of relay devices, the skeleton data being estimated based on a plurality of pieces of detection data corresponding to the plurality of relay devices; a model creation unit that creates a plurality of radio wave propagation models between the information terminal and each of the plurality of relay devices based on the attachment position data indicating an approximate terminal position where the information terminal is attached and a plurality of approximate device positions where the plurality of relay devices are attached, and the skeleton data; a radio wave measuring unit that measures a radio wave state including an arrival direction and an intensity of the radio wave received from each of the plurality of relay devices by sweeping the radio wave in a predetermined sweep range; a position estimation unit that determines estimated positions of the plurality of relay devices based on a corrected direction and a corrected intensity obtained by correcting the direction of arrival and the intensity using the radio wave propagation model; having The radio wave measurement unit sweeps radio waves in the sweep range determined based on the estimated position previously determined by the position estimation unit. Communication systems.

2. the model creation unit estimates a position away from the approximate terminal position in the skeleton data by a distance corresponding to the approximate terminal position as a terminal position that is the position of the information terminal, and estimates a position away from the approximate device position in the skeleton data by a distance corresponding to the approximate device position as a device position that is the position of the relay device, and creates the radio wave propagation model based on the terminal position, the device position, and a skeleton position and orientation indicated by the skeleton data within a predetermined range from the terminal position and the device position. The communication system according to claim 1 .

3. the model creation unit creates the radio wave propagation model in which an influence of a reflected wave generated when the radio wave emitted from the device position is reflected by a part of the person's body corresponding to the skeleton data while the radio wave is propagating to the terminal position, or an influence of a diffracted wave generated when the radio wave emitted from the device position goes around behind the part of the person's body corresponding to the skeleton data while the radio wave is propagating to the terminal position, is reflected. The communication system according to claim 2 .

4. the model creation unit specifies the terminal position and the device position based on a result of specifying a relationship between a body of a person shown in captured image data of a person wearing the plurality of relay devices and the positions of the plurality of relay devices. The communication system according to claim 2 .

5. the model creation unit creates the radio wave propagation model further based on data indicating clothing worn by a person wearing the plurality of relay devices. A communication system according to any one of claims 1 to 4.

6. the model creation unit creates the radio wave propagation model further based on shapes and positions of objects captured in captured image data of the surroundings of the plurality of relay devices. A communication system according to any one of claims 1 to 4.

7. the skeleton estimation unit generates the skeleton data by inputting the plurality of pieces of detection data corresponding to the plurality of relay devices to a skeleton estimation model that outputs the skeleton data by inputting the plurality of pieces of detection data. The communication system according to claim 1 .

8. the skeleton estimation unit generates the skeleton data by using the skeleton estimation model selected from a plurality of skeleton estimation model candidates based on a type or a number of the plurality of relay devices.

8. The communication system according to claim 7.

9. The position estimation unit further estimates an orientation of each of the plurality of relay devices; the radio wave measurement unit determines the sweep range further based on the orientations of the plurality of relay devices estimated by the position estimation unit. The communication system according to claim 1 .

10. An information terminal capable of communicating with a plurality of relay devices, a data receiving unit that receives detection data indicating at least one of an acceleration and an angular velocity of the relay device from each of the plurality of relay devices; a skeleton estimation unit that generates skeleton data indicating skeletons of a person wearing the plurality of relay devices, the skeleton data being estimated based on a plurality of pieces of detection data corresponding to the plurality of relay devices; a model creation unit that creates a plurality of radio wave propagation models between the information terminal and each of the plurality of relay devices based on the attachment position data indicating an approximate terminal position where the information terminal is attached and a plurality of approximate device positions where the plurality of relay devices are attached, and the skeleton data; a radio wave measuring unit that measures the direction of arrival and the strength of radio waves received from each of the plurality of relay devices by sweeping radio waves in a predetermined sweep range; a position estimation unit that determines estimated positions of the plurality of relay devices based on a corrected direction and a corrected intensity obtained by correcting the direction of arrival and the intensity using the radio wave propagation model; having The radio wave measurement unit sweeps radio waves in the sweep range determined based on the estimated position previously determined by the position estimation unit. Information terminal.

11. A program executed by a processor included in an information terminal capable of communicating with a plurality of relay devices, the program causing the processor to: a data receiving unit that receives detection data indicating at least one of an acceleration and an angular velocity of the relay device from each of the plurality of relay devices; a skeleton estimation unit that generates skeleton data indicating skeletons of a person wearing the plurality of relay devices, the skeleton data being estimated based on a plurality of pieces of detection data corresponding to the plurality of relay devices; a model creation unit that creates a plurality of radio wave propagation models between the information terminal and each of the plurality of relay devices based on the attachment position data indicating an approximate terminal position where the information terminal is attached and a plurality of approximate device positions where the plurality of relay devices are attached, and the skeleton data; a radio wave measuring unit that measures a radio wave state including an arrival direction and an intensity of the radio wave received from each of the plurality of relay devices by sweeping the radio wave in a predetermined sweep range; a position estimation unit that determines estimated positions of the plurality of relay devices based on a corrected direction and a corrected intensity obtained by correcting the direction of arrival and the intensity using the radio wave propagation model; Function as a The radio wave measurement unit sweeps radio waves in the sweep range determined based on the estimated position previously determined by the position estimation unit. program.

Images