Communication system, information terminal and program

Abstract

To improve accuracy of estimating a position of a device worn on a body.SOLUTION: An information terminal 1 has: a skeleton estimation unit 151 that generates skeleton data indicating the skeleton of a person wearing a plurality of relay devices 2; a first position estimation unit 152 that determines a first estimation position of each of the plurality of relay devices 2 based on the skeleton data and approximate positions where the plurality of relay devices 2 are worn; a second position estimation unit 154 that determines a second estimation position of each of the plurality of relay devices 2 based on a state of radio waves received from each of the plurality of relay devices 2; and a position identification unit 156 that identifies the position of each of the plurality of relay devices 2 by performing predetermined calculation including the first estimation position and the second estimation position as variables. The skeleton estimation unit generates the skeleton data based on the position of each of the plurality of relay devices most recently identified by the position identification unit and a plurality of pieces of detection data.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to a communication system, an information terminal, and a program. [Background technology]

[0002] 2. Description of the Related Art Conventionally, there is known a technique for correcting a human position estimated by triangulation using Wi-Fi (registered trademark), using information on a position estimated by an inertial sensor (see, for example, Non-Patent Document 1). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] A. Poulose, OS Eyobu and DS Han, "A Combined PDR and Wi-Fi Trilateration Algorithm for Indoor Localization," 2019 International Conference on Artificial Intelligence in Information and Communication Summary of the Invention [Problem to be solved by the invention]

[0004] It has been considered to realize high-speed data communication between a base station and an information terminal by relaying radio waves transmitted from a base station of a mobile phone network to the information terminal by multiple relay devices worn on the body using radio waves in a high frequency band of millimeter wave or higher. In order to maintain good communication between the information terminal and the relay device in an environment where the information terminal and the relay device move due to human movement, it is necessary for the information terminal to estimate the position of the relay device with high accuracy and receive radio waves in the direction corresponding to the estimated position. However, conventional technology has not been able to estimate the position of the relay device taking into account the movement of the body.

[0005] Therefore, the present invention has been made in consideration of these points, and aims to improve the accuracy of estimating the position of a device worn on the body. [Means for solving the problem]

[0006] A communication system according to a first aspect of the present invention includes an information terminal, and a plurality of relay devices that are attached to a human body and capable of communicating with the information terminal and a base station of a mobile phone network, each of the relay devices having a detection unit that outputs detection data indicating at least one of an acceleration and an angular velocity of the relay device. the information terminal includes a data receiving unit that receives the detection data from each of the plurality of relay devices; a skeleton estimation unit that generates skeleton data indicating a skeleton of a person wearing the plurality of relay devices, estimated based on a plurality of the detection data corresponding to the plurality of relay devices; a first position estimation unit that determines a first estimated position of each of the plurality of relay devices based on the skeletal data and wearing position data indicating an approximate position where each of the plurality of relay devices is worn; a radio wave measurement unit that measures a radio wave state including an arrival direction and an intensity of the radio wave received from each of the plurality of relay devices by sweeping radio waves in a sweep range determined based on the first estimated position; a second position estimation unit that determines a second estimated position of the plurality of relay devices based on the radio wave state; and a position identification unit that identifies a position of each of the plurality of relay devices by performing a predetermined calculation including the first estimated position and the second estimated position as variables,

[0007] The first position estimation unit may further estimate an orientation of each of the plurality of relay devices, and the radio wave measurement unit may determine the sweep range further based on the orientation of each of the plurality of relay devices estimated by the first position estimation unit.

[0008] The location determination unit may further include a weight determination unit that determines a first weight of the first estimated location and a second weight of the second estimated location in the calculation performed by the location determination unit based on a first reliability of the detection data or a second reliability of the radio wave state, and the location determination unit may determine a location of each of the multiple relay devices by taking a weighted average of the first estimated location and the second estimated location based on the first weight and the second weight.

[0009] The weight determination unit may determine that the first reliability is lower as the strength of the radio wave is weaker or the variance of the arrival direction of the radio wave is greater, and may decrease a ratio of the second weight to the first weight.

[0010] The weight determination unit may acquire environmental data indicating a temperature or humidity around the plurality of relay devices, and determine that the second reliability is lower as a difference between a temperature or humidity indicated by the acquired environmental data and a predetermined temperature or humidity suitable for operation of the relay device is larger, and increase a ratio of the second weight to the first weight.

[0011] The weight determination unit may determine the first weight and the second weight for each of positions at which the plurality of relay devices are mounted.

[0012] The skeleton estimation unit may generate the skeleton data by inputting a plurality of pieces of detection data corresponding to the plurality of relay devices to a skeleton estimation model that outputs the skeleton data by inputting the plurality of detection data.

[0013] The skeleton estimation unit may generate the skeleton data using the skeleton estimation model selected from a plurality of skeleton estimation model candidates based on a type or a number of the plurality of relay devices.

[0014] The first position estimation unit may estimate, as the first estimated position, a position that is separated from a position in the skeleton data that corresponds to the approximate position by a distance that corresponds to the approximate position.

[0015] The first position estimation unit may correct the first estimated position based on a result of identifying a relationship between a body of a person wearing the multiple relay devices and positions of each of the multiple relay devices, the body being captured in captured image data of the person.

[0016] an information terminal according to a second aspect of the present invention is an information terminal capable of communicating with a plurality of relay devices, the information terminal comprising: a data receiving unit that receives, from each of the plurality of relay devices, detection data indicating at least one of an acceleration and an angular velocity of the relay device; a skeleton estimation unit that generates skeleton data indicating a skeleton of a person wearing the plurality of relay devices, estimated based on a plurality of the detection data corresponding to the plurality of relay devices; a first position estimation unit that determines a first estimated position of each of the plurality of relay devices based on the skeletal data and wearing position data indicating approximate positions at which each of the plurality of relay devices is worn; a radio wave measurement unit that measures a radio wave state including an arrival direction and an intensity of radio waves received from each of the plurality of relay devices by sweeping radio waves in a sweep range determined based on the first estimated positions; a second position estimation unit that determines a second estimated position of the plurality of relay devices based on the radio wave state; and a position identification unit that identifies a position of each of the plurality of relay devices by performing a predetermined calculation including the first estimated position and the second estimated position as variables,

[0017] A program according to a third aspect of the present invention is a program executed by a processor included in an information terminal capable of communicating with a plurality of relay devices, the program being executed by the processor to cause the processor to include: a data receiving unit that receives detection data indicating at least one of an acceleration and an angular velocity of the relay device from each of the plurality of relay devices; a skeleton estimation unit that generates skeleton data indicating a skeleton of a person wearing the plurality of relay devices; a first position estimation unit that determines a first estimated position of each of the plurality of relay devices based on the skeleton data and wearing position data indicating approximate positions at which each of the plurality of relay devices is worn; the first estimated position and the second estimated position of the relay device based on the first estimated position; a second location estimation unit that determines second estimated positions of the relay devices based on the first estimated position; and a location identification unit that identifies the location of each of the relay devices by performing a predetermined calculation including the first estimated position and the second estimated position as variables, and the skeleton estimation unit generates the skeleton data based on the location of each of the relay devices most recently identified by the location identification unit and a plurality of the detection data. Effect of the Invention

[0018] The present invention has the advantage of being able to improve the accuracy of estimating the position of a device worn on the body. [Brief description of the drawings]

[0019] [Figure 1] 1 is a diagram for explaining an overview of a communication system S. [Diagram 2] 1 is a diagram for explaining an overview of a communication system S. [Diagram 3] 1 is a diagram for explaining an overview of the processing flow in a communication system S. [Figure 4] 13 is a diagram showing an example of skeleton data generated by the information terminal 1. FIG. [Diagram 5]FIG. 11 is a diagram for explaining a process of estimating a position. [Figure 6] 2 is a diagram showing configurations of an information terminal 1 and a relay device 2. FIG. [Figure 7] 4 is a flowchart showing a process flow in a communication system S. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] [Outline of Communication System S] 1 and 2 are diagrams for explaining an overview of a communication system S. The communication system S includes an information terminal 1 and a plurality of relay devices 2 attached to the body of a user U who uses the information terminal 1. As shown in Fig. 2, the information terminal 1 transmits and receives data to and from a base station 3 of a mobile phone network via the plurality of relay devices 2.

[0021] The multiple relay devices 2 are attached to, for example, the body of a user U. In the example shown in FIG. 1, the multiple relay devices 2 are attached to both wrists, both ankles, and the head of the user U. The relay devices 2 are devices that can communicate with the information terminal 1 and a base station 3 of a mobile phone network while attached to the human body. The multiple relay devices 2 transmit and receive data to and from the base station 3 using radio waves in a first frequency band (e.g., millimeter waves). The multiple relay devices 2 transmit and receive data to and from the information terminal 1 using radio waves in a second frequency band (e.g., terahertz waves).

[0022] Each of the multiple relay devices 2 transmits data received from the base station 3 to the information terminal 1, and transmits data received from the information terminal 1 to the base station 3. Since the information terminal 1 transmits and receives data to and from the base station 3 via the multiple relay devices 2, the information terminal 1 can transmit and receive data to and from the base station 3 faster than when the information terminal 1 transmits and receives data directly to and from the base station 3.

[0023] 2, the information terminal 1 can change the direction of the antenna in a time-division manner to transmit and receive radio waves to and from the multiple relay devices 2 in multiple different directions. However, if the frequency of the radio waves used when the information terminal 1 transmits and receives data to and from the multiple relay devices 2 is high, the radio waves have strong directivity. Therefore, the information terminal 1 needs to estimate the position of the relay device 2 with high accuracy and point the antenna toward the estimated position.

[0024] Therefore, in order to estimate the position of the relay device 2 with high accuracy, the information terminal 1 periodically estimates the skeletal state of the person wearing the multiple relay devices 2, and estimates the positions of the multiple relay devices 2 based on the estimation result. Then, the information terminal 1 controls the orientation of the antenna based on the estimated positions of the multiple relay devices 2, so that the information terminal 1 can maintain a state in which it can communicate with the multiple relay devices 2 even if the positional relationship between the information terminal 1 and the multiple relay devices 2 changes due to the movement of the person.

[0025] [Outline of processing flow in communication system S] Fig. 3 is a diagram for explaining an outline of a process flow in the communication system S. The process shown in Fig. 3 is executed by the information terminal 1 at a predetermined time interval. The predetermined time interval is set to be equal to or less than the time it takes for the position of the relay device 2 to move a distance that deteriorates the communication quality between the information terminal 1 and the relay device 2, and is, for example, 100 milliseconds.

[0026] The information terminal 1 estimates the latest skeletal state of the user U based on information (e.g., acceleration or angular velocity) indicating the moving state of the relay device 2 measured by inertial sensors of the information terminal 1 and the multiple relay devices 2, and the positions of the information terminal 1 and the multiple relay devices 2 identified at the immediately previous processing timing. At the first processing timing, the positions of the information terminal 1 and the multiple relay devices 2 at the immediately previous processing timing are assumed to be initial positions corresponding to, for example, a state in which the user U is standing upright.

[0027] The information terminal 1, for example, displays a screen for setting an initial position, and accepts input by the user U of the position where the information terminal 1 is held and the body parts where the multiple relay devices 2 are attached, thereby specifying the initial positions of the information terminal 1 and the multiple relay devices 2. The information terminal 1 accepts input of the name of an approximate position indicating a body part, such as "head," "right wrist," "left wrist," "right ankle," and "left ankle," and specifies the position stored in association with the accepted name as the initial position.

[0028] The information terminal 1 estimates the latest skeletal state of the user U by identifying the distance and direction moved by the information terminal 1 and the multiple relay devices 2 between the previous processing timing and the latest processing timing based on at least one of the acceleration and angular velocity (hereinafter referred to as "acceleration or angular velocity") indicated by the detection data input from the sensors of the information terminal 1 and the multiple relay devices 2 (S1 in FIG. 3). As an example, the information terminal 1 generates the latest skeletal data by inputting the positions of the information terminal 1 and the multiple relay devices 2 at the previous processing timing and the acceleration or angular velocity of the information terminal 1 and the multiple relay devices 2 into a skeleton estimation model that outputs skeleton data indicating the state of the skeleton after the change by inputting the distance and direction moved by multiple positions in the skeleton.

[0029] FIG. 4 is a diagram showing an example of skeletal data generated by the information terminal 1. As shown in FIG. 4, the skeletal data is data showing the positional relationship between the main bones and joints of a human being. The skeletal data may be image data, or may be data showing the position coordinates of feature points of a plurality of bones and joints. The skeletal estimation model updates the skeletal data by changing the positions or orientations of the affected joints and bones by moving a specified part in the previous skeletal data by a specified distance in a specified direction. The skeletal estimation model may be a program that generates skeletal data by being executed by a processor included in the information terminal 1, or may be a program that generates skeletal data by being executed by a processor external to the information terminal 1.

[0030] Returning to Fig. 3, the information terminal 1 estimates the latest positions and orientations of the information terminal 1 and the multiple relay devices 2 based on the generated skeleton data (S2 in Fig. 3). The latest positions and orientations of the information terminal 1 and the multiple relay devices 2 are, for example, relative positions and relative orientations of each of the multiple relay devices 2 based on the position of the information terminal 1. The information terminal 1 estimates, as the latest position of the relay device 2, a position that is away from an approximate position, which is a skeleton position corresponding to the names of the mounting parts of the information terminal 1 and the multiple relay devices 2 in the latest skeleton data, by a distance between the approximate position and the positions where the information terminal 1 and the multiple relay devices 2 are mounted.

[0031] FIG. 5 is a diagram for explaining the process of estimating a position. The star H in FIG. 5 indicates the position of the head in the skeleton data. The hexagonal figure in FIG. 5 indicates the relay device 2 worn on the left ear. In this way, even if the wearing position is "head", the position of the head in the skeleton data is different from the position where the relay device 2 is actually worn. Therefore, the information terminal 1 calculates the position coordinates of the information terminal 1 and the multiple relay devices 2 by adding the difference amount based on the distance and direction to the wearing position of the information terminal 1 and the multiple relay devices 2 to the position coordinates corresponding to the wearing part in the skeleton data. As a result, the information terminal 1 can estimate the relative positions of the multiple relay devices 2 with respect to the information terminal 1 with high accuracy.

[0032] In parallel with estimating the relative positions of the multiple relay devices 2 based on the skeleton data, the information terminal 1 estimates the positions and orientations of the multiple relay devices 2 based on radio waves for position estimation transmitted by each of the multiple relay devices 2 (S3 in FIG. 3). The radio waves for position estimation are radio waves transmitted in a predetermined direction of the relay device 2 with a predetermined strength. The radio waves for position estimation may be radio waves including a signal (RSSI signal) indicating the strength of the radio waves received by the relay device 2 from the information terminal 1. The information terminal 1 estimates the orientation of the relay device 2 based on the direction in which the radio waves are received, and estimates the distance to the relay device 2 based on the strength of the received radio waves, thereby identifying the relative positions of the multiple relay devices 2 with respect to the information terminal 1.

[0033] However, when the accuracy of the inertial sensors provided in the information terminal 1 and the multiple relay devices 2 is low, the accuracy of the position based on the skeleton data estimated in S2 is low. Also, when the state of the radio waves received by the information terminal 1 from the multiple relay devices 2 is poor, the accuracy of the position based on the radio waves estimated in S3 is low. Therefore, the information terminal 1 determines the positions and orientations of the multiple relay devices 2 based on the positions (hereinafter sometimes referred to as "first estimated positions") and orientations (hereinafter sometimes referred to as "first estimated orientations") of the multiple relay devices 2 estimated based on the skeleton data and the positions (hereinafter sometimes referred to as "second estimated positions") and orientations (hereinafter sometimes referred to as "second estimated orientations") of the multiple relay devices 2 estimated based on the radio waves (S4 in FIG. 3).

[0034] Although details will be described later, the information terminal 1 calculates the positions and orientations of the multiple relay devices 2 based on a first reliability of the first estimated position and the first estimated orientation based on the skeleton data and a second reliability of the second estimated position and the second estimated orientation based on the radio waves. The information terminal 1 calculates the positions of the multiple relay devices 2 by taking a weighted average of the first estimated position and the second estimated position using weights corresponding to the first reliability of the first estimated position estimated based on the skeleton data and the second reliability of the second estimated position estimated based on the radio waves. The information terminal 1 may calculate the orientations of the multiple relay devices 2 by taking a weighted average of the first estimated orientation and the second estimated orientation using weights corresponding to the first reliability and the second reliability.

[0035] [Configuration of Information Terminal 1 and Relay Device 2] 6 is a diagram showing the configuration of the information terminal 1 and the relay device 2. The information terminal 1 has an inertial sensor 11, a receiving antenna 12, a receiving antenna 13, a storage unit 14, and a control unit 15. The control unit 15 has a skeleton estimation unit 151, a first position estimation unit 152, a radio wave measurement unit 153, a second position estimation unit 154, a weight determination unit 155, and a position identification unit 156. The relay device 2 has an inertial sensor 21, a transmitting antenna 22, a data generation unit 23, and a transmitting antenna 24.

[0036] First, each unit of the relay device 2 will be described. The inertial sensor 21 is a sensor that can measure the acceleration or angular velocity of the relay device 2. The inertial sensor 21 is a detection unit that outputs detection data indicating the acceleration or angular velocity to the skeleton estimation unit 151. The inertial sensor 21 is, for example, a six-axis sensor, but may be another type of sensor. The transmitting antenna 22 transmits the detection data.

[0037] The data generator 23 generates data to be transmitted by the relay device 2 to the information terminal 1. The data generator 23 generates, for example, data for position estimation. The transmitting antenna 24 transmits radio waves for position estimation including the data for position estimation.

[0038] 6, the relay device 2 may further include a communication circuit for transmitting and receiving data to and from the base station 3. The relay device 2 transmits data received from the information terminal 1 to the base station 3 via the communication circuit, and transmits data received from the base station 3 to the information terminal 1 via the communication circuit.

[0039] Next, a description will be given of each unit of the information terminal 1. The inertial sensor 11 is a sensor that can measure the acceleration or angular velocity of the information terminal 1. The inertial sensor 11 is, for example, a six-axis sensor, but may be another type of sensor. The inertial sensor 11 inputs data indicating the detected acceleration or angular velocity to the skeleton estimation unit 151.

[0040] The receiving antenna 12 is an antenna for receiving the acceleration or angular velocity of the relay device 2 from the relay device 2. The receiving antenna 12 is an antenna for receiving data indicating the acceleration or angular velocity transmitted from the relay device 2 by, for example, Bluetooth (registered trademark). The receiving antenna 12 functions as a data receiving unit that receives detection data from each of the multiple relay devices 2. The receiving antenna 12 inputs the received data indicating the acceleration or angular velocity to the skeleton estimation unit 151.

[0041] The receiving antenna 13 is an antenna for receiving data and radio waves for position estimation that the relay device 2 receives from the base station 3. The receiving antenna 13 supports frequencies higher than those that the receiving antenna 12 can receive, and can receive radio waves in the millimeter wave band or terahertz band, for example.

[0042] The storage unit 14 includes a storage medium such as a Read Only Memory (ROM) and a Random Access Memory (RAM). The storage unit 14 stores a program executed by the control unit 15. The storage unit 14 also stores the position and orientation of the relay device 2 in association with a relay device ID for identifying the multiple relay devices 2 and a time.

[0043] The control unit 15 has, for example, a CPU (Central Processing Unit). The control unit 15 executes the programs stored in the storage unit 14 to function as a skeleton estimation unit 151, a first position estimation unit 152, a radio wave measurement unit 153, a second position estimation unit 154, a weight determination unit 155, and a position identification unit 156.

[0044] The skeleton estimation unit 151 generates skeleton data indicating the skeleton of a person wearing the multiple relay devices 2, which is estimated based on the multiple detection data corresponding to the multiple relay devices 2. The skeleton estimation unit 151 generates the skeleton data based on the positions of the multiple relay devices 2 most recently identified by the position identification unit 156 and the multiple detection data.

[0045] Specifically, the skeleton estimation unit 151 generates the latest skeleton data based on the previous positions and orientations of the information terminal 1 and the multiple relay devices 2, the previous skeleton data, the acceleration or angular velocity of the information terminal 1 detected by the inertial sensor 11, and the acceleration or angular velocities of the multiple relay devices 2 indicated by the multiple detection data corresponding to the multiple relay devices 2. The skeleton estimation unit 151 stores the generated skeleton data in the storage unit 14.

[0046] The skeleton estimation unit 151 generates skeleton data by inputting a plurality of detection data corresponding to a plurality of relay devices 2 into a skeleton estimation model that outputs skeleton data by inputting the plurality of detection data. As outlined above, the skeleton estimation unit 151 inputs the positions of the information terminal 1 and the plurality of relay devices 2 and the accelerations or angular velocities of the information terminal 1 and the plurality of relay devices 2 at the immediately previous processing timing into a skeleton estimation model that outputs skeleton data indicating the state of the skeleton after the change by inputting the distances and directions of movement of a plurality of positions in the skeleton. The skeleton estimation unit 151 generates the skeleton data output from the skeleton estimation model as the latest skeleton data.

[0047] It is assumed that an appropriate skeleton estimation model may differ depending on the type or number of the multiple relay devices 2. For example, a skeleton estimation model created by machine learning using detection data output from 10 sensors as training data when 10 relay devices 2 are attached may not be appropriate when only 5 relay devices 2 are attached. Also, a skeleton estimation model created by machine learning using detection data output from a 6-axis inertial sensor as training data may not be appropriate when the relay device 2 has a 3-axis inertial sensor.

[0048] Therefore, the skeleton estimation unit 151 may generate skeleton data using a skeleton estimation model selected from a plurality of skeleton estimation model candidates based on the type or the number of a plurality of relay devices 2. By the skeleton estimation unit 151 operating in this manner, it is possible to improve the accuracy of the latest skeleton data generated by the skeleton estimation unit 151.

[0049] The first position estimation unit 152 determines a first estimated position, which is the position of each of the multiple relay devices 2, based on a plurality of detection data output by the plurality of inertial sensors 21. As an example, the first position estimation unit 152 determines a first estimated position, which is the position of each of the multiple relay devices 2, based on mounting position data indicating an approximate position where each of the multiple relay devices 2 is mounted and skeleton data generated by the skeleton estimation unit 151 based on the plurality of detection data. For example, as described with reference to FIG. 5, the first position estimation unit 152 estimates a position that is away from a position corresponding to the approximate position in the skeleton data by a distance corresponding to the approximate position as the first estimated position.

[0050] Specifically, the first position estimation unit 152 first acquires wearing part data indicating approximate positions of the wearing parts of the multiple relay devices 2 set by the user U via, for example, an operation device included in the information terminal 1. The wearing part data is data including information for identifying the wearing parts, such as "right wrist", "left wrist", etc.

[0051] As described above, for example, the first position estimation unit 152 calculates the position coordinates of the information terminal 1 and the multiple relay devices 2 by adding a difference amount based on the distance and direction to the mounting positions of the information terminal 1 and the multiple relay devices 2 to the coordinates of the approximate positions in the skeleton data corresponding to the mounting positions of the information terminal 1 and the multiple relay devices 2. The first position estimation unit 152 notifies the radio wave measurement unit 153 and the position identification unit 156 of the determined first estimated position.

[0052] Since the skeleton data is generated by estimating the most probable skeleton state based on the accelerations or angular velocities measured in a plurality of relay devices 2, even if there is an error in the accelerations or angular velocities measured in some of the relay devices 2, the influence of the error is unlikely to appear in the skeleton data. Therefore, the first position estimation unit 152 estimates the position of the relay device 2 using the skeleton data, thereby improving the accuracy of the first estimated position.

[0053] The first position estimation unit 152 may specify the orientation of each of the multiple relay devices 2 based on the skeleton data and the wearing position data indicating the approximate positions where each of the multiple relay devices 2 is worn. The first position estimation unit 152 specifies the orientation of each of the multiple relay devices 2 at the first estimated position based on, for example, the orientation of a bone near the first estimated position determined based on the wearing position data in the skeleton data created by the skeleton estimation unit 151.

[0054] The first position estimation unit 152 may obtain captured image data of a person wearing a plurality of relay devices 2 from an external device, and correct the first estimated position based on the result of identifying the relationship between the body of the person shown in the captured image data and the positions of the plurality of relay devices 2. For example, assume that the user U sets the wearing part of the relay device 2 to "right wrist", but the relay device 2 is actually worn on the right elbow. In such a case, when the first position estimation unit 152 determines that the approximate position is "right elbow" based on the captured image data, it corrects the first estimated position corresponding to the right wrist to the position corresponding to the right elbow in the skeleton data. By operating in this manner, the first position estimation unit 152 can correctly estimate the position of the relay device 2 even if the user U makes a mistake in setting the wearing position or makes a mistake in the wearing position of the relay device 2.

[0055] The first position estimation unit 152 may estimate the position of the relay device 2 without using the skeleton data. For example, the first position estimation unit 152 may determine the first estimated position by determining a movement amount and a movement direction from the previous position of the relay device 2 specified by the position specifying unit 156 based on the acceleration or angular velocity of the relay device 2 output by the inertial sensor 21, and adding the specified movement amount in the specified movement direction to the previous position of the relay device 2.

[0056] The radio wave measurement unit 153 sweeps radio waves in a sweep range determined based on the first estimated position, thereby measuring radio wave conditions including the arrival direction and strength of radio waves received from each of the multiple relay devices 2. Specifically, the radio wave measurement unit 153 controls the receiving antenna 13 to sequentially orient the receiving antenna 13 to multiple directions within the sweep range corresponding to each of the multiple first estimated positions, among multiple directions to which the receiving antenna 13 can be oriented. While sequentially orienting the receiving antenna 13 to multiple directions, the radio wave measurement unit 153 measures the strength of radio waves received by the receiving antenna 13 in each direction, and determines the direction with the greatest strength as the arrival direction of the radio waves. The radio wave measurement unit 153 identifies the arrival direction and strength of radio waves transmitted by each of the multiple relay devices 2 based on the relay device ID included in the position estimation signal transmitted by each of the multiple relay devices 2.

[0057] The radio wave measurement unit 153 may determine the sweep range further based on the orientation of each of the multiple relay devices 2 estimated by the first position estimation unit 152. When the orientation of the relay device 2 is such that a radio wave transmitted by the relay device 2 may be reflected by any part of the body and reach the information terminal 1, the radio wave measurement unit 153 determines the sweep range to include the direction from which the reflected wave arrives.

[0058] The second position estimation unit 154 determines second estimated positions, which are the positions of the multiple relay devices 2, based on the radio wave state. The second position estimation unit 154 determines that the relay device 2 is located at a distance corresponding to the strength of the radio waves measured by the radio wave measurement unit 153, in the arrival direction of the radio waves measured by the radio wave measurement unit 153, based on the position of the information terminal 1. The second position estimation unit 154 specifies, for each of the multiple relay devices 2, a second estimated position of the relay device 2 based on the position of the information terminal 1. The second position estimation unit 154 notifies the weight determination unit 155 of the second estimated position.

[0059] The weight determination unit 155 determines a first weight for a first estimated position and a second weight for a second estimated position in a calculation executed by the position determination unit 156 to identify the position of each of the multiple relay devices 2, based on the first reliability of the detection data output by the inertial sensor 11 and the inertial sensor 21 or the second reliability of the radio wave state received by the receiving antenna 13. For example, the weight determination unit 155 determines that the second reliability is lower as the strength of the radio wave received by the receiving antenna 13 is weaker or the variance of the arrival direction of the radio wave is greater, and reduces the ratio of the second weight to the first weight.

[0060] On the other hand, the weight determination unit 155 may determine that the first reliability is lower as the difference between the temperature or humidity around the relay device 2 and a predetermined temperature or humidity suitable for the operation of the relay device 2 is larger, and may increase the ratio of the second weight to the first weight. In order to identify the temperatures or humidity around the multiple relay devices 2, the weight determination unit 155 may obtain environmental data indicating the temperature or humidity from a temperature sensor or humidity sensor (not shown), or may obtain environmental data input by the user U.

[0061] The weight determination unit 155 may increase the ratio of the second weight to the first weight when the acceleration or angular velocity output from the inertial sensor 11 and the inertial sensor 21 indicates an abnormal value outside a preset normal range. The weight determination unit 155 may decrease the ratio of the second weight to the first weight as the intensity of the radio wave received by the receiving antenna 12 decreases.

[0062] Note that the temperature may differ depending on the positions of the multiple relay devices 2, and the state of the propagation path of the radio waves from the relay device 2 to the information terminal 1 may differ. Therefore, the weight determination unit 155 may determine the first weight and the second weight for each position where the multiple relay devices 2 are attached. The weight determination unit 155 may determine the first weight and the second weight for each of the multiple relay devices 2.

[0063] The position identifying unit 156 performs a predetermined calculation including the first estimated position and the second estimated position as variables to identify the position of each of the multiple relay devices 2. For example, the position identifying unit 156 determines the average value or median value of the first estimated position and the second estimated position as the position of each of the multiple relay devices 2.

[0064] However, as described above, the first reliability of the first estimated position based on the detection data output by the inertial sensor 11 and the inertial sensor 21 may be low depending on the surrounding environment of the inertial sensor 11 and the inertial sensor 21. Also, the second reliability of the second estimated position based on the radio wave state received by the receiving antenna 13 may be low depending on the surrounding environment of the information terminal 1 and the relay device 2.

[0065] Therefore, the position identifying unit 156 functions as a Kalman filter in cooperation with the weight determining unit 155, for example, and identifies the positions of the multiple relay devices 2 with higher accuracy than the first estimated position and the second estimated position. The position identifying unit 156 identifies the position of each of the multiple relay devices 2 by calculating a weighted average of the first estimated position and the second estimated position based on the first weight and the second weight.

[0066] Specifically, the position identifying unit 156 identifies the position of each of the multiple relay devices 2 by, for example, the following arithmetic expression 1. When the weight determining unit 155 determines different first weights and second weights for each position at which the multiple relay devices 2 are attached, the position identifying unit 156 identifies the position for each position at which the multiple relay devices 2 are attached by arithmetic expression 1. When the weight determining unit 155 determines different first weights and second weights for each of the multiple relay devices 2, the position identifying unit 156 identifies the position for each of the multiple relay devices 2 by arithmetic expression 1. [Equation 1] Position of relay device 2 = first estimated position x first weight + second estimated position x second weight However, the first weight + the second weight = 1

[0067] The position determination unit 156 thus performs a weighted average based on the reliability of the first estimated position based on the detection data of the inertial sensors 11 and 21, and the second estimated position based on radio waves, thereby improving the accuracy of the determined position of each of the multiple relay devices 2.

[0068] [Processing flow in communication system S] Fig. 7 is a flowchart showing the flow of processing in the communication system S. The flowchart shown in Fig. 7 starts from the point in time when the information terminal 1 and the relay device 2 are powered on.

[0069] The relay device 2 transmits detection data indicating the acceleration or angular velocity detected by the inertial sensor 21 (S21), and transmits radio waves for position estimation (S22). The information terminal 1 receives the detection data transmitted by the relay device 2 through the receiving antenna 12 (S11). The skeleton estimation unit 151 estimates a current skeleton state based on the previous skeleton state, the previous positions of the multiple relay devices 2, and the acceleration or angular velocity of the multiple relay devices 2 indicated by the detection data, and generates skeleton data (S12). Next, the first position estimation unit 152 estimates the positions of the multiple relay devices 2 (first estimated positions) based on the generated skeleton data (S13).

[0070] The radio wave measurement unit 153 receives radio waves for position estimation transmitted by the relay device 2 by sweeping radio waves in a direction based on the estimated positions of the multiple relay devices 2 (S14), and the second position estimation unit 154 estimates the position (second estimated position) of each of the multiple relay devices 2 based on the radio wave state indicated by the received radio waves for position estimation (S15).

[0071] The weight determination unit 155 calculates a first reliability of the first estimated position estimated in S13 based on the temperature or humidity around the multiple relay devices 2, and calculates a second reliability of the second estimated position estimated in S15 based on the state of the radio waves received by the receiving antenna 13. The weight determination unit 155 determines a first weight corresponding to the first estimated position estimated in S13 and a second weight corresponding to the second estimated position estimated in S15 based on the first reliability and the second reliability (S16).

[0072] The position identifying unit 156 identifies the position of each of the multiple relay devices 2 by weighted averaging using the first weight and the second weight (S17). The latest position identified by the position identifying unit 156 is used when the skeleton estimation unit 151 estimates the skeleton state at the next processing timing.

[0073] [Effects of communication system S] As described above, the first position estimation unit 152 determines the first estimated position of each of the multiple relay devices 2 based on the mounting position data indicating the approximate position where each of the multiple relay devices 2 is mounted and the skeleton data generated based on the acceleration or angular velocity of the multiple relay devices 2. Furthermore, the second position estimation unit 154 determines the second estimated position of each of the multiple relay devices 2 based on the state of the position estimation radio waves transmitted by the multiple relay devices 2. Then, the position identification unit 156 identifies the position of each of the multiple relay devices 2 by performing a predetermined calculation including the first estimated position and the second estimated position as variables.

[0074] Since the information terminal 1 is configured in this manner, the positions of the multiple relay devices 2 are identified by taking into consideration the results of estimating the positions of the multiple relay devices 2 using multiple methods, so that the information terminal 1 can identify the positions of the multiple relay devices 2 with higher accuracy than in the case where the positions of the multiple relay devices 2 are estimated using only one method.

[0075] Furthermore, this invention will make it possible to contribute to Goal 9 of the United Nations' Sustainable Development Goals (SDGs), which is to "build resilient infrastructure, promote industry, innovation and infrastructure."

[0076] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. For example, all or part of the device can be configured by distributing or integrating functionally or physically in any unit. In addition, new embodiments resulting from any combination of multiple embodiments are also included in the embodiments of the present invention. The effect of the new embodiment resulting from the combination combines the effect of the original embodiment. [Explanation of symbols]

[0077] 1 Information terminal 2. Relay Device 3 base station 11 Inertial Sensors 12 Receiving antenna 13 Receiving antenna 14 Storage section 15 Control section 21 Inertial Sensor 22 Transmitting Antenna 23 Data Generation Division 24 Transmitting Antenna 151 Skeleton Estimation Section 152 1st position estimation section 153 Radio wave measurement unit 154 Second position estimation section 155 Decision Section 156 Location identification part

Claims

1. An information terminal and a plurality of relay devices capable of communicating with the information terminal and a base station of a mobile phone network while being worn on a person's body, each of the plurality of relay devices includes a detection unit that outputs detection data indicating at least one of an acceleration and an angular velocity of the relay device; The information terminal includes: a data receiving unit that receives the detection data from each of the plurality of relay devices; a skeleton estimation unit that generates skeleton data indicating skeletons of a person wearing the plurality of relay devices, the skeleton data being estimated based on a plurality of pieces of detection data corresponding to the plurality of relay devices; a first position estimator that determines a first estimated position of each of the plurality of relay devices based on mounting position data indicating an approximate position where each of the plurality of relay devices is mounted and the skeleton data; a radio wave measurement unit that measures a radio wave state including an arrival direction and an intensity of radio waves received from each of the plurality of relay devices by sweeping radio waves in a sweep range determined based on the first estimated position; a second location estimator that determines second estimated locations of the plurality of relay devices based on the radio wave conditions; a location identification unit that identifies a location of each of the plurality of relay devices by performing a predetermined calculation including the first estimated location and the second estimated location as variables; having The skeleton estimation unit generates the skeleton data based on the positions of the plurality of relay devices most recently identified by the position identification unit and a plurality of pieces of the detection data.

2. The first position estimation unit further estimates an orientation of each of the plurality of relay devices; the radio wave measurement unit determines the sweep range further based on the orientations of the plurality of relay devices estimated by the first position estimation unit. The communication system according to claim 1 .

3. a weight determination unit that determines a first weight of the first estimated position and a second weight of the second estimated position in the calculation executed by the position identification unit, based on a first reliability of the detection data or a second reliability of the radio wave state; the position identification unit identifies a position of each of the plurality of relay devices by performing a weighted average of the first estimated position and the second estimated position based on the first weight and the second weight. The communication system according to claim 1 .

4. the weight determination unit determines that the first reliability is lower as the strength of the radio wave is lower or as the variance of the arrival direction of the radio wave is larger, and reduces a ratio of the second weight to the first weight. The communication system according to claim 3.

5. the weight determination unit acquires environmental data indicating a temperature or humidity around the plurality of relay devices, and determines that the second reliability is lower as a difference between a temperature or humidity indicated by the acquired environmental data and a predetermined temperature or humidity suitable for an operation of the relay device increases, and increases a ratio of the second weight to the first weight. The communication system according to claim 3.

6. the weight determination unit determines the first weight and the second weight for each position at which the plurality of relay devices are mounted. The communication system according to claim 3.

7. the skeleton estimation unit generates the skeleton data by inputting the plurality of pieces of detection data corresponding to the plurality of relay devices to a skeleton estimation model that outputs the skeleton data by inputting the plurality of pieces of detection data. The communication system according to claim 1 .

8. the skeleton estimation unit generates the skeleton data by using the skeleton estimation model selected from a plurality of skeleton estimation model candidates based on a type or a number of the plurality of relay devices.

8. The communication system according to claim 7.

9. the first position estimation unit estimates, as the first estimated position, a position that is separated from a position in the skeleton data that corresponds to the approximate position by a distance that corresponds to the approximate position; The communication system according to claim 1 .

10. the first position estimation unit corrects the first estimated position based on a result of specifying a relationship between a body of a person appearing in captured image data obtained by capturing an image of a person wearing the plurality of relay devices and positions of the plurality of relay devices; The communication system according to claim 1 .

11. An information terminal capable of communicating with a plurality of relay devices, a data receiving unit that receives detection data indicating at least one of an acceleration and an angular velocity of the relay device from each of the plurality of relay devices; a skeleton estimation unit that generates skeleton data indicating skeletons of a person wearing the plurality of relay devices, the skeleton data being estimated based on a plurality of pieces of detection data corresponding to the plurality of relay devices; a first position estimator that determines a first estimated position of each of the plurality of relay devices based on mounting position data indicating an approximate position where each of the plurality of relay devices is mounted and the skeleton data; a radio wave measurement unit that measures a radio wave state including an arrival direction and an intensity of radio waves received from each of the plurality of relay devices by sweeping radio waves in a sweep range determined based on the first estimated position; a second location estimator that determines second estimated locations of the plurality of relay devices based on the radio wave conditions; a location identification unit that identifies a location of each of the plurality of relay devices by performing a predetermined calculation including the first estimated location and the second estimated location as variables; having the skeleton estimation unit generates the skeleton data based on the positions of the plurality of relay devices most recently identified by the position identification unit and the plurality of pieces of detection data. Information terminal.

12. A program executed by a processor included in an information terminal capable of communicating with a plurality of relay devices, the program causing the processor to: a data receiving unit that receives detection data indicating at least one of an acceleration and an angular velocity of the relay device from each of the plurality of relay devices; a skeleton estimation unit that generates skeleton data indicating a skeleton of a person wearing the plurality of relay devices; a first position estimator that determines a first estimated position of each of the plurality of relay devices based on mounting position data indicating an approximate position where each of the plurality of relay devices is mounted and the skeleton data; a radio wave measurement unit that measures a radio wave state including an arrival direction and an intensity of radio waves received from each of the plurality of relay devices by sweeping radio waves in a sweep range determined based on the first estimated position; a second location estimator that determines second estimated locations of the plurality of relay devices based on the radio wave conditions; a location identification unit that identifies a location of each of the plurality of relay devices by performing a predetermined calculation including the first estimated location and the second estimated location as variables; Functioning as a the skeleton estimation unit generates the skeleton data based on the positions of the plurality of relay devices most recently identified by the position identification unit and the plurality of pieces of detection data. program.

Images