Communication systems, information terminals, and programs

Abstract

To improve accuracy of estimating a position of a device worn on a body.SOLUTION: An information terminal 1 has: a skeleton estimation unit 151 that generates skeleton data indicating the skeleton of a person wearing a plurality of relay devices 2; a first position estimation unit 152 that determines a first estimation position of each of the plurality of relay devices 2 based on the skeleton data and approximate positions where the plurality of relay devices 2 are worn; a second position estimation unit 154 that determines a second estimation position of each of the plurality of relay devices 2 based on a state of radio waves received from each of the plurality of relay devices 2; and a position identification unit 156 that identifies the position of each of the plurality of relay devices 2 by performing predetermined calculation including the first estimation position and the second estimation position as variables. The skeleton estimation unit generates the skeleton data based on the position of each of the plurality of relay devices most recently identified by the position identification unit and a plurality of pieces of detection data.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The present invention relates to a communication system, an information terminal, and a program.

Background Art

[0002] Conventionally, a technique is known for correcting the position of a human estimated by three-point measurement using Wi-Fi (registered trademark) with the information on the position estimated by an inertial sensor (see, for example, Non-Patent Document 1).

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] It has been considered to realize high-speed data communication between a base station and an information terminal by relaying radio waves transmitted from a base station of a mobile phone network by a plurality of relay devices worn on the body to the information terminal by radio waves in a high-frequency band of millimeter wave band or higher. In order to maintain a good communication state between the information terminal and the relay device in an environment where the information terminal and the relay device move due to human movement, it is necessary for the information terminal to accurately estimate the position of the relay device and receive radio waves in the direction corresponding to the estimated position. However, in the conventional technology, it has been impossible to estimate the position of the relay device in consideration of the movement of the body.

[0005] Therefore, the present invention has been made in view of these points, and aims to improve the accuracy of estimating the position of a device worn on the body. [Means for solving the problem]

[0006] A communication system according to a first aspect of the present invention comprises an information terminal and a plurality of relay devices that are worn on a person's body and can communicate with the information terminal and a base station of a mobile phone network. Each of the plurality of relay devices has a detection unit that outputs detection data indicating at least one of the acceleration and angular velocity of the relay device. The information terminal includes: a data receiving unit that receives the detection data from each of the plurality of relay devices; a skeleton estimation unit that generates skeleton data indicating the skeleton of a person to whom the plurality of relay devices are attached, estimated based on a plurality of the detection data corresponding to the plurality of relay devices; a first position estimation unit that determines a first estimated position for each of the plurality of relay devices based on attachment position data indicating the approximate position where each of the plurality of relay devices is attached, and the skeleton data; a radio wave measurement unit that measures the radio wave state, including the direction of arrival and intensity of radio waves received from each of the plurality of relay devices, by sweeping radio waves within a sweep range determined based on the first estimated position; a second position estimation unit that determines a second estimated position for the plurality of relay devices based on the radio wave state; and a position identification unit that identifies the position of each of the plurality of relay devices by performing a predetermined calculation that includes the first estimated position and the second estimated position as variables. The skeleton estimation unit generates the skeleton data based on the position of each of the plurality of relay devices identified immediately before by the position identification unit and a plurality of the detection data.

[0007] The first position estimation unit may further estimate the orientation of each of the plurality of relay devices, and the radio wave measurement unit may determine the sweep range based on the orientation of each of the plurality of relay devices estimated by the first position estimation unit.

[0008] The system further includes a weight determination unit that determines a first weight for the first estimated position and a second weight for the second estimated position in the calculation performed by the position determination unit based on a first confidence level of the detection data or a second confidence level of the radio wave state, and the position determination unit may determine the position of each of the plurality of relay devices by weighting the first estimated position and the second estimated position based on the first weight and the second weight.

[0009] The weight determination unit may determine that the lower the intensity of the radio waves, or the greater the dispersion in the direction of arrival of the radio waves, the lower the first confidence level, and may reduce the ratio of the second weight to the first weight.

[0010] The weight determination unit may acquire environmental data indicating the temperature or humidity around the multiple relay devices, and may determine that the second reliability is lower if the difference between the temperature or humidity indicated by the acquired environmental data and a predetermined temperature or humidity suitable for the operation of the relay devices is large, and increase the ratio of the second weight to the first weight.

[0011] The weight determination unit may determine the first weight and the second weight for each position where the plurality of relay devices are attached.

[0012] The skeleton estimation unit may generate the skeleton data by inputting a plurality of detection data corresponding to the plurality of relay devices into a skeleton estimation model that outputs the skeleton data.

[0013] The skeleton estimation unit may generate the skeleton data using the skeleton estimation model selected from a plurality of candidate skeleton estimation models based on the type or number of the plurality of relay devices.

[0014] The first position estimation unit may estimate the first estimated position as a position that is located at a distance corresponding to the approximate position in the skeletal data from the position corresponding to the approximate position.

[0015] The first position estimation unit may correct the first estimated position based on the result of identifying the relationship between the person's body captured in the image data of a person wearing the multiple relay devices and the position of each of the multiple relay devices.

[0016] An information terminal in a second aspect of the present invention is an information terminal capable of communicating with a plurality of relay devices, comprising: a data receiving unit that receives detection data from each of the plurality of relay devices indicating at least one of the acceleration and angular velocity of the relay device; a skeleton estimation unit that generates skeleton data indicating the skeleton of a person to whom the plurality of relay devices are attached, estimated based on a plurality of the detection data corresponding to the plurality of relay devices; a first position estimation unit that determines a first estimated position for each of the plurality of relay devices based on mounting position data indicating the approximate position where each of the plurality of relay devices is attached, and the skeleton data; a radio wave measurement unit that measures the radio wave state, including the direction of arrival and intensity of radio waves received from each of the plurality of relay devices, by sweeping radio waves within a sweep range determined based on the first estimated position; a second position estimation unit that determines a second estimated position for the plurality of relay devices based on the radio wave state; and a position identification unit that identifies the position of each of the plurality of relay devices by performing a predetermined calculation that includes the first estimated position and the second estimated position as variables, wherein the skeleton estimation unit generates the skeleton data based on the position of each of the plurality of relay devices identified immediately before by the position identification unit and a plurality of the detection data.

[0017] A program according to a third aspect of the present invention is a program executed by a processor in an information terminal capable of communicating with a plurality of relay devices, wherein the processor, when executed, causes the processor to: receive detection data from each of the plurality of relay devices indicating at least one of the acceleration and angular velocity of the relay device; generate skeletal estimation data indicating the skeleton of a person to which the plurality of relay devices are attached; determine a first estimated position of each of the plurality of relay devices based on mounting position data indicating the approximate position where each of the plurality of relay devices is attached and the skeletal data; The skeletal estimation unit functions as a radio wave measurement unit that measures the radio wave state, including the direction of arrival and intensity of radio waves received from each of the plurality of relay devices, by sweeping radio waves within a sweep range determined based on the first estimated position; a second position estimation unit that determines the second estimated position of the plurality of relay devices based on the radio wave state; and a position identification unit that identifies the position of each of the plurality of relay devices by performing a predetermined calculation that includes the first estimated position and the second estimated position as variables. The skeletal estimation unit generates the skeletal data based on the position of each of the plurality of relay devices identified immediately before by the position identification unit and a plurality of the detection data. [Effects of the Invention]

[0018] The present invention has the effect of improving the accuracy of estimating the position of a device worn on the body. [Brief explanation of the drawing]

[0019] [Figure 1] This is a diagram illustrating the overview of communication system S. [Figure 2] This is a diagram illustrating the overview of communication system S. [Figure 3] This diagram illustrates the general flow of processing in communication system S. [Figure 4] This figure shows an example of the skeletal data generated by information terminal 1. [Figure 5]This is a diagram for explaining the process of estimating a position. [Figure 6] This is a diagram showing the configurations of the information terminal 1 and the relay device 2. [Figure 7] This is a flowchart showing the flow of processing in the communication system S.

Embodiments for Carrying Out the Invention

[0020] [Overview of Communication System S] FIG. 1 and FIG. 2 are diagrams for explaining the overview of the communication system S. The communication system S includes an information terminal 1 and a plurality of relay devices 2 worn on the body of a user U who uses the information terminal 1. As shown in FIG. 2, the information terminal 1 transmits and receives data to and from a base station 3 of a mobile phone network via the plurality of relay devices 2.

[0021] The plurality of relay devices 2 are, for example, worn on the body of the user U. In the example shown in FIG. 1, the plurality of relay devices 2 are worn on both wrists, both ankles, and the head of the user U. The relay device 2 is a device that can communicate with the information terminal 1 and the base station 3 of the mobile phone network while being worn on a person's body. The plurality of relay devices 2 transmit and receive data to and from the base station 3 using radio waves in a first frequency band (for example, millimeter waves). The plurality of relay devices 2 transmit and receive data to and from the information terminal 1 using radio waves in a second frequency band (for example, terahertz waves).

[0022] Each of the plurality of relay devices 2 transmits the data received from the base station 3 to the information terminal 1 and transmits the data received from the information terminal 1 to the base station 3. Since the information terminal 1 transmits and receives data to and from the base station 3 via the plurality of relay devices 2, it can transmit and receive data to and from the base station 3 at a higher speed than when the information terminal 1 directly transmits and receives data to and from the base station 3.

[0023] As shown in Figure 2, the information terminal 1 can change the direction of its antenna in a time-division manner and transmit and receive radio waves to and from multiple relay devices 2 in multiple different directions. However, if the frequency of the radio waves used by the information terminal 1 when transmitting and receiving data with multiple relay devices 2 is high, the directivity of the radio waves is strong. Therefore, the information terminal 1 needs to estimate the position of the relay devices 2 with high accuracy and point its antenna at the estimated position.

[0024] Therefore, in order to estimate the position of the relay devices 2 with high accuracy, the information terminal 1 periodically estimates the skeletal state of a person wearing multiple relay devices 2, and estimates the position of the multiple relay devices 2 based on the estimation results. Then, by controlling the direction of the antenna based on the estimated positions of the multiple relay devices 2, the information terminal 1 can maintain a state where communication with the multiple relay devices 2 is possible even if the positional relationship between the information terminal 1 and the multiple relay devices 2 changes due to the person's movement.

[0025] [Overview of the processing flow in communication system S] Figure 3 is a diagram illustrating the general flow of processing in the communication system S. The processing shown in Figure 3 is executed by the information terminal 1 at predetermined time intervals. The predetermined time interval is set to be less than or equal to the time required for the relay device 2 to move by a distance that would degrade the communication quality between the information terminal 1 and the relay device 2, for example, 100 milliseconds.

[0026] Information terminal 1 estimates the latest skeletal state of user U based on information indicating the movement state of relay devices 2 (e.g., acceleration or angular velocity) measured by inertial sensors on information terminal 1 and multiple relay devices 2, and the positions of information terminal 1 and multiple relay devices 2 identified at the previous processing timing. At the first processing timing, the positions of information terminal 1 and multiple relay devices 2 at the previous processing timing are assumed to be the initial positions corresponding to, for example, user U standing upright.

[0027] For example, the information terminal 1 displays a screen for setting the initial position and accepts input from user U regarding the position where the information terminal 1 is being held and the body parts to which the multiple relay devices 2 are attached, thereby determining the initial positions of the information terminal 1 and the multiple relay devices 2. The information terminal 1 accepts input of approximate location names indicating body parts, such as "head," "right wrist," "left wrist," "right ankle," and "left ankle," and identifies the location stored in association with the accepted name as the initial position.

[0028] Information terminal 1 estimates the latest skeletal state of user U by determining the distance and direction that information terminal 1 and the multiple relay devices 2 have moved between the previous processing timing and the latest processing timing, based on at least one of the acceleration and angular velocity (hereinafter referred to as "acceleration or angular velocity") indicated by the detection data input from sensors on information terminal 1 and the multiple relay devices 2 (S1 in Figure 3). As an example, information terminal 1 generates the latest skeletal data by inputting the positions of information terminal 1 and the multiple relay devices 2 at the previous processing timing, and the acceleration or angular velocity of information terminal 1 and the multiple relay devices 2, into a skeletal estimation model that outputs skeletal data indicating the changed skeletal state by inputting the distance and direction that multiple positions in the skeletal system have moved.

[0029] Figure 4 shows an example of skeletal data generated by information terminal 1. As shown in Figure 4, skeletal data is data that shows the positional relationships of major bones and joints in a human body. Skeletal data may be image data, or it may be data that shows the positional coordinates of feature points of multiple bones and joints. The skeletal estimation model updates the skeletal data by changing the position or orientation of the affected joints and bones by moving a specified part in the immediately preceding skeletal data by a specified distance in a specified direction. The skeletal estimation model may be a program that generates skeletal data by being executed by the processor of information terminal 1, or it may be a program that generates skeletal data by being executed by a processor outside of information terminal 1.

[0030] Returning to Figure 3, the information terminal 1 estimates the latest position and orientation of the information terminal 1 and the multiple relay devices 2 based on the generated skeletal data (S2 in Figure 3). The latest position and orientation of the information terminal 1 and the multiple relay devices 2 are, for example, the relative position and relative orientation of each of the multiple relay devices 2 with respect to the position of the information terminal 1. The information terminal 1 estimates the latest position of the relay device 2 as a position that is separated from the approximate position, which is the position of the skeleton corresponding to the name of the mounting part of the information terminal 1 and the multiple relay devices 2 in the latest skeletal data, by the distance between the approximate position and the position where the information terminal 1 and the multiple relay devices 2 are mounted.

[0031] Figure 5 is a diagram illustrating the process of estimating position. In Figure 5, the star H indicates the position of the head in the skeletal data. The hexagonal shape in Figure 5 indicates the relay device 2 attached to the left ear. Thus, even if the attachment position is "head," the position of the head in the skeletal data differs from the actual position where the relay device 2 is attached. Therefore, the information terminal 1 calculates the position coordinates of the information terminal 1 and the multiple relay devices 2 by adding a difference amount based on the distance and direction to the attachment positions of the information terminal 1 and the multiple relay devices 2 to the position coordinates corresponding to the attachment site in the skeletal data. As a result, the information terminal 1 can estimate the relative positions of the multiple relay devices 2 with high accuracy, relative to the information terminal 1.

[0032] Information terminal 1 estimates the relative positions of multiple relay devices 2 based on skeletal data, and simultaneously estimates the positions and orientations of multiple relay devices 2 based on position estimation radio waves transmitted by each of the multiple relay devices 2 (S3 in Figure 3). Position estimation radio waves are radio waves transmitted at a predetermined intensity in a predetermined direction from the relay device 2. Position estimation radio waves may also include radio waves that include a signal (RSSI signal) indicating the intensity of the radio waves received by the relay device 2 from information terminal 1. Information terminal 1 determines the relative positions of multiple relay devices 2 with respect to information terminal 1 by estimating the orientation of the relay device 2 based on the direction from which the radio waves were received and estimating the distance to the relay device 2 based on the intensity of the received radio waves.

[0033] Incidentally, when the accuracy of the inertial sensors installed on the information terminal 1 and the multiple relay devices 2 is low, the accuracy of the position based on the skeletal data estimated in S2 is low. Also, when the radio wave conditions received by the information terminal 1 from the multiple relay devices 2 are poor, the accuracy of the position based on the radio waves estimated in S3 is low. Therefore, the information terminal 1 determines the position and orientation of the multiple relay devices 2 based on the position (hereinafter sometimes referred to as the "first estimated position") and orientation (hereinafter sometimes referred to as the "first estimated orientation") of the multiple relay devices 2 estimated based on the skeletal data, and the position (hereinafter sometimes referred to as the "second estimated position") and orientation (hereinafter sometimes referred to as the "second estimated orientation") of the multiple relay devices 2 estimated based on the radio waves (S4 in Figure 3).

[0034] As will be described in detail later, the information terminal 1 calculates the position and orientation of multiple relay devices 2 based on a first confidence level for the first estimated position and first estimated orientation based on skeletal data, and a second confidence level for the second estimated position and second estimated orientation based on radio waves. The information terminal 1 calculates the position of multiple relay devices 2 by weighting the first estimated position and the second estimated position using weights corresponding to the first confidence level for the first estimated position estimated based on skeletal data and the second confidence level for the second estimated position estimated based on radio waves. The information terminal 1 may also calculate the orientation of multiple relay devices 2 by weighting the first estimated orientation and the second estimated orientation using weights corresponding to the first confidence level and the second confidence level.

[0035] [Configuration of Information Terminal 1 and Relay Device 2] Figure 6 shows the configuration of information terminal 1 and relay device 2. Information terminal 1 includes an inertial sensor 11, a receiving antenna 12, a receiving antenna 13, a storage unit 14, and a control unit 15. The control unit 15 includes a skeletal estimation unit 151, a first position estimation unit 152, a radio wave measurement unit 153, a second position estimation unit 154, a weight determination unit 155, and a position identification unit 156. Relay device 2 includes an inertial sensor 21, a transmitting antenna 22, a data generation unit 23, and a transmitting antenna 24.

[0036] First, let's describe the various parts of the relay device 2. The inertial sensor 21 is a sensor that can measure the acceleration or angular velocity of the relay device 2. The inertial sensor 21 is a detection unit that outputs detection data indicating acceleration or angular velocity to the skeletal estimation unit 151. The inertial sensor 21 is, for example, a 6-axis sensor, but it may be of other types. The transmitting antenna 22 transmits the detection data.

[0037] The data generation unit 23 generates data that the relay device 2 transmits to the information terminal 1. The data generation unit 23 generates, for example, position estimation data. The transmitting antenna 24 transmits position estimation radio waves containing the position estimation data.

[0038] Although not shown in Figure 6, the relay device 2 may further have a communication circuit for sending and receiving data to and from the base station 3. The relay device 2 transmits data received from the information terminal 1 to the base station 3 via the communication circuit, and transmits data received from the base station 3 to the information terminal 1 via the communication circuit.

[0039] Next, the various parts of the information terminal 1 will be described. The inertial sensor 11 is a sensor that can measure the acceleration or angular velocity of the information terminal 1. The inertial sensor 11 is, for example, a 6-axis sensor, but it may be of other types. The inertial sensor 11 inputs the data indicating the detected acceleration or angular velocity to the skeletal estimation unit 151.

[0040] The receiving antenna 12 is an antenna for receiving acceleration or angular velocity from the relay device 2. The receiving antenna 12 is an antenna for receiving data indicating acceleration or angular velocity transmitted from the relay device 2, for example, via Bluetooth®. The receiving antenna 12 functions as a data receiving unit that receives detection data from each of the multiple relay devices 2. The receiving antenna 12 inputs the received data indicating acceleration or angular velocity to the skeletal estimation unit 151.

[0041] The receiving antenna 13 is an antenna for receiving data and position estimation radio waves received by the relay device 2 from the base station 3. The receiving antenna 13 supports frequencies higher than those that the receiving antenna 12 can receive, and can receive radio waves in the millimeter wave band or terahertz band, for example.

[0042] The storage unit 14 has storage media such as ROM (Read Only Memory) and RAM (Random Access Memory). The storage unit 14 stores the program executed by the control unit 15. The storage unit 14 also stores the position and orientation of the relay devices 2 in association with the relay device ID and time for identifying the multiple relay devices 2.

[0043] The control unit 15 includes, for example, a CPU (Central Processing Unit). By executing a program stored in the memory unit 14, the control unit 15 functions as a skeleton estimation unit 151, a first position estimation unit 152, a radio wave measurement unit 153, a second position estimation unit 154, a weight determination unit 155, and a position identification unit 156.

[0044] The skeleton estimation unit 151 generates skeleton data representing the skeleton of a person wearing multiple relay devices 2, estimated based on multiple detection data corresponding to multiple relay devices 2. The skeleton estimation unit 151 generates skeleton data based on the position of each of the multiple relay devices 2 identified immediately before by the position identification unit 156 and the multiple detection data.

[0045] Specifically, the skeleton estimation unit 151 generates the latest skeleton data based on the position and orientation of the information terminal 1 and the multiple relay devices 2 immediately before, the previous skeleton data, the acceleration or angular velocity of the information terminal 1 detected by the inertial sensor 11, and the acceleration or angular velocity of the multiple relay devices 2 indicated by the multiple detection data corresponding to the multiple relay devices 2. The skeleton estimation unit 151 stores the generated skeleton data in the storage unit 14.

[0046] The skeleton estimation unit 151 generates skeleton data by inputting multiple detection data corresponding to multiple relay devices 2 into a skeleton estimation model that outputs skeleton data. As outlined above, the skeleton estimation unit 151 inputs the positions of the information terminal 1 and multiple relay devices 2 at the previous processing timing, as well as the acceleration or angular velocity of the information terminal 1 and multiple relay devices 2, into a skeleton estimation model that outputs skeleton data indicating the state of the skeleton after a change by inputting the distance and direction in which multiple positions in the skeleton have moved. The skeleton estimation unit 151 generates the skeleton data output from the skeleton estimation model as the latest skeleton data.

[0047] It is conceivable that the appropriate skeletal estimation model may differ depending on the type or number of relay devices 2. For example, a skeletal estimation model created by machine learning using detection data output from 10 sensors when 10 relay devices 2 are installed may not be suitable when only 5 relay devices 2 are installed. Similarly, a skeletal estimation model created by machine learning using detection data output from a 6-axis inertial sensor may not be suitable when the relay device 2 has a 3-axis inertial sensor.

[0048] Therefore, the skeleton estimation unit 151 may generate skeleton data using a skeleton estimation model selected from a plurality of candidate skeleton estimation models based on the type or number of multiple relay devices 2. By operating in this manner, the accuracy of the latest skeleton data generated by the skeleton estimation unit 151 can be improved.

[0049] The first position estimation unit 152 determines a first estimated position, which is the position of each of the multiple relay devices 2, based on multiple detection data output by the multiple inertial sensors 21. For example, the first position estimation unit 152 determines a first estimated position, which is the position of each of the multiple relay devices 2, based on mounting position data indicating the approximate position where each of the multiple relay devices 2 is mounted, and skeleton data generated by the skeleton estimation unit 151 based on the multiple detection data. As explained with reference to Figure 5, for example, the first position estimation unit 152 estimates a position as the first estimated position that is a distance corresponding to the approximate position from the position corresponding to the approximate position in the skeleton data.

[0050] Specifically, the first position estimation unit 152 first acquires attachment location data that indicates the approximate locations of the attachment locations of multiple relay devices 2, which have been set by the user U, for example, via an operating device on the information terminal 1. The attachment location data is data that includes information for identifying the attachment location, such as "right wrist" or "left wrist".

[0051] The first position estimation unit 152 calculates the position coordinates of the information terminal 1 and the multiple relay devices 2 by adding a difference amount based on the distance and direction to the mounting positions of the information terminal 1 and the multiple relay devices 2 to the approximate position coordinates corresponding to the mounting locations of the information terminal 1 and the multiple relay devices 2 in the skeletal data, as described above. The first position estimation unit 152 notifies the radio wave measurement unit 153 and the position identification unit 156 of the determined first estimated position.

[0052] Since the skeletal data is generated by estimating the most likely skeletal state based on acceleration or angular velocity measured at multiple relay devices 2, even if there are errors in the acceleration or angular velocity measured at some of the relay devices 2, the effects of these errors are less likely to appear in the skeletal data. Therefore, the accuracy of the first estimated position is improved when the first position estimation unit 152 estimates the position of the relay devices 2 using the skeletal data.

[0053] The first position estimation unit 152 may determine the orientation of each of the multiple relay devices 2 based on mounting position data indicating the approximate position where each of the multiple relay devices 2 is attached, and skeletal data. For example, the first position estimation unit 152 may determine the orientation of each of the multiple relay devices 2 at the first estimated position based on the orientation of bones near the first estimated position determined based on the mounting position data in the skeletal data created by the skeletal estimation unit 151.

[0054] The first position estimation unit 152 may acquire image data from an external device showing a person wearing multiple relay devices 2, and correct the first estimated position based on the results of identifying the relationship between the person's body in the image data and the position of each of the multiple relay devices 2. For example, suppose user U sets the attachment site for relay device 2 to "right wrist," but in reality, relay device 2 is attached to the right elbow. In such a case, if the first position estimation unit 152 determines that the approximate position is "right elbow" based on the image data, it corrects the first estimated position corresponding to the right wrist to the position corresponding to the right elbow in the skeletal data. By operating in this manner, the first position estimation unit 152 can correctly estimate the position of relay device 2 even if user U makes a mistake in setting the attachment position or attaching relay device 2 to the wrong position.

[0055] The first position estimation unit 152 may estimate the position of the relay device 2 without using skeletal data. For example, the first position estimation unit 152 may determine the first estimated position by determining the amount of movement and direction of movement from the position of the relay device 2 previously determined by the position determination unit 156, based on the acceleration or angular velocity of the relay device 2 output by the inertial sensor 21, and adding the determined amount of movement in the direction of the determined direction of movement to the position of the relay device 2 immediately before.

[0056] The radio wave measurement unit 153 measures the radio wave conditions, including the direction of arrival and intensity of radio waves received from each of the multiple relay devices 2, by sweeping the radio waves within a sweep range determined based on the first estimated position. Specifically, the radio wave measurement unit 153 controls the receiving antenna 13 to sequentially point it in multiple directions within the sweep range corresponding to each of the multiple first estimated positions, from among the multiple directions the receiving antenna 13 can point in. While sequentially pointing the receiving antenna 13 in multiple directions, the radio wave measurement unit 153 measures the intensity of the radio waves received by the receiving antenna 13 in each direction, and determines the direction with the greatest intensity as the direction of arrival of the radio waves. Based on the relay device ID included in the position estimation signal transmitted by each of the multiple relay devices 2, the radio wave measurement unit 153 identifies the direction of arrival and intensity of the radio waves transmitted by each of the multiple relay devices 2.

[0057] The radio wave measurement unit 153 may further determine the sweep range based on the orientation of each of the multiple relay devices 2 estimated by the first position estimation unit 152. The radio wave measurement unit 153 includes the direction from which the reflected waves arrive in the sweep range when the orientation of the relay device 2 is such that radio waves transmitted by the relay device 2 may be reflected by any part of the body and reach the information terminal 1.

[0058] The second position estimation unit 154 determines a second estimated position, which is the position of multiple relay devices 2, based on the radio wave conditions. The second position estimation unit 154 determines that, with the position of the information terminal 1 as a reference, the relay device 2 is located at a distance corresponding to the radio wave intensity measured by the radio wave measurement unit 153, in the direction from which the radio waves measured by the radio wave measurement unit 153 are coming. For each of the multiple relay devices 2, the second position estimation unit 154 identifies a second estimated position of the relay device 2 with respect to the position of the information terminal 1. The second position estimation unit 154 notifies the weight determination unit 155 of the second estimated position.

[0059] The weight determination unit 155 determines the first weight for the first estimated position and the second weight for the second estimated position in the calculation performed by the position identification unit 156 to identify the position of each of the multiple relay devices 2, based on the first confidence level of the detection data output by the inertial sensors 11 and 21 or the second confidence level of the radio wave conditions received by the receiving antenna 13. For example, the weight determination unit 155 determines that the second confidence level is lower the smaller the intensity of the radio waves received by the receiving antenna 13, or the greater the dispersion in the direction of arrival of the radio waves, and reduces the ratio of the second weight to the first weight.

[0060] On the other hand, the weight determination unit 155 may determine that the greater the difference between the temperature or humidity around the relay device 2 and a predetermined temperature or humidity suitable for the operation of the relay device 2, the lower the first confidence level and may increase the ratio of the second weight to the first weight. In order to determine the temperature or humidity around multiple relay devices 2, the weight determination unit 155 may acquire environmental data indicating temperature or humidity from a temperature sensor or humidity sensor (not shown), or it may acquire environmental data input by the user U.

[0061] The weight determination unit 155 may increase the ratio of the second weight to the first weight if the acceleration or angular velocity output by the inertial sensors 11 and 21 shows abnormal values ​​outside a preset normal range. The weight determination unit 155 may decrease the ratio of the second weight to the first weight as the intensity of the radio waves received by the receiving antenna 12 decreases.

[0062] Furthermore, the temperature may differ depending on the location of the multiple relay devices 2, and the state of the radio wave propagation path from the relay devices 2 to the information terminal 1 may differ. Therefore, the weight determination unit 155 may determine a first weight and a second weight for each location where the multiple relay devices 2 are installed. The weight determination unit 155 may also determine a first weight and a second weight for each of the multiple relay devices 2.

[0063] The position determination unit 156 determines the position of each of the multiple relay devices 2 by performing a predetermined calculation that includes the first estimated position and the second estimated position as variables. For example, the position determination unit 156 uses the average or median of the first estimated position and the second estimated position as the position of each of the multiple relay devices 2.

[0064] However, as described above, depending on the environment surrounding the inertial sensors 11 and 21, the first confidence level of the first estimated position based on the detection data output by the inertial sensors 11 and 21 may be low. Also, depending on the environment surrounding the information terminal 1 and relay device 2, the second confidence level of the second estimated position based on the radio wave conditions received by the receiving antenna 13 may be low.

[0065] Therefore, the position determination unit 156 functions as a Kalman filter in conjunction with the weight determination unit 155, for example, to determine the positions of multiple relay devices 2 with higher accuracy than the first estimated position and the second estimated position. The position determination unit 156 determines the position of each of the multiple relay devices 2 by weighting the first estimated position and the second estimated position based on the first weight and the second weight.

[0066] Specifically, the position identification unit 156 identifies the position of each of the multiple relay devices 2 using, for example, the following calculation formula 1. If the weight determination unit 155 determines different first and second weights for each of the multiple relay devices 2 installed, the position identification unit 156 identifies the position for each of the multiple relay devices 2 installed using the calculation formula 1. If the weight determination unit 155 determines different first and second weights for each of the multiple relay devices 2, the position identification unit 156 identifies the position for each of the multiple relay devices 2 using the calculation formula 1. [Equation 1] The position of relay device 2 = 1st estimated position × 1st weight + 2nd estimated position × 2nd weight However, the first weight + the second weight = 1

[0067] The positioning unit 156 improves the accuracy of the position of each of the identified relay devices 2 by performing a weighted average based on the reliability of the first estimated position based on the detection data of the inertial sensor 11 and the inertial sensor 21, and the second estimated position based on radio waves.

[0068] [Processing flow in communication system S] Figure 7 is a flowchart showing the processing flow in the communication system S. The flowchart shown in Figure 7 starts from the moment the power to the information terminal 1 and the relay device 2 is turned on.

[0069] The relay device 2 transmits detection data indicating the acceleration or angular velocity detected by the inertial sensor 21 (S21) and transmits radio waves for position estimation (S22). The information terminal 1 receives the detection data transmitted by the relay device 2 with the receiving antenna 12 (S11). The skeleton estimation unit 151 estimates the current state of the skeleton and generates skeleton data based on the previous state of the skeleton, the previous positions of the multiple relay devices 2, and the acceleration or angular velocity of the multiple relay devices 2 indicated by the detection data (S12). Subsequently, the first position estimation unit 152 estimates the positions (first estimated positions) of the multiple relay devices 2 based on the generated skeleton data (S13).

[0070] The radio wave measurement unit 153 receives position estimation radio waves transmitted by the relay devices 2 by sweeping the radio waves in a direction based on the estimated positions of the multiple relay devices 2 (S14), and the second position estimation unit 154 estimates the position of each of the multiple relay devices 2 (second estimated position) based on the radio wave conditions indicated by the received position estimation radio waves (S15).

[0071] The weight determination unit 155 calculates a first confidence level for the first estimated position estimated in S13 based on the temperature or humidity around the multiple relay devices 2, and calculates a second confidence level for the second estimated position estimated in S15 based on the state of the radio waves received by the receiving antenna 13. Based on the first and second confidence levels, the weight determination unit 155 determines a first weight corresponding to the first estimated position estimated in S13 and a second weight corresponding to the second estimated position estimated in S15 (S16).

[0072] The positioning unit 156 determines the position of each of the multiple relay devices 2 by weighted averaging using the first weight and the second weight (S17). The latest position determined by the positioning unit 156 is used by the skeleton estimation unit 151 when estimating the state of the skeleton at the next processing timing.

[0073] [Effects of communication system S] As described above, the first position estimation unit 152 determines the first estimated position of each of the multiple relay devices 2 based on mounting position data indicating the approximate position where each of the multiple relay devices 2 is mounted, and skeletal data generated based on the acceleration or angular velocity of the multiple relay devices 2. The second position estimation unit 154 determines the second estimated position of the multiple relay devices 2 based on the state of the position estimation radio waves transmitted by the multiple relay devices 2. Then, the position identification unit 156 identifies the position of each of the multiple relay devices 2 by performing a predetermined calculation that includes the first estimated position and the second estimated position as variables.

[0074] With the information terminal 1 configured in this way, the locations of multiple relay devices 2 are determined by considering the results of estimating the locations of multiple relay devices 2 using multiple methods. Therefore, the information terminal 1 can determine the locations of multiple relay devices 2 with higher accuracy than when the locations of multiple relay devices 2 are estimated using only one method.

[0075] Furthermore, this invention will make it possible to contribute to Goal 9 of the United Nations-led Sustainable Development Goals (SDGs), "Build resilient infrastructure, promote inclusive and sustainable industrialization and foster innovation."

[0076] Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of its gist. For example, all or part of the apparatus can be configured by functionally or physically distributing and integrating in any unit. Furthermore, new embodiments resulting from any combination of multiple embodiments are also included in the embodiments of the present invention. The effects of the new embodiments resulting from the combinations are combined with the effects of the original embodiments. [Explanation of Symbols]

[0077] 1. Information terminal 2. Relay device 3 base station 11. Inertial Sensor 12 Receiving antenna 13 Receiving antenna 14 Storage section 15 Control Unit 21 Inertial Sensor 22 Transmitting Antennas 23 Data Generation Unit 24 Transmitting Antennas 151 Skeleton Estimation Section 152 1st position estimation section 153 Radio wave measurement section 154 Second position estimation section 155 Decision Section 156 Location identification part

Claims

1. The system comprises an information terminal and a plurality of relay devices that can communicate with the information terminal and a base station of a mobile phone network while being worn on a person's body. Each of the plurality of relay devices has a detection unit that outputs detection data indicating at least one of the acceleration and angular velocity of the relay device. The aforementioned information terminal is A data receiving unit that receives the detection data from each of the multiple relay devices, A skeleton estimation unit that generates skeleton data indicating the skeleton of a person to whom the multiple relay devices are attached, estimated based on the multiple detection data corresponding to the multiple relay devices, A first position estimation unit determines a first estimated position for each of the multiple relay devices based on mounting position data indicating the approximate position where each of the multiple relay devices is mounted, and the skeletal data. A radio wave measuring unit measures the radio wave conditions, including the direction of arrival and intensity of radio waves received from each of the plurality of relay devices, by sweeping radio waves within a sweep range determined based on the first estimated position, A second position estimation unit that determines the second estimated position of the plurality of relay devices based on the radio wave conditions, A position identification unit that identifies the position of each of the plurality of relay devices by performing a predetermined calculation that includes the first estimated position and the second estimated position as variables, It has, The skeletal estimation unit is a communication system that generates skeletal data based on the position of each of the multiple relay devices identified immediately before by the position identification unit and the multiple detection data.

2. The first position estimation unit further estimates the orientation of each of the plurality of relay devices, The radio wave measurement unit further determines the sweep range based on the orientation of each of the plurality of relay devices estimated by the first position estimation unit. The communication system according to claim 1.

3. The system further includes a weight determination unit that determines a first weight for the first estimated position and a second weight for the second estimated position in the calculation performed by the position determination unit, based on a first confidence level of the detection data or a second confidence level of the radio wave state. The position identification unit identifies the position of each of the plurality of relay devices by weighting the first estimated position and the second estimated position based on the first weight and the second weight. The communication system according to claim 1.

4. The weight determination unit determines that the lower the intensity of the radio waves, or the greater the dispersion in the direction of arrival of the radio waves, the lower the first confidence level, and reduces the ratio of the second weight to the first weight. The communication system according to claim 3.

5. The weight determination unit acquires environmental data indicating the temperature or humidity around the plurality of relay devices, and determines that the greater the difference between the temperature or humidity indicated by the acquired environmental data and a predetermined temperature or humidity suitable for the operation of the relay devices, the lower the second reliability, and increases the ratio of the second weight to the first weight. The communication system according to claim 3.

6. The weight determination unit determines the first weight and the second weight for each position where the plurality of relay devices are attached. The communication system according to claim 3.

7. The skeleton estimation unit generates the skeleton data by inputting a plurality of detection data corresponding to the plurality of relay devices into a skeleton estimation model that outputs the skeleton data. The communication system according to claim 1.

8. The skeleton estimation unit generates the skeleton data using the skeleton estimation model selected from a plurality of candidate skeleton estimation models based on the type or number of the plurality of relay devices. The communication system according to claim 7.

9. The first position estimation unit estimates a position as the first estimated position that is located at a distance corresponding to the approximate position in the skeletal data from the position corresponding to the approximate position. The communication system according to claim 1.

10. The first position estimation unit corrects the first estimated position based on the result of identifying the relationship between the person's body captured in the image data taken by the person wearing the multiple relay devices and the position of each of the multiple relay devices. The communication system according to claim 1.

11. An information terminal capable of communicating with multiple relay devices, A data receiving unit that receives detection data from each of the plurality of relay devices indicating at least one of the acceleration and angular velocity of the relay device, A skeleton estimation unit that generates skeleton data indicating the skeleton of a person to whom the multiple relay devices are attached, estimated based on the multiple detection data corresponding to the multiple relay devices, A first position estimation unit determines a first estimated position for each of the multiple relay devices based on mounting position data indicating the approximate position where each of the multiple relay devices is mounted, and the skeletal data. A radio wave measuring unit measures the radio wave conditions, including the direction of arrival and intensity of radio waves received from each of the plurality of relay devices, by sweeping radio waves within a sweep range determined based on the first estimated position, A second position estimation unit that determines the second estimated position of the plurality of relay devices based on the radio wave conditions, A position identification unit that identifies the position of each of the plurality of relay devices by performing a predetermined calculation that includes the first estimated position and the second estimated position as variables, It has, The skeletal estimation unit generates the skeletal data based on the position of each of the multiple relay devices identified immediately before by the position identification unit and the multiple detection data. Information terminal.

12. A program executed by a processor in an information terminal capable of communicating with multiple relay devices, wherein the execution of the program causes the processor to... A data receiving unit that receives detection data from each of the plurality of relay devices indicating at least one of the acceleration and angular velocity of the relay device, A skeleton estimation unit that generates skeleton data showing the skeleton of a person wearing the aforementioned multiple relay devices, A first position estimation unit determines a first estimated position for each of the multiple relay devices based on mounting position data indicating the approximate position where each of the multiple relay devices is mounted, and the skeletal data. A radio wave measuring unit measures the radio wave conditions, including the direction of arrival and intensity of radio waves received from each of the plurality of relay devices, by sweeping radio waves within a sweep range determined based on the first estimated position, A second position estimation unit that determines the second estimated position of the plurality of relay devices based on the radio wave conditions, A position identification unit that identifies the position of each of the plurality of relay devices by performing a predetermined calculation that includes the first estimated position and the second estimated position as variables, To make it function as, The skeletal estimation unit generates the skeletal data based on the position of each of the multiple relay devices identified immediately before by the position identification unit and the multiple detection data. program.

Images