Communication systems, information terminals, and programs

Abstract

To improve accuracy of estimating a position of a device worn on a body.SOLUTION: An information terminal 1 has: a skeleton estimation unit 151 that generates skeleton data indicating the skeleton of a person wearing a plurality of relay devices 2; a model creation unit 152 that creates a plurality of radio wave propagation models between the information terminal 1 and the plurality of relay devices 2 based on the skeleton data and wearing position data indicating an approximate terminal position where the information terminal 1 is worn and a plurality of approximate device positions where the plurality of relay devices 2 are worn; and a second position estimation unit 155 that determines estimation positions of the plurality of relay devices 2 based on a correction direction and correction strength obtained by correcting the arrival direction and strength of the radio waves received from the plurality of relay devices 2 using the radio wave propagation model.SELECTED DRAWING: Figure 7

Description

Technical Field

[0005] ,

[0001] The present invention relates to a communication system, an information terminal, and a program.

Background Art

[0002] Conventionally, a technique for correcting the position of a human estimated by three-point measurement using Wi-Fi (registered trademark) using the information on the position estimated by an inertial sensor is known (see, for example, Non-Patent Document 1).

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

[0006] Therefore, the present invention has been made in view of these points, and aims to improve the accuracy of estimating the position of a device worn on the body. [Means for solving the problem]

[0007] A communication system according to a first aspect of the present invention comprises an information terminal and a plurality of relay devices that can communicate with the information terminal and a base station of a mobile phone network while being worn on a person's body. Each of the plurality of relay devices has a detection unit that outputs detection data of at least one of the acceleration and angular velocity of the relay device. The information terminal includes: a data receiving unit that receives the detection data from each of the plurality of relay devices; a skeleton estimation unit that generates skeleton data indicating the skeleton of a person to whom the plurality of relay devices are attached, estimated based on the plurality of detection data corresponding to the plurality of relay devices; a model creation unit that creates a plurality of radio wave propagation models between the information terminal and each of the plurality of relay devices based on the approximate terminal position to which the information terminal is attached, the approximate device positions to which each of the plurality of relay devices is attached, and the skeleton data; a radio wave measurement unit that measures the radio wave state, including the direction of arrival and intensity of the radio waves received from each of the plurality of relay devices, by sweeping the radio waves within a predetermined sweep range; and a position estimation unit that determines the estimated position of the plurality of relay devices based on the corrected direction and corrected intensity obtained by correcting the direction of arrival and intensity using the radio wave propagation model, wherein the radio wave measurement unit sweeps the radio waves within the sweep range determined based on the estimated position determined by the position estimation unit immediately before.

[0008] The model creation unit may estimate a terminal position, which is the location of the information terminal, at a distance corresponding to the terminal approximate position in the skeletal data, and estimate a device position, which is the location of the relay device, at a distance corresponding to the device approximate position in the skeletal data, and create the radio wave propagation model based on the terminal position, the device position, and the position and orientation of the skeleton indicated by the skeletal data within a predetermined range from the terminal position and the device position.

[0009] The model creation unit may create a radio wave propagation model that reflects the effects of reflected waves generated when radio waves emitted from the device location are reflected by a part of the human body corresponding to the skeletal data as they propagate to the terminal location, or diffracted waves generated when radio waves emitted from the device location bend around the back of a part of the human body corresponding to the skeletal data as they propagate to the terminal location.

[0010] The model creation unit may further determine the terminal position and the device position based on the results of determining the relationship between the person's body captured in the image data taken by a person wearing the multiple relay devices and the position of each of the multiple relay devices.

[0011] The model creation unit may create the radio wave propagation model based on data indicating the clothing worn by the person wearing the multiple relay devices.

[0012] The model creation unit may further create the radio wave propagation model based on the shapes and positions of objects captured in the image data of the surrounding area of ​​the multiple relay devices.

[0013] The skeleton estimation unit may generate the skeleton data by inputting a plurality of detection data corresponding to the plurality of relay devices into a skeleton estimation model that outputs the skeleton data.

[0014] The skeleton estimation unit may generate the skeleton data using the skeleton estimation model selected from a plurality of candidate skeleton estimation models based on the type or number of the plurality of relay devices.

[0015] The position estimation unit may further estimate the orientation of each of the multiple relay devices, and the radio wave measurement unit may determine the sweep range based on the orientation of each of the multiple relay devices estimated by the position estimation unit.

[0016] An information terminal according to a second aspect of the present invention is an information terminal capable of communicating with a plurality of relay devices, comprising: a data receiving unit that receives detection data from each of the plurality of relay devices indicating at least one of the acceleration and angular velocity of the relay device; a skeleton estimation unit that generates skeleton data indicating the skeleton of a person to whom the plurality of relay devices are attached, estimated based on a plurality of the detection data corresponding to the plurality of relay devices; a model creation unit that creates a plurality of radio wave propagation models between the information terminal and each of the plurality of relay devices based on mounting position data indicating the approximate terminal position to which the information terminal is attached, the approximate device positions to which each of the plurality of relay devices is attached, and the skeleton data; a radio wave measurement unit that measures the direction of arrival and intensity of radio waves received from each of the plurality of relay devices by sweeping radio waves within a predetermined sweep range; and a position estimation unit that determines the estimated position of the plurality of relay devices based on a corrected direction and corrected intensity obtained by correcting the direction of arrival and intensity using the radio wave propagation model, wherein the radio wave measurement unit sweeps radio waves within the sweep range determined based on the estimated position determined immediately before by the position estimation unit.

[0017] A program according to a third aspect of the present invention is a program executed by a processor in an information terminal capable of communicating with a plurality of relay devices, wherein the processor, when executed, causes the processor to receive detection data from each of the plurality of relay devices indicating at least one of the acceleration and angular velocity of the relay device; a skeleton estimation unit that generates skeleton data indicating the skeleton of a person to which the plurality of relay devices are attached, estimated based on a plurality of the detection data corresponding to the plurality of relay devices; the approximate terminal position to which the information terminal is attached; and the approximate device positions to which each of the plurality of relay devices is attached. The system functions as follows: a model creation unit that creates multiple radio wave propagation models between the information terminal and each of the multiple relay devices based on mounting position data and skeletal data; a radio wave measurement unit that measures the radio wave state, including the direction of arrival and intensity of radio waves received from each of the multiple relay devices, by sweeping radio waves within a predetermined sweep range; and a position estimation unit that determines the estimated positions of the multiple relay devices based on a corrected direction and corrected intensity obtained by correcting the direction of arrival and intensity using the radio wave propagation model. The radio wave measurement unit sweeps radio waves within the sweep range determined based on the estimated position determined by the position estimation unit immediately prior to the sweep. [Effects of the Invention]

[0018] The present invention has the effect of improving the accuracy of estimating the position of a device worn on the body. [Brief explanation of the drawing]

[0019] [Figure 1] This is a diagram illustrating the overview of communication system S. [Figure 2] This is a diagram illustrating the overview of communication system S. [Figure 3] This diagram illustrates the general flow of processing in communication system S. [Figure 4] This figure shows an example of the skeletal data generated by information terminal 1. [Figure 5]This is a diagram for explaining a radio wave propagation model. [Figure 6] This is a diagram for explaining a process of estimating a position. [Figure 7] This is a diagram showing configurations of the information terminal 1 and the relay device 2. [Figure 8] This is a flowchart showing a process flow in the communication system S.

Embodiments for Carrying Out the Invention

[0020] [Overview of Communication System S] FIG. 1 and FIG. 2 are diagrams for explaining an overview of the communication system S. The communication system S includes an information terminal 1 and a plurality of relay devices 2 attached to the body of a user U who uses the information terminal 1. As shown in FIG. 2, the information terminal 1 transmits and receives data to and from a base station 3 of a mobile phone network via the plurality of relay devices 2.

[0021] The plurality of relay devices 2 are, for example, attached to the body of the user U. In the example shown in FIG. 1, the plurality of relay devices 2 are attached to both wrists, both ankles and the head of the user U. The relay device 2 is a device that can communicate with the information terminal 1 and the base station 3 of the mobile phone network while being attached to a human body. The plurality of relay devices 2 transmit and receive data to and from the base station 3 using radio waves in a first frequency band (for example, millimeter waves). The plurality of relay devices 2 transmit and receive data to and from the information terminal 1 using radio waves in a second frequency band (for example, terahertz waves).

[0022] Each of the plurality of relay devices 2 transmits the data received from the base station 3 to the information terminal 1 and transmits the data received from the information terminal 1 to the base station 3. Since the information terminal 1 transmits and receives data to and from the base station 3 via the plurality of relay devices 2, the information terminal 1 can transmit and receive data to and from the base station 3 at a higher speed than when the information terminal 1 directly transmits and receives data to and from the base station 3.

[0023] As shown in Figure 2, the information terminal 1 can change the direction of its antenna in a time-division manner and transmit and receive radio waves to and from multiple relay devices 2 in multiple different directions. However, if the frequency of the radio waves used by the information terminal 1 when transmitting and receiving data with multiple relay devices 2 is high, the directivity of the radio waves is strong. Therefore, the information terminal 1 needs to estimate the position of the relay devices 2 with high accuracy and point its antenna at the estimated position.

[0024] Therefore, in order to estimate the position of relay devices 2 with high accuracy, information terminal 1 estimates the positions of multiple relay devices 2 based on position estimation radio waves received from multiple relay devices 2. However, the radio waves used for communication between information terminal 1 and relay devices 2 are reflected by user U's body before reaching the destination, or diffracted by user U's body before reaching the destination. The propagation environment near user U changes dynamically with changes in user U's posture. When the propagation environment changes with changes in user U's posture, the intensity of the radio waves received by information terminal 1 changes, causing errors in position estimation.

[0025] To minimize the impact of errors, the information terminal 1 creates a radio wave propagation model that corresponds to the radio wave propagation environment which changes according to the user U's posture (i.e., the positions of the information terminal 1 and the multiple relay devices 2). Based on the radio wave propagation model, the information terminal 1 corrects the characteristics of the received radio waves and uses the corrected results to estimate the positions of the multiple relay devices 2, thereby improving the accuracy of the estimated positions. Furthermore, by controlling the orientation of the antenna based on the estimated positions of the multiple relay devices 2, the information terminal 1 can maintain a state where communication with the multiple relay devices 2 is possible even if the positional relationship between the information terminal 1 and the multiple relay devices 2 changes due to human movement.

[0026] [Overview of the processing flow in communication system S] Figure 3 is a diagram illustrating the general flow of processing in the communication system S. The processing shown in Figure 3 is executed by the information terminal 1 at predetermined time intervals. The predetermined time interval is set to be less than or equal to the time required for the relay device 2 to move by a distance that would degrade the communication quality between the information terminal 1 and the relay device 2, for example, 100 milliseconds.

[0027] Information terminal 1 estimates the latest skeletal state of user U based on information indicating the movement state of relay devices 2 (e.g., acceleration or angular velocity) measured by inertial sensors on information terminal 1 and multiple relay devices 2, and the positions of information terminal 1 and multiple relay devices 2 identified at the previous processing timing. At the first processing timing, the positions of information terminal 1 and multiple relay devices 2 at the previous processing timing are assumed to be the initial positions corresponding to, for example, user U standing upright.

[0028] For example, the information terminal 1 displays a screen for setting the initial position and accepts input from user U regarding the position where the information terminal 1 is being held and the body parts to which the multiple relay devices 2 are attached, thereby determining the initial positions of the information terminal 1 and the multiple relay devices 2. The information terminal 1 accepts input of approximate location names indicating body parts, such as "head," "right wrist," "left wrist," "right ankle," and "left ankle," and identifies the location stored in association with the accepted name as the initial position.

[0029] Information terminal 1 estimates the latest skeletal state of user U by determining the distance and direction that information terminal 1 and the multiple relay devices 2 have moved between the previous processing timing and the latest processing timing, based on at least one of the acceleration and angular velocity (hereinafter referred to as "acceleration or angular velocity") indicated by the detection data input from sensors on information terminal 1 and the multiple relay devices 2 (S1 in Figure 3). As an example, information terminal 1 generates the latest skeletal data by inputting the positions of information terminal 1 and the multiple relay devices 2 at the previous processing timing, and the acceleration or angular velocity of information terminal 1 and the multiple relay devices 2, into a skeletal estimation model that outputs skeletal data indicating the changed skeletal state by inputting the distance and direction that multiple positions in the skeletal system have moved.

[0030] Figure 4 shows an example of skeletal data generated by information terminal 1. As shown in Figure 4, skeletal data is data that shows the positional relationships of major bones and joints in a human body. Skeletal data may be image data, or it may be data that shows the positional coordinates of feature points of multiple bones and joints. The skeletal estimation model updates the skeletal data by changing the position or orientation of the affected joints and bones by moving a specified part in the immediately preceding skeletal data by a specified distance in a specified direction. The skeletal estimation model may be a program that generates skeletal data by being executed by the processor of information terminal 1, or it may be a program that generates skeletal data by being executed by a processor outside of information terminal 1.

[0031] Returning to Figure 3, information terminal 1 creates a radio wave propagation model based on the generated skeletal data (S2 in Figure 3). Figure 5 is a diagram illustrating the radio wave propagation model. As shown in Figure 5, for example, based on the radio waves transmitted by relay device 2 toward information terminal 1, direct waves, reflected waves, and diffracted waves are generated, and these radio waves reach information terminal 1. Therefore, information terminal 1 creates a radio wave propagation model for each of the multiple relay devices 2, showing the intensity and phase of the reflected and diffracted waves relative to the direct wave.

[0032] Furthermore, in parallel with creating the radio wave propagation model, the information terminal 1 may estimate the latest position and orientation of the information terminal 1 and the multiple relay devices 2 based on the generated skeletal data (S3 in Figure 3). The latest position and orientation of the information terminal 1 and the multiple relay devices 2 are, for example, the relative position and relative orientation of each of the multiple relay devices 2 with respect to the position of the information terminal 1. The information terminal 1 estimates the latest position of the relay device 2 as a position that is separated from the approximate position, which is the position of the skeleton corresponding to the name of the mounting part of the information terminal 1 and the multiple relay devices 2 in the latest skeletal data, by the distance between the approximate position and the position where the information terminal 1 and the multiple relay devices 2 are mounted.

[0033] Figure 6 is a diagram illustrating the process of estimating position. In Figure 6, the star H indicates the position of the head in the skeletal data. The hexagonal shape in Figure 6 indicates the relay device 2 attached to the left ear. Thus, even if the attachment position is "head," the position of the head in the skeletal data differs from the actual position where the relay device 2 is attached. Therefore, the information terminal 1 calculates the position coordinates of the information terminal 1 and the multiple relay devices 2 by adding a difference amount based on the distance and direction to the attachment positions of the information terminal 1 and the multiple relay devices 2 to the position coordinates corresponding to the attachment site in the skeletal data. As a result, the information terminal 1 can estimate the relative positions of the multiple relay devices 2 with high accuracy, relative to the information terminal 1.

[0034] The information terminal 1 estimates the relative positions of multiple relay devices 2 based on skeletal data, and simultaneously estimates the positions and orientations of multiple relay devices 2 based on the created radio wave propagation model and position estimation radio waves transmitted by each of the multiple relay devices 2 (S4 in Figure 3). The position estimation radio waves are radio waves transmitted at a predetermined intensity in a predetermined direction from the relay device 2. The position estimation radio waves may also include a signal (RSSI signal) indicating the intensity of the radio waves received by the relay device 2 from the information terminal 1. The information terminal 1 determines the relative positions of multiple relay devices 2 with respect to the information terminal 1 by estimating the orientation of the relay device 2 based on the direction from which the radio waves were received and estimating the distance to the relay device 2 based on the intensity of the received radio waves.

[0035] Incidentally, when the radio wave conditions received by the information terminal 1 from multiple relay devices 2 are poor, the accuracy of the position estimated in S4 based on the radio waves is low. Also, when the accuracy of the inertial sensors provided on the information terminal 1 and the multiple relay devices 2 is low, the accuracy of the position estimated in S3 based on the skeletal data is low. Therefore, when the information terminal 1 estimates the position (hereinafter sometimes referred to as the "first estimated position") and orientation (hereinafter sometimes referred to as the "first estimated orientation") of the multiple relay devices 2 based on the skeletal data, it may determine the position and orientation of the multiple relay devices 2 based on the position (hereinafter sometimes referred to as the "second estimated position") and orientation (hereinafter sometimes referred to as the "second estimated orientation") estimated based on the radio waves, as well as the first estimated position and first estimated orientation (S5 in Figure 3).

[0036] As will be described in detail later, the information terminal 1 calculates the position and orientation of multiple relay devices 2 based on a first confidence level for the first estimated position and first estimated orientation based on skeletal data, and a second confidence level for the second estimated position and second estimated orientation based on radio waves. The information terminal 1 calculates the position of multiple relay devices 2 by weighting the first estimated position and the second estimated position using weights corresponding to the first confidence level for the first estimated position estimated based on skeletal data and the second confidence level for the second estimated position estimated based on radio waves. The information terminal 1 may also calculate the orientation of multiple relay devices 2 by weighting the first estimated orientation and the second estimated orientation using weights corresponding to the first confidence level and the second confidence level.

[0037] [Configuration of Information Terminal 1 and Relay Device 2] Figure 7 shows the configuration of information terminal 1 and relay device 2. Information terminal 1 includes an inertial sensor 11, a receiving antenna 12, a receiving antenna 13, a storage unit 14, and a control unit 15. The control unit 15 includes a skeleton estimation unit 151, a model creation unit 152, a first position estimation unit 153, a radio wave measurement unit 154, a second position estimation unit 155, a weight determination unit 156, and a position identification unit 157. Relay device 2 includes an inertial sensor 21, a transmitting antenna 22, a data generation unit 23, and a transmitting antenna 24.

[0038] First, let's describe the various parts of the relay device 2. The inertial sensor 21 is a sensor that can measure the acceleration or angular velocity of the relay device 2. The inertial sensor 21 is a detection unit that outputs detection data indicating acceleration or angular velocity to the skeletal estimation unit 151. The inertial sensor 21 is, for example, a 6-axis sensor, but it may be of other types.

[0039] The data generation unit 23 generates data that the relay device 2 transmits to the information terminal 1. The data generation unit 23 generates, for example, position estimation data. The transmitting antenna 24 transmits position estimation radio waves containing the position estimation data.

[0040] Although not shown in Figure 7, the relay device 2 may further have a communication circuit for sending and receiving data to and from the base station 3. The relay device 2 transmits data received from the information terminal 1 to the base station 3 via the communication circuit, and transmits data received from the base station 3 to the information terminal 1 via the communication circuit.

[0041] Next, the various parts of the information terminal 1 will be described. The inertial sensor 11 is a sensor that can measure the acceleration or angular velocity of the information terminal 1. The inertial sensor 11 is, for example, a 6-axis sensor, but it may be of other types. The inertial sensor 11 inputs the data indicating the detected acceleration or angular velocity to the skeletal estimation unit 151.

[0042] The receiving antenna 12 is an antenna for receiving acceleration or angular velocity from the relay device 2. The receiving antenna 12 is an antenna for receiving data indicating acceleration or angular velocity transmitted from the relay device 2, for example, via Bluetooth®. The receiving antenna 12 functions as a data receiving unit that receives detection data from each of the multiple relay devices 2. The receiving antenna 12 inputs the received data indicating acceleration or angular velocity to the skeletal estimation unit 151.

[0043] The receiving antenna 13 is an antenna for receiving data and position estimation radio waves received by the relay device 2 from the base station 3. The receiving antenna 13 supports frequencies higher than those that the receiving antenna 12 can receive, and can receive radio waves in the millimeter wave band or terahertz band, for example.

[0044] The storage unit 14 has storage media such as ROM (Read Only Memory) and RAM (Random Access Memory). The storage unit 14 stores the program executed by the control unit 15. The storage unit 14 also stores the position and orientation of the relay devices 2 in association with the relay device ID and time for identifying the multiple relay devices 2.

[0045] The control unit 15 includes, for example, a CPU (Central Processing Unit). By executing a program stored in the memory unit 14, the control unit 15 functions as a skeleton estimation unit 151, a model creation unit 152, a first position estimation unit 153, a radio wave measurement unit 154, a second position estimation unit 155, a weight determination unit 156, and a position identification unit 157.

[0046] The skeleton estimation unit 151 generates skeleton data representing the skeleton of a person wearing multiple relay devices 2, estimated based on multiple detection data corresponding to multiple relay devices 2. The skeleton estimation unit 151 generates skeleton data based on the position of each of the multiple relay devices 2 identified immediately before by the position identification unit 157 and the multiple detection data.

[0047] Specifically, the skeleton estimation unit 151 generates the latest skeleton data based on the position and orientation of the information terminal 1 and the multiple relay devices 2 immediately before, the previous skeleton data, the acceleration or angular velocity of the information terminal 1 detected by the inertial sensor 11, and the acceleration or angular velocity of the multiple relay devices 2 indicated by the multiple detection data corresponding to the multiple relay devices 2. The skeleton estimation unit 151 stores the generated skeleton data in the storage unit 14.

[0048] The skeleton estimation unit 151 generates skeleton data by inputting multiple detection data corresponding to multiple relay devices 2 into a skeleton estimation model that outputs skeleton data. As outlined above, the skeleton estimation unit 151 inputs the positions of the information terminal 1 and multiple relay devices 2 at the previous processing timing, as well as the acceleration or angular velocity of the information terminal 1 and multiple relay devices 2, into a skeleton estimation model that outputs skeleton data indicating the state of the skeleton after a change by inputting the distance and direction in which multiple positions in the skeleton have moved. The skeleton estimation unit 151 generates the skeleton data output from the skeleton estimation model as the latest skeleton data.

[0049] It is conceivable that the appropriate skeletal estimation model may differ depending on the type or number of relay devices 2. For example, a skeletal estimation model created by machine learning using detection data output from 10 sensors when 10 relay devices 2 are installed may not be suitable when only 5 relay devices 2 are installed. Similarly, a skeletal estimation model created by machine learning using detection data output from a 6-axis inertial sensor may not be suitable when the relay device 2 has a 3-axis inertial sensor.

[0050] Therefore, the skeleton estimation unit 151 may generate skeleton data using a skeleton estimation model selected from a plurality of candidate skeleton estimation models based on the type or number of multiple relay devices 2. By operating in this manner, the accuracy of the latest skeleton data generated by the skeleton estimation unit 151 can be improved.

[0051] The model creation unit 152 creates multiple radio wave propagation models between the information terminal 1 and each of the multiple relay devices 2 based on mounting position data indicating the approximate terminal position where the information terminal 1 is attached, the approximate device positions where each of the multiple relay devices 2 is attached, and skeletal data generated by the skeletal estimation unit 151. The approximate terminal position and approximate device position are represented by the names of body parts such as "right hand," "right wrist," and "right ear," and the model creation unit 152 receives input for the approximate terminal position and approximate device position from, for example, user U. The model creation unit 152 may also acquire the positions specified by user U in a body image displayed on the screen of the information terminal 1, such as in Figure 1, as the approximate terminal position and approximate device position.

[0052] The model creation unit 152 estimates the terminal position, which is the location of information terminal 1, as a position located a distance from the approximate terminal position in the skeletal data, corresponding to the distance corresponding to the approximate terminal position, using a method described with reference to Figure 6, for example. The model creation unit 152 also estimates the device position, which is the location of relay device 2, as a position located a distance from the approximate device position in the skeletal data, corresponding to the distance corresponding to the approximate device position. The model creation unit 152 creates a radio wave propagation model based on the terminal position, the device position, and the position and orientation of the skeleton indicated by the skeletal data within a predetermined range from the terminal position and the device position. The model creation unit 152 creates a radio wave propagation model that reflects the effects of reflected waves generated when radio waves emitted from the location of relay device 2 are reflected by a part of the human body corresponding to the skeletal data as they propagate to the terminal position, or diffracted waves generated when radio waves emitted from the location of relay device 2 are bent behind a part of the human body corresponding to the skeletal data as they propagate to the terminal position.

[0053] The model creation unit 152 may be a model created by machine learning using terminal position, device position, and skeletal position and orientation, along with a known propagation model, as training data. In this case, the model creation unit 152 outputs a radio wave propagation model upon receiving input of terminal position, device position, and skeletal data identified in the previous processing timing.

[0054] The model creation unit 152 may further determine the terminal position and device position based on the results of identifying the relationship between the person's body in the captured image data of a person wearing multiple relay devices 2 and the position of each of the multiple relay devices 2. The model creation unit 152 may acquire captured image data of a person wearing multiple relay devices 2 from an external device and correct the estimated terminal position and device position based on the results of identifying the relationship between the person's body in the captured image data and the position of each of the multiple relay devices 2.

[0055] For example, suppose user U sets the attachment location for relay device 2 to "right wrist," but in reality, relay device 2 is attached to the right elbow. In such a case, if the model creation unit 152 determines that the approximate position is "right elbow" based on the captured image data, it corrects the device position corresponding to the right wrist to the position corresponding to the right elbow in the skeletal data. By operating in this way, the model creation unit 152 can create an appropriate radio wave propagation model even if user U makes a mistake in setting the attachment location or attaching relay device 2 to the wrong location.

[0056] Incidentally, the intensity of reflected and diffracted waves changes based on the material and shape of the clothing worn by user U. Therefore, the model creation unit 152 may create a radio wave propagation model based on data indicating the clothing worn by a person wearing multiple relay devices 2. The data indicating the clothing is, for example, set in advance by user U and stored in the storage unit 14. By configuring the model creation unit 152 in this way, it is possible to prevent the accuracy of the radio wave propagation model from decreasing due to the clothing worn by user U.

[0057] Furthermore, the generation state of reflected and diffracted waves changes depending on the presence and shape of objects around the user U. Therefore, the model creation unit 152 may create a radio wave propagation model based on the shapes and positions of objects captured in the image data of the surroundings of the multiple relay devices 2. By configuring the model creation unit 152 in this way, it is possible to suppress differences in the accuracy of the radio wave propagation model depending on the conditions around the user U.

[0058] The first position estimation unit 153 determines a first estimated position, which is the position of each of the multiple relay devices 2, based on multiple detection data output by the multiple inertial sensors 21. For example, the first position estimation unit 153 determines a first estimated position, which is the position of each of the multiple relay devices 2, based on mounting position data indicating the approximate position where each of the multiple relay devices 2 is mounted, and skeleton data generated by the skeleton estimation unit 151 based on the multiple detection data. As explained with reference to Figure 6, for example, the first position estimation unit 153 estimates a position as the first estimated position that is a distance corresponding to the approximate position from the position corresponding to the approximate position in the skeleton data.

[0059] Specifically, the first position estimation unit 153 first acquires attachment location data that indicates the approximate locations of the attachment locations of multiple relay devices 2, which have been set by the user U, for example, via an operating device on the information terminal 1. The attachment location data is data that includes information for identifying the attachment location, such as "right wrist" or "left wrist".

[0060] The first position estimation unit 153 calculates the position coordinates of the information terminal 1 and the multiple relay devices 2 by adding a difference amount based on the distance and direction to the mounting positions of the information terminal 1 and the multiple relay devices 2 to the approximate position coordinates corresponding to the mounting locations of the information terminal 1 and the multiple relay devices 2 in the skeletal data, as described above. The first position estimation unit 153 notifies the position identification unit 157 of the determined first estimated position.

[0061] Since the skeletal data is generated by estimating the most likely skeletal state based on acceleration or angular velocity measured at multiple relay devices 2, even if there are errors in the acceleration or angular velocity measured at some of the relay devices 2, the effects of these errors are less likely to appear in the skeletal data. Therefore, the accuracy of the first estimated position is improved when the first position estimation unit 153 uses the skeletal data to estimate the position of the relay devices 2.

[0062] The first position estimation unit 153 may determine the orientation of each of the multiple relay devices 2 based on mounting position data indicating the approximate position where each of the multiple relay devices 2 is attached, and skeletal data. For example, the first position estimation unit 153 determines the orientation of each of the multiple relay devices 2 at the first estimated position based on the orientation of bones near the first estimated position determined based on the mounting position data in the skeletal data created by the skeletal estimation unit 151.

[0063] The first position estimation unit 153 may acquire image data from an external device showing a person wearing multiple relay devices 2, and correct the first estimated position based on the results of identifying the relationship between the person's body in the image data and the position of each of the multiple relay devices 2. For example, suppose user U sets the attachment site for the relay device 2 to "right wrist," but the relay device 2 is actually attached to the right elbow. In such a case, if the first position estimation unit 153 determines that the approximate position is "right elbow" based on the image data, it corrects the first estimated position corresponding to the right wrist to the position corresponding to the right elbow in the skeletal data. By operating in this manner, the first position estimation unit 153 can correctly estimate the position of the relay device 2 even if user U makes a mistake in setting the attachment position or attaching the relay device 2 to the wrong position.

[0064] The first position estimation unit 153 may estimate the position of the relay device 2 without using skeletal data. For example, the first position estimation unit 153 may determine the first estimated position by determining the amount of movement and direction of movement from the position of the relay device 2 previously determined by the position determination unit 157, based on the acceleration or angular velocity of the relay device 2 output by the inertial sensor 21, and adding the determined amount of movement in the direction of the determined direction of movement to the position of the relay device 2 immediately before.

[0065] The radio wave measurement unit 154 measures the radio wave conditions, including the direction of arrival and intensity of the radio waves received from each of the multiple relay devices 2, by sweeping the radio waves within a sweep range determined based on the positions of the multiple relay devices 2 identified by, for example, the second position estimation unit 155 or the position identification unit 157 at the immediately preceding processing timing.

[0066] Specifically, the radio wave measurement unit 154 controls the receiving antenna 13 to sequentially orient it in multiple directions within the sweep range corresponding to each of the multiple second estimated positions, from among the multiple directions in which the receiving antenna 13 can orient itself. While sequentially orienting the receiving antenna 13 in multiple directions, the radio wave measurement unit 154 measures the intensity of the radio waves received by the receiving antenna 13 in each direction, and determines the direction with the greatest intensity as the direction of arrival of the radio waves. Based on the relay device ID included in the position estimation signal transmitted by each of the multiple relay devices 2, the radio wave measurement unit 154 identifies the direction of arrival and intensity of the radio waves transmitted by each of the multiple relay devices 2.

[0067] The radio wave measurement unit 154 may further determine the sweep range based on the orientation of each of the multiple relay devices 2 identified by the second position estimation unit 155 or the position identification unit 157 at the immediately preceding processing timing. The radio wave measurement unit 154 includes the direction from which the reflected waves arrive in the sweep range when the orientation of the relay device 2 is such that radio waves transmitted by the relay device 2 may be reflected by any part of the body and reach the information terminal 1.

[0068] The second position estimation unit 155 determines a second estimated position, which is the position of multiple relay devices 2, based on the radio wave conditions. The second position estimation unit 155 determines that, with the position of the information terminal 1 as a reference, the relay devices 2 are located at a distance corresponding to the intensity of the radio waves measured by the radio wave measurement unit 154, in the direction of arrival of the radio waves measured by the radio wave measurement unit 154. The second position estimation unit 155 determines the second estimated position of the multiple relay devices 2 based on the corrected direction and corrected intensity obtained by correcting the direction of arrival and intensity of the position estimation radio waves using a radio wave propagation model.

[0069] The second position estimation unit 155 determines the second estimated position by, for example, inputting the direction of arrival and intensity of the position estimation radio waves into a radio wave propagation model and obtaining the correction direction and correction intensity output from the radio wave propagation model. The second position estimation unit 155 may also obtain the correction direction and correction intensity by inputting the direction of arrival and intensity of the position estimation radio waves into a calculation formula shown by the radio wave propagation model. For each of the multiple relay devices 2, the second position estimation unit 155 identifies the second estimated position of the relay device 2 based on the position of the information terminal 1. The second position estimation unit 155 notifies the weight determination unit 156 of the second estimated position.

[0070] The weight determination unit 156 determines the first weight for the first estimated position and the second weight for the second estimated position in the calculation performed by the position identification unit 157 to identify the position of each of the multiple relay devices 2, based on the first confidence level of the detection data output by the inertial sensors 11 and 21 or the second confidence level of the radio wave conditions received by the receiving antenna 13. For example, the weight determination unit 156 determines that the second confidence level is lower the smaller the intensity of the radio waves received by the receiving antenna 13, or the greater the dispersion of the direction of arrival of the radio waves, and reduces the ratio of the second weight to the first weight.

[0071] On the other hand, the weight determination unit 156 may determine that the greater the difference between the temperature or humidity around the relay device 2 and a predetermined temperature or humidity suitable for the operation of the relay device 2, the lower the first confidence level and may increase the ratio of the second weight to the first weight. In order to determine the temperature or humidity around multiple relay devices 2, the weight determination unit 156 may acquire environmental data indicating temperature or humidity from a temperature sensor or humidity sensor (not shown), or it may acquire environmental data input by the user U.

[0072] The weight determination unit 156 may increase the ratio of the second weight to the first weight if the acceleration or angular velocity output by the inertial sensors 11 and 21 shows abnormal values ​​outside a preset normal range. The weight determination unit 156 may decrease the ratio of the second weight to the first weight as the intensity of the radio waves received by the receiving antenna 12 decreases.

[0073] Furthermore, the temperature may differ depending on the location of the multiple relay devices 2, and the state of the radio wave propagation path from the relay devices 2 to the information terminal 1 may differ. Therefore, the weight determination unit 156 may determine a first weight and a second weight for each location where the multiple relay devices 2 are installed. The weight determination unit 156 may also determine a first weight and a second weight for each of the multiple relay devices 2.

[0074] The location identification unit 157 identifies the location of each of the multiple relay devices 2 by performing a predetermined calculation that includes the first estimated location and the second estimated location as variables. For example, the location identification unit 157 uses the average or median of the first estimated location and the second estimated location as the location of each of the multiple relay devices 2.

[0075] However, as described above, depending on the environment surrounding the inertial sensors 11 and 21, the first confidence level of the first estimated position based on the detection data output by the inertial sensors 11 and 21 may be low. Also, depending on the environment surrounding the information terminal 1 and relay device 2, the second confidence level of the second estimated position based on the radio wave conditions received by the receiving antenna 13 may be low.

[0076] Therefore, the position determination unit 157 functions as a Kalman filter in conjunction with, for example, the weight determination unit 156, to determine the positions of multiple relay devices 2 with higher accuracy than the first estimated position and the second estimated position. The position determination unit 157 determines the position of each of the multiple relay devices 2 by weighting the first estimated position and the second estimated position based on the first weight and the second weight.

[0077] Specifically, the position identification unit 157 identifies the position of each of the multiple relay devices 2 using, for example, the following calculation formula 1. If the weight determination unit 156 determines different first and second weights for each of the multiple relay devices 2 installed, the position identification unit 157 identifies the position for each of the multiple relay devices 2 installed using calculation formula 1. If the weight determination unit 156 determines different first and second weights for each of the multiple relay devices 2, the position identification unit 157 identifies the position for each of the multiple relay devices 2 using calculation formula 1. [Equation 1] The position of relay device 2 = 1st estimated position × 1st weight + 2nd estimated position × 2nd weight However, the first weight + the second weight = 1

[0078] The positioning unit 157 improves the accuracy of the position of each of the identified relay devices 2 by performing a weighted average based on the reliability of the first estimated position based on the detection data of the inertial sensors 11 and 21, and the second estimated position based on radio waves.

[0079] [Processing flow in communication system S] Figure 8 is a flowchart showing the processing flow in the communication system S. The flowchart shown in Figure 8 starts from the moment the power to the information terminal 1 and the relay device 2 is turned on.

[0080] The relay device 2 transmits detection data indicating the acceleration or angular velocity detected by the inertial sensor 21 (S21) and transmits radio waves for position estimation (S22). The information terminal 1 receives the detection data transmitted by the relay device 2 with the receiving antenna 12 (S11). The skeleton estimation unit 151 estimates the current state of the skeleton and generates skeleton data based on the previous state of the skeleton, the previous positions of the multiple relay devices 2, and the acceleration or angular velocity of the multiple relay devices 2 indicated by the detection data (S12). Subsequently, the first position estimation unit 153 estimates the positions (first estimated positions) of the multiple relay devices 2 based on the generated skeleton data (S13).

[0081] In parallel with the operation of the first position estimation unit 153, the model creation unit 152 creates a radio wave propagation model (S14). The radio wave measurement unit 154 receives position estimation radio waves transmitted by the relay devices 2 by sweeping the radio waves in a direction based on the estimated positions of the multiple relay devices 2, and the second position estimation unit 155 estimates the position of each of the multiple relay devices 2 (second estimated position) based on the radio wave propagation model and the radio wave conditions indicated by the received position estimation radio waves (S15).

[0082] The positioning unit 157 determines the position of each of the multiple relay devices 2 by weighted averaging using the first weight and the second weight (S16). The latest position determined by the positioning unit 157 is used when the model creation unit 152 creates a radio wave propagation model and when the skeleton estimation unit 151 generates skeleton data at the next processing timing.

[0083] [Effects of communication system S] As explained above, the information terminal 1 creates multiple radio wave propagation models between the information terminal 1 and each of the multiple relay devices 2 based on mounting position data indicating the approximate terminal position on which the information terminal 1 is attached, the approximate device positions on which each of the multiple relay devices 2 is attached, and skeletal data. The second position estimation unit 155 then determines the estimated positions of the multiple relay devices 2 based on the corrected direction and corrected intensity obtained by correcting the direction and intensity of the radio waves used for position estimation using the radio wave propagation model. With the information terminal 1 configured in this way, even if the user U's posture changes, the positions of the multiple relay devices 2 are estimated by taking into account the effects of reflected and diffracted waves according to the posture, thus improving the estimation accuracy.

[0084] In the above description, the control unit 15 has a first position estimation unit 153, and the position identification unit 157 performs a weighted average of the first estimated position and the second estimated position. However, the control unit 15 may not have a first position estimation unit 153, and the position identification unit 157 may determine the second estimated position to be the position of the relay device 2.

[0085] Furthermore, this invention will make it possible to contribute to Goal 9 of the United Nations-led Sustainable Development Goals (SDGs), "Build resilient infrastructure, promote inclusive and sustainable industrialization and foster innovation."

[0086] Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of its gist. For example, all or part of the apparatus can be configured by functionally or physically distributing and integrating in any unit. Furthermore, new embodiments resulting from any combination of multiple embodiments are also included in the embodiments of the present invention. The effects of the new embodiments resulting from the combinations are combined with the effects of the original embodiments. [Explanation of Symbols]

[0087] 1. Information terminal 2. Relay device 3 base station 11. Inertial Sensor 12 Receiving antenna 13 Receiving antenna 14 Storage section 15 Control Unit 21 Inertial Sensor 22 Transmitting Antennas 23 Data Generation Unit 24 Transmitting Antennas 151 Skeleton Estimation Section 152 Model Creation Department 153 1st position estimation section 154 Radio wave measurement section 155 Second position estimation section 156 Decision Section 157 Location identification part

Claims

1. The system comprises an information terminal and a plurality of relay devices that can communicate with the information terminal and a base station of a mobile phone network while being worn on a person's body. Each of the plurality of relay devices has a detection unit that outputs detection data of at least one of the acceleration and angular velocity of the relay device. The aforementioned information terminal is A data receiving unit that receives the detection data from each of the multiple relay devices, A skeleton estimation unit that generates skeleton data indicating the skeleton of a person to whom the multiple relay devices are attached, estimated based on the multiple detection data corresponding to the multiple relay devices, A model creation unit creates multiple radio wave propagation models between the information terminal and each of the multiple relay devices based on mounting position data indicating the approximate terminal location where the information terminal is mounted, the approximate device locations where each of the multiple relay devices is mounted, and the skeletal data. A radio wave measuring unit measures the radio wave conditions, including the direction of arrival and intensity of radio waves received from each of the plurality of relay devices, by sweeping radio waves within a predetermined sweep range. A position estimation unit determines the estimated positions of the plurality of relay devices based on the corrected direction and corrected intensity obtained by correcting the aforementioned direction of arrival and the aforementioned intensity using the radio wave propagation model, It has, The radio wave measurement unit sweeps the radio waves within the sweep range determined based on the estimated position determined by the position estimation unit immediately beforehand. Communication system.

2. The model creation unit estimates a terminal position, which is the location of the information terminal, at a distance corresponding to the terminal approximate position in the skeletal data, and estimates a device position, which is the location of the relay device, at a distance corresponding to the device approximate position in the skeletal data, and creates the radio wave propagation model based on the terminal position, the device position, and the position and orientation of the skeleton indicated by the skeletal data within a predetermined range from the terminal position and the device position. The communication system according to claim 1.

3. The model creation unit creates a radio wave propagation model that reflects the effects of reflected waves generated when radio waves emitted from the device location are reflected by a part of the human body corresponding to the skeletal data as they propagate to the terminal location, or diffracted waves generated when radio waves emitted from the device location bend around the back of a part of the human body corresponding to the skeletal data as they propagate to the terminal location. The communication system according to claim 2.

4. The model creation unit further determines the terminal position and the device position based on the results of determining the relationship between the person's body captured in the image data taken by the person wearing the multiple relay devices and the position of each of the multiple relay devices. The communication system according to claim 2.

5. The model creation unit further creates the radio wave propagation model based on data indicating the clothing worn by the person wearing the multiple relay devices. A communication system according to any one of claims 1 to 4.

6. The model creation unit further creates the radio wave propagation model based on the shapes and positions of objects captured in the image data of the surrounding area of ​​the multiple relay devices. A communication system according to any one of claims 1 to 4.

7. The skeleton estimation unit generates the skeleton data by inputting a plurality of detection data corresponding to the plurality of relay devices into a skeleton estimation model that outputs the skeleton data. The communication system according to claim 1.

8. The skeleton estimation unit generates the skeleton data using the skeleton estimation model selected from a plurality of candidate skeleton estimation models based on the type or number of the plurality of relay devices. The communication system according to claim 7.

9. The position estimation unit further estimates the orientation of each of the plurality of relay devices, The radio wave measurement unit further determines the sweep range based on the orientation of each of the plurality of relay devices estimated by the position estimation unit. The communication system according to claim 1.

10. An information terminal capable of communicating with multiple relay devices, A data receiving unit that receives detection data from each of the plurality of relay devices indicating at least one of the acceleration and angular velocity of the relay device, A skeleton estimation unit that generates skeleton data indicating the skeleton of a person to whom the multiple relay devices are attached, estimated based on the multiple detection data corresponding to the multiple relay devices, A model creation unit creates multiple radio wave propagation models between the information terminal and each of the multiple relay devices based on mounting position data indicating the approximate terminal location where the information terminal is mounted, the approximate device locations where each of the multiple relay devices is mounted, and the skeletal data. A radio wave measuring unit that measures the direction of arrival and intensity of radio waves received from each of the multiple relay devices by sweeping radio waves within a predetermined sweep range, A position estimation unit determines the estimated positions of the plurality of relay devices based on the corrected direction and corrected intensity obtained by correcting the aforementioned direction of arrival and the aforementioned intensity using the radio wave propagation model, It has, The radio wave measurement unit sweeps the radio waves within the sweep range determined based on the estimated position determined by the position estimation unit immediately beforehand. Information terminal.

11. A program executed by a processor in an information terminal capable of communicating with multiple relay devices, wherein the execution of the program causes the processor to... A data receiving unit that receives detection data from each of the plurality of relay devices indicating at least one of the acceleration and angular velocity of the relay device, A skeleton estimation unit that generates skeleton data indicating the skeleton of a person to whom the multiple relay devices are attached, estimated based on the multiple detection data corresponding to the multiple relay devices, A model creation unit creates multiple radio wave propagation models between the information terminal and each of the multiple relay devices based on mounting position data indicating the approximate terminal location where the information terminal is mounted, the approximate device locations where each of the multiple relay devices is mounted, and the skeletal data. A radio wave measuring unit measures the radio wave conditions, including the direction of arrival and intensity of radio waves received from each of the plurality of relay devices, by sweeping radio waves within a predetermined sweep range. A position estimation unit determines the estimated positions of the plurality of relay devices based on the corrected direction and corrected intensity obtained by correcting the aforementioned direction of arrival and the aforementioned intensity using the radio wave propagation model, To make it function as, The radio wave measurement unit sweeps the radio waves within the sweep range determined based on the estimated position determined by the position estimation unit immediately beforehand. program.

Images