Information Processing Apparatus, Display System, Display Control Method, and Program

Abstract

The present invention provides an information processing device enabling a test subject to easily recognize for themselves whether they are breathing correctly. An acquisition unit (2) acquires image data indicating at least the chest and the abdomen of the test subject. A detection unit (4) uses the image data to detect a displacement of each of the chest and the abdomen of the test subject. A display control unit (6) causes a motion image of each of the chest and the abdomen to be displayed on the basis of the detected displacements of the chest and the abdomen. If the motion of the chest and the motion of the abdomen are synchronized with each other, the display control unit (6) causes the chest motion image and the abdomen motion image to have the same display appearance. If the motion of the chest and the motion of the abdomen are not synchronized with each other, the display control unit (6) causes the chest motion image and the abdomen motion image to have mutually different display appearances.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, a display system, a display control method, and a program. to the muscle

Background Art

[0002] For improving and maintaining health, it is desirable to breathe with the correct breathing method. And in order to confirm whether one is breathing with the correct breathing method, movements of the chest and abdomen are detected. In relation to this, Patent Document 1 discloses measuring the movement of the chest and the movement of the abdomen from an image obtained by photographing the chest and abdomen of a patient who is breathing. Further, Non-Patent Document 1 discloses performing non-contact measurement of natural breathing movement by projecting pattern light onto the trunk.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Non-Patent Documents

[0004]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] ​In respiratory training using correct breathing methods, it is important that the anterior-posterior movement of the chest and the anterior-posterior movement of the abdomen of the subject undergoing training are synchronized with each other. And it is desirable that the subject himself / herself can easily recognize whether he / she is performing correct breathing. Here, the technologies according to the above-mentioned patent documents and non-patent documents are for doctors and the like to diagnose the respiratory state of patients. That is, the technologies according to the above-mentioned patent documents and non-patent documents do not assume that patients grasp their own respiratory state. Therefore, even if the patient (subject) views the analysis screen disclosed in the above-mentioned patent documents and non-patent documents, it is difficult to recognize whether he / she is performing correct breathing.

[0006] An object of the present disclosure is to solve such problems, and to provide an information processing apparatus, a display control method, and a program that enable a subject to easily recognize whether he / she is performing correct breathing.

Means for Solving the Problems

[0007] The information processing apparatus according to the present disclosure includes an acquisition unit that acquires image data showing at least the chest and abdomen of a subject undergoing respiratory training, a detection unit that detects the displacement amount of each of the chest and abdomen of the subject using the image data, and a display control unit that controls to display a movement image showing the anterior-posterior movement of the subject of each of the chest and abdomen based on the detected displacement amounts of the chest and abdomen. The display control unit controls such that when the movement of the chest and the movement of the abdomen are synchronized with each other, the display form of the chest movement image, which is the movement image of the chest, and the display form of the abdomen movement image, which is the movement image of the abdomen, are the same as each other, and when the movement of the chest and the movement of the abdomen are not synchronized with each other, the display form of the chest movement image and the display form of the abdomen movement image are different from each other.

[0008] In addition, the display control method according to the present disclosure acquires image data showing at least the chest and abdomen of a subject performing breathing training, detects the displacement amounts of the chest and abdomen of the subject using the image data, and based on the detected displacement amounts of the chest and abdomen, controls so that movement images showing the movement of the subject in the front-rear direction of the chest and abdomen are displayed respectively. When the movement of the chest and the movement of the abdomen are synchronized with each other, the display form of the chest movement image, which is the movement image of the chest, and the display form of the abdomen movement image, which is the movement image of the abdomen, are the same as each other. When the movement of the chest and the movement of the abdomen are not synchronized with each other, control is performed so that the display form of the chest movement image and the display form of the abdomen movement image are different from each other.

[0009] In addition, the program according to the present disclosure causes a computer to execute steps of acquiring image data showing at least the chest and abdomen of a subject performing breathing training, detecting the displacement amounts of the chest and abdomen of the subject using the image data, and based on the detected displacement amounts of the chest and abdomen, controlling so that movement images showing the movement of the subject in the front-rear direction of the chest and abdomen are displayed respectively. When the movement of the chest and the movement of the abdomen are synchronized with each other, the display form of the chest movement image, which is the movement image of the chest, and the display form of the abdomen movement image, which is the movement image of the abdomen, are the same as each other. When the movement of the chest and the movement of the abdomen are not synchronized with each other, control is performed so that the display form of the chest movement image and the display form of the abdomen movement image are different from each other.

Advantages of the Invention

[0010] According to the present disclosure, it is possible to provide an information processing apparatus, a display control method, and a program that enable a subject to easily recognize whether the subject is performing correct breathing.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Embodiments for Carrying out the Invention

[0012] (Outline of Embodiments According to the Present Disclosure) Prior to the description of the embodiments of the present disclosure, an outline of the embodiments according to the present disclosure will be described. FIG. 1 is a diagram showing an outline of the information processing apparatus 1 according to the embodiment of the present disclosure. The information processing apparatus 1 is a computer such as a server or a personal computer, for example.

[0013] The information processing apparatus 1 includes an acquisition unit 2, a detection unit 4, and a display control unit 6. The acquisition unit 2 has a function as acquisition means. The detection unit 4 has a function as detection means. The display control unit 6 has a function as display control means.

[0014] FIG. 2 is a flowchart showing a display control method (information processing method; support method) executed by the information processing apparatus 1 according to the embodiment of the present disclosure. The acquisition unit 2 acquires image data showing at least the chest and abdomen of the subject (step S2). Here, the "subject" is a person who performs breathing training. The acquisition unit 2 may acquire image data from, for example, an imaging device (camera) that photographs the subject. The image data may be, for example, 2 - dimensional image data such as an RGB image, or 3 - dimensional image data such as a distance image (3 - dimensional point cloud data). Note that hereinafter, the term "image" also means "image data showing an image" which is a processing target in information processing.

[0015] The detection unit 4 detects the displacement amounts of the chest and abdomen of the subject using the image data (step S4). The display control unit 6 performs control so that the motion images of the chest and abdomen are displayed based on the detected displacement amounts of the chest and abdomen (step S6). The display control unit 6 may perform control to display the motion image on a display device separate from the information processing device 1. Alternatively, the display control unit 6 may perform control to display the motion image on a display or the like provided in the information processing device 1.

[0016] Note that the "motion image" is an image showing the movement of the chest and abdomen of the subject in the front-rear direction. Note that the "front-rear direction" means the front direction or the rear direction with respect to the subject. That is, when the subject is in the supine position (supine position), the front direction corresponds to the upward direction and the rear direction corresponds to the downward direction. Further, hereinafter, the motion image showing the movement of the chest in the front-rear direction may be referred to as the "chest motion image". Also, the motion image showing the movement of the abdomen in the front-rear direction may be referred to as the "abdomen motion image".

[0017] Here, when the movement of the chest and the movement of the abdomen are synchronized with each other, the display control unit 6 performs control so that the display form of the chest motion image and the display form of the abdomen motion image are the same as each other. Also, when the movement of the chest and the movement of the abdomen are not synchronized with each other, the display control unit 6 performs control so that the display form of the chest motion image and the display form of the abdomen motion image are different from each other. Here, "the display forms are the same" means, for example, that the color expressions are the same, but is not limited to this. That is, the "display form" is not limited to the "color expression".

[0018] For example, when the display form is the "color expression", the display control unit 6 performs control so that the color expression of the chest motion image and the color expression of the abdomen motion image are the same as each other when the movement of the chest and the movement of the abdomen are synchronized with each other. Also, when the movement of the chest and the movement of the abdomen are not synchronized with each other, the display control unit 6 performs control so that the color expression of the chest motion image and the color expression of the abdomen motion image are different from each other.

[0019] Note that "movement" can be defined, for example, as the amount of displacement in the front - rear direction of each of the chest and abdomen with respect to a reference position. In this case, for example, the displacement amount of the chest becomes a positive (+) value (a value of 0 or more) when the position of the chest is in front of the reference position (chest reference position), and becomes a negative (-) value (a value less than 0) when the position of the chest is behind the reference position. Similarly, for example, the displacement amount of the abdomen becomes a positive (+) value (a value of 0 or more) when the position of the abdomen is in front of the reference position (abdomen reference position), and becomes a negative (-) value (a value less than 0) when the position of the abdomen is behind the reference position. The reference position will be described later.

[0020] Also, "movement" can be defined, for example, by the direction of movement of each of the chest and abdomen. Here, the "direction of movement" is the direction of change (increase or decrease) in the displacement amount of each of the chest and abdomen. In this case, for example, the direction of movement of the chest becomes the forward direction (positive direction) when the displacement amount of the chest is increasing (for example, in the inhalation state), and becomes the rearward direction (negative direction) when the displacement amount of the chest is decreasing (for example, in the exhalation state). Similarly, for example, the direction of movement of the abdomen becomes the forward direction (positive direction) when the displacement amount of the abdomen is increasing (for example, in the inhalation state), and becomes the rearward direction (negative direction) when the displacement amount of the abdomen is decreasing (for example, in the exhalation state).

[0021] As described above, in order to improve and maintain the health state, it is desirable to breathe using the correct breathing method. And in order to breathe correctly, it is desirable to continuously perform correct breathing training based on the guidance of a training instructor such as a therapist (hereinafter simply referred to as "instructor"). For example, when performing breathing training according to the correct breathing method, the health state such as physical functions and mental state such as low back pain can be improved.

[0022] Here, in breathing training, it is considered that the training effect is good when the subject breathes so as to satisfy the condition that the front-back movements (exercises) of the chest and abdomen are synchronized with each other ( "synchronization of the chest and abdomen"). Furthermore, it is considered that the training effect is good when the subject breathes so as to satisfy the condition that the ribs rotate sufficiently inward when exhaling (that is, the displacement amount in the left-right direction of the chest becomes sufficiently small when exhaling; "inward rotation of the ribs"). However, it is difficult for the subject himself / herself to confirm the above. That is, it is difficult for the subject to recognize his / her own breathing state.

[0023] On the other hand, since the information processing apparatus 1 according to the present disclosure is configured as described above, it is possible to display a chest movement image and an abdominal movement image so that it is visually easy to understand whether or not the movement of the chest and the movement of the abdomen are synchronized with each other. Therefore, the information processing apparatus 1 according to the present disclosure enables the subject himself / herself to easily recognize whether or not he / she is breathing correctly.

[0024] In addition, even when using a display system including an imaging device, a display device, and the information processing apparatus 1, it is possible for the subject himself / herself to easily recognize whether or not he / she is breathing correctly. Also, even when using a display control method executed by the information processing apparatus 1 and a program for executing the display control method, it is possible for the subject himself / herself to easily recognize whether or not he / she is breathing correctly.

[0025] (Embodiment 1) Hereinafter, embodiments will be described with reference to the drawings. For clarity of explanation, the following description and drawings are appropriately omitted and simplified. Also, in each drawing, the same elements are denoted by the same reference numerals, and redundant explanations are omitted as necessary.

[0026] FIG. 3 and FIG. 4 are diagrams showing the display system 20 according to Embodiment 1. FIG. 3 illustrates a state in which the display system 20 is being used for the breathing training of the subject 90. FIG. 4 is a functional block diagram showing the configuration of the display system 20. As shown in FIG. 3, preferably, the subject 90 performs breathing training in a supine position (lying on the back), but the posture of the subject 90 is not limited to the supine position. In the embodiments described below, it is assumed that the subject 90 performs breathing training in the supine position.

[0027] The display system 20 includes at least one imaging device 30, a display device 40, and an information processing device 100. The information processing device 100 corresponds to the information processing device 1 shown in FIG. 1. The information processing device 100 is communicably connected to the imaging device 30 and the display device 40 via a wired or wireless network.

[0028] The imaging device 30 photographs the subject 90 who is performing breathing training. The imaging device 30 can be installed at a position where it can photograph the chest 92 and abdomen 94 of the subject 90. When the subject 90 performs breathing training in the supine position, the imaging device 30 can be installed, for example, above the chest 92 and abdomen 94 of the subject 90. That is, the imaging device 30 can be installed at a position facing the subject 90 in the supine position. Note that the subject 90 may perform breathing training while wearing clothes. In this case, the chest 92 corresponds to the part of the subject 90's chest in the state of wearing clothes. Similarly, the abdomen 94 corresponds to the part of the subject 90's abdomen in the state of wearing clothes.

[0029] The imaging device 30 is, for example, a camera. The imaging device 30 may be a 2D camera (e.g., an RGB camera, etc.), a 3D camera (e.g., a depth sensor, LiDAR (Light Detection and Ranging), a stereo camera, etc.), or a camera including both of these (e.g., an RGB-D camera, etc.). The imaging device 30 may measure the distance to the imaged object, for example, by the ToF (Time of Flight) method.

[0030] By using the imaging device 30, the position of the subject 90 can be detected. Also, by using the imaging device 30, the movement of the subject 90 can be detected. For example, by using the imaging device 30, motion capture or the like can be realized. Further, by using the imaging device 30, skeletal data indicating the skeleton (joints) of the photographed subject 90 may be generated. The skeletal data is data indicating the positions of the joints of the subject 90. The skeletal data can be obtained, for example, when the imaging device 30 (or the information processing device 100) recognizes the joints of a person in motion.

[0031] The imaging device 30 generates image data showing at least the chest 92 and abdomen 94 of the subject 90 by photographing the subject 90. That is, the image data can show the chest 92 and abdomen 94 of the subject 90 and the images (photographed images) around them. The photographed image may be a moving image or a still image. Also, as described above, the image data may be, for example, 2D image data such as an RGB image, or 3D image data such as a distance image (3D point cloud data). Alternatively, the image data may be data showing an image in which a 2D image and a 3D image are combined. Therefore, the image data can show the 3D coordinates (position information) of the positions on the surface of the photographed subject 90 by 3D point cloud data or the like. Also, the image data may include the above-described skeletal data. The imaging device 30 transmits the generated image data to the information processing device 100.

[0032] Note that by using the image data, the positions of the chest 92 and abdomen 94 of the subject 90 can be detected. Also, by using the image data, the movements of the chest 92 and abdomen 94 of the subject 90 can be detected. Therefore, the imaging device 30 can also function as a detection device capable of detecting the position (and movement) of the subject 90.

[0033] The information processing apparatus 100 acquires image data from the imaging apparatus 30. The information processing apparatus 100 detects the displacement amounts of the chest 92 and the abdomen 94 of the subject 90 respectively using the acquired image data. Further, the information processing apparatus 100 performs control so that moving images of the chest 92 and the abdomen 94 (the "chest moving image" and the "abdomen moving image") are displayed based on the detected displacement amounts of the chest 92 and the abdomen 94. Details will be described later.

[0034] The display device 40 displays the moving images of the chest 92 and the abdomen 94 (the "chest moving image" and the "abdomen moving image") under the control of the information processing apparatus 100. The display device 40 displays information regarding the training of the subject 90. Preferably, the display device 40 displays an image for the subject 90. The display device 40 may be configured to display an image for the subject 90 when installed above the head of the subject 90. For example, the display device 40 may be configured to display an image for the subject 90 when a camera built into the display device 40 detects the face of the subject 90.

[0035] The display device 40 is arranged to display an image at a position visible to the subject 90. The display device 40 has, for example, a display for displaying an image. The display device 40 has, for example, an LCD (Liquid Crystal Display), but is not limited thereto. The display device 40 may be realized by an organic EL (Electro-Luminescence) display or a projector or the like. The display device 40 may be, for example, a smartphone or a tablet terminal or the like. Details of the content displayed by the display device 40 will be described later.

[0036] Figure 4 shows the configuration of the information processing apparatus 100 according to Embodiment 1. As shown in Figure 4, the information processing apparatus 100 includes, as its main hardware components, a control unit 102, a storage unit 104, a communication unit 106, and an interface unit 108 (IF; Interface). The control unit 102, the storage unit 104, the communication unit 106, and the interface unit 108 are interconnected via a data bus or the like. Note that the imaging device 30 and the display device 40 may also have the hardware configuration of the information processing apparatus 100 shown in Figure 4.

[0037] The control unit 102 is a processor such as a CPU (Central Processing Unit), for example. The control unit 102 has a function as an arithmetic unit that performs control processing, arithmetic processing, and the like. The storage unit 104 is a storage device such as a memory or a hard disk, for example. The storage unit 104 is, for example, a ROM (Read Only Memory) or a RAM (Random Access Memory). The storage unit 104 has a function for storing control programs, arithmetic programs, and the like executed by the control unit 102. Further, the storage unit 104 has a function for temporarily storing processing data and the like. The storage unit 104 may include a database.

[0038] The communication unit 106 performs necessary processing for communicating with the imaging device 30 and the display device 40 (and other devices) via a network. The communication unit 106 may include a communication port, a router, a firewall, and the like. The interface unit 108 (IF; Interface) is, for example, a user interface (UI). The interface unit 108 includes an input device such as a keyboard, a touch panel, or a mouse, and an output device such as a display or a speaker. The interface unit 108 receives an operation of inputting data by a user (operator) and outputs information to the user. The interface unit 108 may display information related to the breathing training of the subject 90. For example, the interface unit 108 may display an image for the instructor. The image for the instructor may include, for example, a setting screen related to breathing training.

[0039] The information processing apparatus 100 according to Embodiment 1 includes, as components, an image acquisition unit 112, a position specification unit 114, a displacement amount detection unit 116, a movement direction detection unit 118, a width detection unit 120, a display control unit 130, a result determination unit 140, and an output control unit 150. The image acquisition unit 112 corresponds to the acquisition unit 2 shown in FIG. 1. The image acquisition unit 112 has a function as an image acquisition means (acquisition means). The position specification unit 114 has a function as a position specification means. The displacement amount detection unit 116 corresponds to the detection unit 4 shown in FIG. 1. The displacement amount detection unit 116 has a function as a displacement amount detection means (detection means, calculation means). The movement direction detection unit 118 has a function as a movement direction detection means (detection means, calculation means). The width detection unit 120 has a function as a width detection means. The display control unit 130 corresponds to the display control unit 6 shown in FIG. 1. The display control unit 130 has a function as a display control means. The result determination unit 140 has a function as a result determination means (determination means). The output control unit 150 has a function as an output control means. Specific functions of each component will be described later.

[0040] Note that each of the above-described components can be realized, for example, by causing a program to be executed under the control of the control unit 102. More specifically, each component can be realized by the control unit 102 executing a program stored in the storage unit 104. Alternatively, necessary programs may be recorded on an arbitrary non-volatile recording medium and installed as needed to realize each component. Also, each component is not limited to being realized by software based on a program, and may be realized by any combination of hardware, firmware, and software. Further, each component may be realized using a user-programmable integrated circuit such as an FPGA (field-programmable gate array) or a microcomputer. In this case, a program composed of the above-described components may be realized using this integrated circuit.

[0041] FIG. 5 is a flowchart showing a display control method executed by the information processing apparatus 100 according to Embodiment 1. Note that the display control method can also be said to be an information processing method for performing display control. Further, the display control method can also be said to be a support method for performing display for supporting breathing training for a user such as the subject 90.

[0042] The image acquisition unit 112 acquires an image of the subject 90 (step S102). The image acquisition unit 112 acquires (receives) image data showing at least the chest 92 and the abdomen 94 of the subject 90 from the imaging device 30 using the communication unit 106.

[0043] Here, in the process of S102, the image acquisition unit 112 acquires an image of the subject 90 in a resting state. Specifically, the instructor instructs the subject 90 to relax and be at rest, and the imaging device 30 photographs the subject 90 in that state. Thereby, the image acquisition unit 112 acquires an image of the subject 90 during normal breathing, that is, during resting breathing. As described above, the image of the subject 90 may be a moving image or a still image.

[0044] The position specifying unit 114 specifies the ranges of the chest 92 and the abdomen 94 and the respective reference points in the acquired image (step S104). Specifically, the position specifying unit 114 specifies a region (chest region) corresponding to the chest 92 in the acquired image. Similarly, the position specifying unit 114 specifies a region (abdomen region) corresponding to the abdomen 94 in the acquired image. Thereby, in the image acquired during breathing training, the regions corresponding to the chest 92 and the abdomen 94 can be specified.

[0045] Regarding the identification of the region corresponding to the chest 92, the position identification unit 114 may identify the region corresponding to the chest 92 in the image, for example, using the skeletal data included in the image data. Further, the position identification unit 114 may identify the region corresponding to the chest 92 in the image, for example, using a learned model learned by machine learning. This learned model is learned to take an image of a subject as an input and output the chest region in the image. Further, the position identification unit 114 may identify the region corresponding to the chest 92, for example, by an operation by a user such as an instructor. In this case, the user may select the region corresponding to the chest 92 on the image of the subject 90 displayed on the touch panel, for example, by an operation such as tracing the region corresponding to the chest 92 with a finger on the touch panel. The position identification unit 114 may identify the region corresponding to the abdomen 94 in substantially the same manner as the method described above. For example, the position identification unit 114 may identify the region corresponding to the sternum (and its surroundings) of the subject 90 as the chest region. Further, the position identification unit 114 may identify the region corresponding to the umbilicus (and its surroundings) of the subject 90 as the abdominal region.

[0046] FIG. 6 is a diagram illustrating a captured image 52 acquired by the image acquisition unit 112 according to the first embodiment. Here, when the captured image 52 has three-dimensional information, the pixels constituting the captured image 52 may include the position information (distance information) of the portion of the subject corresponding to the pixel. The captured image 52 includes a subject image 90Im that is an image of the subject 90. Further, in the subject image 90Im of the captured image 52, a chest region 92Im and an abdominal region 94Im identified by the position identification unit 114 are shown. In the captured image 52, for example, the subject image 90Im may be shown in red, the chest region 92Im may be shown in green, and the abdominal region 94Im may be shown in blue.

[0047] In addition, the position specifying unit 114 may specify the anterior-posterior direction and the left-right direction of the subject 90. For example, the position specifying unit 114 may use the skeletal data to specify the anterior-posterior direction and the left-right direction of the subject 90. Alternatively, the position specifying unit 114 may specify the up-down direction of the subject 90 from the direction of the center line of the subject 90 (arrow A1 in FIG. 6) recognized by recognizing the head and lower limb portions of the subject 90. Further, the position specifying unit 114 may specify the left-right direction of the subject 90 from the direction of the line connecting the two shoulders recognized by recognizing the two shoulders of the subject 90 (arrow A2 in FIG. 6). Then, the position specifying unit 114 may specify the direction perpendicular to the specified up-down direction and left-right direction as the anterior-posterior direction. Also, the position specifying unit 114 may specify the direction of the face recognized by recognizing the face of the subject 90 as the front direction. Alternatively, when the subject 90 is in the supine position on a horizontal plane, the position specifying unit 114 may specify the direction along the vertical direction as the anterior-posterior direction.

[0048] In addition, the position specifying unit 114 specifies the reference positions of the chest 92 and the abdomen 94 respectively using the captured image 52 acquired when the subject 90 is in a resting state. Specifically, the position specifying unit 114 specifies the reference position of the chest 92 using the position information corresponding to the chest region specified in the captured image 52 in the resting state. Similarly, the position specifying unit 114 specifies the reference position of the abdomen 94 using the position information corresponding to the abdominal region specified in the captured image 52 in the resting state. Note that the position specifying unit 114 may specify the reference position of the trunk (body trunk) including the chest 92 and the abdomen 94 of the subject 90.

[0049] Here, the position specifying unit 114 specifies the reference positions in the front-rear direction of the chest 92 and the abdomen 94. The position specifying unit 114 specifies the chest reference position which is the reference position in the front-rear direction of the chest 92. Similarly, the position specifying unit 114 specifies the abdominal reference position which is the reference position in the front-rear direction of the abdomen 94. For example, the chest reference position may be the average position in the front-rear direction of the surface (front surface) of the chest 92 between exhalation and inhalation in a resting state. Similarly, the abdominal reference position may be the average position in the front-rear direction of the surface (front surface) of the abdomen 94 between exhalation and inhalation in a resting state. Note that the position specifying unit 114 may specify the reference position in the front-rear direction (trunk reference position) of the trunk including the chest 92 and the abdomen 94.

[0050] Here, the chest reference position may be, for example, the average position (first chest reference position) of the positions in the front-rear direction (corresponding to "height" in the supine position) at each location on the entire surface of the chest 92 at rest. In this case, the chest reference position can be indicated by one value. Also, the chest reference position may be, for example, the position in the front-rear direction (second chest reference position) of one or more specific locations (for example, the central location of the sternum of the chest 92) on the surface of the chest 92 at rest. In this case, the chest reference position can be indicated by the number of values (N values) corresponding to the number of specific locations. Also, the chest reference position may be, for example, the position in the front-rear direction (third chest reference position) of each of M locations arranged at predetermined intervals in the vertical direction (corresponding to arrow A1 in FIG. 6) on the surface of the chest 92 at rest. In this case, the chest reference position can be indicated by M values. Note that in the third chest reference position, each of the "positions in the front-rear direction of M locations arranged at predetermined intervals in the vertical direction" may be the average position (or the most anterior position) in the front-rear direction of the area divided by dividing the chest region at predetermined intervals in the vertical direction. Also, the chest reference position may be the position in the front-rear direction (fourth chest reference position) of each of n locations on the entire surface of the chest 92 at rest. The same applies to the abdominal reference position and the trunk reference position.

[0051] Further, the position specifying unit 114 specifies the reference positions (reference widths) in the left - right direction (width direction) of the chest 92 and the abdomen 94 respectively, using the captured image 52 obtained when the subject 90 is in a resting state. The position specifying unit 114 specifies the chest reference width which is the reference width in the left - right direction of the chest 92. Similarly, the position specifying unit 114 specifies the abdomen reference width which is the reference width in the left - right direction of the abdomen 94. For example, the chest reference width may be the average width of the chest 92 between exhalation and inhalation in the resting state. Similarly, the abdomen reference width may be the average width of the abdomen 94 between exhalation and inhalation in the resting state. Here, the chest reference width may be, for example, the distance between the left end and the right end of the chest 92 in the resting state. Similarly, the abdomen reference width may be, for example, the distance between the left end and the right end of the abdomen 94 in the resting state.

[0052] In addition, the position specifying unit 114 may set a threshold value Th1 for the width of the chest 92 during exhalation. The threshold value Th1 corresponds to the width of the chest 92 when the breath is fully exhaled during exhalation. Therefore, if the width of the chest 92 narrows to the threshold value Th1 during exhalation, it can be said that the internal rotation of the ribs is sufficiently performed. Therefore, the threshold value Th1 can also be said to be the target value for the chest width during exhalation. The threshold value Th1 is appropriately set by an instructor who has confirmed the breathing state of the subject 90. The threshold value Th1 is a value smaller than the chest reference width. Therefore, if the chest reference width is set to 0, the threshold value Th1 becomes a negative value.

[0053] The image acquisition unit 112 acquires an image of the subject 90 who is performing breathing training (step S106). Specifically, the instructor instructs the subject 90 to perform breathing training, and the imaging device 30 captures the subject 90 in that state. Thereby, the image acquisition unit 112 acquires an image of the subject 90 who is performing breathing training. Preferably, the image acquisition unit 112 acquires a moving image of the subject 90 who is performing breathing training. Thereby, as will be described later, the information processing device 100 can detect the transition of the displacement amount etc. of the chest 92 and the abdomen 94.

[0054] Note that the processes of S106 and S110 to S132 described later can be executed for each frame of the acquired moving image. That is, while the subject 90 is performing breathing training, the processes of S106 to S132 are continuously executed. In other words, for a certain frame of the moving image obtained by photographing the subject 90 who is performing breathing training, the processes of S110 to S132 are executed, and when the next frame is acquired, the processes of S110 to S132 are executed for the next frame. Note that, if necessary, the processes of S140 to S142 described later may also be executed for each frame.

[0055] The displacement amount detection unit 116 detects the displacement amounts from the reference positions of the chest 92 and the abdomen 94 respectively (step S110). Specifically, the displacement amount detection unit 116 detects the changes in the positions of the chest 92 and the abdomen 94 in the front-rear direction (corresponding to "height" in the supine position). More specifically, the displacement amount detection unit 116 specifies the chest region and the abdomen region in the captured image 52 of the subject 90 acquired in the process of S106. This specification may be performed using the process of specifying the regions corresponding to the chest 92 and the abdomen 94 respectively in S104.

[0056] Also, the displacement amount detection unit 116 detects the positions of the chest 92 and the abdomen 94 in the front-rear direction (corresponding to "height" in the supine position) in the same manner as when the reference positions in the front-rear direction of the chest 92 and the abdomen 94 are specified in the process of S104. Specifically, the displacement amount detection unit 116 uses the position information corresponding to the chest region specified in the captured image 52 acquired in the process of S106 to detect the position of the chest 92 in the front-rear direction (chest position). Similarly, the displacement amount detection unit 116 uses the position information corresponding to the abdomen region specified in the captured image 52 acquired in the process of S106 to detect the position of the abdomen 94 in the front-rear direction (abdomen position).

[0057] Furthermore, the displacement detection unit 116 calculates the displacement amount of the chest position with respect to the chest reference position specified in S104. The displacement detection unit 116 calculates the difference between the chest position and the chest reference position as the displacement amount (chest displacement amount). Here, when the chest position is anterior to the chest reference position, that is, when the chest 92 is more expanded than at rest, the sign of the displacement amount is positive (+). On the other hand, when the chest position is posterior to the chest reference position, that is, when the chest 92 is more contracted than at rest, the sign of the displacement amount is negative (-). Therefore, the displacement detection unit 116 calculates the displacement amount of the chest 92 by subtracting the value of the chest reference position from the value of the chest position.

[0058] In addition, the displacement detection unit 116 calculates the displacement amount of the abdominal position with respect to the abdominal reference position specified in S104. The displacement detection unit 116 calculates the difference between the abdominal position and the abdominal reference position as the displacement amount (abdominal displacement amount). Here, when the abdominal position is anterior to the abdominal reference position, that is, when the abdomen 94 is more expanded than at rest, the sign of the displacement amount is positive (+). On the other hand, when the abdominal position is posterior to the abdominal reference position, that is, when the abdomen 94 is more contracted than at rest, the sign of the displacement amount is negative (-). Therefore, the displacement detection unit 116 calculates the displacement amount of the abdomen 94 by subtracting the value of the abdominal reference position from the value of the abdominal position.

[0059] Note that the displacement detection unit 116 may detect the displacement amount in the front-rear direction of the trunk including the chest 92 and the abdomen 94 in the same manner as the chest 92 and the abdomen 94. In this case, the displacement detection unit 116 detects the trunk position, which is the position in the front-rear direction of the trunk (corresponding to "height" in the supine position), and calculates the difference obtained by subtracting the trunk reference position from the detected trunk position as the displacement amount (trunk displacement amount) of the trunk.

[0060] When the chest reference position is the average position of the entire surface of the chest 92 at rest (the above-mentioned first chest reference position), the displacement amount detection unit 116 calculates the average position of the positions in the front-rear direction at each location on the surface of the chest 92 as the chest position (the first chest position). Then, the displacement amount detection unit 116 calculates the difference between the calculated first chest position and the first chest reference position. Also, when the chest reference position is the position in the front-rear direction at a specific location on the surface of the chest 92 at rest (the above-mentioned second chest reference position), the displacement amount detection unit 116 detects the position in the front-rear direction (the second chest position) of the specific location in the chest region specified in the process of S110. Then, the displacement amount detection unit 116 calculates the difference (displacement amount) between the detected second chest position and the second chest reference position at the corresponding location. In this case, further, the displacement amount detection unit 116 may calculate the sum or average value of the N differences obtained for each of the N specific locations as the displacement amount. Also, when the chest reference position is the above-mentioned third chest reference position, the displacement amount detection unit 116 calculates the positions in the front-rear direction of each of the M locations arranged at predetermined intervals in the vertical direction of the chest region specified in the process of S110 as the chest positions (the third chest positions). Then, the displacement amount detection unit 116 calculates the difference (displacement amount) between the calculated third chest position and the third chest reference position at the corresponding location. In this case, further, the displacement amount detection unit 116 may calculate the sum or average value of the M differences obtained for each of the M calculated locations as the displacement amount. Also, when the chest reference position is the above-mentioned fourth chest reference position, the displacement amount detection unit 116 calculates the positions in the front-rear direction of each of the n locations of the entire chest region specified in the process of S110 as the chest positions (the fourth chest positions). Then, the displacement amount detection unit 116 calculates the difference (displacement amount) between the fourth chest position and the fourth chest reference position at the corresponding location. Then, further, the displacement amount detection unit 116 may calculate the sum of the calculated n differences (displacement amounts) as the total displacement amount. The same applies to the abdominal displacement amount and the body displacement amount.

[0061] The display control unit 130 performs control for displaying the displacement amount image (step S112). Note that the displacement amount image is the moving image described above and is an image indicating the displacement amount in the front-rear direction. The display control unit 130 performs control for displaying the chest displacement amount image and the abdominal displacement amount image. Here, the "chest displacement amount image" is the chest moving image described above and is an image (displacement amount image) indicating the displacement amount from the chest reference position of the chest 92. That is, the chest displacement amount image is an image indicating the displacement amount (chest displacement amount) of the chest position with respect to the chest reference position. Also, the "abdominal displacement amount image" is the abdominal moving image described above and is an image (displacement amount image) indicating the displacement amount from the abdominal reference position of the abdomen 94. That is, the abdominal displacement amount image is an image indicating the displacement amount (abdominal displacement amount) of the abdominal position with respect to the abdominal reference position.

[0062] Here, when the chest displacement amount and the abdominal displacement amount are synchronized with each other, the display control unit 130 performs control so that the display forms of the chest displacement amount image and the abdominal displacement amount image are the same as each other. On the other hand, when the chest displacement amount and the abdominal displacement amount are not synchronized with each other, the display control unit 130 performs control so that the display forms of the chest displacement amount image and the abdominal displacement amount image are different from each other.

[0063] Here, the case where "the chest displacement amount and the abdominal displacement amount are synchronized with each other" means the case where the signs (positive or negative) of the chest displacement amount and the abdominal displacement amount are the same as each other. Conversely, the case where "the chest displacement amount and the abdominal displacement amount are not synchronized with each other" means the case where the signs (positive or negative) of the chest displacement amount and the abdominal displacement amount are different from each other. Therefore, when the signs of the chest displacement amount and the abdominal displacement amount are the same as each other, the display control unit 130 performs control so that the display forms of the chest displacement amount image and the abdominal displacement amount image are the same as each other. On the other hand, when the signs of the chest displacement amount and the abdominal displacement amount are different from each other, the display control unit 130 performs control so that the display forms of the chest displacement amount image and the abdominal displacement amount image are different from each other.

[0064] Note that the "display form" is, for example, color representation. In this case, when the chest displacement amount and the abdominal displacement amount are synchronized with each other, the display control unit 130 performs control so that the color representation of the chest displacement amount image and the color representation of the abdominal displacement amount image are the same as each other. That is, when the sign of the chest displacement amount and the sign of the abdominal displacement amount are the same as each other, the display control unit 130 performs control so that the color representation of the chest displacement amount image and the color representation of the abdominal displacement amount image are the same as each other. On the other hand, when the sign of the chest displacement amount and the sign of the abdominal displacement amount are different from each other, the display control unit 130 performs control so that the color representation of the chest displacement amount image and the color representation of the abdominal displacement amount image are different from each other. Note that the display form is not limited to color representation. In Embodiment 1, the "display form" is assumed to be color representation.

[0065] For example, for each of the chest displacement amount image and the abdominal displacement amount image, when the sign of the displacement amount is "positive", the display control unit 130 performs control so that the displacement amount is displayed in red. Therefore, when the sign of the chest displacement amount is "positive", the chest displacement amount image is displayed in red. Similarly, when the sign of the abdominal displacement amount is "positive", the abdominal displacement amount image is displayed in red. On the other hand, for each of the chest displacement amount image and the abdominal displacement amount image, when the sign of the displacement amount is "negative", the display control unit 130 performs control so that the displacement amount is displayed in blue. Therefore, when the sign of the chest displacement amount is "negative", the chest displacement amount image is displayed in blue. Similarly, when the sign of the abdominal displacement amount is "negative", the abdominal displacement amount image is displayed in blue.

[0066] The display control unit 130 generates data (displacement amount image display instruction data) indicating an instruction to display the displacement amount image, and transmits it to the display device 40. The displacement amount image display instruction data may indicate the displacement amount and the color representation to be displayed according to the sign of the displacement amount. In particular, the displacement amount image display instruction data may indicate the color representation to be displayed according to the chest displacement amount and the sign of the chest displacement amount, and the color representation to be displayed according to the abdominal displacement amount and the abdominal displacement amount. Further, the displacement amount image display instruction data may indicate the color representation to be displayed according to the body displacement amount and the sign of the body displacement amount. Thereby, the display device 40 displays the displacement amount image in a color representation according to the sign of the displacement amount.

[0067] Further, the display control unit 130 performs control so that the displacement amount image changes and is displayed in real time in accordance with the progress of the breathing training. That is, the display control unit 130 causes the display device 40 to display the displacement amount image that changes in accordance with the progress of the breathing training of the subject 90. In other words, the display control unit 130 performs control so that the displacement amount image (moving image) that changes in accordance with the progress of the breathing training of the subject 90 is displayed during the progress of the breathing training. Further in other words, the display control unit 130 performs control so that the displacement amount image corresponding to the displacement amount at a certain timing during the breathing training is displayed at that timing. For example, when the displacement amount is detected at time t1, the displacement amount image may be displayed at time t1 + Δt. Here, Δt is a minute time. Δt can be determined according to the processing capabilities of the information processing device 100 and the display device 40 and the communication environment between the information processing device 100 and the display device 40.

[0068] Also, each time a frame of the captured image of the subject 90 performing the breathing training is acquired, the display control unit 130 generates displacement amount image display instruction data and transmits it to the display device 40. Thereby, the display control unit 130 can cause the display device 40 to display the displacement amount image that changes as the breathing training progresses.

[0069] FIGS. 7 to 11 are diagrams illustrating a display screen 200 that is displayed on the display device 40 under the control of the display control unit 130 according to Embodiment 1 and shows the displacement amounts in the front-rear direction of the chest 92 and the abdomen 94. FIG. 7 shows the display screen 200 when both the sign of the displacement amount in the front-rear direction of the chest 92 (chest displacement amount) and the sign of the displacement amount in the front-rear direction of the abdomen 94 (abdomen displacement amount) are positive. Further, FIG. 7 shows the display screen 200 that displays the displacement amount at a certain timing during the breathing training. The display screen 200 illustrated in FIG. 7 is displayed on the display device 40 at that timing. That is, the display screen 200 illustrated in FIG. 7 is displayed on the display device 40 in real time during the breathing training.

[0070] The display screen 200 illustrated in FIG. 7 displays a chest displacement amount screen 210, an abdominal displacement amount screen 220, and a body displacement amount screen 230. Note that the display screen 200 may display only the chest displacement amount screen 210 and the abdominal displacement amount screen 220 and not display the body displacement amount screen 230. Alternatively, the display screen 200 may display only the body displacement amount screen 230 and not display the chest displacement amount screen 210 and the abdominal displacement amount screen 220. This also applies to FIG. 8 and the like described below.

[0071] In FIG. 7, on the chest displacement amount screen 210, a chest displacement amount image 212rd indicating that the sign of the chest displacement amount is positive is displayed. Also, on the abdominal displacement amount screen 220, an abdominal displacement amount image 222rd indicating that the sign of the abdominal displacement amount is positive is displayed. Here, the chest displacement amount image 212rd and the abdominal displacement amount image 222rd are drawn in red, which is the same color representation as each other. Note that the vertical length of the chest displacement amount image 212rd and the abdominal displacement amount image 222rd indicates the scalar value of the displacement amount. That is, the chest displacement amount image 212rd indicates the chest displacement amount at the timing when the display screen 200 (chest displacement amount image 212rd) is displayed. Similarly, the abdominal displacement amount image 222rd indicates the abdominal displacement amount at the timing when the display screen 200 (abdominal displacement amount image 222rd) is displayed.

[0072] Here, as time elapses, the chest displacement amount and the abdominal displacement amount change. Therefore, on the display screen 200, the chest displacement amount image 212 and the abdominal displacement amount image 222 change as time elapses. This also applies to FIG. 8 and the like.

[0073] Here, in the example of FIG. 7, since the signs of the chest displacement amount and the abdominal displacement amount are both positive, the chest displacement amount and the abdominal displacement amount are synchronized with each other. And the chest displacement amount image 212rd and the abdominal displacement amount image 222rd are both drawn in red. Therefore, in the example of FIG. 7, the chest displacement amount image 212rd and the abdominal displacement amount image 222rd are displayed such that it is visually easily recognizable that the chest displacement amount and the abdominal displacement amount are synchronized with each other. Therefore, the subject 90 who has seen the display screen 200 shown in FIG. 7 can easily recognize that the movement of the chest 92 and the movement of the abdomen 94 are synchronized with each other. Therefore, the subject 90 can easily recognize his / her own breathing state.

[0074] Also, in FIG. 7, the trunk displacement amount screen 230 displays a set of points (dot sequence) corresponding to the three-dimensional point group of the trunk of the subject 90 during breathing training. In the trunk displacement amount screen 230, the upward direction in FIG. 7 corresponds to the front direction of the subject 90, and the downward direction corresponds to the rear direction of the subject 90. Also, the trunk displacement amount screen 230 displays the range corresponding to the chest 92 of the subject 90 and the range corresponding to the abdomen 94.

[0075] In FIG. 7, the trunk reference position dot sequence 232bk is shown by a dot sequence composed of a plurality of dots each formed by a black circle. The trunk reference position dot sequence 232bk indicates the trunk reference position. In the display screen 200, the trunk reference position dot sequence 232bk is drawn in black. Each dot constituting the trunk reference position dot sequence 232bk indicates the position in the front-rear direction (corresponding to "height" in the supine position) at each location in the vertical direction (left-right direction in FIG. 7) of the trunk surface of the subject 90 at rest corresponding to that dot.

[0076] Also, in FIG. 7, the body position dot sequence 234rd is shown as a dot sequence composed of a plurality of dots each formed as a white circle. Each dot constituting the body position dot sequence 234rd indicates the position in the front-rear direction (corresponding to "height" in the supine position) at each location in the vertical direction (left-right direction in FIG. 7) of the body surface of the subject 90 during training, corresponding to that dot. That is, the body position dot sequence 234rd indicates the amount of body displacement at the timing when the display screen 200 (body position dot sequence 234rd) is displayed. That is, the body position dot sequence 234rd corresponds to the displacement amount image described above. And the body position dot sequence 234rd is drawn in red when the body position is in front of the body reference position (that is, when the body displacement amount is positive).

[0077] Here, as time passes, the amount of body displacement changes. Therefore, on the display screen 200, the body position dot sequence 234 changes as time passes. This is the same in FIG. 8 and the like.

[0078] Here, as shown in the chest displacement amount image 212rd and the abdominal displacement amount image 222rd, at the timing when the display screen 200 of FIG. 7 is displayed, the surfaces of the chest 92 and the abdomen 94 are both located on the front side of the reference position. Therefore, at this timing, in the body displacement amount screen 230 illustrated in FIG. 7, the entire body position dot row 234rd is located above the body reference position dot row 232bk. Therefore, on the display screen 200, the entire body position dot row 234rd is drawn in red. In this way, in the body displacement amount screen 230, since the entire body position dot row 234rd is drawn in red, it is displayed in a visually recognizable manner that the chest displacement amount and the abdominal displacement amount are synchronized with each other. Therefore, the subject 90 who views the display screen 200 can easily recognize that the movement of the chest 92 and the movement of the abdomen 94 are synchronized with each other. Also, by schematically showing the movement of the chest 92 and the movement of the abdomen 94 as in the body displacement amount screen 230, the subject 90 can easily visually recognize how the movement of the chest 92 and the movement of the abdomen 94 are synchronized with each other.

[0079] Note that in the above-mentioned Patent Document 1, it is disclosed that the diagnosis of a patient with COPD (chronic obstructive pulmonary disease) is supported by detecting the phase difference (especially the deviation of the expiratory peak) between the respiratory rate of the chest and the respiratory rate of the abdomen of the patient. However, at the end phase of forced expiration, since the change rate of the surface position of the chest and the abdomen becomes extremely small, it is difficult to detect the above-mentioned phase difference. On the other hand, in Embodiment 1, the displacement amount of the surface position of the chest and the displacement amount of the surface position of the abdomen can be compared. Therefore, even at the end phase of forced expiration, it is possible to appropriately recognize whether the movement of the chest and the movement of the abdomen are synchronized. Therefore, it is possible to efficiently support exercises for enhancing motor function.

[0080] FIG. 8 shows a display screen 200 when the signs of the displacement amounts in the front-rear direction (chest displacement amount) of the chest 92 and the signs of the displacement amounts in the front-rear direction (abdominal displacement amount) of the abdomen 94 are both negative. Also, similar to FIG. 7, FIG. 8 shows a display screen 200 that displays the displacement amounts at a certain timing during respiratory training. The display screen 200 illustrated in FIG. 8 is displayed on the display device 40 at that timing.

[0081] Similar to FIG. 7, the display screen 200 illustrated in FIG. 8 displays a chest displacement amount screen 210, an abdominal displacement amount screen 220, and a torso displacement amount screen 230. In FIG. 8, on the chest displacement amount screen 210, a chest displacement amount image 212bu indicating that the sign of the chest displacement amount is negative is displayed. Also, on the abdominal displacement amount screen 220, an abdominal displacement amount image 222bu indicating that the sign of the abdominal displacement amount is negative is displayed. Here, the chest displacement amount image 212bu and the abdominal displacement amount image 222bu are drawn in blue, which is the same color representation as each other. Note that the vertical lengths of the chest displacement amount image 212bu and the abdominal displacement amount image 222bu indicate the scalar values of the displacement amounts. That is, similar to the chest displacement amount image 212rd, the chest displacement amount image 212bu indicates the chest displacement amount at the timing when the display screen 200 (chest displacement amount image 212bu) is displayed. Similarly, the abdominal displacement amount image 222bu indicates the abdominal displacement amount at the timing when the display screen 200 (abdominal displacement amount image 222bu) is displayed.

[0082] Here, in the example of FIG. 8, since the signs of the chest displacement amount and the abdominal displacement amount are both negative, the chest displacement amount and the abdominal displacement amount are synchronized with each other. And the chest displacement amount image 212bu and the abdominal displacement amount image 222bu are both drawn in blue. Therefore, in the example of FIG. 8, the chest displacement amount image 212bu and the abdominal displacement amount image 222bu are displayed in such a way that it is visually easily recognizable that the chest displacement amount and the abdominal displacement amount are synchronized with each other. Therefore, the subject 90 who sees the display screen 200 shown in FIG. 8 can easily recognize that the movement of the chest 92 and the movement of the abdomen 94 are synchronized with each other. Therefore, the subject 90 can easily recognize his / her own respiratory state.

[0083] Also, similar to FIG. 7, in FIG. 8, the body displacement amount screen 230 displays a set of points (dot sequence) corresponding to the three-dimensional point cloud of the body of the subject 90 during breathing training. The body reference position dot sequence 232bk is substantially the same as that shown in FIG. 7. Also, in FIG. 8, the body position dot sequence 234bu is shown as a dot sequence composed of a plurality of dots each formed by an unfilled triangle. Similar to the body position dot sequence 234rd, the body position dot sequence 234bu indicates the body displacement amount at the timing when the display screen 200 (body position dot sequence 234bu) is displayed. That is, the body position dot sequence 234bu corresponds to the displacement amount image described above. And the body position dot sequence 234bu is drawn in blue when the body position is behind the body reference position (that is, when the body displacement amount is negative).

[0084] Here, as shown in the chest displacement amount image 212bu and the abdominal displacement amount image 222bu, at the timing when the display screen 200 of FIG. 8 is displayed, the surfaces of the chest 92 and the abdomen 94 are both located behind the reference position. Therefore, in the body displacement amount screen 230 illustrated in FIG. 8, the entire body position dot sequence 234bu is located below the body reference position dot sequence 232bk. Therefore, on the display screen 200, the entire body position dot sequence 234bu is drawn in blue. Thus, since the entire body position dot sequence 234bu is drawn in blue in the body displacement amount screen 230, it is visually easily recognizable that the chest displacement amount and the abdominal displacement amount are synchronized with each other. Therefore, the subject 90 who views the display screen 200 can easily recognize that the movement of the chest 92 and the movement of the abdomen 94 are synchronized with each other. Also, similar to FIG. 7, by schematically showing the movement of the chest 92 and the movement of the abdomen 94 as in the body displacement amount screen 230, the subject 90 can visually easily recognize how the movement of the chest 92 and the movement of the abdomen 94 are synchronized. Therefore, the subject 90 can easily recognize his / her own breathing state.

[0085] FIG. 9 shows the display screen 200 when the sign of the displacement amount in the front-rear direction of the chest 92 (chest displacement amount) and the sign of the displacement amount in the front-rear direction of the abdomen 94 (abdomen displacement amount) are different from each other. FIG. 9 shows the display screen 200 when the sign of the chest displacement amount is negative and the sign of the abdomen displacement amount is positive. Also, similar to FIG. 7 and the like, FIG. 9 shows the display screen 200 that displays the displacement amount at a certain timing during the breathing training. The display screen 200 illustrated in FIG. 9 is displayed on the display device 40 at that timing.

[0086] Similar to FIG. 7 and the like, the display screen 200 illustrated in FIG. 9 displays a chest displacement amount screen 210, an abdomen displacement amount screen 220, and a body displacement amount screen 230. In FIG. 9, on the chest displacement amount screen 210, a chest displacement amount image 212bu indicating that the sign of the chest displacement amount is negative is displayed in blue. Also, on the abdomen displacement amount screen 220, an abdomen displacement amount image 222rd indicating that the sign of the abdomen displacement amount is positive is displayed in red. Therefore, the chest displacement amount image 212bu and the abdomen displacement amount image 222rd are drawn with different color expressions.

[0087] Here, in the example of FIG. 9, since the sign of the chest displacement amount is negative and the sign of the abdomen displacement amount is positive, the chest displacement amount and the abdomen displacement amount are not synchronized with each other. And the chest displacement amount image 212bu is drawn in blue, and the abdomen displacement amount image 222rd is drawn in red. That is, the chest displacement amount image 212bu and the abdomen displacement amount image 222rd are drawn with different color expressions. Therefore, in the example of FIG. 9, the chest displacement amount image 212bu and the abdomen displacement amount image 222rd are displayed in such a way that it is visually easily recognizable that the chest displacement amount and the abdomen displacement amount are not synchronized with each other. Therefore, the subject 90 who sees the display screen 200 shown in FIG. 9 can easily recognize that the movement of the chest 92 and the movement of the abdomen 94 are not synchronized with each other.

[0088] Also, similar to FIG. 7, in FIG. 9, the body displacement amount screen 230 displays a set of points (dot sequence) corresponding to the three-dimensional point cloud of the body of the subject 90 during breathing training. The body reference position dot sequence 232bk is substantially the same as that shown in FIG. 7. Also, in FIG. 9, the body displacement amount screen 230 displays a body position dot sequence 234rd drawn in red and a body position dot sequence 234bu drawn in blue.

[0089] Here, as shown in the chest displacement amount image 212bu and the abdominal displacement amount image 222rd, at the timing when the display screen 200 of FIG. 9 is displayed, the surface of the chest 92 is located posterior to the reference position, and the surface of the abdomen 94 is located anterior to the reference position. Therefore, in the body displacement amount screen 230 illustrated in FIG. 9, at the location corresponding to the chest 92, the body position dot sequence 234bu drawn in blue is displayed below the body reference position dot sequence 232bk. On the other hand, at the location corresponding to the abdomen 94, the body position dot sequence 234rd drawn in red is displayed above the body reference position dot sequence 232bk. Thus, in the body displacement amount screen 230 illustrated in FIG. 9, in the body part located anterior to the reference position, the body position dot sequence 234rd is displayed above the body reference position dot sequence 232bk. On the other hand, in the body part located posterior to the reference position, the body position dot sequence 234bu is displayed below the body reference position dot sequence 232bk.

[0090] In this way, on the body displacement amount screen 230, since the body position dot sequences 234rd and 234bu are displayed with different color representations, it is visually easily recognizable that the chest displacement amount and the abdominal displacement amount are not synchronized with each other. Therefore, the subject 90 who views the display screen 200 can easily recognize that the movement of the chest 92 and the movement of the abdomen 94 are not synchronized with each other. Also, similar to FIG. 7, by schematically showing the movement of the chest 92 and the movement of the abdomen 94 as in the body displacement amount screen 230, the subject 90 can visually easily recognize how the movement of the chest 92 and the movement of the abdomen 94 are not synchronized. Therefore, the subject 90 can easily recognize his / her own breathing state.

[0091] Also, the display screen 200 illustrated in FIGS. 7 to 9 displays the displacement amount at the timing when this display screen 200 is displayed. Therefore, the display screen 200 displays the displacement amount in real time during the breathing training of the subject 90. Therefore, the subject 90 who views the display screen 200 can immediately recognize whether the breathing method at the current timing is correct during his / her own breathing training. That is, the subject 90 who views the display screen 200 shown in FIGS. 7 and 8 can immediately recognize that the breathing method at the current timing is correct during his / her own breathing training. On the other hand, the subject 90 who views the display screen 200 shown in FIG. 9 can immediately recognize that the breathing method at the current timing is incorrect during his / her own breathing training. Therefore, the subject 90 can easily recognize his / her own breathing state in real time.

[0092] Also, on the display screen 200, as the breathing training progresses over time, the display of the chest displacement amount image 212, the abdominal displacement amount image 222, and the body position dot sequence 234 changes. Therefore, the subject 90 who views the display screen 200 can visually easily recognize how his / her chest 92 and abdomen 94 (and body) are moving as the breathing training progresses over time.

[0093] FIG. 10 and FIG. 11 are diagrams illustrating a display screen 200 on which a displacement amount history screen 240 for displaying a history of displacement amounts is displayed. FIG. 10 illustrates a case where the displacement amount history screen 240 is displayed on the display screen 200 illustrated in FIG. 7. FIG. 11 illustrates a case where the displacement amount history screen 240 is displayed on the display screen 200 illustrated in FIG. 9. Note that the displacement amount history screen 240 may be displayed separately from the chest displacement amount screen 210, the abdominal displacement amount screen 220, and the body displacement amount screen 230 shown in FIG. 7 and the like.

[0094] The displacement amount history screen 240 displays a graph with the vertical axis representing the displacement amounts of the chest 92 and the abdomen 94 respectively, and the horizontal axis representing the transition of the number of frames of the captured moving image. That is, in the graph shown on the displacement amount history screen 240, the horizontal axis indicates the passage of time. Therefore, the displacement amount history screen 240 shows the time-series data of the chest displacement amount and the abdominal displacement amount. For example, when the frame rate of the imaging device 30 is 15 fps (frames per second), the data at the frame number "600" on the horizontal axis corresponds to the data 40 seconds after the start of shooting. Also, on the displacement amount history screen 240, the points corresponding to the chest 92 and the points corresponding to the abdomen 94 respectively indicate the displacement amount of the chest 92 (chest displacement amount) and the displacement amount of the abdomen 94 (abdominal displacement amount) at the timing when the corresponding number of frames was obtained.

[0095] Note that the displacement amount history screen 240 may display an image of the history of the displacement amount that changes over time during the breathing training. Therefore, the display of the displacement amount history screen 240 changes over time. Also, in FIGS. 10 and 11, the chest displacement amount and the abdominal displacement amount are shown in the same graph, but they may be shown in separate graphs.

[0096] In the displacement history screen 240 illustrated in FIG. 10, the phase of the graph shape (waveform) indicating the change in the chest displacement amount and the phase of the graph shape (waveform) indicating the change in the abdominal displacement amount are generally the same. That is, the abdominal displacement amount also decreases at substantially the same timing as the timing when the chest displacement amount decreases, and the abdominal displacement amount also increases at substantially the same timing as the timing when the chest displacement amount increases. Therefore, it can be recognized from the displacement history screen 240 illustrated in FIG. 10 that the movement (displacement amount) of the chest 92 and the movement (displacement amount) of the abdomen 94 are synchronized. Therefore, the subject 90 can easily recognize his / her own breathing state.

[0097] On the other hand, in the displacement history screen 240 illustrated in FIG. 11, the phase of the graph shape (waveform) indicating the change in the chest displacement amount and the phase of the graph shape (waveform) indicating the change in the abdominal displacement amount do not match. Specifically, at the location indicated by arrow B, the abdominal displacement amount is increasing at the timing when the chest displacement amount is decreasing. Therefore, it can be recognized from the displacement history screen 240 illustrated in FIG. 11 that asynchronous breathing occurs where the movement of the chest 92 and the movement of the abdomen 94 are not synchronized at the location indicated by arrow B. Therefore, the subject 90 can easily recognize his / her own breathing state.

[0098] Returning to the description of the flowchart shown in FIG. 5. The movement direction detection unit 118 detects the movement direction of each of the chest 92 and the abdomen 94 (step S120). Here, the "movement direction" is the direction in which the surfaces of the chest 92 and the abdomen 94 are moving. That is, the movement direction indicates whether the surfaces of the chest 92 and the abdomen 94 are moving forward or backward. In other words, the movement direction is the direction of the change (increase or decrease) in the displacement amount. That is, when the displacement amount is increasing, that is, when the subject 90 is inhaling, the movement direction is forward (positive direction). On the other hand, when the displacement amount is decreasing, that is, when the subject 90 is exhaling, the movement direction is backward (negative direction). More specifically, the movement direction can also be said to be the direction of the change rate of the displacement amount.

[0099] Specifically, the movement direction detection unit 118 subtracts the displacement amount of the chest 92 at a predetermined time before from the current displacement amount of the chest 92. When the value obtained by the subtraction is 0 or more, the movement direction detection unit 118 determines that the movement direction of the chest 92 is the positive direction (forward direction). On the other hand, when the value obtained by the subtraction is less than 0, the movement direction detection unit 118 determines that the movement direction of the chest 92 is the negative direction (backward direction).

[0100] Similarly, the movement direction detection unit 118 subtracts the displacement amount of the abdomen 94 at a predetermined time before from the current displacement amount of the abdomen 94. When the value obtained by the subtraction is 0 or more, the movement direction detection unit 118 determines that the movement direction of the abdomen 94 is the positive direction (forward direction). On the other hand, when the value obtained by the subtraction is less than 0, the movement direction detection unit 118 determines that the movement direction of the abdomen 94 is the negative direction (backward direction).

[0101] Note that the "predetermined time" may be, for example, 1 second. In this case, the movement direction may correspond to the direction of the change rate of the displacement amount. Alternatively, the "predetermined time" may be the reciprocal of the frame rate. In this case, the movement direction regarding the frame to be processed corresponds to the direction of the change from the displacement amount detected for the frame immediately before the frame to be processed. Therefore, in this case, the movement direction detection unit 118 detects the movement direction by subtracting the displacement amount regarding the frame immediately before the frame to be processed from the displacement amount regarding the frame to be processed.

[0102] The display control unit 130 performs control for displaying the direction image (step S122). The display control unit 130 performs control for displaying the direction image, which is the moving image described above and indicates the moving direction in the front-rear direction. The display control unit 130 performs control for displaying the chest direction image and the abdominal direction image. Here, the "chest direction image" is the chest moving image described above and is an image (direction image) indicating the moving direction (chest moving direction) of the chest 92. That is, the chest direction image is an image indicating the direction of change (increase or decrease) in the displacement amount of the chest 92. Also, the "abdominal direction image" is the abdominal moving image described above and is an image (direction image) indicating the moving direction (abdominal moving direction) of the abdomen 94. That is, the abdominal direction image is an image indicating the direction of change (increase or decrease) in the displacement amount of the abdomen 94.

[0103] Here, when the chest moving direction and the abdominal moving direction are synchronized with each other, the display control unit 130 performs control so that the display form of the chest direction image and the display form of the abdominal direction image are the same as each other. On the other hand, when the chest moving direction and the abdominal moving direction are not synchronized with each other, the display control unit 130 performs control so that the display form of the chest direction image and the display form of the abdominal direction image are different from each other.

[0104] Here, the case where "the chest moving direction and the abdominal moving direction are synchronized with each other" means the case where the chest moving direction (positive direction or negative direction) and the abdominal moving direction (positive direction or negative direction) are the same as each other. Conversely, the case where "the chest moving direction and the abdominal moving direction are not synchronized with each other" means the case where the chest moving direction (positive direction or negative direction) and the abdominal moving direction (positive direction or negative direction) are different from each other. Therefore, when the chest moving direction and the abdominal moving direction are the same as each other, the display control unit 130 performs control so that the display form of the chest direction image and the display form of the abdominal direction image are the same as each other. On the other hand, when the chest moving direction and the abdominal moving direction are different from each other, the display control unit 130 performs control so that the display form of the chest direction image and the display form of the abdominal direction image are different from each other.

[0105] Note that the "display form" is, for example, color representation. In this case, when the chest movement direction and the abdominal movement direction are synchronized with each other, the display control unit 130 performs control so that the color representation of the chest direction image and the color representation of the abdominal direction image are the same as each other. That is, when the chest movement direction and the abdominal movement direction are the same as each other, the display control unit 130 performs control so that the color representation of the chest direction image and the color representation of the abdominal direction image are the same as each other. On the other hand, when the chest movement direction and the abdominal movement direction are different from each other, the display control unit 130 performs control so that the color representation of the chest direction image and the color representation of the abdominal direction image are different from each other. Note that the display form is not limited to color representation. In Embodiment 1, the "display form" is assumed to be color representation.

[0106] For example, for each of the chest direction image and the abdominal direction image, when the movement direction is the "positive direction (forward direction)", the display control unit 130 performs control so that the movement direction is displayed in red. Therefore, when the chest movement direction is the "positive direction (forward direction)", the chest direction image is displayed in red. Similarly, when the abdominal movement direction is the "positive direction (forward direction)", the abdominal direction image is displayed in red. On the other hand, for each of the chest direction image and the abdominal direction image, when the movement direction is the "negative direction (backward direction)", the display control unit 130 performs control so that the movement direction is displayed in blue. Therefore, when the chest movement direction is the "negative direction (backward direction)", the chest direction image is displayed in blue. Similarly, when the abdominal movement direction is the "negative direction (backward direction)", the abdominal direction image is displayed in blue.

[0107] The display control unit 130 generates data (direction image display instruction data) indicating an instruction to display a direction image and transmits it to the display device 40. The direction image display instruction data may indicate the movement direction and the color representation to be displayed according to the movement direction. In particular, the direction image display instruction data may indicate the chest movement direction and the color representation to be displayed according to the chest movement direction, and the abdominal movement direction and the color representation to be displayed according to the abdominal movement direction. Further, the direction image display instruction data may indicate the body movement direction and the color representation to be displayed according to the body movement direction. Thereby, the display device 40 displays the direction image in a color representation according to the movement direction.

[0108] In addition, the display control unit 130 performs control so that the direction image changes and is displayed in real time according to the progress of the breathing training. That is, the display control unit 130 causes the display device 40 to display a direction image that changes according to the progress of the breathing training of the subject 90. In other words, the display control unit 130 performs control so that a direction image (moving image) that changes according to the progress of the breathing training of the subject 90 is displayed during the progress of the breathing training. Further in other words, the display control unit 130 performs control so that a direction image corresponding to the change in the displacement amount at a certain timing during the breathing training is displayed at that timing.

[0109] In addition, each time a frame of the captured image of the subject 90 performing the breathing training is acquired, the display control unit 130 generates direction image display instruction data and transmits it to the display device 40. Thereby, the display control unit 130 can cause the display device 40 to display a direction image that changes as the breathing training progresses.

[0110] FIGS. 12 to 14 are diagrams illustrating a display screen 200 that is displayed on the display device 40 under the control of the display control unit 130 according to Embodiment 1 and shows the moving directions in the front-rear direction of the chest 92 and the abdomen 94. FIG. 12 shows the display screen 200 when both the moving direction in the front-rear direction of the chest 92 (chest moving direction) and the moving direction in the front-rear direction of the abdomen 94 (abdomen moving direction) are in the positive direction (front direction).

[0111] In addition, FIG. 12 shows the display screen 200 that displays the moving direction (direction of change in displacement amount) at a certain timing during the breathing training. The display screen 200 illustrated in FIG. 12 is displayed on the display device 40 at that timing. That is, the display screen 200 illustrated in FIG. 12 is displayed on the display device 40 in real time during the breathing training.

[0112] The display screen 200 illustrated in FIG. 12 displays a chest-direction screen 250 and an abdomen-direction screen 260. Note that the display screen 200 may display the chest-direction screen 250 and the abdomen-direction screen 260 together with the chest displacement amount screen 210, the abdomen displacement amount screen 220, and the body displacement amount screen 230 illustrated in FIG. 7 and the like. This also applies to FIG. 13 and the like.

[0113] In FIG. 12, a chest-direction image 252rd indicating that the chest movement direction is the positive direction is displayed on the chest-direction screen 250. Also, an abdomen-direction image 262rd indicating that the abdomen movement direction is the positive direction is displayed on the abdomen-direction screen 260. Here, the chest-direction image 252rd and the abdomen-direction image 262rd are drawn in red, which is the same color representation as each other. That is, the chest-direction image 252rd indicates the chest movement direction at the timing when the display screen 200 (chest-direction image 252rd) is displayed. Similarly, the abdomen-direction image 262rd indicates the abdomen movement direction at the timing when the display screen 200 (abdomen-direction image 262rd) is displayed. Note that the vertical length of the chest-direction image 252 and the abdomen-direction image 262 may indicate the difference in displacement amounts, that is, the scalar value of the change speed of the displacement amounts. This also applies to FIG. 13 and the like.

[0114] Here, as time elapses, the chest movement direction and the abdomen movement direction change. Therefore, on the display screen 200, the chest-direction image 252 and the abdomen-direction image 262 change as time elapses. This also applies to FIG. 13 and the like.

[0115] Here, in the example of FIG. 12, since both the chest movement direction and the abdominal movement direction are in the positive direction (forward direction), the chest movement direction and the abdominal movement direction are synchronized with each other. And the chest direction image 252rd and the abdominal direction image 262rd are both drawn in red. Therefore, in the example of FIG. 12, the chest direction image 252rd and the abdominal direction image 262rd are displayed in such a way that it is visually easy to recognize that the chest movement direction and the abdominal movement direction are synchronized with each other. Therefore, the subject 90 who has seen the display screen 200 shown in FIG. 12 can easily recognize that the movement of the chest 92 and the movement of the abdomen 94 are synchronized with each other. Therefore, the subject 90 can easily recognize his / her own breathing state.

[0116] FIG. 13 shows the display screen 200 when both the forward and backward movement direction of the chest 92 (chest movement direction) and the forward and backward movement direction of the abdomen 94 (abdominal movement direction) are in the negative direction (backward direction). Also, similar to FIG. 12, FIG. 13 shows the display screen 200 that displays the movement direction (direction of change in displacement amount) at a certain timing during the breathing training. The display screen 200 illustrated in FIG. 13 is displayed on the display device 40 at that timing.

[0117] Similar to FIG. 12, the display screen 200 illustrated in FIG. 13 displays the chest direction screen 250 and the abdominal direction screen 260. In FIG. 13, on the chest direction screen 250, a chest direction image 252bu indicating that the chest movement direction is in the negative direction is displayed. Also, on the abdominal direction screen 260, an abdominal direction image 262bu indicating that the abdominal movement direction is in the negative direction is displayed. Here, the chest direction image 252bu and the abdominal direction image 262bu are drawn in blue, which is the same color representation as each other. That is, the chest direction image 252bu indicates the chest movement direction at the timing when the display screen 200 (chest direction image 252bu) is displayed. Similarly, the abdominal direction image 262bu indicates the abdominal movement direction at the timing when the display screen 200 (abdominal direction image 262bu) is displayed.

[0118] Here, in the example of FIG. 13, since both the chest movement direction and the abdominal movement direction are in the negative direction (rear direction), the chest movement direction and the abdominal movement direction are synchronized with each other. And the chest direction image 252bu and the abdominal direction image 262bu are both drawn in blue. Therefore, in the example of FIG. 13, the chest direction image 252bu and the abdominal direction image 262bu are displayed in such a way that it is visually easy to recognize that the chest movement direction and the abdominal movement direction are synchronized with each other. Therefore, the subject 90 who has seen the display screen 200 shown in FIG. 13 can easily recognize that the movement of the chest 92 and the movement of the abdomen 94 are synchronized with each other. Therefore, the subject 90 can easily recognize his / her own breathing state.

[0119] FIG. 14 shows the display screen 200 when the movement direction in the front-rear direction (chest movement direction) of the chest 92 and the movement direction in the front-rear direction (abdominal movement direction) of the abdomen 94 are different from each other. FIG. 14 shows the display screen 200 when the chest movement direction is in the negative direction (rear direction) and the abdominal movement direction is in the positive direction (front direction). Also, similar to FIG. 12 etc., FIG. 14 shows the display screen 200 that displays the movement direction (the direction of change in the displacement amount) at a certain timing during the breathing training. The display screen 200 illustrated in FIG. 14 is displayed on the display device 40 at that timing.

[0120] Similar to FIG. 12 etc., the display screen 200 illustrated in FIG. 14 displays the chest direction screen 250 and the abdominal direction screen 260. In FIG. 14, on the chest direction screen 250, the chest direction image 252bu indicating that the chest movement direction is in the negative direction is displayed in blue. Also, on the abdominal direction screen 260, the abdominal direction image 262rd indicating that the abdominal movement direction is in the positive direction is displayed in red. Here, the chest direction image 252bu and the abdominal direction image 262rd are drawn with different color expressions.

[0121] Here, in the example of FIG. 14, since the chest movement direction is the negative direction (backward) and the abdominal movement direction is the positive direction (forward), the chest movement direction and the abdominal movement direction are not synchronized with each other. And the chest direction image 252bu is drawn in blue, and the abdominal direction image 262rd is drawn in red. Therefore, in the example of FIG. 14, the chest direction image 252bu and the abdominal direction image 262rd are displayed in such a way that it is visually easy to recognize that the chest movement direction and the abdominal movement direction are not synchronized with each other. Therefore, the subject 90 who has seen the display screen 200 shown in FIG. 14 can easily recognize that the movement of the chest 92 and the movement of the abdomen 94 are not synchronized with each other. Therefore, the subject 90 can easily recognize his own breathing state.

[0122] Also, the display screen 200 illustrated in FIGS. 12 to 14 displays the movement direction (the direction of change in the displacement amount) at the timing when this display screen 200 is displayed. Therefore, the display screen 200 displays the movement direction in real time during the breathing training of the subject 90. Therefore, the subject 90 who has seen the display screen 200 can immediately recognize whether the breathing method at the current timing is correct during his own breathing training. That is, the subject 90 who has seen the display screen 200 shown in FIGS. 12 and 13 can immediately recognize that the breathing method at the current timing is correct during his own breathing training. On the other hand, the subject 90 who has seen the display screen 200 shown in FIG. 14 can immediately recognize that the breathing method at the current timing is incorrect during his own breathing training. Therefore, the subject 90 can easily recognize his own breathing state in real time.

[0123] Return to the description of the flowchart shown in FIG. 5. The width detection unit 120 detects changes in the widths of the chest 92 and the abdomen 94 respectively (step S130). That is, the width detection unit 120 detects the displacement amounts in the left-right direction of the chest 92 and the abdomen 94 respectively. Specifically, the width detection unit 120 detects the positions of the chest 92 and the abdomen 94 in the left-right direction (width direction). More specifically, the width detection unit 120 identifies the chest region and the abdomen region in the captured image 52 of the subject 90 obtained in the process of S106. This identification may be performed using the process of identifying the regions corresponding to the chest 92 and the abdomen 94 respectively in S104.

[0124] Also, the width detection unit 120 detects the positions of the chest 92 and the abdomen 94 in the left-right direction in the same manner as when the reference positions in the left-right direction of the chest 92 and the abdomen 94 are identified in the process of S104. Specifically, the width detection unit 120 detects the width of the chest 92 (chest width) using the position information corresponding to the chest region identified in the captured image 52 obtained in the process of S106. Similarly, the width detection unit 120 detects the width of the abdomen 94 (abdomen width) using the position information corresponding to the abdomen region identified in the captured image 52 obtained in the process of S106.

[0125] Furthermore, the width detection unit 120 calculates the displacement amount of the chest width with respect to the chest reference width identified in S104. The width detection unit 120 calculates the difference between the chest width and the chest reference width as the displacement amount in the left-right direction (chest width increase / decrease amount). Here, when the chest width is larger (wider) than the chest reference width, the sign of the displacement amount in the left-right direction is positive (+). On the other hand, when the chest width is smaller (narrower) than the chest reference width, the sign of the displacement amount in the left-right direction is negative (-). Therefore, the width detection unit 120 calculates the displacement amount of the chest 92 in the left-right direction by subtracting the value of the chest reference width from the value of the chest width. The chest width may be, for example, the distance between the left end and the right end of the chest 92.

[0126] Furthermore, the width detection unit 120 calculates the displacement amount of the abdominal width with respect to the abdominal reference width specified in S104. The width detection unit 120 calculates the difference between the abdominal width and the abdominal reference width as the displacement amount in the left - right direction (abdominal width increase / decrease amount). Here, when the abdominal width is larger (wider) than the abdominal reference width, the sign of the displacement amount in the left - right direction is positive (+). On the other hand, when the abdominal width is smaller (narrower) than the abdominal reference width, the sign of the displacement amount in the left - right direction is negative (-). Therefore, the width detection unit 120 calculates the displacement amount in the left - right direction of the abdomen 94 by subtracting the value of the abdominal reference width from the value of the abdominal width. Similarly, the abdominal width may be, for example, the distance between the left end and the right end of the abdomen 94. Note that the width detection unit 120 does not necessarily need to calculate the displacement amount in the left - right direction of the abdomen 94. That is, the width detection unit 120 only needs to calculate at least the displacement amount in the left - right direction of the chest 92.

[0127] The display control unit 130 performs control for displaying the width image (step S132). Note that the width image is the above - described motion image and is an image showing the increase and decrease of the width from the reference width, that is, the change (displacement amount in the left - right direction). That is, the display control unit 130 performs control so that the displacement amount in the left - right direction of the chest 92 and the abdomen 94 of the subject 90 is displayed. The display control unit 130 performs control for displaying the chest width image and the abdominal width image. Here, the "chest width image" is the above - described chest motion image and is an image (width image) showing the change (increase / decrease) of the width of the chest 92 from the chest reference width. That is, the chest width image is an image showing the displacement amount (chest width increase / decrease amount) of the chest width with respect to the chest reference width. Also, the "abdominal width image" is the above - described abdominal motion image and is an image (width image) showing the change (increase / decrease) of the width of the abdomen 94 from the abdominal reference width. That is, the abdominal width image is an image showing the displacement amount (abdominal width increase / decrease amount) of the abdominal width with respect to the abdominal reference width.

[0128] Here, the display control unit 130 may perform control so that the display form of the chest width image changes depending on whether the chest width increase / decrease amount is below the threshold Th1 (<0) or not. For example, the display control unit 130 may perform control so that the chest width image is displayed in blue when the chest width increase / decrease amount is not below the threshold Th1, and the chest width image is displayed in red when the chest width increase / decrease amount is below the threshold Th1.

[0129] The display control unit 130 generates data (width image display instruction data) indicating an instruction to display the width image and transmits it to the display device 40. The width image display instruction data may indicate the increase / decrease amount. In particular, the width image display instruction data may indicate the chest width increase / decrease amount and the abdominal width increase / decrease amount. Further, the width image display instruction data may indicate the color representation of the chest width image drawn according to the comparison between the chest width increase / decrease amount and the threshold Th1 (<0).

[0130] Also, the display control unit 130 performs control so that the width image changes and is displayed in real time as the breathing training progresses. That is, the display control unit 130 causes the display device 40 to display a width image that changes according to the progress of the breathing training of the subject 90. In other words, the display control unit 130 performs control so that a width image (moving image) that changes according to the progress of the breathing training of the subject 90 is displayed during the progress of the breathing training. More specifically, the display control unit 130 performs control so that a width image corresponding to the width increase / decrease amount at a certain timing during the breathing training is displayed at that timing.

[0131] Also, the display control unit 130 generates width image display instruction data each time a frame of the captured image of the subject 90 performing the breathing training is acquired, and transmits it to the display device 40. Thereby, the display control unit 130 can cause the display device 40 to display a width image that changes as the breathing training progresses.

[0132] FIG. 15 is a diagram illustrating a display screen 200 that is displayed on the display device 40 under the control of the display control unit 130 according to the first embodiment and shows the displacement amounts in the left - right direction of the chest 92 and the abdomen 94. The display screen 200 illustrated in FIG. 15 displays a chest width screen 270 and an abdomen width screen 280 together with the chest displacement amount screen 210 and the abdomen displacement amount screen 220 illustrated in FIG. 7. Note that the display screen 200 may display the chest width screen 270 and the abdomen width screen 280 separately from the chest displacement amount screen 210 and the abdomen displacement amount screen 220.

[0133] On the chest width screen 270, a chest width image 272 indicating the chest width increase - decrease amount (the displacement amount in the left - right direction of the chest 92) is displayed in blue. Also, on the abdomen width screen 280, an abdomen width image 282 indicating the abdomen width increase - decrease amount (the displacement amount in the left - right direction of the abdomen 94) is displayed in blue. Further, a threshold value Th1 is displayed on the chest width screen 270. Then, on the chest width screen 270, when the chest width increase - decrease amount is less than the threshold value Th1, the chest width image 272 may be changed to, for example, red. Also, the color representation of the chest width image 272 may change according to the difference between the chest width increase - decrease amount and the threshold value Th1. For example, during exhalation, as the chest width increase - decrease amount (the chest width image 272) approaches the threshold value Th1, it may gradually change from blue to red.

[0134] As described above, under the control of the display control unit 130, the display device 40 displays a chest width image 272 indicating the displacement amount in the left - right direction of the chest 92. Therefore, the subject 90 can visually and easily recognize the movement of the chest 92 in the left - right direction during breathing training by looking at the chest width image 272 (chest width screen 270). Further, the chest width screen 270 displays a threshold value Th1. Therefore, when the chest width image 272 indicates that the chest width is below the threshold value Th1 during exhalation, the subject 90 can visually and easily recognize that the internal rotation of the ribs is sufficiently performed. On the other hand, when the chest width image 272 does not indicate that the chest width is below the threshold value Th1 during exhalation, the subject 90 can visually and easily recognize that the internal rotation of the ribs is not sufficiently performed. Therefore, the subject 90 can easily recognize his / her own breathing state.

[0135] Also, the display screen 200 illustrated in FIG. 15 displays the displacement amount (chest width) in the left - right direction of the chest 92 at the timing when this display screen 200 is displayed. Therefore, the display screen 200 displays the chest width in real - time during the breathing training of the subject 90. Therefore, the subject 90 who sees the display screen 200 can immediately recognize whether the breathing method at the current timing is correct during his / her own breathing training. Therefore, the subject 90 can easily recognize his / her own breathing state in real - time.

[0136] FIGS. 16 and 17 are diagrams illustrating a display screen for a leader according to Embodiment 1. Note that the display screen for a leader may be displayed on the display device 40. Also, the display screen for a leader may be displayed on a display device carried by the leader, which is different from the display device 40 visually recognized by the subject 90. Further, the display screen for a leader may be displayed on the interface unit 108 of the information processing device 100.

[0137] FIG. 16 illustrates a leader display screen 300 which is a first example of the leader display screen. The leader display screen 300 displays a chest displacement amount screen 210, an abdominal displacement amount screen 220, and a body displacement amount screen 230 shown in FIG. 7 and the like. Further, the leader display screen 300 displays a setting screen 310, an RGB image display screen 322, and a distance image display screen 324. The setting screen 310 is used for settings for respiratory training. The RGB image display screen 322 displays an RGB image obtained by the imaging device 30 photographing the subject 90. The distance image display screen 324 displays a distance image obtained by the imaging device 30 photographing the subject 90. The leader can confirm whether the subject 90 is appropriately photographed by the imaging device 30 by visually recognizing the RGB image display screen 322 and the distance image display screen 324.

[0138] FIG. 17 illustrates a leader display screen 400 which is a second example of the leader display screen. The leader display screen 400 displays a chest displacement amount screen 210, an abdominal displacement amount screen 220, a chest width screen 270, and an abdominal width screen 280 shown in FIG. 15. Further, the leader display screen 300 displays a setting screen 410, an RGB image display screen 322, a distance image display screen 324, and a photographed image display screen 420. The setting screen 410 is used for settings for respiratory training. The photographed image display screen 420 displays a photographed image 52.

[0139] In this way, by separately displaying the leader display screens 300 and 400 from the display screen 200 that the subject 90 visually recognizes, it is possible to suppress the subject 90 from visually recognizing information that is not necessary for the subject 90. Therefore, the subject 90 can concentrate on performing respiratory training.

[0140] Returning to the description of the flowchart shown in FIG. 5. The result determination unit 140 determines the evaluation of the result of respiratory training (step S140). Then, the result determination unit 140 generates evaluation information regarding the evaluation of the training result.

[0141] For example, the result determination unit 140 determines whether the movement of the chest 92 and the movement of the abdomen 94 are synchronized. Then, the result determination unit 140 generates evaluation information indicating whether the movement of the chest 92 and the movement of the abdomen 94 are synchronized.

[0142] Specifically, the result determination unit 140 determines whether the signs of the chest displacement amount and the abdomen displacement amount are the same as each other. When the signs of the chest displacement amount and the abdomen displacement amount are the same as each other, the result determination unit 140 may determine that the movement of the chest 92 and the movement of the abdomen 94 are synchronized. On the other hand, when the signs of the chest displacement amount and the abdomen displacement amount are different from each other, the result determination unit 140 may determine that the movement of the chest 92 and the movement of the abdomen 94 are not synchronized.

[0143] In addition, the result determination unit 140 determines whether the chest movement direction and the abdomen movement direction are the same as each other. When the chest movement direction and the abdomen movement direction are the same as each other, the result determination unit 140 may determine that the movement of the chest 92 and the movement of the abdomen 94 are synchronized. On the other hand, when the chest movement direction and the abdomen movement direction are different from each other, the result determination unit 140 may determine that the movement of the chest 92 and the movement of the abdomen 94 are not synchronized. Alternatively, when the signs of the chest displacement amount and the abdomen displacement amount are the same as each other and the chest movement direction and the abdomen movement direction are the same as each other, the result determination unit 140 may determine that the movement of the chest 92 and the movement of the abdomen 94 are synchronized. On the other hand, when the signs of the chest displacement amount and the abdomen displacement amount are different from each other or the chest movement direction and the abdomen movement direction are different from each other, the result determination unit 140 may determine that the movement of the chest 92 and the movement of the abdomen 94 are not synchronized.

[0144] Alternatively, the result determination unit 140 may determine whether the movement of the chest 92 and the movement of the abdomen 94 are synchronized using a trained model trained by machine learning such as a neural network. The trained model is trained, for example, by supervised learning. For example, the trained model is trained to take the chest displacement amount and the abdominal displacement amount, and the chest movement direction and the abdominal movement direction as inputs, and output a determination result of the synchronization between the chest and the abdomen when the input data is obtained. Note that, in the learning stage, the determination result of synchronization, which is the correct label, can be set by an instructor or the like. Thus, by using the trained model, it is possible to more accurately determine whether the movement of the chest 92 and the movement of the abdomen 94 are synchronized.

[0145] Further, for example, the result determination unit 140 determines whether the balance between the chest displacement amount and the abdominal displacement amount is good. The result determination unit 140 generates evaluation information indicating whether the balance between the chest displacement amount and the abdominal displacement amount is good. The result determination unit 140 determines whether the balance between the chest displacement amount and the abdominal displacement amount is good using the ratio of the chest displacement amount to the abdominal displacement amount.

[0146] Specifically, when the abdominal displacement amount is x1 and the chest displacement amount is x2, the ratio s is represented by the following formula (1). That is, s is the ratio of the chest displacement amount x2 to the abdominal displacement amount x1. s = x2 / x1 ···(1)

[0147] Here, when s < 0, that is, when the signs of x1 and x2 do not match, the result determination unit 140 determines that the balance between the chest displacement amount and the abdominal displacement amount is not good. Also, when s > 0, that is, when the signs of x1 and x2 match, the result determination unit 140 determines that the balance between the chest displacement amount and the abdominal displacement amount is not good when s < Th2. On the other hand, the result determination unit 140 determines that the balance between the chest displacement amount and the abdominal displacement amount is good when s ≧ Th2. Note that Th2 is a predetermined constant greater than 0, such as 0.8. Th2 is adjusted according to the acquisition status of the breathing method for each subject 90.

[0148] Also, for example, the result determination unit 140 determines whether the internal rotation of the ribs of the subject 90 is sufficiently performed during exhalation. The result determination unit 140 generates evaluation information indicating whether the internal rotation of the ribs of the subject 90 is sufficiently performed during exhalation. Specifically, the result determination unit 140 determines whether the amount of increase or decrease in chest width during exhalation is less than the threshold value Th1. When the amount of increase or decrease in chest width during exhalation is less than the threshold value Th1, the result determination unit 140 determines that the internal rotation of the ribs is sufficiently performed. On the other hand, when the amount of increase or decrease in chest width during exhalation is not less than the threshold value Th1, the result determination unit 140 determines that the internal rotation of the ribs is not sufficiently performed.

[0149] Also, for example, the result determination unit 140 may evaluate the above-described evaluation of the training result (for example, the presence or absence of synchronization between the chest and the abdomen) by comparing it with the past training results. That is, the result determination unit 140 evaluates by comparing the current training result of the subject 90 with the past training result of the subject 90. Then, the result determination unit 140 generates evaluation information regarding the comparison between the past training result of the subject 90 and the current training result of the subject 90. That is, the result determination unit 140 generates evaluation information indicating whether the current training result of the subject has improved as compared with the past training result of the subject.

[0150] For example, in the past training, it is assumed that breathing was performed 10 times and the synchronization between the chest and the abdomen continued for 3 times. In this case, the result determination unit 140 may determine that the breathing method of the subject 90 has improved when the synchronization between the chest and the abdomen continues for 3 times or more in this training. The result determination unit 140 may similarly evaluate the current training result of the subject 90 by comparing it with the past training result of the subject 90 regarding the balance between the abdominal displacement amount and the chest displacement amount and the internal rotation of the ribs.

[0151] The output control unit 150 performs control so that evaluation information regarding the evaluation of the training result is output (step S142). The output control unit 150 generates data (evaluation information output instruction data) indicating an instruction for outputting the evaluation information by a method as described later, and transmits it to the display device 40. Thereby, the output control unit 150 performs control so that the evaluation information is output. The display device 40 outputs the evaluation information according to the evaluation information output instruction data.

[0152] For example, the output control unit 150 may perform control so that a message (character string) or an image corresponding to the evaluation information is displayed on the display device 40. Also, the output control unit 150 may perform control so that a voice corresponding to the evaluation information is output to the display device 40. Also, the output control unit 150 may perform control so that vibrations corresponding to the evaluation information are generated on the display device 40. Also, the output control unit 150 may perform control so that the color of the display screen 200 changes corresponding to the evaluation information.

[0153] Also, the output control unit 150 may perform control so that the evaluation information is output in real time during the breathing training. That is, the output control unit 150 may perform control so that the evaluation information is output at the timing when a training result that can change with the progress of the breathing training is obtained. In this case, the output control unit 150 may perform control so that the evaluation information corresponding to the moving image displayed on the display screen 200 is displayed together with the moving image. Alternatively, the output control unit 150 may perform control so that evaluation information (for example, an improvement method) that summarizes the training result is output after the training is completed.

[0154] The output control unit 150 may perform control so that evaluation information indicating whether the movement of the chest 92 and the movement of the abdomen 94 are synchronized is output. In this case, when the movement of the chest 92 and the movement of the abdomen 94 are synchronized, the output control unit 150 may perform control so that the frame or background of the display screen 200 is displayed in green. On the other hand, when the movement of the chest 92 and the movement of the abdomen 94 are not synchronized, the output control unit 150 may perform control so that the frame or background of the display screen 200 is displayed in red. Also, for example, when the movement of the chest 92 and the movement of the abdomen 94 are synchronized, the output control unit 150 may perform control so that a message such as "The chest and the abdomen are synchronized" is output on the display screen 200 as a character string or voice. On the other hand, when the movement of the chest 92 and the movement of the abdomen 94 are not synchronized, the output control unit 150 may perform control so that a message such as "The chest and the abdomen are not synchronized" is output on the display screen 200 as a character string or voice. At this time, the output control unit 150 may perform control so that an alarm sound is output. Also, for example, when the movement of the chest 92 and the movement of the abdomen 94 are synchronized, the output control unit 150 may perform control so that vibrations corresponding to the synchronization of the movement of the chest 92 and the movement of the abdomen 94 are generated in the display device 40. Note that the output control unit 150 may similarly perform control so that evaluation information is output regarding the balance between the abdominal displacement amount and the chest displacement amount, and the internal rotation of the ribs.

[0155] Further, the output control unit 150 may perform control so that a message for inducing breathing training is output. For example, when the training result is good (for example, the chest and abdomen are synchronized), the output control unit 150 may perform control so that a message such as "You're on the right track" is output as a character string or voice. On the other hand, when the training result is not good (for example, the chest and abdomen are not synchronized), the output control unit 150 may perform control so that a message such as "Keep it up" is output as a character string or voice. Also, when the internal rotation of the ribs is not sufficiently performed, the output control unit 150 may perform control so that a message such as "Please exhale completely" is output as a character string or voice. Further, when the current training result of the subject has improved compared to the past training result of the subject, the output control unit 150 may perform control so that a message such as "You're on the right track" is output as a character string or voice.

[0156] Further, when the training result is not good (for example, the chest and abdomen are not synchronized), the output control unit 150 may perform control so that an alarm is output by an image, voice, or the like. Also, the output control unit 150 may perform control so that the number of alarms in one training session and the transition of the number of alarms are displayed. Further, the output control unit 150 may perform control so that the items of the alarm are output. Note that the "items of the alarm" indicate the content (for example, the synchronization of the chest and abdomen) for which the alarm was output. Also, the output control unit 150 may perform control so that the target value and guidance on the breathing method are output.

[0157] In this way, by outputting the evaluation information under the control of the output control unit 150, it is possible to induce the subject 90's motivation for respiratory training. Further, by outputting the evaluation information regarding the comparison between the results of the subject 90's past training and the results of the subject 90's current training under the control of the output control unit 150, the subject 90 can easily recognize whether his / her breathing method has been improved. Therefore, it is possible to further induce the subject 90's motivation for respiratory training.

[0158] The information processing apparatus 100 determines whether to end the training (step S150). Specifically, the information processing apparatus 100 determines whether an operation to end the training has been performed by a user (for example, an instructor or the like). For example, when the "measurement stop" button is operated on the setting screen 310 of the instructor display screen 300 illustrated in FIG. 16, the information processing apparatus 100 determines to end the training. On the other hand, when the "measurement stop" button is not operated on the setting screen 310 of the instructor display screen 300 illustrated in FIG. 16, the information processing apparatus 100 determines to continue the training. When it is determined to continue the training (NO in S150), the processing flow returns to S106, and the processing of S106 to S142 is repeated. On the other hand, when it is determined to end the training (YES in S150), the processing flow ends.

[0159] (Modification example) Note that the present invention is not limited to the above-described embodiments, and can be appropriately modified without departing from the spirit. For example, one or more of the steps in the above-described flowchart can be appropriately omitted. Also, the order of one or more of the steps in the above-described flowchart can be appropriately changed. For example, in the flowchart shown in FIG. 5, the processes of S120 and S122 may be omitted. Also, the processes of S130 and S132 may be before the processes of S110 and S112. Also, the processes of S112, S122, and S132 may be executed together after all the processes of S110, S120, and S130 are completed. Also, a part of the processes of S140 and S142 may be executed after the training is completed.

[0160] Also, in the above-described embodiment, it is assumed that the chest displacement amount image and the abdominal displacement amount image are displayed on the display device 40 in real time during training, but the configuration is not limited to this. The chest displacement amount image and the abdominal displacement amount image may be displayed after the training is completed. In this case, the display device 40 may display, as a moving image, how the chest displacement amount image and the abdominal displacement amount image change according to the temporal transition of the training.

[0161] Also, the display system 20 may include a plurality of imaging devices 30. In this case, the subject 90 is photographed using the plurality of imaging devices 30. Thereby, since the subject 90 can be photographed from a plurality of viewpoints, it is possible to suppress the occurrence of a blind spot of the subject 90 during photographing. Therefore, it becomes possible to detect the displacement amount and the like with higher accuracy.

[0162] Also, the display system 20 may be realized by a device in which two or more of the imaging device 30, the display device 40, and the information processing device 100 are integrally configured. For example, the subject 90 may perform breathing training using one device (such as a smartphone) having the imaging device 30, the display device 40, and the information processing device 100. Thereby, it becomes possible to perform breathing training without special equipment. For example, the subject 90 can easily perform breathing training at home or the like.

[0163] In addition, in devices such as smartphones, it may not be possible to acquire skeletal data. In this case, the subject 90 may perform an operation to specify the chest region and the abdominal region in the self-photographed image 52. Further, when using a single device having the imaging device 30, the display device 40, and the information processing device 100, breathing training may be performed using a device having the imaging device 30 that cannot acquire three-dimensional data. For example, by installing the device in the lateral direction of the subject 90 and photographing the subject 90, it is possible to detect the displacement amount and the like of the chest 92 and the abdomen 94 of the subject 90. And the device that realizes the display system 20 may perform only the detection of the displacement amount and the like during the breathing training, and display a displacement amount image or the like after the training ends. Note that this device may output evaluation information by voice during the breathing training.

[0164] Further, in the above-described embodiment, the display form showing the moving image (displacement amount image, direction image, width increase / decrease image) is a red or blue color representation, but it is not limited thereto. The color representation can be arbitrarily set by the user. Further, the display form is not limited to the color representation. For example, the display form may be a grayscale (shades of black) or a plurality of types of patterns (hatching) or the like. That is, the display form may be such that when the movement of the chest and the movement of the abdomen are synchronized with each other, the chest movement image and the abdomen movement image become visually the same. In other words, the display form may be such that it can visually distinguish the movement of the chest and the movement of the abdomen.

[0165] When the above program is loaded into a computer, it includes a set of instructions (or software code) for causing the computer to perform one or more functions described in the embodiments. The program may be stored in a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, the computer-readable medium or tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD), or other memory technologies, CD-ROM, digital versatile disk (DVD), Blu-ray (registered trademark) disk, or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage, or other magnetic storage devices. The program may be transmitted on a transitory computer-readable medium or a communication medium. By way of example and not limitation, the transitory computer-readable medium or communication medium includes electrical, optical, acoustic, or other forms of propagated signals.

[0166] Some or all of the above embodiments may be described as follows, but are not limited thereto. (Appendix 1) An acquisition means for acquiring image data showing at least the chest and abdomen of a subject performing breathing training; A detection means for detecting the displacement amounts of the chest and abdomen of the subject using the image data; Display control means for performing control so that a motion image showing the forward and backward motion of the subject of the chest and abdomen respectively is displayed based on the detected displacement amounts of the chest and abdomen; having The display control means performs control such that when the motion of the chest and the motion of the abdomen are synchronized with each other, the display form of the chest motion image, which is the motion image of the chest, and the display form of the abdomen motion image, which is the motion image of the abdomen, are the same as each other, and when the motion of the chest and the motion of the abdomen are not synchronized with each other, the display form of the chest motion image and the display form of the abdomen motion image are different from each other. An information processing apparatus. (Appendix 2) The display control means controls so that a chest displacement amount image showing the displacement amount in the front - rear direction of the chest is displayed as the chest motion image, and an abdominal displacement amount image showing the displacement amount in the front - rear direction of the abdomen is displayed as the abdominal motion image. controls so that when the sign of the displacement amount of the chest and the sign of the displacement amount of the abdomen are the same, the display form of the chest displacement amount image and the display form of the abdominal displacement amount image are the same, and when the sign of the displacement amount of the chest and the sign of the displacement amount of the abdomen are different, the display form of the chest displacement amount image and the display form of the abdominal displacement amount image are different. The information processing apparatus according to Appendix 1. (Appendix 3) The display control means controls so that the motion image that changes according to the progress of the subject's breathing training is displayed during the progress of the training. The information processing apparatus according to Appendix 1 or 2. (Appendix 4) The display control means further controls so that the displacement amount in the left - right direction of the subject's chest is displayed. The information processing apparatus according to any one of Appendices 1 to 3. (Appendix 5) Output control means for controlling so that evaluation information regarding the evaluation of the result of the training is output. The information processing apparatus according to any one of Appendices 1 to 4, further comprising the above. (Appendix 6) The output control means controls so that the evaluation information regarding the comparison between the result of the subject's past training and the result of the current training is output. The information processing apparatus according to Appendix 5. (Appendix 7) The output control means controls so that the evaluation information indicating whether the motion of the chest and the motion of the abdomen are synchronized is output. The information processing apparatus according to Appendix 5 or 6. (Appendix 8) The output control means performs control so that the evaluation information indicating that the balance between the displacement amount of the abdomen and the displacement amount of the chest is good is output when the ratio of the displacement amount of the chest in the front-rear direction to the displacement amount of the abdomen in the front-rear direction is equal to or greater than a predetermined threshold value. The information processing apparatus according to any one of Appendices 5 to 7. (Appendix 9) The display control means performs control so that a chest direction image indicating the direction of change in the displacement amount of the chest is displayed as the chest motion image, and an abdominal direction image indicating the direction of change in the displacement amount of the abdomen is displayed as the abdominal motion image, and performs control so that the display form of the chest direction image and the display form of the abdominal direction image are the same when the direction of change in the displacement amount of the chest and the direction of change in the displacement amount of the abdomen are the same, and the display form of the chest direction image and the display form of the abdominal direction image are different when the direction of change in the displacement amount of the chest and the direction of change in the displacement amount of the abdomen are different. The information processing apparatus according to any one of Appendices 1 to 8. (Appendix 10) The display control means performs control so that the color representation of the chest motion image and the color representation of the abdominal motion image are the same when the motion of the chest and the motion of the abdomen are synchronized with each other, and the color representation of the chest motion image and the color representation of the abdominal motion image are different when the motion of the chest and the motion of the abdomen are not synchronized with each other. The information processing apparatus according to any one of Appendices 1 to 9. (Appendix 11) Position specifying means for specifying a chest reference position that is the reference position in the front-rear direction of the chest and an abdominal reference position that is the reference position in the front-rear direction of the abdomen, further having When the movement of the chest with respect to the chest reference position and the movement of the abdomen with respect to the abdomen reference position are synchronized, the display forms of the chest movement image and the abdomen movement image are the same as each other, and when the movement of the chest with respect to the chest reference position and the movement of the abdomen with respect to the abdomen reference position are not synchronized, the display forms of the chest movement image and the abdomen movement image are different from each other, and control is performed. The information processing apparatus according to any one of Appendices 1 to 10. (Appendix 12) The information processing apparatus according to any one of Appendices 1 to 11, At least one imaging device that images the subject, A display device that displays the movement image, and has The acquisition means acquires the image data from the imaging device, The display control means controls the display device. Display system. (Appendix 13) Acquire image data showing at least the chest and abdomen of a subject performing breathing training, Using the image data, detect the displacement amounts of the chest and abdomen of the subject respectively, Based on the detected displacement amounts of the chest and abdomen, control is performed so that movement images showing the movement of the subject in the front-rear direction of the chest and abdomen are displayed respectively, When the movement of the chest and the movement of the abdomen are synchronized with each other, the display form of the chest movement image, which is the movement image of the chest, and the display form of the abdomen movement image, which is the movement image of the abdomen, are the same as each other, and when the movement of the chest and the movement of the abdomen are not synchronized with each other, the display form of the chest movement image and the display form of the abdomen movement image are different from each other, and control is performed. Display control method. (Appendix 14) Control is performed so that a chest displacement amount image showing the displacement amount in the front-rear direction of the chest is displayed as the chest movement image, and an abdomen displacement amount image showing the displacement amount in the front-rear direction of the abdomen is displayed as the abdomen movement image. When the signs of the displacement amounts of the chest and the abdomen are the same, the display forms of the chest displacement amount image and the abdomen displacement amount image are the same, and when the signs of the displacement amounts of the chest and the abdomen are different from each other, control is performed so that the display forms of the chest displacement amount image and the abdomen displacement amount image are different from each other. The display control method according to Supplementary Note 13. (Supplementary Note 15) Control is performed so that the moving image that changes according to the progress of the breathing training of the subject is displayed during the progress of the training. The display control method according to Supplementary Note 13 or 14. (Supplementary Note 16) Furthermore, control is performed so that the displacement amount in the left-right direction of the chest of the subject is displayed. The display control method according to any one of Supplementary Notes 13 to 15. (Supplementary Note 17) Control is performed so that evaluation information regarding the evaluation of the result of the training is output. The display control method according to any one of Supplementary Notes 13 to 16. (Supplementary Note 18) Control is performed so that the evaluation information regarding the comparison between the result of the subject's past training and the result of the current training is output. The display control method according to Supplementary Note 17. (Supplementary Note 19) Control is performed so that the evaluation information indicating whether the movement of the chest and the movement of the abdomen are synchronized is output. The display control method according to Supplementary Note 17 or 18. (Supplementary Note 20) Control is performed so that the evaluation information indicating that the balance between the displacement amount of the abdomen and the displacement amount of the chest is good when the ratio of the displacement amount in the front-rear direction of the chest to the displacement amount in the front-rear direction of the abdomen is equal to or greater than a predetermined threshold value is output. The display control method according to any one of Supplementary Notes 17 to 19. (Supplementary Note 21) Control is performed such that a chest direction image indicating the direction of change in the displacement amount of the chest is displayed as the chest movement image, and an abdominal direction image indicating the direction of change in the displacement amount of the abdomen is displayed as the abdominal movement image. Control is performed such that when the direction of change in the displacement amount of the chest and the direction of change in the displacement amount of the abdomen are the same, the display forms of the chest direction image and the abdominal direction image are the same, and when the direction of change in the displacement amount of the chest and the direction of change in the displacement amount of the abdomen are different, the display forms of the chest direction image and the abdominal direction image are different. The display control method according to any one of Appendices 13 to 20. (Appendix 22) Control is performed such that when the movement of the chest and the movement of the abdomen are synchronized with each other, the color representations of the chest movement image and the abdominal movement image are the same, and when the movement of the chest and the movement of the abdomen are not synchronized with each other, the color representations of the chest movement image and the abdominal movement image are different. The display control method according to any one of Appendices 13 to 21. (Appendix 23) Identify the chest reference position, which is the reference position in the front-rear direction of the chest, and the abdominal reference position, which is the reference position in the front-rear direction of the abdomen. Control is performed such that when the movement of the chest relative to the chest reference position and the movement of the abdomen relative to the abdominal reference position are synchronized, the display forms of the chest movement image and the abdominal movement image are the same, and when the movement of the chest relative to the chest reference position and the movement of the abdomen relative to the abdominal reference position are not synchronized, the display forms of the chest movement image and the abdominal movement image are different. The display control method according to any one of Appendices 13 to 22. (Appendix 24) A step of acquiring image data showing at least the chest and abdomen of a subject who performs breathing training. A step of detecting the displacement amount of each of the chest and abdomen of the subject using the image data. Based on the detected displacement amounts of the chest and abdomen, control is performed so that movement images indicating the movement of the subject in the front-rear direction of the chest and abdomen are displayed. When the movement of the chest and the movement of the abdomen are synchronized with each other, the display form of the chest movement image, which is the movement image of the chest, and the display form of the abdomen movement image, which is the movement image of the abdomen, are the same as each other. When the movement of the chest and the movement of the abdomen are not synchronized with each other, control is performed so that the display form of the chest movement image and the display form of the abdomen movement image are different from each other, and A program that causes a computer to execute.

[0167] Although the present invention has been described with reference to the embodiments above, the present invention is not limited thereto. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the invention.

[0168] This application claims priority based on Japanese Patent Application No. 2021-136990 filed on August 25, 2021, and incorporates the entire disclosure thereof herein.

Explanation of Signs

[0169] 1 Information processing apparatus 2 Acquisition unit 4 Detection unit 6 Display control unit 20 Display system 30 Imaging device 40 Display device 52 Captured image 90 Subject 92 Chest 94 Abdomen 100 Information processing apparatus 112 Image acquisition unit 114 Position specifying unit 116 Displacement amount detection unit 118 Movement direction detection unit 120 Width detection unit 130 Display control unit 140 Result determination unit 150 Output control unit 200 Display screen 210 Chest Displacement Amount Screen 212 Chest Displacement Amount Image 220 Abdomen Displacement Amount Screen 222 Abdomen Displacement Amount Image 230 Trunk Displacement Amount Screen 232bk Trunk Reference Position Dot Array 234 Trunk Position Dot Array 240 Displacement Amount History Screen 250 Chest Direction Screen 252 Chest Direction Image 260 Abdomen Direction Screen 262 Abdomen Direction Image 270 Chest Width Screen 272 Chest Width Image 280 Abdomen Width Screen 282 Abdomen Width Image 300 Instructor Display Screen 310 Setting Screen 322 RGB Image Display Screen 324 Distance Image Display Screen 400 Instructor Display Screen 410 Setting Screen 420 Captured Image Display Screen

Claims

1. An acquisition means for acquiring image data showing at least the chest and abdomen of a subject undergoing breathing training; A detection means for detecting the displacement amount of each of the chest and abdomen of the subject using the image data; Display control means for performing control so that movement images showing the movement of the subject in the front-rear direction of each of the chest and abdomen are displayed based on the detected displacement amounts of the chest and abdomen; comprising: When the movement of the chest and the movement of the abdomen are synchronized with each other, the display control means makes the display form of the chest movement image, which is the movement image of the chest, the same as the display form of the abdomen movement image, which is the movement image of the abdomen. When the movement of the chest and the movement of the abdomen are not synchronized with each other, the display control means performs control so that the display form of the chest movement image and the display form of the abdomen movement image are different from each other. An information processing apparatus.

2. The display control means: Performs control so that a chest displacement amount image showing the displacement amount of the chest in the front-rear direction is displayed as the chest movement image, and an abdomen displacement amount image showing the displacement amount of the abdomen in the front-rear direction is displayed as the abdomen movement image. When the signs of the displacement amount of the chest and the sign of the displacement amount of the abdomen are the same as each other, the display control means makes the display form of the chest displacement amount image the same as the display form of the abdomen displacement amount image. When the signs of the displacement amount of the chest and the sign of the displacement amount of the abdomen are different from each other, the display control means performs control so that the display form of the chest displacement amount image and the display form of the abdomen displacement amount image are different from each other. The information processing apparatus according to Claim 1.

3. The display control means performs control so that the movement image that changes in accordance with the progress of the breathing training of the subject is displayed during the progress of the training. The information processing apparatus according to Claim 1 or 2.

4. The display control means further performs control so that the displacement amount of the subject's chest in the left-right direction is displayed. The information processing apparatus according to Claim 1 or 2.

5. Output control means for performing control so that evaluation information regarding the evaluation of the result of the training is output; The information processing apparatus according to Claim 1, further comprising:

6. The output control means performs control so that the evaluation information regarding the comparison between the result of the subject's past training and the result of the current training is output. The information processing apparatus according to Claim 5.

7. The output control means performs control so that the evaluation information indicating whether the movement of the chest and the movement of the abdomen are synchronized is output. The information processing apparatus according to claim 5 or 6.

8. The output control means performs control so that the evaluation information indicating that the balance between the displacement amount of the abdomen and the displacement amount of the chest is good is output when the ratio of the displacement amount of the chest in the front-rear direction to the displacement amount of the abdomen in the front-rear direction is equal to or greater than a predetermined threshold value. The information processing apparatus according to claim 5 or 6.

9. The display control means performs control so that a chest direction image indicating the direction of change in the displacement amount of the chest is displayed as the chest movement image, and an abdominal direction image indicating the direction of change in the displacement amount of the abdomen is displayed as the abdominal movement image, and performs control so that the display form of the chest direction image and the display form of the abdominal direction image are the same when the direction of change in the displacement amount of the chest and the direction of change in the displacement amount of the abdomen are the same as each other, and the display form of the chest direction image and the display form of the abdominal direction image are different from each other when the direction of change in the displacement amount of the chest and the direction of change in the displacement amount of the abdomen are different from each other. The information processing apparatus according to claim 1 or 2.

10. The display control means performs control so that the color representation of the chest movement image and the color representation of the abdominal movement image are the same when the movement of the chest and the movement of the abdomen are synchronized with each other, and the color representation of the chest movement image and the color representation of the abdominal movement image are different from each other when the movement of the chest and the movement of the abdomen are not synchronized with each other. The information processing apparatus according to claim 1 or 2.

11. Position specifying means for specifying a chest reference position that is the reference position of the chest in the front-rear direction and an abdominal reference position that is the reference position of the abdomen in the front-rear direction, further comprising: The display control means performs control so that the display form of the chest movement image and the display form of the abdominal movement image are the same when the movement of the chest with respect to the chest reference position and the movement of the abdomen with respect to the abdominal reference position are synchronized, and the display form of the chest movement image and the display form of the abdominal movement image are different from each other when the movement of the chest with respect to the chest reference position and the movement of the abdomen with respect to the abdominal reference position are not synchronized. The information processing apparatus according to claim 1 or 2.

12. The information processing apparatus according to claim 1 or 2, and at least one imaging device for photographing the subject. A display device that displays the moving image, having the acquisition means acquires the image data from the imaging device, the display control means controls the display device, display system.

13. Acquire image data showing at least the chest and abdomen of a subject undergoing respiratory training, Detect the displacement amount of each of the chest and abdomen of the subject using the image data, Based on the detected displacement amounts of the chest and abdomen, perform control so that moving images showing the movement of the subject in the front-rear direction of each of the chest and abdomen are displayed, When the movement of the chest and the movement of the abdomen are synchronized with each other, the display form of the chest movement image, which is the movement image of the chest, and the display form of the abdomen movement image, which is the movement image of the abdomen, are the same as each other, and when the movement of the chest and the movement of the abdomen are not synchronized with each other, perform control so that the display form of the chest movement image and the display form of the abdomen movement image are different from each other, Display control method.

14. A step of acquiring image data showing at least the chest and abdomen of a subject undergoing respiratory training, A step of detecting the displacement amount of each of the chest and abdomen of the subject using the image data, Based on the detected displacement amounts of the chest and abdomen, perform control so that moving images showing the movement of the subject in the front-rear direction of each of the chest and abdomen are displayed, and when the movement of the chest and the movement of the abdomen are synchronized with each other, the display form of the chest movement image, which is the movement image of the chest, and the display form of the abdomen movement image, which is the movement image of the abdomen, are the same as each other, and when the movement of the chest and the movement of the abdomen are not synchronized with each other, perform control so that the display form of the chest movement image and the display form of the abdomen movement image are different from each other, and A program for causing a computer to execute.

Images