Image processing device, image processing method, and program

Abstract

To prevent a user from getting injured due to overexertion.SOLUTION: An image processing system in an embodiment has an acquisition unit, an identification unit, a calculation unit and a display control unit. The acquisition unit acquires an optical image of a subject. The identification unit identifies the position of a muscle of the subject based on the optical image. The calculation unit calculates a blood flow rate in the muscle based on the optical image. The display control unit allows a display unit to display the blood flow rate in the muscle.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The embodiments disclosed in this specification and the drawings relate to an image processing device, an image processing method, and a program. [Background technology]

[0002] It is difficult to visually determine whether fatigue has accumulated in muscles, and it is also difficult to know which muscles are fatigued, so fatigue can accumulate in muscles without you realizing it, and muscles that have accumulated too much fatigue can suffer from muscle tears.

[0003] On the other hand, there are known technologies that estimate the joints of the human body from depth images and then further estimate the skeleton from the estimated joints, as well as technologies that use an infrared camera to combine the contours of multiple blood vessels into one to estimate three-dimensional vascular structure. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-6250 Summary of the Invention [Problem to be solved by the invention]

[0005] The problem to be solved by the embodiments disclosed in this specification and the drawings is to prevent a user from getting injured due to excessive exercise. However, the problem to be solved by the embodiments disclosed in this specification and the drawings is not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described below can also be positioned as other problems. [Means for solving the problem]

[0006] An image processing device according to an embodiment includes an acquisition unit, an identification unit, a calculation unit, and a display control unit. The acquisition unit acquires an optical image of a subject. The identification unit identifies the position of a muscle in the subject based on the optical image. The calculation unit calculates a blood flow rate in the muscle based on the optical image. The display control unit causes a display unit to display the blood flow rate in the muscle. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a diagram illustrating an example of a usage scene of a terminal device 100 to which an image processing device according to a first embodiment is applied. [Figure 2] 1 is a diagram illustrating an example of the configuration of a terminal device 100 to which an image processing device according to a first embodiment is applied. [Figure 3] 4 is a flowchart showing the flow of a series of processes by the processing circuit 120 according to the first embodiment. [Figure 4] FIG. 10 is a diagram showing an example of displaying information according to fatigue level. [Figure 5] 10 is a flowchart showing the flow of a series of processes by a processing circuit 120 according to the second embodiment. [Figure 6] FIG. 10 is a diagram illustrating an example of an image processing system including a terminal device 100 and a server 200 according to a third embodiment. [Figure 7] FIG. 11 is a sequence diagram showing the flow of processing in an image processing system according to a third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, an image processing apparatus, an image processing method, and a program according to an embodiment will be described with reference to the drawings.

[0009] (First embodiment) [Usage scenarios and overview] FIG. 1 is a diagram illustrating an example of a usage scenario of a terminal device 100 to which an image processing device according to a first embodiment is applied. For example, a user may use a camera-equipped terminal device 100 to capture an image of a part of their own body (e.g., a calf as shown in the figure) or the whole body as part of managing their daily exercise routine. The terminal device 100 may be, for example, a smartphone or a tablet device. A user may use the terminal device 100 to capture an image of another user's whole body or part of their own body instead of or in addition to their own body. The terminal device 100 analyzes the captured image of the whole or part of the body to estimate muscle-related indices such as the fatigue level and inflammation level of the subject's muscles, and presents the estimated indices to the user. This allows the user to prevent injuries such as muscle strains caused by excessive exercise.

[0010] [Terminal device configuration] 2 is a diagram illustrating an example of the configuration of a terminal device 100 to which the image processing device according to the first embodiment is applied. The terminal device 100 includes, for example, a camera 111, a communication interface 112, an input interface 113, an output interface 114, a memory 115, and a processing circuit 120.

[0011] The camera 111 is, for example, a multi-lens camera and includes two or more hyperspectral cameras. A hyperspectral camera may typically be a camera that captures images by dispersing light into the wavelength band of visible light and other wavelength bands such as infrared light. The camera 111 may also include a thermal camera. For example, when the camera 111 captures an image of the user's body, it generates an optical image of the body and outputs the image to the processing circuit 120. In the following description, the "optical image" will be simply referred to as an "image." Infrared rays are an example of "electromagnetic waves of a predetermined wavelength."

[0012] The communication interface 112 communicates with external devices via a communication network NW. The communication network NW may refer to any information communication network that uses electrical communication technology. For example, the communication network NW includes a wireless LAN (Local Area Network), a wired LAN, the Internet, a telephone communication network, an optical fiber communication network, a cable communication network, a satellite communication network, and the like. The communication interface 112 includes, for example, a network interface card (NIC) and an antenna for wireless communication.

[0013] The input interface 113 accepts various input operations from an operator, converts the accepted input operations into electrical signals, and outputs the electrical signals to the processing circuit 120. For example, the input interface 113 is typically a touch panel, but is not limited to this, and may include a mouse, a keyboard, a trackball, a switch, a button, a joystick, etc. The input interface 113 may also be a user interface that accepts audio input from a microphone, etc. If the input interface 113 is a touch panel, the input interface 113 may also have the display function of a display 113a included in the output interface 114, which will be described later.

[0014] In this specification, the input interface 113 is not limited to an interface having physical operation parts such as a mouse, a keyboard, etc. For example, an example of the input interface 113 also includes an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the device and outputs this electrical signal to a control circuit.

[0015] The output interface 114 includes, for example, a display 113a and a speaker 113b. The display 113a displays various types of information. For example, the display 113a displays images generated by the processing circuit 120 and a GUI (Graphical User Interface) for receiving various input operations from an operator. For example, the display 113a is an LCD (Liquid Crystal Display), an organic EL (Electro Luminescence) display, or the like. The speaker 113b outputs information input from the processing circuit 120 as sound.

[0016] The memory 115 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory), a flash memory, a hard disk, or an optical disk. These non-transitory storage media may be realized by other storage devices connected via a communication network NW, such as a NAS (Network Attached Storage) or an external storage server device. The memory 115 may also include other non-transitory storage media such as a ROM (Read Only Memory) or a register. The memory 115 stores programs executed by a hardware processor, anatomical skeleton data, and the like.

[0017] The anatomical skeleton data is a database used for skeleton estimation, which will be described later, and specifically, is a database that defines correspondences such as which bones are connected to which joints, the lengths between joints, etc. The anatomical skeleton data may be prepared individually according to age, sex, race, etc.

[0018] The processing circuit 120 includes, for example, an acquisition function 121, a determination function 122, a calculation function 123, and an output control function 124. The processing circuit 120 realizes these functions by, for example, a hardware processor (computer) executing a program stored in a memory 115 (storage circuit). The acquisition function 121 is an example of an "acquisition unit," and the determination function 122 is an example of an "determination unit." The calculation function 123 is an example of a "calculation unit," "first calculation unit," or "second calculation unit," and the output control function 124 is an example of a "display control unit."

[0019] The hardware processor in the processing circuit 120 refers to a circuit such as a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD) or a complex programmable logic device (CPLD), or a field programmable gate array (FPGA)). Instead of storing the program in the memory 115, the program may be directly embedded in the circuit of the hardware processor. In this case, the hardware processor realizes its function by reading and executing the program embedded in the circuit. The program may be stored in the memory 115 in advance, or may be stored in a non-transitory storage medium such as a DVD or CD-ROM, and installed in the memory 115 from the non-transitory storage medium when the non-transitory storage medium is inserted into a drive device (not shown) of the user interface 10. The hardware processor is not limited to being configured as a single circuit, but may be configured as a single hardware processor by combining multiple independent circuits to realize each function, or multiple components may be integrated into a single hardware processor to realize each function.

[0020] [Processing flow of terminal device (image processing device)] A series of processes performed by the processing circuit 120 of the terminal device 100 will be described below with reference to the flowchart. Fig. 3 is a flowchart showing the flow of a series of processes performed by the processing circuit 120 according to the first embodiment.

[0021] First, the user captures an image of part or all of his or her body using the terminal device 100 (step S100). In response, the acquisition function 121 acquires a plurality of images capturing a part or all of the user's body from the camera 111 (multi-lens camera). That is, the acquisition function 121 acquires a plurality of images captured from different viewpoints by the multi-lens camera. The images may be still images or moving images.

[0022] Next, the identification function 122 identifies the three-dimensional positions of the user's joints based on the multiple images (step S102). For example, the identification function 122 may identify the three-dimensional positions of the joints using a three-dimensional position estimation technique that applies stereoscopic vision using a multi-lens camera. "Identify" may also be rephrased as "estimate" or "detect."

[0023] Next, the identification function 122 identifies the user's skeleton based on the anatomical skeleton data stored in the memory 115 and the three-dimensional positions of the joints (step S104). For example, the identification function 122 may estimate the bones connecting the joints and their lengths in light of the correspondence relationship between the anatomical skeleton data, such as which bones are connected to which joints, and identify the user's skeleton based on how the bones are connected to each other.

[0024] Next, based on the joints and skeleton whose three-dimensional positions have been identified, the identification function 122 identifies where and how each muscle is located within the area of ​​the body shown in the image, i.e., the three-dimensional position of each muscle (step S106).

[0025] On the other hand, the calculation function 123 detects hemoglobin in the muscle during the systole when blood vessels contract and during the diastole when blood vessels expand based on multiple images (time-series frames) of different time phases (step S108), and calculates the amount of hemoglobin during each period (step S110).

[0026] It is known that hemoglobin changes brightness when exposed to light. Near-infrared light, in particular, among infrared rays, can reach deep within a subject (approximately 2 cm from the surface) and is capable of detecting hemoglobin in muscles. Therefore, the calculation function 123 detects hemoglobin on an image using a near-infrared optical image (hereinafter referred to as a near-infrared image) obtained by a hyperspectral camera. Furthermore, since the three-dimensional positions of bones and muscles are identified by the identification function 122, the calculation function 123 calculates the amount of hemoglobin distributed in the area corresponding to the muscle on the image.

[0027] Next, the calculation function 123 calculates the difference between the amount of hemoglobin in the muscle during the vasoconstriction period and the amount of hemoglobin in the muscle during the vasodiastole period, and calculates the blood flow rate in the muscle based on the difference in hemoglobin amount (step S112). In other words, the calculation function 123 calculates the amount of hemoglobin for each of the time-series frames captured during a period of one heartbeat or more that includes the vasoconstriction period and the vasodiastole period, and calculates the blood flow rate in each muscle from the difference between the maximum and minimum hemoglobin amounts among the hemoglobin amounts in each frame.

[0028] In this case, calculation function 123 may calculate the blood flow rate of each muscle based on the distribution of hemoglobin in the muscle, instead of or in addition to the amount of hemoglobin in the muscle. For example, calculation function 123 calculates the distribution of hemoglobin in the muscle during both vasoconstriction and vasodilation, calculates the thickness of the blood vessels in the muscle during vasoconstriction based on the distribution of hemoglobin in the muscle during vasoconstriction, and calculates the thickness of the blood vessels in the muscle during vasodilation based on the distribution of hemoglobin in the muscle during vasodilation. Then, calculation function 123 calculates the blood flow rate of the muscle based on the difference between the thickness of the blood vessel during vasoconstriction (minimum thickness) and the thickness of the blood vessel during vasodilation (maximum thickness).

[0029] Next, the calculation function 123 calculates a muscle-related index for each muscle based on the blood flow amount of each muscle (step S114). The muscle-related index is, for example, the degree of muscle fatigue.

[0030] For example, when a muscle is fatigued, blood flow to that muscle is stagnant, and the difference in blood flow between when the blood vessels contract and when they dilate is thought to be small. In other words, when muscle fatigue occurs, blood flow is expected to decrease. Therefore, the calculation function 123 may calculate a higher level of fatigue for that muscle as the blood flow to that muscle decreases, and may calculate a lower level of fatigue for that muscle as the blood flow to that muscle increases. More specifically, the maximum value of blood flow may be converted to a minimum value of fatigue (e.g., 0), and the minimum value of blood flow may be converted to a maximum value of fatigue (e.g., 100).

[0031] When calculating muscle-related indices (such as fatigue level), calculation function 123 may correct the blood flow rate for each muscle based on statistical values ​​of the blood flow rates for all muscles. Examples of statistical values ​​include the average, median, maximum, and minimum values. For example, if there are three muscles A, B, and C, calculation function 123 may calculate the average value of the blood flow rate for muscle A, the blood flow rate for muscle B, and the blood flow rate for muscle C, and normalize the blood flow rate for each muscle using this average value.

[0032] Next, the output control function 124 outputs the calculation result of the calculation function 123 (step S116). For example, the output control function 124 may cause the display 113a of the output interface 114 to display information corresponding to the fatigue level. Furthermore, instead of or in addition to displaying the information corresponding to the fatigue level on the display 113a, the output control function 124 may transmit the information corresponding to the fatigue level to an external device (for example, another terminal device available to the personal trainer) via the communication interface 112. This ends the processing of this flowchart.

[0033] 4 is a diagram showing an example of displaying information according to the fatigue level. As shown in the figure, when a user captures an image of their own legs using the terminal device 100, the display 113a of the terminal device 100 displays numerical values ​​(30, 50, 60, 70 in the figure) representing the fatigue level of each muscle in the legs, such as the hamstrings and gastrocnemius, and also displays a color map superimposed on the image, in which the color becomes darker as the fatigue level increases and the color becomes lighter as the fatigue level decreases.

[0034] Furthermore, the output control function 124 may output the time or number of days to rest to recover from the fatigue level as information according to the fatigue level. For example, the output control function 124 outputs the time or number of days based on the statistical value of the fatigue level of each muscle.

[0035] Furthermore, the output control function 124 may output a recommended recuperation method for recovering from the fatigue level as information according to the fatigue level. For example, the output control function 124 may output information such as "Icing" is required because the fatigue level of XXX muscle is high.

[0036] Furthermore, the output control function 124 may output a comparison result between a fatigue level calculated based on an image captured before the prescribed treatment and a fatigue level calculated based on an image captured after the prescribed treatment. For example, the output control function 124 may compare the fatigue level before the "massage" with the fatigue level after the "massage" and output the extent to which the fatigue level has recovered before and after the "massage."

[0037] The output control function 124 may also output a comparison result between the fatigue level of the left and right parts of the user's body. For example, when doing squats, the same amount of load is basically applied to both legs, so the muscle fatigue levels of the left and right legs should be the same. However, if there is a significant difference between the fatigue levels of the left and right parts after the squat (if the difference is equal to or greater than a threshold), the output control function 124 may output information indicating that there is a difference in fatigue level between the left and right parts.

[0038] According to the first embodiment described above, the terminal device 100 (an example of an image processing device) acquires an image capturing a part or all of a user's body, identifies the three-dimensional positions of the user's joints based on the image, and identifies the user's skeleton based on the anatomical skeleton data and the three-dimensional positions of the joints. The terminal device 100 identifies the three-dimensional positions of each of the user's muscles based on the user's joints and skeleton.

[0039] On the other hand, the terminal device 100 detects hemoglobin in the muscle during the systole when blood vessels contract and during the diastole when blood vessels expand, based on multiple images (time-series frames) of different time phases, and calculates the amount of hemoglobin in each period. The terminal device 100 calculates the difference between the amount of hemoglobin in the muscle during the vasoconstriction period and the amount of hemoglobin in the muscle during the vasodiastole, and calculates the blood flow rate in the muscle based on the difference in hemoglobin amount.

[0040] Furthermore, the terminal device 100 calculates the fatigue level for each muscle based on the blood flow rate of each muscle and outputs information corresponding to the fatigue level. This allows the user to take action such as resting if the fatigue level is high, thereby preventing injuries such as muscle tears caused by excessive exercise.

[0041] (Modification of the first embodiment) Modifications of the first embodiment will be described below. For example, when a user holds the camera 111 of the terminal device 100 against a muscle for which blood flow volume is to be calculated and captures the image, the terminal device 100 may perform the processes of S108 to S112 (detection of hemoglobin to calculation of blood flow volume) only for the range captured by the camera 111. Furthermore, the terminal device 100 may automatically identify the positions of the joints, skeleton, and / or muscles from the image captured by the camera 111, or, for example, when a user specifies a desired position on the image, the terminal device 100 may identify the positions of the joints, skeleton, and / or muscles only in the area specified by the user. In this case, the user does not need to move their body to identify the positions of the joints, skeleton, and / or muscles.

[0042] The terminal device 100 may also calculate the degree of muscle bleeding as a muscle-related index based on the distribution of hemoglobin in the muscle. For example, when a muscle is torn and bleeding occurs, the shape of the hemoglobin distribution is no longer tubular along the blood vessels. Furthermore, when a hematoma (blood clot) forms in the muscle due to an external factor, the amount of hemoglobin does not change, and the hemoglobin distribution becomes clot-shaped rather than tubular. Therefore, when the hemoglobin distribution shape is not tubular along the blood vessels, the terminal device 100 can estimate that bleeding is due to a muscle tear or a hematoma has occurred due to an external factor, and therefore may calculate the degree of bleeding to be higher than when the hemoglobin distribution shape is tubular along the blood vessels.

[0043] Furthermore, the terminal device 100 may calculate the blood flow rate of the body surface in addition to the blood flow rate of the muscles, and calculate the inflammation level of the body surface based on the blood flow rate of the body surface. For example, if a person is bedridden and stays in the same position for a long time, the blood flow near the body surface may stagnate, causing inflammation and ultimately leading to skin necrosis. Therefore, the terminal device 100 may output information indicating that the lower the blood flow rate of the body surface, the higher the inflammation level of the body surface, and the higher the blood flow rate of the body surface, the lower the inflammation level of the body surface.

[0044] (Second embodiment) The second embodiment will be described below. The second embodiment differs from the first embodiment described above in that the blood flow rate, which is the basis for indicators such as fatigue level and bleeding level, is corrected based on at least one of the user's attributes, vital signs, and medical test results. The following description will focus on the differences from the first embodiment, and will omit a description of the points in common with the first embodiment. In the description of the second embodiment, the same parts as in the first embodiment will be described with the same reference numerals.

[0045] FIG. 5 is a flowchart showing the flow of a series of processes performed by the processing circuit 120 according to the second embodiment.

[0046] First, the user captures an image of part or all of his or her body using the terminal device 100 (step S200). In response, the acquisition function 121 acquires a plurality of images capturing part or all of the user's body from the camera 111 (multi-lens camera).

[0047] Next, the identification function 122 identifies the three-dimensional positions of the user's joints based on the multiple images (step S202).

[0048] Next, the identification function 122 identifies the user's skeleton based on the anatomical skeleton data stored in the memory 115 and the three-dimensional positions of the joints (step S204).

[0049] Next, based on the joints and skeleton whose three-dimensional positions have been identified, the identification function 122 identifies where and how each muscle is located within the area of ​​the body shown in the image, i.e., the three-dimensional position of each muscle (step S206).

[0050] On the other hand, the calculation function 123 detects hemoglobin in the muscle during the systole when blood vessels contract and during the diastole when blood vessels expand based on multiple images (time-series frames) of different time phases (step S208), and calculates the amount of hemoglobin during each period (step S210).

[0051] Next, the calculation function 123 calculates the difference between the amount of hemoglobin in the muscle during vasoconstriction and the amount of hemoglobin in the muscle during vasodiastole, and calculates the blood flow rate in the muscle based on the difference in hemoglobin amount (step S212).

[0052] Next, the calculation function 123 determines whether the user has input at least one of the user's attributes, vital signs, and medical test results to the input interface 113 (step S214).

[0053] Examples of user attributes include age and gender. Examples of vital signs include body temperature, pulse rate, blood pressure, and respiration. Examples of medical test results include cholesterol levels, blood sugar levels, blood cell counts, and protein levels. Some or all of the vital signs may be included in the medical test results, or conversely, some or all of the medical test results may be included in the vital signs. Information such as the user's attributes, vital signs, and medical test results may be input to the terminal device 100 from an external device via the communication interface 112. Furthermore, if the camera 111 includes a thermal camera, the user's body temperature may be obtained from an image captured by the camera 111.

[0054] When at least one of the user's attributes, vital signs, and medical test results is input to the input interface 113, the calculation function 123 corrects the muscle blood flow using the input information (step S216). For example, the calculation function 123 may correct the blood flow to be smaller the older the user is (the more elderly the user is), or the less favorable the vital signs or medical test results are, the smaller the blood flow.

[0055] If at least one of the user's attributes, vital signs, and medical test results is not input to the input interface 113, the calculation function 123 omits the process of S216.

[0056] Next, the calculation function 123 calculates muscle-related indices for each muscle based on the blood flow volume of each muscle (step S218). The muscle-related indices are the muscle fatigue level and inflammation level, as described above.

[0057] Next, the output control function 124 outputs the calculation result of the calculation function 123 (step S220). For example, the output control function 124 may cause information according to the fatigue level to be displayed on the display 113a of the output interface 114. This ends the processing of this flowchart.

[0058] According to the second embodiment described above, the terminal device 100 (an example of an image processing device) acquires an image of a part or all of a user's body, identifies the three-dimensional positions of the user's joints based on the image, and identifies the user's skeleton based on the anatomical skeleton data and the three-dimensional positions of the joints. The terminal device 100 identifies the three-dimensional positions of each of the user's muscles based on the user's joints and skeleton.

[0059] On the other hand, the terminal device 100 detects hemoglobin in the muscle during the systole when blood vessels contract and during the diastole when blood vessels expand, based on multiple images (time-series frames) of different time phases, and calculates the amount of hemoglobin in each period. The terminal device 100 calculates the difference between the amount of hemoglobin in the muscle during the vasoconstriction period and the amount of hemoglobin in the muscle during the vasodiastole, and calculates the blood flow rate in the muscle based on the difference in hemoglobin amount.

[0060] Furthermore, when at least one of the user's attributes, vital signs, and medical test results is input to the input interface 113, the terminal device 100 uses this information to correct the muscle blood flow rate.

[0061] The terminal device 100 then calculates the fatigue level for each muscle based on the blood flow rate of each muscle and outputs information corresponding to the fatigue level. As a result, similar to the first embodiment, the user can take action such as resting if the fatigue level is high, thereby preventing injuries such as muscle strains caused by excessive exercise.

[0062] (Third embodiment) The third embodiment will be described below. In the first or second embodiment described above, the processing is described as being completed by the terminal device 100 alone. In contrast, the third embodiment differs from the first and second embodiments described above in that part of the processing executed by the terminal device 100 is executed by the server 200, which is one of the external devices. The following description will focus on the differences from the first and second embodiments, and will omit a description of the points in common with the first and second embodiments. In the description of the third embodiment, parts that are the same as those in the first and second embodiments will be described with the same reference numerals.

[0063] 6 is a diagram illustrating an example of an image processing system including a terminal device 100 and a server 200 according to the third embodiment. The server 200 is, for example, a so-called application server that causes the terminal device 100 to execute an application program. For example, the server 200 communicates with the terminal device 100 on which the application program is running via the network NW, acquires from the terminal device 100 an image of the user's own body captured using the terminal device 100, and calculates muscle blood flow and indexes such as fatigue level and inflammation level on behalf of the terminal device 100. In this case, the server 200 is another example of an "image processing device."

[0064] 7 is a sequence diagram showing the flow of processing in the image processing system in the third embodiment. First, the terminal device 100 waits until the user captures an image of part or all of his or her body (step S300), and when the image capture is complete, transmits the image of the user's body to the server 200 (step S302).

[0065] When the server 200 receives the image from the terminal device 100, it identifies the three-dimensional position of the muscle, calculates the blood flow rate of the muscle, and calculates indices such as the degree of fatigue and the degree of inflammation based on the blood flow rate of the muscle (step S304).

[0066] Then, the server 200 transmits the calculation results of the indices such as the muscle blood flow rate, fatigue level, and inflammation level to the terminal device 100 (step S306).

[0067] When the terminal device 100 receives the calculation result from the server 200, it displays it on the display 113a of the output interface 114 (step S308), which ends the processing of this sequence.

[0068] Although several embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, as well as within the scope of the invention and its equivalents as defined in the claims. [Explanation of symbols]

[0069] 100... terminal device (image processing device), 111... camera, 112... communication interface, 113... input interface, 114... output interface, 114a... display, 115... memory, 120... processing circuit, 121... acquisition function, 122... identification function, 123... calculation function, 124... output control function

Claims

1. an acquisition unit that acquires an optical image of the subject; an identification unit that identifies a position of a muscle in the subject based on the optical image; a calculation unit that calculates the blood flow rate of the muscle whose position has been identified based on the optical image; a display control unit that displays the muscle blood flow on a display unit, The calculation unit calculating the amount of hemoglobin in the muscle during a systole when the blood vessels of the subject contract and during a diastole when the blood vessels of the subject expand, based on the optical images having different time phases and obtained using electromagnetic waves of a predetermined wavelength; calculating a blood flow volume in the muscle based on a difference between the amount of hemoglobin in the muscle during the systole and the amount of hemoglobin in the muscle during the diastole; Image processing device.

2. The identification unit Identifying a position of a joint in the subject based on the plurality of optical images; identifying a skeleton of the subject based on the joints; Identifying the location of the muscles based on the joints and skeleton. The image processing device according to claim 1 .

3. The calculation unit further calculating a distribution of hemoglobin in the muscle during each of the systole and the diastole based on the plurality of optical images; calculating a blood vessel size of the muscle based on the hemoglobin distribution in the muscle during the systole and the hemoglobin distribution in the muscle during the diastole; calculating the blood flow volume of the muscle based on the difference between the thickness of the blood vessel in the systole and the thickness of the blood vessel in the diastole; The image processing device according to claim 1 .

4. The calculation unit further calculates an index related to the muscle based on the blood flow amount. The image processing device according to claim 1 .

5. The calculation unit Calculating the muscle fatigue level as the index, The smaller the blood flow rate, the greater the fatigue level. The greater the blood flow rate, the smaller the fatigue level. The image processing device according to claim 4 .

6. the display control unit causes the display unit to display information according to the fatigue level. The image processing device according to claim 5 .

7. the display control unit causes the display unit to display the time or number of days for which rest is required to recover from the fatigue level. The image processing device according to claim 6 .

8. the display control unit causes the display unit to display a recommended recuperation method for recovering from the fatigue level.

8. The image processing device according to claim 6 or 7.

9. the display control unit causes the display unit to display a comparison result between the fatigue level calculated based on an optical image of the subject taken before the prescribed medical treatment and the fatigue level calculated based on an optical image of the subject taken after the prescribed medical treatment. The image processing device according to any one of claims 6 to 8.

10. the identifying unit identifies the position of each of a plurality of muscles of the subject based on the optical image; the calculation unit calculates the blood flow rate for each muscle based on the optical image, and calculates a fatigue level for each muscle as the index based on the blood flow rate for each muscle; the display control unit causes the display unit to display a comparison result between the fatigue level of the left side region of the subject and the fatigue level of the right side region of the subject. The image processing device according to any one of claims 6 to 9.

11. The calculation unit corrects the blood flow rate based on a statistical value of the blood flow rate for each muscle. The image processing device according to any one of claims 4 to 10.

12. The acquisition unit further acquires at least one of attributes, vital signs, and medical test results of the subject; the calculation unit corrects the blood flow rate based on at least one of the attributes, the vital signs, and the medical test results. The image processing device according to any one of claims 4 to 11.

13. The calculation unit further calculating an index related to the muscle based on the blood flow rate; Calculating the index of bleeding in the muscle based on the distribution of hemoglobin in the muscle. The image processing device according to claim 1 .

14. The calculation unit further calculating a blood flow rate on the body surface of the subject based on the optical image; Calculating the inflammation level of the body surface based on the blood flow volume of the body surface. The image processing device according to any one of claims 4 to 13.

15. an acquisition unit that acquires a plurality of optical images of an object, the optical images being at different time phases; an identification unit that identifies muscle positions in the subject based on the plurality of optical images; a first calculation unit that calculates a blood flow amount in the muscle whose position has been identified, based on the plurality of optical images, during a systole when the blood vessels of the subject contract and during a diastole when the blood vessels of the subject expand; a second calculation unit that calculates a fatigue level of the muscle based on a difference between the blood flow rate of the muscle in the systole and the blood flow rate of the muscle in the diastole; An image processing device comprising:

16. Acquire an optical image of the subject, Identifying a muscle location in the subject based on the optical image; Calculating the blood flow rate of the muscle whose location has been identified based on the optical image; Displaying the blood flow rate of the muscle on a display unit; calculating the amount of hemoglobin in the muscle during a systole when the blood vessels of the subject contract and during a diastole when the blood vessels of the subject expand, based on the optical images having different time phases and obtained using electromagnetic waves of a predetermined wavelength; calculating a blood flow volume in the muscle based on a difference between the amount of hemoglobin in the muscle during the systole and the amount of hemoglobin in the muscle during the diastole; Image processing methods.

17. acquiring a plurality of optical images of the subject, the optical images being at different time phases from one another; Identifying a location of a muscle in the subject based on the plurality of optical images; calculating a blood flow rate of the muscle whose position has been identified during a systole when the blood vessels of the subject contract and during a diastole when the blood vessels of the subject expand based on the plurality of optical images; calculating a fatigue level of the muscle based on a difference between the blood flow rate of the muscle during the contraction period and the blood flow rate of the muscle during the diastole period; Image processing methods.

18. On the computer, acquiring an optical image of the subject; Identifying a muscle location in the subject based on the optical image; calculating a blood flow rate in the muscle based on the optical image; displaying the blood flow rate of the muscle on a display unit; calculating the amount of hemoglobin in the muscle during a systole when blood vessels of the subject contract and during a diastole when blood vessels of the subject expand, based on the optical images having different time phases and obtained using electromagnetic waves of a predetermined wavelength; calculating a blood flow volume in the muscle based on a difference between the amount of hemoglobin in the muscle during the systole and the amount of hemoglobin in the muscle during the diastole; A program to execute.

19. On the computer, Acquiring a plurality of optical images of the subject, the optical images being at different time phases; Identifying a location of a muscle in the subject based on the plurality of optical images; calculating, based on the plurality of optical images, a blood flow amount in the muscle whose position has been identified during a systole when the blood vessels of the subject contract and during a diastole when the blood vessels of the subject expand; calculating a fatigue level of the muscle based on a difference between the blood flow rate of the muscle during the systole and the blood flow rate of the muscle during the diastole; A program to execute.

Images