Boot method

Abstract

A camera (400) for capturing a person's walking posture comprises a detection unit (421) for detecting the direction of the camera and a display unit (422) for displaying a guide line indicating the position of the person walking on a screen display unit. The display unit (422) displays different guide lines depending on the direction of the camera detected by the detection unit (421).

Description

Technical Field

[0001] The present invention relates to a guiding method, a photographing device, and a recording medium. Background Art

[0002] It is known to perform walking analysis of a user based on data.

[0003] For example, Patent Document 1 describes a walking analysis device that analyzes a person's walking. Patent Document 1 describes a walking analysis device comprising: a data acquisition unit that acquires two types of image data from a depth sensor; a skeletal information generation unit that generates skeletal information based on the image data acquired by the data acquisition unit; a correction processing unit that corrects the skeletal information generated by the skeletal information generation unit; and an analysis processing unit that uses the corrected skeletal information to analyze the user's walking.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: International Publication No. 2017 / 170832 Summary of the Invention

[0007] Problems to be solved by the invention

[0008] The technology described in Patent Document 1 requires the use of a depth sensor (3D sensor), but there is a demand for performing walking analysis based on image data acquired by a camera without using a 3D sensor such as a depth sensor.

[0009] In order to improve the accuracy of walking analysis based on image data, it is preferable to make the shooting conditions of the image data as consistent as possible. However, the image data to be analyzed are acquired at different timings. Therefore, it is difficult to make the shooting conditions consistent.

[0010] This leads to a problem that it is difficult to make the shooting conditions uniform when acquiring image data.

[0011] Therefore, an object of the present invention is to provide a guidance method, an imaging device, and a recording medium that solve the problem that it is difficult to make imaging conditions consistent when acquiring image data.

[0012] Technical solutions to problems

[0013] To achieve this object, a guidance method as one embodiment of the present disclosure adopts the following structure:

[0014] The camera that captures the person's walking posture detects the direction of the camera,

[0015] The camera displays a guide line indicating the position where the person is walking on the screen display unit.

[0016] When the guide line is displayed on the screen display unit, the imaging device displays different guide lines according to the detected direction of the imaging device.

[0017] In addition, a camera device as another embodiment of the present disclosure has the following structure:

[0018] The camera captures the person's walking posture, wherein:

[0019] The shooting device comprises:

[0020] a detection unit for detecting a direction of the camera; and

[0021] The display unit displays a guide line indicating a position where the person is walking on the screen display unit,

[0022] The display unit displays different guide lines according to the direction of the imaging device detected by the detection unit.

[0023] In addition, as another aspect of the present disclosure, a recording medium,

[0024] A program is recorded, which enables a camera for photographing a person walking to implement the following components:

[0025] a detection unit for detecting a direction of the camera; and

[0026] The display unit displays a guide line indicating a position where the person is walking on the screen display unit,

[0027] The display unit displays different guide lines according to the direction of the imaging device detected by the detection unit.

[0028] Effects of the Invention

[0029] According to the above-described configurations, it is possible to make the shooting conditions uniform when acquiring image data. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is a diagram showing a configuration example of a walking posture measurement system in the first embodiment of the present disclosure.

[0031] Figure 2 This figure shows an example of capturing a walking posture in the front-back direction.

[0032] Figure 3 This figure shows an example of capturing a person walking in the left-right direction.

[0033] Figure 4 Yes Figure 1A block diagram of an example of the structure of a smartphone is shown.

[0034] Figure 5 Yes Figure 4 A block diagram of a configuration example of a shooting assistance unit is shown.

[0035] Figure 6 1 and 2 are diagrams showing an example of shooting assistance display when the camera is held horizontally.

[0036] Figure 7 3 is a diagram showing an example of shooting assistance display when the camera is held vertically.

[0037] Figure 8 Yes Figure 5 FIG. 1 is a diagram showing an operation example of the angle adjustment information output unit.

[0038] Figure 9 Yes Figure 1 A block diagram of a configuration example of a walking posture measurement device shown in FIG.

[0039] Figure 10 Yes Figure 9 An example of skeleton information is shown in FIG.

[0040] Figure 11 Yes Figure 9 An example of measurement result information is shown in FIG.

[0041] Figure 12 It is a diagram for explaining the processing of the actual measurement value calculation unit.

[0042] Figure 13 It is a diagram for explaining the processing of the actual measurement value calculation unit.

[0043] Figure 14 It is a diagram for explaining the processing of the actual measurement value calculation unit.

[0044] Figure 15 It is a diagram for explaining the processing of the actual measurement value calculation unit.

[0045] Figure 16 It is a diagram for explaining the processing of the actual measurement value calculation unit.

[0046] Figure 17 It is a diagram for explaining the processing of the actual measurement value calculation unit.

[0047] Figure 18 It is a diagram for explaining the processing of the actual measurement value calculation unit.

[0048] Figure 19 It is a diagram for explaining the processing of the actual measurement value calculation unit.

[0049] Figure 20 This is a flowchart showing an example of the operation of the photography assistance unit in a smartphone.

[0050] Figure 21 This is a flowchart showing an example of the operation of the walking posture measurement device.

[0051] Figure 22 This is a flowchart showing an example of a process for calculating actual measurement value information.

[0052] Figure 23 This is a diagram for explaining a tracking example of the walking posture measurement device in the second embodiment of the present disclosure.

[0053] Figure 24 This is a block diagram showing a configuration example of a walking posture measurement device in a second embodiment of the present disclosure.

[0054] Figure 25 Yes Figure 24 A diagram showing an example of the structure of a tracking unit is shown.

[0055] Figure 26 Yes Figure 25 FIG. 1 is a diagram showing an example of a graphic generated by the included graphic generating unit.

[0056] Figure 27 Yes Figure 25 FIG. 1 is a diagram showing an example of a graphic generated by the included graphic generating unit.

[0057] Figure 28 This is a flowchart showing an example of the operation of the tracking unit.

[0058] Figure 29 This is a diagram showing an example of the hardware configuration of the imaging device in the third embodiment of the present disclosure.

[0059] Figure 30 This is a block diagram showing a structural example of an imaging device.

[0060] Figure 31 This is a block diagram showing a configuration example of an information processing device in a fourth embodiment of the present disclosure.

[0061] Figure 32 This is a block diagram showing a configuration example of an information processing device in the fifth embodiment of the present disclosure.

[0062] Figure 33 This is a block diagram showing a configuration example of a tracking device in a sixth embodiment of the present disclosure. DETAILED DESCRIPTION

[0063] [First embodiment]

[0064] Regarding the first embodiment of the present disclosure, refer to Figures 1 to 22 Provide explanation. Figure 11 is a diagram showing a configuration example of the walking posture measurement system 100 . Figure 2 This figure shows an example of capturing a walking posture in the front-back direction. Figure 3 This figure shows an example of capturing a person walking in the left-right direction. Figure 4 It is a block diagram showing a configuration example of the smartphone 200 . Figure 5 2 is a block diagram showing a configuration example of the imaging support unit 212 . Figure 6 1 and 2 are diagrams showing an example of shooting assistance display when the camera is held horizontally. Figure 7 1 and 2 are diagrams showing an example of shooting assistance display when the camera is held vertically. Figure 8 This is a diagram showing an example of the operation of the angle adjustment information output unit 2124. Figure 9 3 is a block diagram showing a configuration example of the walking posture measurement device 300 . Figure 10 This is a diagram showing an example of skeleton information 334. Figure 11 This is a diagram showing an example of the measurement result information 336. Figure 12 It is a diagram for explaining the processing of the actual value calculation unit 343. Figures 13 to 19 It is a diagram for explaining the processing of the actual value calculation unit 343. Figure 20 This is a flowchart showing an example of the operation of the imaging assistance unit 212 in the smartphone 200 . Figure 21 This is a flowchart showing an example of the operation of the walking posture measurement device 300 . Figure 22 This is a flowchart showing an example of a process of calculating the actual measurement value information 335 .

[0065] In the first embodiment of the present disclosure, a walking posture measurement system 100 for measuring a person's walking posture based on a dynamic image obtained using a camera such as a smartphone 200 is described. In the walking posture measurement system 100, a dynamic image (i.e., a plurality of image data) representing a person's walking posture in the front-back direction of the image and a dynamic image representing a person's walking posture in the left-right direction of the image are captured using the smartphone 200. Then, based on the plurality of image data obtained as dynamic images through the capture, the walking posture measurement system 100 measures walking posture such as stride length, walking speed, and straightness corresponding to shaking of the head during walking. In addition, as described later, the walking posture measurement system 100 includes a structure for making the shooting conditions as consistent as possible when acquiring image data using the smartphone 200, a structure for measuring actual measurement values ​​based on the image data, and the like.

[0066] Figure 1 FIG. 1 shows a configuration example of the walking posture measurement system 100. Figure 1 The walking posture measurement system 100 includes, for example, a smartphone 200 and a walking posture measurement device 300. Figure 1As shown, the smartphone 200 and the walking posture measurement device 300 are connected to each other via, for example, wireless or wired communication.

[0067] The smartphone 200 functions as a camera for capturing images of people walking. The smartphone 200 may be a smartphone having common functions such as a camera, a touch panel 201 for displaying a screen, and various sensors such as a GPS sensor and an acceleration sensor.

[0068] In the case of this embodiment, if Figure 2 As shown, while holding the smartphone 200 vertically (in a vertical holding position), the photographer captures a person walking in the front and back directions, such as from the depth of the screen to the front. In other words, while holding the smartphone 200 vertically with its short side horizontal to the ground, the photographer captures a person walking in the front and back directions. Figure 3 As shown, while holding smartphone 200 horizontally (horizontally held), the photographer captures a person walking in the left-right direction, such as from the left side of the screen to the right side. In other words, while holding smartphone 200 horizontally with its long side parallel to the ground, the photographer captures a person walking in the front-back direction. Thus, in walking posture measurement system 100, the photographer captures dynamic images (a plurality of image data) of a type corresponding to the orientation of smartphone 200. Furthermore, the person walking in the front-back direction and the person walking in the left-right direction can be captured twice, for example, using a single smartphone 200, or simultaneously using two smartphones 200.

[0069] Figure 4 The characteristic configuration example of the smartphone 200 in this embodiment is shown. Figure 4 The smartphone 200 has a general structure of a smartphone such as an acceleration sensor and a gyro sensor for detecting the direction (vertical or horizontal) of the smartphone 200, and also has a motion image capturing unit 210 for measurement. Figure 4 As shown, the measurement motion capturing section 210 includes an image data capturing section 211 and a capturing assistance section 212 .

[0070] For example, the smartphone 200 includes a CPU (Central Processing Unit) and a storage device. The smartphone 200 implements the aforementioned processing unit by, for example, executing a program stored in the storage device through the computing device.

[0071] The measuring action shooting unit 210 shoots the walking posture of a person, which is the measuring action in this embodiment, based on the photographer's operation of the smartphone 200. In addition, the measuring action shooting unit 210 includes a function for assisting shooting so that shooting can be performed under the same conditions as much as possible. For example, the measuring action shooting unit 210 is a shooting application that has a camera operation function for shooting the walking posture of a person, and has a guidance function for making the shooting conditions such as the walking direction, angle, and size reflected on the screen as consistent as possible when shooting the walking posture of a person. As described above, the measuring action shooting unit 210 includes an image data shooting unit 211 and a shooting assistance unit 212.

[0072] The image data capturing unit 211 captures a person using the camera of the smartphone 200 to obtain a moving image (a plurality of image data). The image data capturing unit 211 can also associate information indicating the date and time the image data was acquired, information acquired by the later-described capturing assistance unit 212, and the like with the moving image (image data) captured by the image data capturing unit 211.

[0073] The imaging assistance unit 212 assists the image data imaging unit 211 in making imaging conditions as consistent as possible when the image data imaging unit 211 acquires image data. Figure 5 An example of the configuration included in the imaging support unit 212 is shown. Figure 5 The shooting assistance unit 212 includes, for example, a guide line display unit 2121 , an angle information display unit 2122 , a height information input unit 2123 , and an angle adjustment information output unit 2124 .

[0074] The guide line display unit 2121 displays guide lines 2011 on the touch panel 201, which serve as a reference for the position of the person's feet, etc., when photographing the person. By making the person walk as closely as possible along the guide lines 2011 displayed on the touch panel 201, the direction, angle, and size of the person's walking posture can be made consistent when photographing the person's walking posture. Alternatively, a marker can be placed in the real world to align with the position indicated by the guide lines 2011, and the person can be made to walk using the marker.

[0075] In the present embodiment, the guide line display unit 2121 displays different guide lines 2011 on the touch panel 201 depending on whether the smartphone 200 is held in portrait or landscape orientation. For example, if the smartphone 200 is judged to be held in landscape orientation based on information obtained from an acceleration sensor, etc., the guide line display unit 2121 displays different guide lines 2011 on the touch panel 201. Figure 6As shown, the guide line display unit 2121 displays the guide line 2011 for photographing a person walking in the left-right direction of the screen on the touch panel 201. In other words, the guide line display unit 2121 displays the guide line 2011 for guiding a person walking in the left-right direction on the touch panel 201. Figure 6 As can be seen, the touch panel 201 displays the area being photographed by the camera of the smartphone 200, i.e., the photographing area, and also displays a guide line 2011. In addition, when it is determined based on information obtained from the acceleration sensor etc. that the smartphone 200 is held vertically, as shown in FIG. Figure 7 As shown, the guide line display unit 2121 displays a guide line 2011 for photographing a person walking in the inward and outward directions of the screen on the touch panel 201. In other words, the guide line display unit 2121 displays a guide line 2011 for guiding a person walking in the inward and outward directions on the touch panel 201.

[0076] In addition, the position where the guide line display unit 2121 displays the guide line 2011 is predetermined. For example, when the smartphone 200 is held horizontally, Figure 6 As shown, the guide line display unit 2121 displays the guide line 2011 below the center of the shooting area (for example, in the middle of the lower half of the area). Figure 7 As shown in FIG, the guide line display unit 2121 displays the guide line 2011 at the center of the imaging area. The position where the guide line display unit 2121 displays the guide line 2011 may be a position other than the above-mentioned example.

[0077] The angle information display unit 2122 obtains information indicating the angle of the smartphone 200, such as the left-right tilt and the inside-out tilt, from the smartphone 200's accelerometer, gyroscope, and other sensors. The angle information display unit 2122 then displays the obtained information on the touch panel 201 as angle information 2012 indicating the angle of the smartphone 200. When photographing a person walking, it is preferable to shoot with the smartphone 200 tilted as little as possible. By displaying the angle information 2012 on the touch panel 201 using the angle information display unit 2122, the photographer can correct the angle of the smartphone 200 when photographing the person walking, allowing the desired photographic position of the smartphone 200 without tilting the smartphone 200.

[0078] In the present embodiment, the angle information display unit 2122 displays information indicating the tilt in the left-right direction and information indicating the tilt in the inside-out direction on the touch panel 201. In other words, the angle information display unit 2122 displays information indicating the tilt of the smartphone 200 in the horizontal and vertical directions. For example, Figure 6 、 Figure 7 As shown, angle information display unit 2122 displays angle information 2012 at a predetermined position on touch panel 201. Furthermore, the display of angle information 2012 by angle information display unit 2122 may be the same (display corresponding to the tilt) when smartphone 200 is held in portrait orientation and when smartphone 200 is held in landscape orientation.

[0079] The height information input unit 2123 receives input of height information indicating the height h of the smartphone 200 from the ground from a person, and displays the height display unit 2013 indicating the input height h on the touch panel 201. The height h input by the height information input unit 2123 can be used when the walking posture measurement device 300 calculates the measured value W.

[0080] like Figure 6 、 Figure 7 As shown, the altitude information input unit 2123 displays the altitude display unit 2013 at a predetermined position on the touch panel 201. The altitude information input unit 2123 then receives input of information indicating the altitude h from the person operating the smartphone 200, for example, by touching the altitude display unit 2013. The altitude information input unit 2123 displays the received information indicating the altitude h on the altitude display unit 2013. The display of the altitude display unit 2013 by the altitude information input unit 2123 can be the same whether the smartphone 200 is held in portrait orientation or landscape orientation, with the display position changing.

[0081] The angle adjustment information output unit 2124 outputs information for adjusting the inclination of the smartphone 200. For example, the angle adjustment information output unit 2124 outputs information corresponding to the inclination of the smartphone 200 that varies depending on how the smartphone 200 is tilted. Specifically, for example, the angle adjustment information output unit 2124 outputs a sound adjusted according to the inclination of the smartphone 200 as information for adjusting the inclination of the smartphone 200.

[0082] Figure 8 An example of the processing of the angle adjustment information output unit 2124 is shown. Figure 8 For example, the angle adjustment information output unit 2124 outputs a sound whose length is adjusted according to the left-right tilt of the smartphone 200, and outputs a sound whose pitch is adjusted according to the inside-out tilt of the smartphone 200. In this way, the angle adjustment information output unit 2124 performs different adjustments according to the tilt of the smartphone 200. For example, referring to Figure 8The angle adjustment information output unit 2124 adjusts the length of the sound so that the more the smartphone 200 is tilted to the left, the shorter the length of the sound. Furthermore, the angle adjustment information output unit 2124 adjusts the length of the sound so that the more the smartphone 200 is tilted to the right, the longer the length of the sound. Furthermore, the angle adjustment information output unit 2124 adjusts the pitch so that the pitch decreases as the smartphone 200 is tilted toward the front (e.g., toward the photographer). Furthermore, the angle adjustment information output unit 2124 adjusts the pitch so that the pitch increases as the smartphone 200 is tilted toward the rear (e.g., toward the side opposite to the photographer).

[0083] According to the above configuration, when smartphone 200 is tilted in any direction, angle adjustment information output unit 2124 adjusts at least one of the length and pitch of the sound according to the tilt, thereby outputting two sounds. On the other hand, when smartphone 200 is not tilted, the length and pitch of the sound are not adjusted, so angle adjustment information output unit 2124 outputs a single sound. By configuring angle adjustment information output unit 2124 to output the adjusted sound, for example, even when the photographer has difficulty viewing touch panel 201, the angle of smartphone 200 can be easily adjusted.

[0084] Furthermore, the information output by the angle adjustment information output unit 2124 for adjusting the tilt of the smartphone 200 is not necessarily limited to being in the form of sound. For example, the angle adjustment information output unit 2124 may illuminate a light or vibrate the smartphone 200 instead of sound. For example, the angle adjustment information output unit 2124 may be configured to illuminate a light or vibrate the smartphone 200 when the smartphone is nearing the correct angle, i.e., when the smartphone is not tilted. Furthermore, the angle adjustment information output unit 2124 may combine the above processes, such as outputting sound based on the left-right tilt and illuminating the light based on the front-back tilt, or outputting both sound and illuminating the light based on the left-right tilt.

[0085] The above is an example of the structure of the smartphone 200 that is characteristic of this embodiment.

[0086] The walking posture measurement device 300 is a server device that measures walking posture such as stride length, walking speed, and straightness based on moving images (ie, a plurality of image data) captured by the smartphone 200 . Figure 9 FIG. 3 shows a configuration example of the walking posture measurement device 300. Figure 9 The walking posture measurement device 300 includes, as main components, a screen display unit 310 , a communication I / F unit 320 , a storage unit 330 , and a calculation processing unit 340 .

[0087] Screen display unit 310 is comprised of a screen display device such as a touch panel or liquid crystal display. Based on instructions from processing unit 340, screen display unit 310 can display image information 333, skeletal information 334, measurement result information 336, and a display in which the positions of bones indicated by skeletal information 334 are superimposed on the image data included in image information 333.

[0088] Communication I / F unit 320 is composed of a data communication circuit and performs data communication with an external device or smartphone 200 connected via a communication line.

[0089] The storage unit 330 is a storage device such as a hard disk or memory. It stores processing information and programs 337 required for various processes in the calculation processing unit 340. Programs 337 are read into the calculation processing unit 340 and executed to implement the various processing units. Programs 337 are pre-loaded from external devices or recording media via data input / output functions such as the communication I / F unit 320 and stored in the storage unit 330. Examples of the main information stored in the storage unit 330 include a learned model 331, camera setting information 332, image information 333, skeleton information 334, measured value information 335, and measurement result information 336.

[0090] The learned model 331 is a learned model used by the skeleton recognition unit 342 when performing skeleton recognition. The learned model 331 is generated in advance, for example, by performing machine learning using training data such as image data in which skeleton coordinates are input, on an external device, etc., and is obtained from the external device, etc., via the communication I / F unit 320, etc., and stored in the storage unit 330.

[0091] Furthermore, the learned model 331 may be updated through a relearning process using additional training data or the like.

[0092] The camera setting information 332 includes information indicating parameters of the camera included in the smartphone 200 used when the smartphone 200 photographs a person walking. The camera setting information 332 includes, for example, information indicating the vertical field angle θ and the horizontal field angle ψ of the camera.

[0093] The camera setting information 332 is acquired in advance from the smartphone 200 or the like via the communication I / F unit 320 or the like and stored in the storage unit 330. The camera setting information 332 may also be acquired from the smartphone 200 along with the image data when the image data is acquired from the smartphone 200 and stored in the storage unit 330.

[0094] Image information 333 includes image data (moving images) captured by the camera of smartphone 200. For example, image information 333 associates image data, information indicating the date and time when the smartphone 200 captured the image data, and information indicating the height input by the height information input unit 2123 with each unit of the moving image. Furthermore, image information 333 associates moving images of a person walking left and right with those of a person walking front and back. As described later, measurement unit 344 measures the corresponding walking posture based on the moving images of a person walking left and right and the moving images of a person walking front and back.

[0095] The skeleton information 334 includes information indicating the coordinates of each part of the person recognized by the skeleton recognition unit 342 . Figure 10 An example of skeleton information 334 is shown. Figure 10 In the skeleton information 334, for example, for each person to be recognized, time and positional information of each part are associated. Time represents the time elapsed since the start of the moving image capture, the time when the moving image was captured, etc. Furthermore, the positional information of each part includes information indicating the coordinates of each part in the image data, such as the position of the pelvis.

[0096] Furthermore, the parts included in the position information of each part are parts corresponding to the learned model 331. For example, Figure 10 In the example, the pelvis, the center of the spine, ... are given. The position information of each part can include about 30 parts such as the right shoulder, ..., the left elbow, ..., the right knee, ... (parts other than the examples may also be included). The parts included in the position information of each part can also be Figure 10 Parts other than those shown in the examples.

[0097] The measured value information 335 includes information indicating the measured value W calculated by the measured value calculator 343. For example, the measured value W is associated with a reference line or identification information of the image data in the measured value information 335. The measured value information 335 may also include information indicating a stride length, etc. The processing performed by the measured value calculator 343 will be described in detail later.

[0098] The measurement result information 336 indicates the result of the walking posture measurement measured by the measurement unit 344 . Figure 11 An example of the measurement result information 336 is shown. Figure 11In the measurement result information 336, for each person being measured, for example, time, walking speed, stride length, and straightness are associated. Here, time refers to the time elapsed since the start of the dynamic image capture or the time when the dynamic image was captured. In addition, walking speed refers to the speed at which the person walks. In addition, stride length refers to the length between the toes (or heels) of the person's right and left feet when walking. In addition, straightness refers to the degree of shaking or trembling of the person's head or body when walking. In addition, the measurement result information 336 may also include information such as the distance indicating the time it takes to take a step.

[0099] Here, among the various pieces of information included in measurement result information 336, walking speed and stride length are information measured by actual value calculation unit 343 and measurement unit 344 based on a dynamic image (image data) of a person walking in the left-right direction. Furthermore, among the various pieces of information included in measurement result information 336, straightness is information measured by measurement unit 344 based on a dynamic image (image data) of a person walking in the front-back direction. For example, as described above, measurement result information 336 includes both information measured by measurement unit 344 based on a dynamic image of a person walking in the left-right direction and information measured by measurement unit 344 based on a dynamic image of a person walking in the front-back direction.

[0100] The processing unit 340 includes a microprocessor such as an MPU and its peripheral circuits. The processing unit 340 reads and executes a program 337 from the storage unit 330, thereby implementing various processing units by cooperating with the hardware and program 337. Examples of the main processing units implemented by the processing unit 340 include an image acquisition unit 341, a skeleton recognition unit 342, a measured value calculation unit 343, a measurement unit 344, and an output unit 345.

[0101] The image acquisition unit 341 acquires a moving image (a plurality of image data) acquired by the smartphone 200 from the smartphone 200 via the communication I / F unit 320. The image acquisition unit 341 then associates the acquired image data with information indicating, for example, the acquisition date and time of the image data and the altitude, and stores the associated image data in the storage unit 330 as image information 333.

[0102] In this embodiment, the image acquisition unit 341 acquires a moving image (image data) of a person walking in the left-right direction and a moving image (image data) of a person walking in the front-back direction from the smartphone 200 in a manner that associates or can associate the moving image. The image acquisition unit 341 then associates the two acquired moving images and stores them in the storage unit 330 as image information 333.

[0103] The skeleton recognition unit 342 uses the learned model 331 to identify the skeleton of the person being measured for walking posture in the image data. For example, the skeleton recognition unit 342 identifies various parts of the body, such as the upper spine, right shoulder, left shoulder, right elbow, left elbow, right wrist, left wrist, right hand, left hand, etc. The skeleton recognition unit 342 also calculates the coordinates of each identified part in the image data. The skeleton recognition unit 342 then associates the results of the identification / calculation with the identification information used to identify the person, thereby storing the information as skeleton information 334 for each person in the storage unit 330.

[0104] Furthermore, the parts recognized by the skeleton recognition unit 342 correspond to the learned model 331 (the training data used when learning the learned model 331 ). Therefore, the skeleton recognition unit 342 can also recognize parts other than those exemplified above based on the learned model 331 .

[0105] The measured value calculation unit 343 calculates the measured value W from one end of the screen to the other end of the reference line based on the height h of the camera of the smartphone 200 from the ground, the vertical field of view angle θ and the horizontal field of view angle ψ of the camera of the smartphone 200, and the ratio α of the reference line at an arbitrary height on the screen relative to half the screen. Furthermore, the measured value calculation unit 343 can use the calculated measured value W to calculate a person's stride, etc. The measured value calculation unit 343 then stores the calculated measured value W and other values ​​as measured value information 335 in the storage unit 330. Furthermore, the measured value calculation unit 343 can calculate the measured value W for each frame of a moving image (i.e., each piece of image data).

[0106] Generally, from image data that cannot obtain depth values ​​from a camera such as a smartphone 200, only the planar position on the screen can be obtained, and the actual length in the real world cannot be immediately determined. The measured value calculation unit 343 can calculate the actual length (i.e., the length in the real world), namely the measured value W, by using the various values ​​described above.

[0107] Figure 12 The mathematical formula used by the actual value calculation unit 343 to calculate the actual value W is shown in FIG. Figure 12 As shown, the measured value calculation unit 343 calculates the measured value W by solving the equation shown in Mathematical Formula 1. Specifically, the measured value calculation unit 343 calculates the measured value W based on the height h of the camera of the smartphone 200 from the ground, the parameters of the camera of the smartphone 200, and the ratio α of the reference line for calculating the measured value W relative to half the screen.

[0108] [Mathematical formula 1]

[0109]

[0110] Here, the measured value W represents, for example, the actual length of a baseline in the image data from one end of the screen to the other. Furthermore, h represents the height of the camera (smartphone 200) above the ground. The value of height h is input by the photographer when acquiring image data using the height information input unit 2123. Furthermore, θ represents the vertical angle of view, and ψ represents the horizontal angle of view. α represents the ratio of a baseline at any height on the screen to half the screen.

[0111] Below, refer to Figures 13 to 18 , mathematical formula 1 is explained in detail.

[0112] For example, Figure 13 As shown, a camera of a smartphone 200 located at a height h is used to shoot a person at an arbitrary height, that is, a reference line, on the screen. Figure 14 As shown in FIG, the position of the smartphone 200 at a height h is defined as the origin O. Furthermore, the point where the smartphone 200 touches the floor when it is vertically lowered toward the floor is defined as point G, and the position of the person is defined as point P. Furthermore, the angle formed by the line connecting the origin O and point P and the ground (or a line horizontal to the ground) is defined as angle

[0113] In the above case, if the distance from point G to point P is d, then Figure 15 As shown, Mathematical Formula 2 becomes Mathematical Formula 3.

[0114] [Mathematical formula 2]

[0115]

[0116] [Mathematical formula 3]

[0117]

[0118] In addition, if Figure 16 As shown, the angle If the ratio of the position of an arbitrary point P on the screen is defined as the vertical viewing angle θ, then Equation 4 is obtained.

[0119] [Formula 4]

[0120]

[0121] In addition, if you look from directly above Figure 14 The scene shown becomes Figure 17 As shown. Figure 17 In the example, one end of a line of height on an arbitrary screen passing through point P is set to point Q. In this case, Figure 18As shown, if the width between point P and point Q is set to width w, then Equation 5 becomes Equation 6. Furthermore, since Equation 3 and Equation 4 give Equation 7, substituting Equation 7 into Equation 6 yields Equation 8.

[0122] [Formula 5]

[0123]

[0124] [Formula 6]

[0125]

[0126] [Formula 7]

[0127]

[0128] [Formula 8]

[0129]

[0130] Therefore, since the total horizontal length of the screen at a distance d from the smartphone 200, namely the measured value W, is 2w, the above-mentioned mathematical formula 8 is multiplied by 2 to obtain the above-mentioned mathematical formula 1. As can be seen from this, the measured value W can be calculated by calculating mathematical formula 1.

[0131] In addition, the arbitrary height baseline on the screen that serves as the basis for calculating the ratio α can be determined, for example, based on the position of the person's feet in the image data. That is, the measured value calculation unit 343 can perform the process of calculating the ratio α after determining the baseline based on the position of the person's feet in the image data. For example, the measured value calculation unit 343 can use a line with a slope of 0 that passes through the average value of the Y coordinates of the left and right feet of the person, the Y coordinate of any one foot, etc. as the baseline (it can also be a line other than the example). In addition, the above-mentioned processing performed by the measured value calculation unit 343 can also be performed, for example, per frame (i.e., per each image data). In addition, the height of the baseline on the screen can also be predetermined based on the position of the guide line 2011 displayed by the guide line display unit 2121, etc.

[0132] In addition, the actual value calculation unit 343 can calculate the stride length, etc. based on the calculated actual value W. For example, the actual value calculation unit 343 can calculate the stride length, etc. based on the calculated actual value W. Figure 19 The stride length is calculated based on the image data of the person walking in the left-right direction, the calculated measured value W, and the resolution of the image data. Specifically, for example, when the resolution of the image data is set to 1920×1080, the measured value k per pixel on the baseline is k=W / 1920. Therefore, for example, Figure 19If the difference between the x-coordinates of the left and right feet (e.g., the toes identified by the skeleton recognition unit 342) in the image data shown is 100 pixels, the actual stride length can be calculated by the measured value calculation unit 343 by calculating 100×k. In this way, the measured value calculation unit 343 can calculate the stride length based on the calculated measured value W, the resolution, and the number of pixels.

[0133] For example, as described above, the measured value calculation unit 343 can calculate the measured value W of the length from one end of the screen to the other end of the reference line, or calculate the stride of the person based on the measured value W.

[0134] The measurement unit 344 measures the walking posture of the person using the calculation results of the actual value calculation unit 343 and the recognition results of the skeleton recognition unit 342. The measurement unit 344 stores the measurement results in the storage unit 330 as measurement result information 336.

[0135] As described above, the measuring unit 344 can perform measurements based on dynamic images of a person walking in the left-right direction and measurements based on dynamic images of a person walking in the inside-out direction. For example, the measuring unit 344 can calculate walking speed, distance, etc. based on dynamic images (image data) of a person walking in the left-right direction. For example, the measuring unit 344 can calculate the moving distance between frames of the part identified by the skeleton recognition unit 342, and calculate the walking speed based on the calculated moving distance and the time of each frame (image data). When calculating the walking speed, the measuring unit 344 can also use the calculation results of the actual measurement value calculation unit 343 such as stride length. In addition, the measuring unit 344 can calculate straightness, etc. based on dynamic images (image data) of a person walking in the inside-out direction. For example, the measuring unit 344 can calculate straightness based on the shaking of the coordinates of the head identified by the skeleton recognition unit 342.

[0136] Furthermore, the measuring unit 344 may be configured to perform measurements other than those exemplified above.

[0137] The output unit 345 can output the skeleton information 334, the measured value information 335, the measurement result information 336, and information obtained by superimposing the skeleton information 334 on the dynamic image included in the image information 333. For example, the output unit 345 outputs the above-mentioned information by displaying it on the screen display unit 310 or transmitting it to an external device connected via the communication I / F unit 320.

[0138] The above is an example of the configuration of the walking posture measurement device 300 .

[0139] Next, refer to Figures 20 to 22 , an example of the operation of the walking posture measurement system 100 will be described. Figure 20 , an operation example of the angle adjustment information output unit 2124 is described.

[0140] Figure 20 An example of the operation of the angle adjustment information output unit 2124 is shown. Figure 20 If the smartphone 200 is tilted in the front-to-back direction (step S101, yes), the angle adjustment information output unit 2124 adjusts the pitch according to the tilt (step S102). Specifically, the further the smartphone 200 is tilted in the front-to-back or rear-to-back direction, the larger the pitch adjustment will be. The angle adjustment information output unit 2124 then outputs the corrected sound (step S103).

[0141] Furthermore, when the smartphone 200 is tilted left or right (step S104, yes), the angle adjustment information output unit 2124 adjusts the length of the sound according to the degree of tilt (step S102). Specifically, the further the smartphone 200 is tilted left or right, the greater the angle adjustment information output unit 2124 adjusts the length of the sound. The angle adjustment information output unit 2124 then outputs the corrected sound (step S103).

[0142] The output of the sound by the angle adjustment information output unit 2124 is terminated if it satisfies the termination condition (step S107: Yes). Examples of termination conditions include the start of video capture, the termination of video capture, the lapse of a predetermined time after the smartphone 200 stops tilting and the sound becomes consistent, or a person stopping the output. Termination conditions may also include conditions other than those listed above.

[0143] The above is an example of the operation of the angle adjustment information output unit 2124. Figure 21 , an example of the overall operation of the walking posture measurement device 300 will be described. Figure 21 FIG. 3 shows an example of the overall operation of the walking posture measurement device 300. Figure 21 The image acquisition unit 341 acquires the moving images (a plurality of image data) acquired by the smartphone 200 from the smartphone 200 via the communication I / F unit 320 (step S201). In the present embodiment, the image acquisition unit 341 acquires the moving images (image data) of a person walking in the left-right direction and the moving images (image data) of a person walking in the front-back direction from the smartphone 200 in a manner that associates or can associate the moving images (image data) of the person walking in the back-front direction.

[0144] The skeleton recognition unit 342 recognizes the skeleton of the person whose walking posture is to be measured in the image data using the learned model 331 (step S202 ).

[0145] The measurement unit 344 obtains the measured value W and other values ​​indicated by the measured value information 335 (step S203). The measured value W indicated by the measured value information 335 may be pre-calculated by the measured value calculation unit 343, for example, or calculated by the skeleton recognition unit 342 in parallel with the skeleton recognition process or after the skeleton recognition process.

[0146] The measuring unit 344 measures the walking posture of the person (step S204) using the calculation results of the measured value calculating unit 343. For example, the measuring unit 344 performs measurement based on a dynamic image of the person walking in the left-right direction and a dynamic image of the person walking in the front-back direction.

[0147] The output unit 345 outputs the skeleton information 334 , the measured value information 335 , the measurement result information 336 , and information obtained by superimposing the skeleton information 334 on the moving image included in the image information 333 (step S205 ).

[0148] The above is an example of the overall operation of the walking posture measurement device 300. Figure 22 , an example of the process of calculating the measured value W by the measured value calculation unit 343 is described.

[0149] Reference Figure 22 The measured value calculation unit 343 obtains information indicating the height input from the height information input unit 2123 and included in the image information 333. Furthermore, the measured value calculation unit 343 refers to the camera setting information 332 and obtains information indicating the vertical field angle θ and the horizontal field angle ψ of the camera (step S301).

[0150] The measured value calculation unit 343 determines a reference line for calculating the measured value W and calculates a ratio α (step S302). For example, the measured value calculation unit 343 determines the reference line based on the position of the person's feet in the image data. Furthermore, the measured value calculation unit 343 calculates a ratio α of the reference line relative to half the screen based on the determined reference line.

[0151] The measured value calculator 343 calculates the measured value W based on the height h, the vertical viewing angle θ, the horizontal viewing angle ψ, and the ratio α (step S303). For example, the measured value calculator 343 calculates the measured value W by solving the equation shown in the above-mentioned mathematical formula 1. The measured value calculator 343 then stores the calculated measured value W as measured value information 335 in the storage unit 330.

[0152] The measured value calculation unit 343 also calculates the difference in x-coordinate values ​​of the left and right feet (e.g., toes) in the image data, and then calculates the stride length based on the difference in x-coordinate values, the measured value W, and the resolution (step S304).

[0153] The above is an example of processing by the actual value calculation unit 343.

[0154] As described above, the walking posture measurement device 300 includes the skeleton recognition unit 342 and the measurement unit 344. With this configuration, the measurement unit 344 can measure the walking posture based on the results of skeleton recognition by the skeleton recognition unit 342. As a result, even without using a depth sensor or the like, the measurement unit 344 can measure the walking posture based on image data acquired using a camera or the like included in the smartphone 200.

[0155] Furthermore, the walking posture measurement device 300 is configured to acquire a dynamic image showing a person walking in the image's front-to-back direction and a dynamic image showing a person walking in the image's left-to-right direction. With this configuration, the measurement unit 344 can perform measurements based on the dynamic image showing a person walking in the image's front-to-back direction, as well as the dynamic image showing a person walking in the image's left-to-right direction. As a result, even without using a depth sensor or other device, the measurement unit 344 can measure various walking postures, which would be difficult to measure from a single dynamic image, based on image data acquired using a camera, etc., included in the smartphone 200.

[0156] Furthermore, the walking posture measurement device 300 includes a measured value calculation unit 343. With this configuration, the measured value calculation unit 343 can calculate measured values. Consequently, it is possible to calculate stride length, etc., based on moving images captured using the camera of the smartphone 200. Thus, the walking posture measurement device 300 can measure correct walking posture based on moving images captured using the camera of the smartphone 200.

[0157] Furthermore, the smartphone 200 includes a shooting assistance unit 212. With this configuration, the smartphone 200, with the assistance of the shooting assistance unit 212, can acquire a dynamic image showing a person walking in the image's front-to-back direction, and a dynamic image showing a person walking in the image's left-to-right direction. As a result, the smartphone 200 can acquire these two types of dynamic images while maintaining the same shooting conditions as closely as possible. This improves the accuracy of the measurement performed by the measurement unit 344 when performing measurements using these two types of dynamic images. Furthermore, with the configuration described above, it is also possible to suppress shooting conditions when acquiring dynamic images at different timings. That is, it is also possible to maintain consistent shooting conditions when capturing image data in the same direction acquired at various timings. As a result, image data in the same direction can be compared with high precision, improving the accuracy of analysis.

[0158] Furthermore, the shooting assist unit 212 includes an angle adjustment information output unit 2124. With this configuration, the angle adjustment information output unit 2124 can output information such as sound for adjusting the tilt of the smartphone 200. As a result, even in situations where the screen of the smartphone 200 is difficult to see, the angle of the smartphone 200 can be easily adjusted. This makes it easier to maintain consistent shooting conditions and improve the accuracy of the measurement performed by the measurement unit 344.

[0159] In this embodiment, the smartphone 200 is used as the camera. However, a camera other than the smartphone 200 may be used to acquire moving images. In other words, the camera included in the walking posture measurement system 100 is not limited to the smartphone 200.

[0160] In this embodiment, the function of the walking posture measurement device 300 is implemented by a single information processing device. However, the function of the walking posture measurement device 300 may also be implemented by multiple information processing devices connected via a network, for example. In other words, the function of the walking posture measurement device 300 is not limited to being implemented by a single information processing device and may also be implemented on a cloud, for example.

[0161] In addition, the shooting assistance unit 212 may also have Figure 5 All the multiple functions exemplified in Figure 5 For example, the imaging assistance unit 212 may not display the angle information 2012 on the angle information display unit 2122 but may only function as the angle adjustment information output unit 2124 .

[0162] Furthermore, the auxiliary functions of the imaging assistance unit 212 described in this embodiment can also be applied to systems other than the walking posture measurement system 100. The auxiliary functions of the imaging assistance unit 212 can also be applied to various scenarios where it is necessary to achieve the most consistent imaging conditions when acquiring image data. Similarly, the calculation process of the measured value W by the measured value calculation unit 343 can also be applied to systems other than the walking posture measurement system 100. The calculation process of the measured value W by the measured value calculation unit 343 can be applied to various scenarios where measured values ​​are calculated based on image data.

[0163] [Second embodiment]

[0164] Next, regarding the second embodiment of the present invention, refer to Figures 23 to 28 Provide explanation. Figure 23 This is a diagram for explaining a tracking example of the walking posture measurement device 300 . Figure 24 This is a block diagram showing a configuration example of the walking posture measurement device 300 in the second embodiment. Figure 25 It is a diagram showing a configuration example of the tracking unit 346 . Figure 26 、 Figure 27 This is a diagram showing an example of a graphic generated by the inclusion graphic generation unit 3461. Figure 28 This is a flowchart showing an example of the operation of the tracking unit 346.

[0165] In the second embodiment of the present disclosure, a modified example of the walking posture measurement device 300 described in the first embodiment will be described. Figure 23 As shown, when multiple people exist in the image data, if the same person is not accurately tracked between frames, it is impossible to accurately calculate the walking speed using the movement distance between frames. Therefore, in this embodiment, in addition to the components of the walking posture measurement device 300 described in the first embodiment, a tracking unit 346 is further included. The tracking unit 346 is configured to perform tracking based on the recognition results of the skeleton recognition unit 342.

[0166] Figure 24 An example of the configuration of the walking posture measurement device 300 in the second embodiment is shown. Figure 24 In addition to the configuration described in the first embodiment, the walking posture measurement device 300 further includes a tracking unit 346. The characteristic configuration of this embodiment will be described below.

[0167] The tracking unit 346 tracks the person based on the recognition result of the skeleton recognition unit 342. For example, the tracking unit 346 tracks the person by assigning an identification number to the recognized person. In other words, the tracking unit 346 tracks the person by assigning the same identification number to the person identified in the image data of the previous frame and the image data of the current frame. Figure 25 A more detailed configuration example of the tracking unit 346 is shown. Figure 25 The tracking unit 346 includes, for example, an internal graphics generating unit 3461 , an average bone coordinate calculating unit 3462 and a comparative tracking unit 3463 .

[0168] For each person, the inclusion figure generation unit 3461 generates a figure containing the coordinates of all parts identified by the skeleton recognition unit 342 (the coordinates included in the skeleton information 334). For example, the inclusion figure generation unit 3461 generates a minimum convex hull, a rectangle, or a circle that contains all the coordinates. Furthermore, the inclusion figure generation unit 3461 calculates the area of ​​the generated figure.

[0169] For example, the inclusion pattern generation unit 3461 generates an inclusion pattern and calculates its area for each frame of image data. Furthermore, if a single frame of image data contains multiple people, the inclusion pattern generation unit 3461 generates an inclusion pattern and calculates its area for each of the multiple people contained in the image data.

[0170] Figure 26 An example of a graphic including the coordinates of a person walking in and out of the screen. Figure 26 As shown, the inner graphics generating unit 3461 can output any one of the smallest convex hull, rectangle, and circle that contains all the coordinates. Figure 27 An example of a graphic including the coordinates of a person walking in the left-right direction of the screen is shown. Figure 27 The situation shown is also consistent with Figure 26 Similarly to the case shown, the inclusion figure generation unit 3461 can generate any of the smallest convex hull, rectangle, and circle that includes all coordinates.

[0171] Furthermore, for example, it is predetermined which graphic, a convex hull, a rectangle, or a circle, the inclusion graphic generator 3461 generates. For example, the inclusion graphic generator 3461 determines to generate a circle as a graphic including the coordinates.

[0172] The average skeleton coordinate calculation unit 3462 calculates the average value of the coordinates of all parts of each person identified by the skeleton recognition unit 342 (the coordinates included in the skeleton information 334). Thus, the average skeleton coordinate calculation unit 3462 calculates the average skeleton coordinates based on the coordinates of each part of the person identified by the skeleton recognition unit 342.

[0173] Similar to the case of the included graphic generation unit 3461, the average skeletal coordinate calculation unit 3462 calculates the average skeletal coordinates for each frame of image data. Furthermore, when a single frame of image data contains multiple people, the average skeletal coordinate calculation unit 3462 calculates the average skeletal coordinates for each of the multiple people contained in the image data.

[0174] The comparative tracking unit 3463 tracks the person based on the area of ​​the inclusion figure calculated by the inclusion figure generation unit 3461 and the average skeletal coordinates calculated by the average skeletal coordinate calculation unit 3462. For example, the comparative tracking unit 3463 compares the area corresponding to the person being tracked calculated in the current frame with the area calculated one frame previously. Furthermore, based on the results of this comparison, the comparative tracking unit 3463 compares the average skeletal coordinates corresponding to the person being tracked with the average skeletal coordinates calculated one frame previously.

[0175] Specifically, for example, if there is only one area corresponding to each person in the previous frame whose difference with the area corresponding to the person being tracked in the current frame is within the first tolerance, the comparison and tracking unit 3463 determines that the person in the previous frame whose area difference is within the first tolerance is the same person as the person in the current frame. As a result, the comparison and tracking unit 3463 uses, for example, the identification number corresponding to the person in the previous frame determined to be the same as the person in the current frame as the identification number corresponding to the person in the current frame.

[0176] Furthermore, if multiple areas whose differences from the area of ​​the tracking target are calculated one frame ago are within a first tolerance, the comparison tracking unit 3463 compares the average skeletal coordinates of the person being tracked with the average skeletal coordinates of the person one frame ago whose area difference is within the first tolerance. The comparison tracking unit 3463 then determines that the person one frame ago whose average skeletal coordinate difference is within a second tolerance is the same person as the person in the current frame. As a result, the comparison tracking unit 3463, for example, uses the identification number corresponding to the person one frame ago determined to be the same person as the identification number corresponding to the person in the current frame. Furthermore, if multiple people whose average skeletal coordinate differences are within the second tolerance, the comparison tracking unit 3463 can determine that the person whose average skeletal coordinate difference is closest to the second tolerance is the same person. If multiple people whose average skeletal coordinate differences are within the second tolerance, the comparison tracking unit 3463 can also be configured to perform processing other than the above-described example, such as determining that the person is an error.

[0177] If no area whose area difference from the tracking target falls within the first tolerance value has been calculated one frame previously, and if no person whose average skeletal coordinate difference falls within the second tolerance value one frame previously exists, the comparative tracking unit 3463 determines that the person being tracked is a newly identified person. In this case, the comparative tracking unit 3463 assigns a new identification number to the newly identified person.

[0178] Furthermore, the comparison and tracking unit 3463 may use, as the first allowable value, an estimated area value estimated based on the rate of increase or decrease of the area corresponding to a person determined to be the same person in the past several frames. The first allowable value may also be a predetermined value.

[0179] Furthermore, as the second allowable value, the comparison tracking unit 3463 may use an estimated coordinate value estimated based on the average skeletal coordinate movement speed of a person identified as the same person in the past several frames. The second allowable value may also be a predetermined value, similar to the first allowable value.

[0180] The above is an example of the structure of the tracking unit 346. Figure 28 , an example of the operation of the tracking unit 346 is described.

[0181] Reference Figure 28 The inclusion graphic generating unit 3461 generates a graphic for each person that includes the coordinates of all parts identified by the skeleton identifying unit 342 (the coordinates included in the skeleton information 334). The inclusion graphic generating unit 3461 also calculates the area of ​​the generated graphic (step S401).

[0182] The average skeleton coordinate calculation unit 3462 calculates the average skeleton coordinates for each person by calculating the average value of the coordinates of all parts recognized by the skeleton recognition unit 342 (the coordinates included in the skeleton information 334 ).

[0183] The comparison tracking unit 3463 compares the area corresponding to the person being tracked, calculated in the current frame, with the area calculated one frame before (step S403 ).

[0184] If there is only one area of ​​each person one frame ago whose area difference with the area corresponding to the person being tracked in the current frame is within the first allowable value (step S403, 1), the comparison and tracking unit 3463 determines that the person one frame ago and the person in the current frame whose area difference is within the first allowable value are the same person (step S404). As a result, the comparison and tracking unit 3463 uses, for example, the identification number corresponding to the person one frame ago as the identification number of the current frame.

[0185] In addition, if multiple areas whose area difference with the tracking target is within the first allowable value are calculated one frame ago (step S403, multiple), the comparison tracking unit 3463 compares the average skeletal coordinates of the person being tracked with the average skeletal coordinates of the person being tracked one frame ago whose area difference is within the first allowable value (step S405). Then, if a person whose average skeletal coordinate difference with the tracking target is within the second allowable value exists one frame ago (step S405, yes), the comparison tracking unit 3463 determines that the person one frame ago and the person being tracked are the same person (step S404). In addition, if multiple people whose average skeletal coordinate difference with the tracking target is within the second allowable value exist one frame ago, the comparison tracking unit 3463 can, for example, determine that the person whose difference is closest to the second allowable value is the same person. In addition, if no person whose average skeletal coordinate difference with the tracking target is within the second allowable value exists one frame ago (step S405, no), the comparison tracking unit 3463 determines that the person being tracked is a newly identified person (step S406). In this case, the comparison and tracking unit 3463 assigns a new identification number to the newly recognized person.

[0186] If an area whose difference from the area of ​​the tracking target is within the first allowable value has not been calculated in the previous frame (step S403, 0), the comparison tracking unit 3463 determines that the person being tracked is a newly identified person (step S406). In this case, the comparison tracking unit 3463 assigns a new identification number to the newly identified person.

[0187] The above is an example of the operation of the tracking unit 346.

[0188] As described above, the walking posture measurement device 300 in this embodiment includes the tracking unit 346. With this configuration, the tracking unit 346 can track the same person based on the recognition results of the skeleton recognition unit 342. As a result, erroneous calculations of walking speed, etc., can be suppressed, and the accuracy of walking posture measurement can be improved.

[0189] In this embodiment, the example of calculating the area of ​​the inclusion pattern generated by inclusion pattern generator 3461 is described. However, inclusion pattern generator 3461 may also be configured to calculate a value other than the area of ​​the generated inclusion pattern. For example, inclusion pattern generator 3461 may calculate the height, diameter, or other value of the generated inclusion pattern instead of the area. In this case, comparison tracking unit 3463 compares the value of the height, etc. calculated based on the pattern generated by inclusion pattern generator 3461 instead of the area.

[0190] The included pattern generation unit 3461 generates a pattern that is included in all coordinates. However, the included pattern generation unit 3461 may also be configured to generate a pattern that is included in the coordinates corresponding to the upper body of the person, or to generate a pattern that includes a portion of the coordinates identified by the skeleton recognition unit 342. Furthermore, the average skeleton coordinate calculation unit 3462 may calculate average coordinates based on a portion of the coordinates identified by the skeleton recognition unit 342, such as the average coordinates of the coordinates corresponding to the upper body of the person.

[0191] In addition, the comparison tracking unit 3463 may be configured to perform only one of the comparison of the values ​​based on the inclusion pattern generated by the inclusion pattern generating unit 3461 and the comparison of the average skeletal coordinates.

[0192] Furthermore, the tracking method using the recognition results of the skeleton recognition unit 342 described in this embodiment can also be applied to situations other than walking posture measurement, such as tracking a person. In other words, the function of the tracking unit 346 can also be applied to devices other than the walking posture measurement device 300 that require person tracking. Thus, the method for tracking a person using skeleton information described in this embodiment is not limited to walking posture measurement and can be flexibly applied in a variety of situations.

[0193] In addition, in the case of this embodiment, various modifications can be adopted similarly to the first embodiment.

[0194] [Third embodiment]

[0195] Next, refer to Figure 29 、 Figure 30 A third embodiment of the present invention will be described. Figure 29 、 Figure 30 A configuration example of the imaging device 400 is shown.

[0196] The camera 400 captures a person's walking posture. Figure 29 FIG. 4 shows an example of the hardware configuration of the imaging device 400. Figure 29 In addition to a camera for photographing, the photographing device 400 also has the following hardware structure as an example.

[0197] CPU (Central Processing Unit) 401 (computing device)

[0198] ROM (Read Only Memory) 402 (storage device)

[0199] RAM (Random Access Memory) 403 (storage device)

[0200] Program group 404 loaded into RAM 403

[0201] Storage device 405 storing program group 404

[0202] A drive device 406 that reads and writes to a recording medium 410 outside the information processing device

[0203] Communication interface 407 connected to a communication network 411 outside the information processing device

[0204] Input / output interface 408 for inputting and outputting data

[0205] Bus 409 connecting various structural elements

[0206] In addition, the photographing device 400 obtains the program group 404 through the CPU 401 and executes it by the CPU 401, which can be realized as Figure 30The functions of the detection unit 421 and the display unit 422 are shown. Furthermore, the program group 404 is stored in advance in, for example, the storage device 405 or the ROM 402, and the CPU 401 loads the program into the RAM 403 or the like as needed for execution. Alternatively, the program group 404 may be supplied to the CPU 401 via the communication network 411, or may be stored in advance in the recording medium 410, and the drive device 406 reads the program and supplies it to the CPU 401.

[0207] also, Figure 29 4 shows an example of the hardware configuration of the imaging device 400. The hardware configuration of the imaging device 400 is not limited to the above. For example, the imaging device 400 may be configured by excluding a portion of the above-mentioned configuration such as the drive device 406.

[0208] The detection unit 421 detects the direction of the camera 400. For example, the detection unit 421 detects whether the camera 400 is in a portrait or landscape orientation.

[0209] The display unit 422 displays a guide line indicating the position where the person is walking on the screen display unit. For example, the display unit 422 displays different guide lines depending on the direction of the imaging device 400 detected by the detection unit 421.

[0210] Thus, the camera 400 includes a detection unit 421 and a display unit 422. With this configuration, the display unit 422 can display different guide lines depending on the orientation of the camera 400 detected by the detection unit 421. As a result, when acquiring image data corresponding to the orientation of the camera 400, appropriate assistance can be provided that corresponds to each orientation. Consequently, when acquiring multiple sets of image data, the shooting conditions can be kept as consistent as possible.

[0211] Furthermore, the aforementioned camera 400 can be implemented by installing a predetermined program in the camera 400. Specifically, another embodiment of the present invention includes a program that causes the camera 400 to implement a detection unit 421 for detecting the direction of the camera, and a display unit 422 for displaying a guide line indicating the position of a person walking on a screen display unit, wherein the display unit 422 displays different guide lines depending on the direction of the camera detected by the detection unit 421.

[0212] In addition, the guidance method performed by the above-mentioned shooting device 400 is as follows: the shooting device 400 that shoots the walking posture of a person detects the program of the shooting device, and displays a guide line indicating the position where the person is walking on the screen display part. When the guide line is displayed on the screen display part, different guide lines are displayed according to the detected direction of the shooting device.

[0213] Even if the invention is a program or a guidance method having the above-mentioned structure, it can also achieve the above-mentioned object of the present invention because it has the same function and effect as the above-mentioned imaging device 400.

[0214] [Fourth embodiment]

[0215] Next, refer to Figure 31 A fourth embodiment of the present invention will be described. Figure 31 A configuration example of the information processing device 500 is shown.

[0216] The information processing device 500 has, for example, Figure 29 The hardware structure of the photographing device 400 described above is the same as that of the photographing device 400. In addition, the information processing device 500 can realize the program group obtained by the CPU and executed by the CPU. Figure 31 The function shown is the calculation unit 521.

[0217] The calculation unit 521 calculates the actual length at a predetermined position in the image data based on the parameters of the imaging device that acquired the image data and information indicating the height of the imaging device when the image data was acquired.

[0218] Thus, information processing device 500 includes calculation unit 521. With this configuration, calculation unit 521 can calculate the actual length at a predetermined location in image data based on various information. As a result, analysis using the actual length can be performed based on image data captured by a camera such as a smartphone.

[0219] Furthermore, the information processing device 500 described above can be implemented by installing a predetermined program in the information processing device 500. Specifically, a program as another embodiment of the present invention is a program for causing the information processing device 500 to implement a calculation unit 521 that calculates the actual length at a predetermined position in the image data based on parameters of the imaging device that acquired the image data and information indicating the height of the imaging device when the image data was acquired.

[0220] In addition, the calculation method performed by the above-mentioned information processing device 500 is the following method, in which the information processing device 500 obtains parameters of the shooting device used to obtain image data and information indicating the height of the shooting device when the image data is obtained, and calculates the actual length at a specified position in the image data based on the obtained information.

[0221] Even if an invention is an invention of a program or a calculation method having the above-mentioned structure, it can also achieve the above-mentioned object of the present invention because it has the same action and effect as the above-mentioned information processing device 500.

[0222] [Fifth embodiment]

[0223] Next, refer to Figure 32 A fifth embodiment of the present invention will be described. Figure 32 A configuration example of the information processing device 600 is shown.

[0224] The information processing device 600 has, for example, Figure 29 The hardware structure of the photographing device 400 described above is the same as that of the photographing device 400. In addition, the information processing device 600 obtains the program group possessed by the information processing device 600 through the CPU and executes it by the CPU, thereby being able to realize the function of Figure 32 The functions of the detection unit 621 and the output unit 622 are shown.

[0225] The detection unit 621 detects the inclination of the information processing device.

[0226] The output unit 622 outputs information corresponding to the inclination of the information processing device detected by the detection unit 621 , which information varies depending on how the information processing device is tilted.

[0227] Thus, information processing device 600 includes detection unit 621 and output unit 622. With this configuration, output unit 622 can output information corresponding to the inclination of the information processing device detected by detection unit 621. As a result, an operator operating information processing device 600 can correct the inclination based on the output information.

[0228] Furthermore, the above-described information processing device 600 can be implemented by installing a predetermined program in the information processing device 600. Specifically, the program as another embodiment of the present invention is a program for causing the information processing device 600 to implement a detection unit 621 for detecting the inclination of the information processing device 600, and an output unit 622 for outputting information corresponding to the inclination of the information processing device 600 detected by the detection unit 621, which information varies depending on how the information processing device 600 is tilted.

[0229] The calculation method performed by the information processing device 600 is a method in which the information processing device 600 detects the inclination of the information processing device 600 and outputs information corresponding to the detected inclination of the information processing device that varies depending on how the information processing device is tilted.

[0230] Even if the invention is a program or output method having the above-mentioned structure, it can achieve the above-mentioned object of the present invention because it has the same action and effect as the above-mentioned information processing device 600.

[0231] [Sixth embodiment]

[0232] Next, refer to Figure 33A sixth embodiment of the present invention will be described. Figure 33 A configuration example of the tracking device 700 is shown.

[0233] The tracking device 700 has, for example, Figure 29 The hardware structure of the shooting device 400 described above is the same as that of the shooting device 400. In addition, the tracking device 700 obtains the program group possessed by the tracking device 700 through the CPU and executes it by the CPU, which can realize the function of Figure 33 The functions of the acquisition unit 721 and the tracking unit 722 are shown.

[0234] The acquisition unit 721 recognizes the skeleton of a person in the image data to acquire information indicating a plurality of parts of the recognized person.

[0235] The tracking unit 722 tracks the same person across a plurality of image data based on the information acquired by the acquisition unit 721 .

[0236] Thus, the tracking device 700 includes the acquisition unit 721 and the tracking unit 722. With such a configuration, the tracking unit 722 can perform tracking based on the information indicating the part acquired by the acquisition unit 721. This enables easy tracking.

[0237] Furthermore, the tracking device 700 described above can be implemented by installing a predetermined program in the tracking device 700. Specifically, the program as another embodiment of the present invention is a program for causing the tracking device 700 to implement an acquisition unit 721 for acquiring information representing multiple parts of a recognized person by recognizing the skeleton of the person in image data, and a tracking unit 722 for tracking the same person across multiple image data based on the information acquired by the acquisition unit 721.

[0238] In addition, the tracking method performed by the above-mentioned tracking device 700 is as follows: the tracking device 700 obtains information representing multiple parts of the identified person by identifying the skeleton of the person in the image data, and tracks the same person between multiple image data based on the obtained information.

[0239] Even if an invention of a program or tracking method having the above-mentioned structure has the same function and effect as the above-mentioned tracking device 700, it can also achieve the above-mentioned purpose of the present invention.

[0240] <Note>

[0241] Part or all of the above embodiments may be described as the following supplementary notes. The following is an overview of the guidance method of the present invention. However, the present invention is not limited to the following configuration.

[0242] (Note 1)

[0243] A booting method, wherein

[0244] The camera that captures the person's walking posture detects the direction of the camera,

[0245] The camera displays a guide line indicating the position where the person is walking on the screen display unit.

[0246] When the guide line is displayed on the screen display unit, the imaging device displays different guide lines according to the detected direction of the imaging device.

[0247] (Note 2)

[0248] In the boot method described in Supplement 1,

[0249] Different guide lines are displayed depending on whether the camera is in a portrait or landscape orientation.

[0250] (Note 3)

[0251] In the guidance method described in Supplement 1 or Supplement 2,

[0252] When the camera is in a portrait orientation, the guide line is displayed to guide a person walking in or out of the screen.

[0253] (Note 4)

[0254] In any one of the guidance methods of Supplement 1 to Supplement 3,

[0255] When the camera is in landscape orientation, guide lines are displayed to guide people walking in the left-right direction of the screen.

[0256] (Note 5)

[0257] In the guidance method described in any one of Supplementary Notes 1 to 4,

[0258] The tilt of the imaging device is detected, and information indicating the detected tilt is displayed on the screen display unit.

[0259] (Note 6)

[0260] In any one of the guidance methods of Supplement 1 to Supplement 5,

[0261] The tilt of the imaging device is detected, and information corresponding to the detected tilt is output.

[0262] (Note 7)

[0263] In the guidance method described in Supplementary Note 6,

[0264] The device detects the left-right tilt and the inside-out tilt, and outputs different information when the left-right tilt is detected and when the inside-out tilt is detected.

[0265] (Note 8)

[0266] In the guidance method described in Supplement 6 or Supplement 7,

[0267] When the tilt in the left-right direction is detected, and when the tilt in the inside-out direction is detected, sounds adjusted by different methods are output.

[0268] (Note 9)

[0269] A shooting device for shooting a person's walking posture, wherein:

[0270] The shooting device comprises:

[0271] a detection unit for detecting a direction of the camera; and

[0272] The display unit displays a guide line indicating a position where the person is walking on the screen display unit,

[0273] The display unit displays different guide lines according to the direction of the imaging device detected by the detection unit.

[0274] (Note 10)

[0275] A program enables a camera for photographing a person walking to implement the following components:

[0276] a detection unit for detecting a direction of the camera; and

[0277] The display unit displays a guide line indicating a position where the person is walking on the screen display unit,

[0278] The display unit displays different guide lines according to the direction of the imaging device detected by the detection unit.

[0279] (Note 11)

[0280] A calculation method, wherein

[0281] The information processing device acquires parameters of an imaging device used to acquire image data and information indicating the height of the imaging device when the image data was acquired, and calculates an actual length at a predetermined position in the image data based on the acquired information.

[0282] (Note 12)

[0283] In the calculation method described in Supplementary Note 11,

[0284] A reference line for length calculation is determined based on a predetermined standard, and the actual length from one end to the other end of the image data along the determined reference line is calculated.

[0285] (Note 13)

[0286] In the calculation method described in Supplementary Note 12,

[0287] The reference line is determined based on the position of the person's feet in the image data.

[0288] (Note 14)

[0289] In the calculation method described in Supplement 12 or Supplement 13,

[0290] The ratio of the position of the reference line on the screen represented by the image data to half of the screen is calculated, and the actual length of the reference line is calculated based on the calculated ratio, the parameter, and the height.

[0291] (Note 15)

[0292] In any one of the calculation methods in Supplementary Notes 11 to 14,

[0293] The parameters include information indicating a vertical field of view angle and a horizontal field of view angle of the camera.

[0294] (Note 16)

[0295] In any one of the calculation methods in Supplementary Notes 11 to 15,

[0296] The actual length W is calculated using Equation 1 using the vertical field angle θ, horizontal field angle ψ of the camera, the ratio α of the position of the reference line determined on the screen relative to half the screen, and the height h.

[0297] (Note 17)

[0298] In any one of the calculation methods of Supplementary Notes 11 to 16,

[0299] Based on the calculated length and the resolution of the image data, the stride length of the person is calculated.

[0300] (Note 18)

[0301] In the calculation method described in Supplementary Note 17,

[0302] Get the number of pixels between the left and right feet of the person in the image data,

[0303] The person's stride is calculated based on the calculated length, the resolution of the image data, and the number of pixels obtained.

[0304] (Note 19)

[0305] An information processing device, wherein:

[0306] The information processing device includes a calculation unit that calculates an actual length at a predetermined position in the image data based on parameters of an imaging device used to acquire the image data and information indicating a height of the imaging device when the image data was acquired.

[0307] (Note 20)

[0308] A program that causes an information processing device to implement a calculation unit.

[0309] The calculation unit calculates the actual length at a predetermined position in the image data based on parameters of the imaging device used to acquire the image data and information indicating the height of the imaging device when the image data was acquired.

[0310] (Note 21)

[0311] An output method, wherein

[0312] The information processing device detects the inclination of the information processing device,

[0313] Different information corresponding to the inclination of the information processing device is output according to the inclination mode of the information processing device.

[0314] (Note 22)

[0315] In the output method described in Supplementary Note 21,

[0316] Detecting the left-right and inside-out tilts of the information processing device,

[0317] Different information is output when the left-right tilt is detected and when the inside-out tilt is detected.

[0318] (Note 23)

[0319] In the output method described in Supplement 21 or Supplement 22,

[0320] When the tilt in the left-right direction is detected, and when the tilt in the inside-out direction is detected, sounds adjusted by different methods are output.

[0321] (Note 24)

[0322] In the output method described in Supplementary Note 23,

[0323] When the tilt in the left-right direction is detected, a sound is output in which either the pitch or the length of the sound is adjusted. When the tilt in the inside-outside direction is detected, a sound is output in which the pitch or the length of the sound is adjusted using a method different from that when the tilt in the left-right direction is detected.

[0324] (Note 25)

[0325] In any one of the output methods of Supplementary Notes 21 to 24,

[0326] When the information processing device is tilted, two sounds are output.

[0327] When the information processing device is not tilted, one sound is output.

[0328] (Note 26)

[0329] In any one of the output methods of Supplement 1 to Supplement 25,

[0330] Information indicating the inclination of the information processing device is displayed on the screen display unit.

[0331] (Note 27)

[0332] An information processing device comprising:

[0333] a detection unit that detects an inclination of the information processing device; and

[0334] The output unit outputs different information corresponding to the inclination of the information processing device detected by the detection unit according to the inclination of the information processing device.

[0335] (Note 28)

[0336] In the information processing device described in Supplementary Note 27,

[0337] The detection unit detects the inclination of the information processing device in the left-right direction and the inclination in the front-back direction.

[0338] The output unit outputs different information when the detection unit detects the inclination in the left-right direction and when the detection unit detects the inclination in the front-back direction.

[0339] (Note 29)

[0340] A program for causing an information processing device to implement the following components:

[0341] a detection unit that detects an inclination of the information processing device; and

[0342] The output unit outputs different information corresponding to the inclination of the information processing device detected by the detection unit according to the inclination of the information processing device.

[0343] (Note 30)

[0344] In the procedure described in Note 29,

[0345] The detection unit detects the inclination of the information processing device in the left-right direction and the inclination in the front-back direction.

[0346] The output unit outputs different information when the detection unit detects the inclination in the left-right direction and the inclination in the front-back direction.

[0347] (Note 31)

[0348] A tracking method, wherein

[0349] The information processing device obtains information indicating a plurality of parts of a person recognized by recognizing a skeleton of the person in the image data,

[0350] The information processing device tracks the same person across a plurality of image data based on the acquired information.

[0351] (Note 32)

[0352] In the tracking method described in Supplement 31,

[0353] generating an inclusion pattern including at least a portion of the identified part,

[0354] Tracking of the same person is performed based on the value corresponding to the generated included pattern.

[0355] (Note 33)

[0356] In the tracking method described in Supplement 32,

[0357] Tracking of the same person is performed based on the difference in values ​​corresponding to the included pattern between the image data.

[0358] (Note 34)

[0359] In the tracking method described in Supplement 33,

[0360] When the number of differences between the value corresponding to the included graphic of the tracking object and the value corresponding to the included graphic of a person contained in image data different from the image data to which the tracking object belongs is within a prescribed value is one, the person whose difference is within the prescribed value is judged to be the same person as the person being tracked.

[0361] (Note 35)

[0362] In the tracking method described in Supplement 34,

[0363] The predetermined value is determined based on a change in a value corresponding to the included pattern among a plurality of image data.

[0364] (Note 36)

[0365] In any one of the tracking methods of Supplementary Notes 31 to 35,

[0366] calculating an average value of the coordinates of at least a portion of the identified parts,

[0367] And based on the calculated results, the same person is tracked.

[0368] (Note 37)

[0369] In the tracking method described in Supplement 36,

[0370] Tracking of the same person is performed based on the difference between the average values ​​between the image data.

[0371] (Note 38)

[0372] In the tracking method described in Supplement 36 or Supplement 37,

[0373] Tracking of the same person is performed based on a difference between the average value of the tracking target and the average value corresponding to a person included in image data different from the image data to which the tracking target belongs.

[0374] (Note 39)

[0375] A tracking device comprising:

[0376] an acquisition unit that acquires information indicating a plurality of parts of a person recognized by recognizing a skeleton of the person in the image data; and

[0377] The tracking unit tracks the same person across a plurality of image data based on the information acquired by the acquisition unit.

[0378] (Note 40)

[0379] A program for enabling a tracking device to implement the following components:

[0380] an acquisition unit that acquires information indicating a plurality of parts of a person recognized by recognizing a skeleton of the person in the image data; and

[0381] The tracking unit tracks the same person across the plurality of image data based on the information acquired by the acquisition unit.

[0382] The programs described in the above embodiments and supplementary notes are stored in a storage device or recorded in a computer-readable recording medium, such as a removable medium such as a flexible disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0383] The present invention has been described above with reference to the above embodiments, but the present invention is not limited to the above embodiments. Various modifications that can be understood by those skilled in the art can be made to the structure and details of the present invention within the scope of the present invention.

[0384] The present invention also claims the benefit of priority based on Japanese Patent Application No. 2020-053974, filed on March 25, 2020 in Japan, and the entire contents of the patent application are incorporated herein by reference.

[0385] Label Description

[0386] 100 Walking Posture Measurement System

[0387] 200 smartphones

[0388] 210 Measurement motion capture unit

[0389] 211 Image Data Capture Unit

[0390] 212 Shooting Support Department

[0391] 2121 Guide Line Display

[0392] 2122 Angle information display unit

[0393] 2123 Altitude information input unit

[0394] 2124 Angle adjustment information output unit

[0395] 201 Touch Panel

[0396] 2011 Leading Lines

[0397] 2012 Angle Information

[0398] 2013 Height Display Department

[0399] 300 Walking Posture Measurement Device

[0400] 310 screen display unit

[0401] 320 Communication I / F Department

[0402] 330 Storage Department

[0403] 331 Learning Complete Model

[0404] 332 Camera setting information

[0405] 333 Image Information

[0406] 334 Skeleton Information

[0407] 335 Measured value information

[0408] 336 Measurement result information

[0409] 337 Program

[0410] 340 Operation Processing Unit

[0411] 341 Image Acquisition Unit

[0412] 342 Skeleton Recognition Department

[0413] 343 Measured value calculation unit

[0414] 344 Measurement Department

[0415] 345 Output

[0416] 346 Tracking Department

[0417] 3461 In-house Graphics Generation Department

[0418] 3462 Average Bone Coordinate Calculation Unit

[0419] 3463 Comparative Tracking Department

[0420] 400 camera device

[0421] 401 CPU

[0422] 402 ROM

[0423] 403 RAM

[0424] 404 Program Group

[0425] 405 Storage Device

[0426] 406 drive unit

[0427] 407 Communication Interface

[0428] 408 Input and Output Interface

[0429] 409 Bus

[0430] 410 Recording Media

[0431] 411 Communication Network

[0432] 421 Testing Department

[0433] 422 Display

[0434] 500 Information Processing Device

[0435] 521 Computing Department

[0436] 600 Information Processing Device

[0437] 621 Testing Department

[0438] 622 Output

[0439] 700 Tracking Device

[0440] 721 Acquisition Department

[0441] 722 Tracking Department

Claims

1. A booting method, wherein: A shooting device for shooting a person walking in the inside and outside directions of a screen and a person walking in the left and right directions of a screen detects the direction of the shooting device, The imaging device displays a guide line indicating a position where a person serving as an imaging subject is walking, together with an area captured by the camera, that is, a shooting area, on a screen display unit. When the guide line is displayed on the screen display unit, the camera displays different guide lines according to the direction of the camera detected. When the camera is in a portrait orientation, the guide lines for guiding people walking in the inside and outside directions of the screen are displayed; when the camera is in a landscape orientation, the guide lines for guiding people walking in the left and right directions of the screen are displayed. The camera captures a person walking along the guide line.

2. The booting method according to claim 1, wherein: The tilt of the imaging device is detected, and information indicating the detected tilt is displayed on the screen display unit. The booting method according to claim 1 , wherein: The tilt of the imaging device is detected, and information corresponding to the detected tilt is output. The booting method according to claim 3 , wherein: The device detects the left-right tilt and the inside-out tilt, and outputs different information when the left-right tilt is detected and when the inside-out tilt is detected. The booting method according to claim 3 , wherein: When the tilt in the left-right direction is detected, and when the tilt in the inside-out direction is detected, sounds adjusted by different methods are output.

6. A shooting device for shooting a person walking in the inside and outside directions of a picture and a person walking in the left and right directions of a picture, wherein: The shooting device comprises: a detection unit for detecting a direction of the camera; and The display unit displays a guide line indicating the position where the person serving as the photographic subject is walking, together with the photographic area photographed by the camera, on the screen display unit. The display unit displays different guide lines according to the direction of the imaging device detected by the detection unit. When the camera is in a portrait orientation, the guide lines for guiding people walking in the inside and outside directions of the screen are displayed; when the camera is in a landscape orientation, the guide lines for guiding people walking in the left and right directions of the screen are displayed. The camera captures a person walking along the guide line.

Images