Information processing method, information processing device, and computer program

Abstract

This improves the success rate of capturing good video footage of external objects of interest that a person inside a mobile vehicle is interested in, using cameras mounted on the vehicle. [Solution] The information processing method causes the controller (28) to perform the following processes: taking a picture of a person inside the moving object with a first camera or acquiring audio data of the person (S1, S2); identifying the direction in which the person's object of interest is located as the target direction based on the person's video or audio data (S3, S4); and acquiring an image of the scenery outside the moving object in the target direction taken by a second camera mounted on the moving object (S5).

Description

Technical Field

[0001] The present invention relates to an information processing method, an information processing apparatus, and a computer program.

Background Art

[0002] In Patent Document 1, a technique is proposed in which video data representing a video in front of a vehicle is transmitted from an in-vehicle device to an out-of-vehicle terminal and the video is displayed on the out-of-vehicle terminal.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the technique of Patent Document 1, since a video with a fixed shooting range with respect to the direction of the vehicle is shot, it is not always possible to shoot an object of interest of a person inside the moving body. An object of the present invention is to improve the success rate of shooting a good video of an external object of interest that a person inside a moving body is interested in, with a camera provided in the moving body.

Means for Solving the Problems

[0005] In an information processing method according to an aspect of the present invention, a controller is caused to execute a process of shooting a person inside a moving body with a first camera or acquiring voice data of the person, a process of specifying, as a target direction, the direction in which an object of interest of the person exists based on the video or voice data of the person, and a process of acquiring a video of a scene in the target direction outside the moving body shot with a second camera provided in the moving body.

Effects of the Invention

[0006] According to the present invention, the success rate of capturing good images of external objects of interest that a person inside a mobile body is interested in, using a camera mounted on the mobile body, can be improved. [Brief explanation of the drawing]

[0007] [Figure 1] This is a schematic diagram of an example of the overall configuration of the imaging system of the embodiment. [Figure 2] This is a block diagram showing an example of the functional configuration of a controller in an information processing device. [Figure 3] This is a flowchart of an example of an information processing method in an embodiment. [Modes for carrying out the invention]

[0008] Embodiments of the present invention will be described below with reference to the drawings. Note that the drawings are schematic and may differ from actual ones. Furthermore, the embodiments of the present invention described below are illustrative examples of devices and methods for realizing the technical concept of the present invention, and the technical concept of the present invention is not limited to the structure, arrangement, etc., of the components described below. The technical concept of the present invention can be modified in various ways within the technical scope defined by the claims described in the patent claims.

[0009] (composition) Figure 1 is a schematic diagram of an example of the overall configuration of the imaging system of the embodiment. The imaging system 1 comprises an information processing device 20 mounted on a vehicle 2, an external terminal device 3, and a server device 4. The imaging system 1 is a system that uses a camera mounted on the vehicle 2 to capture the scenery outside the vehicle 2 and shares the captured video with the terminal device 3 and the server device 4. The terminal device 3 and the server device 4 are examples of the "external devices" described in the claims. In this specification, the case in which the "mobile body" described in the claims is a vehicle is given as an example, but the mobile body of the present invention is not limited to a vehicle, but may be a ship, aircraft, or any other means of transportation in which a person can appear.

[0010] The information processing device 20 includes an internal camera 21, an external camera 22, a microphone 23, a map database (map DB) 24, a positioning device 25, a human-machine interface (HMI) 26, a communication device 27, and a controller 28.

[0011] The internal camera 21 is installed inside the vehicle 2 and is a camera that takes pictures of the inside of the vehicle 2. The information processing device 20 acquires internal images, which are images of the inside of the vehicle 2, using the internal camera 21. For example, the internal images may be images of people (occupants, driver) inside the vehicle 2. The internal camera 21 is an example of the "first camera" described in the claims.

[0012] The external camera 22 is a camera mounted on the vehicle 2 that captures the scenery outside the vehicle 2. The information processing device 20 acquires external images, which are images of the outside of the vehicle 2, from the external camera 22. For example, the external camera 22 may be a wide-angle camera that captures a relatively wide field of view. For example, the external camera 22 may be an ultra-wide-angle camera or an omnidirectional camera with a field of view of 100 degrees or more. The external camera 22 is an example of the "second camera" described in the claims. Microphone 23 is installed inside vehicle 2 to detect sound signals inside vehicle 2. Information processing device 20 acquires voice data of people inside vehicle 2 using microphone 23.

[0013] Map DB24 stores road map data. For example, Map DB24 may store navigation map data. Road map data may include information about locations around roads (such as place names) and information about features (such as buildings and facilities). The positioning device 25 measures the current position and attitude of the vehicle 2. The positioning device 25 may include, for example, a Global Navigation Satellite System (GNSS) receiver. The GNSS receiver is, for example, a GPS receiver, and measures the current position of the vehicle 2 by receiving radio waves from multiple navigation satellites. The positioning device 25 may also include an inertial navigation system.

[0014] HMI26 is an interface device that exchanges information between a person inside the vehicle 2 and the information processing device 20. For example, HMI26 may include a display device (e.g., a touch panel display) that displays images captured by an external camera 22 or an internal camera 21. Also, HMI26 may include a speaker or buzzer that presents auditory information. Furthermore, HMI26 may include an interface (touch panel, buttons, switches, etc.) that accepts operation input from a person inside the vehicle 2.

[0015] The communication device 27 provides communication functionality between the information processing device 20 and external devices. The communication method used by the communication device 27 may be, for example, wireless communication over a public mobile network, satellite communication, or vehicle-to-infrastructure communication. The information processing device 20 transmits and receives data between the terminal device 3 and the server device 4 via the communication device 27.

[0016] The controller 28 is an electronic control unit (ECU) that controls the operation of the information processing device 20. The controller 28 includes a processor 28a and peripheral components such as a storage device 28b. The processor 28a may be, for example, a CPU (Central Processing Unit) or an MPU (Micro-Processing Unit).

[0017] The storage device 28b may include semiconductor storage devices, magnetic storage devices, optical storage devices, etc. The storage device 28b may include registers, cache memory, and memory such as ROM (Read Only Memory) and RAM (Random Access Memory) used as main memory. The functions of the controller 28 described below are realized, for example, by the processor 28a executing a computer program stored in the storage device 28b.

[0018] Note that the controller 28 may be formed of dedicated hardware for executing each of the information processes described below. For example, the controller 28 may include a functional logic circuit set in a general-purpose semiconductor integrated circuit. For example, the controller 28 may have a programmable logic device (PLD) such as a field-programmable gate array (FPGA).

[0019] The terminal device 3 is an information processing device used by a person outside the vehicle 2 (for example, a person remote from the vehicle 2). The terminal device 3 may be a portable information terminal that can be carried by a person outside the vehicle 2, a small computer that is easy to carry, or an installed desktop computer. The terminal device 3 includes a communication device 30, an HMI 31, a microphone 32, and a controller 33.

[0020] The communication device 30 provides a communication function between the terminal device 3 and an external device. The communication method by the communication device 30 may be, for example, wireless communication using a public mobile communication network, satellite communication, or the like. The terminal device 3 transmits and receives data to and from the information processing device 20 of the vehicle 2 through the communication device 40.

[0021] The HMI 31 is an interface device that exchanges information between the user of the terminal device 3 (that is, a person outside the vehicle 2) and the terminal device 3. For example, the HMI 31 may include a display device (for example, a touch panel display) that displays captured images by the external camera 22 and the internal camera 21 received from the information processing device 20. Also, for example, the HMI 31 may include a speaker or a buzzer that presents auditory information. The HMI 31 may also include an interface (touch panel, button, switch, etc.) that receives operation inputs from the user of the terminal device 3.

[0022] The microphone 32 acquires sound signals around the terminal device 3. The terminal device 3 acquires voice data of the user of the terminal device 3 by the microphone 32. The controller 33 is an electronic control unit that controls the operation of the terminal device 3. The controller 33 includes a processor 34 and peripheral components such as a storage device 35. The processor 34 may be, for example, a CPU or an MPU.

[0023] The storage device 35 may include semiconductor storage devices, magnetic storage devices, optical storage devices, etc. The storage device 35 may include registers, cache memory, and memory such as ROM and RAM used as main memory. The functions of the controller 33 described below are realized, for example, by the processor 34 executing a computer program stored in the storage device 35.

[0024] Server device 4 is an information processing device that collects or stores video footage captured by the external camera 22 and internal camera 21 of the information processing device 20. Server device 4 includes a communication device 40 and a controller 41. The communication device 40 provides communication functionality between the server device 4 and external devices. The communication method used by the communication device 40 may be, for example, wireless communication over a public mobile network, satellite communication, or vehicle-to-infrastructure communication. The server device 4 transmits and receives data to and from the information processing device 20 of the vehicle 2 using the communication device 40.

[0025] The controller 41 is an electronic control unit that controls the operation of the server device 4. The controller 41 includes a processor 42 and peripheral components such as a storage device 43. The processor 42 may be, for example, a CPU or an MPU. The storage device 43 may include semiconductor storage devices, magnetic storage devices, optical storage devices, etc. The storage device 43 may include registers, cache memory, and memory such as ROM and RAM used as main memory. The functions of the controller 41 described below are realized, for example, by the processor 42 executing a computer program stored in the storage device 43.

[0026] Next, the functions of the controller 28 of the information processing device 20 will be described. Figure 2 is a block diagram of an example of the functional configuration of the controller 28. The controller 28 includes a video information acquisition unit 50, an audio information acquisition unit 51, a video analysis unit 52, an audio analysis unit 53, a timing determination unit 54, an editing unit 55, a storage unit 56, and an HMI control unit 57. The video information acquisition unit 50 acquires internal video captured by the internal camera 21 and external video captured by the external camera 22.

[0027] The audio information acquisition unit 51 acquires the sound signal inside the vehicle 2 detected by the microphone 23. The video analysis unit 52 analyzes the internal video captured by the internal camera 21 to detect at least one of the actions or states of a person inside the vehicle 2. For example, the video analysis unit 52 may detect a person pointing their finger inside the vehicle 2. Alternatively, the video analysis unit 52 may detect the direction of gaze, the orientation of the face, or facial expression as a state of a person inside the vehicle 2. The video analysis unit 52 outputs the detection result of at least one of the actions or states of a person inside the vehicle 2 to the timing determination unit 54 and the editing unit 55.

[0028] The voice analysis unit 53 analyzes the sound signals inside the vehicle 2 detected by the microphone 23. For example, the voice analysis unit 53 recognizes the content of speech spoken by a person inside the vehicle 2 from the voice data of that person contained in the sound signals inside the vehicle 2. For example, the voice analysis unit 53 may recognize the voice data of the person inside the vehicle 2 and convert it into linguistic information such as text. The voice analysis unit 53 may analyze the converted linguistic information using natural language processing and extract the content of speech spoken by the person inside the vehicle 2.

[0029] The audio analysis unit 53 also detects the intensity level of the voices of people inside the vehicle 2 based on the sound signals inside the vehicle 2. The voice analysis unit 53 outputs the extracted speech content of the person inside the vehicle 2 to the video analysis unit 52, the timing determination unit 54, and the editing unit 55. It also outputs the detection result of the voice intensity level of the person inside the vehicle 2 to the timing determination unit 54.

[0030] The video analysis unit 52 may determine whether the video of the spoken content output from the audio analysis unit 53 is included in the external video. For example, if the spoken content output from the audio analysis unit 53 is "Oh, it's a fire truck," the video analysis unit 52 may determine whether the video of the fire truck is included in the external video. For example, the video analysis unit 52 may use an image recognition learning model to recognize objects in the external video and determine whether or not the video of the spoken content output from the audio analysis unit 53 is included in the external video. If the video of the spoken content is included in the external video, the video analysis unit 52 may determine the pixel range within the external video that is occupied by the video of the spoken content. The video analysis unit 52 may also output these determination results to the editing unit 55.

[0031] The timing determination unit 54 determines at least one of the start timing and end timing of shooting by the external camera 22, based on at least one of the actions or state of a person inside the vehicle 2 detected by the video analysis unit 52, or the content of speech of a person inside the vehicle 2 extracted by the voice analysis unit 53, or the intensity level of the voice of a person inside the vehicle 2 detected by the voice analysis unit 53.

[0032] The timing determination unit 54 outputs a timing signal to the video information acquisition unit 50 to start or stop shooting with the external camera 22. The video information acquisition unit 50 starts or stops shooting with the external camera 22 based on the timing signal output from the timing determination unit 54. For example, the timing determination unit 54 may start recording with the external camera 22 when it detects an action or state of a person inside the vehicle 2, such as a pointing gesture or a smile. Alternatively, the timing determination unit 54 may start recording with the external camera 22 when it detects a specific speech content or a sound with an intensity above a threshold level from a person inside the vehicle 2.

[0033] Furthermore, the timing determination unit 54 may acquire information about locations and features around the current location of the vehicle 2 based on the current location information of the vehicle 2 acquired from the positioning device 25 and the road map data of the map DB 24. The timing determination unit 54 may start taking pictures with the external camera 22 when the current location of the vehicle 2 approaches a location or feature mentioned in the speech of a person inside the vehicle 2.

[0034] Furthermore, the timing determination unit 54 may store the speech content of a person inside the vehicle 2 and extract speech content that appears at a frequency above a threshold in the conversation of that person. Hereinafter, speech content that appears at a frequency above a threshold will be referred to as "frequently occurring speech content". The timing determination unit 54 may also output the extracted frequently occurring speech content to the video analysis unit 52. The video analysis unit 52 may determine whether or not video of frequently spoken content is included in the external video and output the determination result to the editing unit 55.

[0035] The editing unit 55 and the video analysis unit 52 identify the direction in which the object of interest of the person inside the vehicle 2 is located as the target direction, based on at least one of the actions, state, or speech content of the person inside the vehicle 2. For example, if a person inside vehicle 2 makes a gesture pointing to the front right, or turns their face or gaze towards the front right, the target direction can be identified as the front right of vehicle 2. Also, if a person inside vehicle 2 says, "Look to the right," the target direction can be identified as the front right of vehicle 2.

[0036] The editing unit 55 extracts a portion image from the external image captured by the external camera 22, which is a wide-angle camera, that covers a narrower shooting range centered on the target direction, and acquires it as the target image, which captures the scenery in the target direction (i.e., captures the object of interest of the person inside the vehicle 2). The editing unit 55 may store the acquired image in the storage unit 56 or output it to the display device of the HMI 26. The HMI control unit 57 outputs the acquired image acquired by the editing unit 55 to the display device of the HMI 26. The HMI control unit 57 and HMI 26 are examples of the "reproduction device" described in the claims.

[0037] Furthermore, if the video analysis unit 52 determines that the video contains footage of a person speaking inside vehicle 2, the editing unit 55 may extract the video containing the pixel range within the external video that the video of the spoken content occupies, and acquire it as the target video. Furthermore, if the video analysis unit 52 determines whether or not the video of the frequently departing conversation content is included in the external video, the editing unit 55 may start recording the target video at the time it is determined that the video of the frequently departing conversation content is included in the external video. In this case, the editing unit 55 may automatically start recording the target video even if the person inside the vehicle 2 is not facing the direction in which the subject of the frequently departing conversation content is being filmed.

[0038] The editing unit 55 may acquire internal video captured by the internal camera 21 in synchronization with the target video extracted from the external video. For example, the editing unit 55 may store the target video and the internal video in the storage unit 56 in synchronization, or output them to the display device of the HMI 26. For example, the editing unit 55 may store a composite video, which is a combination of the target video and the internal video, in the storage unit 56, or output it to the display device of the HMI 26. In the following description, the composite video, which is a combination of the target video and the internal video, may be simply referred to as "composite video".

[0039] The editorial department 55 may transmit the acquired video, internal video, synthesized video, or audio data of people inside vehicle 2 to terminal devices 3 or server devices 4 via the communication device 27. For example, the editorial department 55 may share the acquired video, internal video, synthesized video, or audio data of people inside vehicle 2 with users of terminal devices 3.

[0040] The controller 33 of terminal device 3 may play back the acquired video, internal video, synthesized video, or audio data of people inside vehicle 2 received from the information processing device 20 on the display device or speaker of HMI 31, or store it in the storage device 35. The controller 41 of server device 4 may store the acquired video, internal video, synthesized video, or audio data of people inside vehicle 2 received from the information processing device 20 in the storage device 43.

[0041] The information processing device 20 and the terminal device 3 may synchronize the playback of the target video, internal video, and composite video. For example, when the terminal device 3 plays back acquired video, internal video, or composite video that was previously received from the information processing device 20 and stored in the storage device 35 on the HMI 31 of the terminal device 3, the information processing device 20 may play back the same video on the HMI 26 in synchronization with the terminal device 3. For example, when playing back previously stored target video, internal video, or composite video on the HMI 31 of the terminal device 3, the controller 33 of the terminal device 3 may transmit identification information of the video to be played back and current playback position information to the information processing device 20.

[0042] The HMI control unit 57 of the information processing device 20 may read from the storage unit 56 the same video being played back on the HMI 31 of the terminal device 3, based on the identification information received from the terminal device 3. The HMI control unit 57 may play back the video read from the storage unit 56 in synchronization with the video playback on the HMI 31 of the terminal device 3, based on the playback position information received from the terminal device 3.

[0043] When the terminal device 3 and the information processing device 20 synchronize playback of acquired video, internal video, and composite video, the HMI control unit 57 may transmit the voice data of a person inside the vehicle 2, currently acquired by the microphone 23 of the vehicle 2, to the terminal device 3 via the communication device 27. The controller of the terminal device 3 may output the received voice data from the HMI 31. Conversely, the controller 33 of terminal device 3 may transmit the voice data of the terminal device 3 currently acquired by the microphone 32 to the information processing device 20 via the communication device 30. The HMI control unit 57 may output the received voice data from the HMI 26. In other words, when the terminal device 3 and the information processing device 20 synchronize the playback of acquired video, internal video, and synthesized video, they may exchange (transfer) the current voice data of the person inside the vehicle 2 and the current voice data of the user of the terminal device 3.

[0044] (operation) Figure 3 is a flowchart of an example of an information processing method in an embodiment. In step S1, the video information acquisition unit 50 acquires the internal video captured by the internal camera 21. In step S2, the voice information acquisition unit 51 acquires voice data of the person in vehicle 2 detected by the microphone 23.

[0045] In step S3, the video analysis unit 52 and the audio analysis unit 53 analyze the internal video and audio data. In step S4, the editorial department 55 identifies the direction in which the person's object of interest inside vehicle 2 is located as the target direction, based on the results of the analysis of the internal video and audio data. In step S5, the editorial department 55 extracts a portion of the image within the shooting range in the target direction from the external video captured by the external camera 22 and acquires it as the target video. After that, the processing ends.

[0046] (modified version) The above description mentions an example in which a portion of the external image captured by the wide-angle external camera 22 is extracted from the external image, focusing on a narrower shooting range centered on the target direction, and acquired as the target image. It also describes an example in which the image containing the pixels of the speech of a person inside the vehicle 2 is extracted from the external image and acquired as the target image.

[0047] Alternatively, an external camera 22 with adjustable shooting direction (i.e., the optical axis direction of the shooting optical system) may be used. The controller 28 may direct the shooting direction of the external camera 22 toward the target direction identified by the editorial department 55 to capture the target video. Alternatively, the shooting direction of the external camera 22 may be directed toward the direction where the target of the person's speech inside the vehicle 2 is located to capture the target video.

[0048] (Effects of the embodiment) (1) The controller performs the following processes: (1) Capture a person inside the moving object with the first camera or acquire audio data of the person; (2) Based on the person's video or audio data, identify the direction in which the person's object of interest is located as the target direction; and (3) Acquire video footage of the scenery outside the moving object in the target direction using the second camera mounted on the moving object. This improves the success rate of capturing good video footage of external objects of interest that a person inside the moving object is interested in, using cameras mounted on the moving object.

[0049] (2) The controller may determine the target direction based on the actions or state of the person detected from the person's video footage, or the content of the person's speech detected from the person's audio data. This allows the system to identify the target direction based on video and audio data of a person. (3) The controller may detect the object mentioned in the speech content detected from the person's voice data from the external video footage captured by the second camera, and either extract the image of the object from the external video or adjust the shooting direction of the second camera so that the object in the external video can be captured. This allows the camera mounted on the mobile device to capture images of objects mentioned in the speech of a person inside the device.

[0050] (4) The controller may determine at least one of the start timing and end timing of recording by the second camera based on the person's actions or state detected from the person's recorded video, or the person's voice. This allows for filming the object of interest of a person inside a moving object.

[0051] (5) A person's action or state, such as a person pointing or smiling, may be used as a trigger to start filming with the second camera. This makes it possible to determine the timing for photographing the object of interest of a person inside a moving object. (6) A person's voice may be used as a trigger to start recording with the second camera if it contains specific speech content or sounds with an intensity above a certain threshold. This makes it possible to determine the timing for photographing the object of interest of a person inside a moving object.

[0052] (7) The controller may start recording with the second camera when it detects from a person's voice that the frequency of occurrence is above a threshold, and the object indicated by the detected speech content is detected from the external video, which is the video captured by the second camera. This makes it possible to determine the timing for recording video of the subject of interest to the person inside the moving object.

[0053] (8) The controller may acquire the current location information of the moving object and start taking pictures with the second camera when the current location of the moving object approaches a location mentioned in the speech content detected from the person's voice data. This makes it possible to determine the timing for photographing the object of interest of a person inside a moving object. (9) The controller may synchronize the video footage from the first camera and the video footage from the second camera. This allows for the simultaneous acquisition of video footage of the object of interest of a person inside a moving object, as well as video footage of that person at the time the object of interest was filmed.

[0054] (10) The controller may transmit the video footage captured by the first camera, the video footage captured by the second camera, and the acquired audio data of the person to an external device outside the mobile body. This allows these video and audio data to be shared with third parties. (11) The video footage captured by the first camera or the video footage captured by the second camera may be played back in sync between a playback device inside the mobile vehicle and an external device. This allows people inside the vehicle and people outside the vehicle to view these videos simultaneously.

[0055] (12) The controller and the external device store at least one of the video footage captured by the first camera and the video footage captured by the second camera, the external device acquires the user's voice data, and the controller and the external device may exchange the currently acquired voice data when playing back at least one of the video footage captured by the first camera and the video footage captured by the second camera in sync between the playback device inside the mobile device and the external device. This allows for real-time conversational communication when people inside and outside the vehicle simultaneously view these videos. [Explanation of Symbols]

[0056] 1...Shooting system, 2...Vehicle, 3...Terminal device, 4...Server device, 20...Information processing device, 21...Internal camera, 22...External camera, 23, 32...Microphone, 24...Map database, 25...Positioning device, 26, 31...Human-machine interface, 27, 30, 40...Communication device, 28, 33, 41...Controller, 28a, 34, 42...Processor, 28b, 35, 43...Storage device, 50...Video information acquisition unit, 51...Audio information acquisition unit, 52...Video analysis unit, 53...Audio analysis unit, 54...Timing determination unit, 55...Editing unit, 56...Storage unit, 57...HMI control unit

Claims

1. A process of either photographing a person inside a moving object with a first camera, or acquiring audio data of the person, A process to identify the direction in which the person's object of interest is located as the target direction based on the person's video or audio data, The process of acquiring video footage of the scenery outside the mobile body in the direction of the target using a second camera mounted on the mobile body, An information processing method characterized by having a controller execute a command.

2. The information processing method according to claim 1, characterized in that the controller identifies the target direction based on the actions or state of the person detected from the captured video of the person, or the content of speech detected from the voice data of the person.

3. The aforementioned controller, The object mentioned in the speech content detected from the voice data of the person is detected from the external video footage captured by the second camera. To extract the image of the target from the external image, or to adjust the shooting direction of the second camera to photograph the target in the external image, The information processing method according to feature 1.

4. The information processing method according to claim 1, characterized in that the controller determines at least one of the start timing and end timing of shooting by the second camera based on the actions or state of the person detected from the person's captured video footage, or the person's voice.

5. The information processing method according to claim 4, characterized in that the actions or state of the person are such that the person makes a pointing gesture or smiles, which triggers the start of shooting by the second camera.

6. The information processing method according to claim 4, characterized in that a specific utterance or an intensity above a threshold is used as the trigger for the start of shooting by the second camera as the voice of the person.

7. The aforementioned controller, From the voice of the aforementioned person, we detect speech content that appears at a frequency above a threshold, When the object indicated by the detected speech content is detected in the external video footage captured by the second camera, recording of the external video footage is started. The information processing method according to feature 1.

8. The aforementioned controller, The current location information of the moving object is acquired, When the current position of the moving object approaches the location mentioned in the speech content detected from the voice data of the person, the second camera starts taking pictures. The information processing method according to feature 4.

9. The information processing method according to any one of claims 1 to 8, characterized in that the controller acquires the captured video of the first camera and the captured video of the second camera in a synchronized manner.

10. The information processing method according to any one of claims 1 to 8, characterized in that the controller transmits the video footage captured by the first camera, the video footage captured by the second camera, and the acquired voice data of the person to an external device outside the mobile body.

11. The information processing method according to claim 10, characterized in that the video footage captured by the first camera or the video footage captured by the second camera is played back in sync between the playback device inside the mobile body and the external device.

12. The controller and the external device store at least one of the images captured by the first camera and the images captured by the second camera. The external device acquires the user's voice data, The controller and the external device exchange the currently acquired audio data when synchronously playing back at least one of the video footage captured by the first camera and the video footage captured by the second camera between the playback device in the mobile vehicle and the external device. The information processing method according to feature 10.

13. A first camera that photographs the inside of the moving object, A microphone for detecting sound signals inside the vehicle of the aforementioned mobile vehicle, A second camera that photographs the outside of the aforementioned moving object, A controller that performs the following processes: photographing a person inside the mobile body with the first camera or acquiring the person's voice with the microphone; identifying the direction in which the person's object of interest is located as a target direction based on the person's image or speech; and acquiring an image of the scenery outside the mobile body in the target direction with a second camera mounted on the mobile body. An information processing device characterized by comprising:

14. A process of either photographing a person inside a moving object with a first camera, or acquiring the voice of the person, A process to identify the direction in which the object of interest of the person is located as the target direction based on the person's video or speech, The process of acquiring video footage of the scenery outside the mobile body in the direction of the target using a second camera mounted on the mobile body, A computer program characterized by causing a controller to execute a command.

Images