Mobile device

Abstract

To provide a mobile apparatus which can effectively collect sounds of surroundings.SOLUTION: A mobile apparatus 10 according to the present invention has a main body part 12 having a movement mechanism 18, a sound collection unit 20 provided at the main body part 12 and capable of collecting sounds of surroundings, a control unit 24 for controlling the movement mechanism 18 and the sound collection unit 20, and a ventilation apparatus for cooling the control unit 24 by ventilation. If noise of the sound collected by the sound collection unit 20 is equal to or larger than a predetermined value, the control unit 24 stops the ventilation apparatus.SELECTED DRAWING: Figure 1

Description

Technical Field

[0006] , , ,

[0005] , ,

[0001] The present invention relates to a mobile device.

Background Art

[0002] Patent Document 1 discloses an interactive system including a touch panel, a voice input device, and a voice output device. Specifically, in the interactive system described in Patent Document 1, after the start of guidance, the interaction mode is switched and controlled for each interaction according to the user's situation.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, a mobile device equipped with an interactive system such as the above Patent Document 1 is known. Such a mobile device needs to control, in addition to the interactive system, a moving mechanism, a sensor for detecting the surroundings, etc., and the temperature of the control unit may rise due to the processing load. As a countermeasure, a blowing device such as a cooling fan can be provided, but there is a possibility that the operation sound of the blowing device may affect the sound collection of the voice.

[0005] The present invention provides a mobile device capable of effectively collecting ambient sound in a mobile device capable of collecting ambient sound.

Means for Solving the Problems

[0006] The mobile device according to claim 1 includes a main body portion having a moving mechanism, a sound collection portion provided in the main body portion and capable of collecting ambient sound, a control portion for controlling the moving mechanism and the sound collection portion, and a blowing device for cooling the control portion by blowing air. The sound collection unit identifies the audio data collected by a person from the audio data collected by the sound collection unit, and distinguishes between the audio data collected by a person and other noise.If the noise level exceeds a predetermined value, the control unit will stop the blower.

[0007] The mobile device according to claim 1 comprises a main body with a moving mechanism and a sound collection unit. This allows it to move to any location, such as where a person is located, and collect sound. Furthermore, the control unit that controls the moving mechanism and the sound collection unit is cooled by airflow from a blower. This suppresses the rise in the heat of the control unit even when the processing load on the control unit increases.

[0008] Furthermore, if the noise level of the sound collected by the sound collection unit exceeds a predetermined value, the control unit will stop the blower. This suppresses the generation of noise caused by the operation of the blower, allowing for effective sound collection.

[0009] The mobile device according to claim 2 is, in claim 1, The device comprises a housing that constitutes the main body, and is configured to be transformable from a normal mode in which the backrest and seat are stored inside the housing in a folded state, to a wheelchair mode in which the backrest and seat are extended and the user can sit on the seat and use it as a wheelchair.

[0015] Claim 3 The mobile device relating to claim 1 estimates the meaning of a sound by inputting the sound data collected by the sound collection unit to a machine learning model that has been trained based on the sound data and the meaning that the sound data represents.

[0016] Claim 3 In the mobile device related to this, the meaning of the speech can be estimated with greater accuracy than when the meaning is estimated based solely on stored speech, by using a machine learning model to estimate the meaning of the speech. [Effects of the Invention]

[0017] According to the mobile device of the present invention, it is possible to effectively collect ambient sounds in a mobile device capable of collecting ambient sounds. [Brief explanation of the drawing]

[0018] [Figure 1] This is a perspective view of the mobile device according to the embodiment, showing the state in normal mode. [Figure 2] It is a perspective view of a mobile device according to an embodiment, showing the state in wheelchair mode. [Figure 3] It is a block diagram showing the hardware configuration of the mobile device in the embodiment. [Figure 4] It is a block diagram showing the functional configuration of the mobile device according to the embodiment. [Figure 5] It is a flowchart showing an example of the flow of sound collection processing in the embodiment. [Figure 6] It is a flowchart showing an example of the flow of notification processing in the embodiment.

Embodiments for Carrying Out the Invention

[0019] The mobile device 10 according to the embodiment will be described with reference to the drawings. In each figure, the arrows UP, FR, and RH shown as appropriate indicate the upward, forward (travel direction), and rightward directions of the mobile device 10, respectively. Hereinafter, when simply describing the front-rear, left-right, and up-down directions, unless otherwise specified, they indicate the front-rear, left-right, and up-down of the mobile device 10.

[0020] (Overall Configuration of Mobile Device 10)

[0021] As shown in FIG. 1, the mobile device 10 of the present embodiment includes a main body portion 12. The main body portion 12 is formed in a substantially rectangular parallelepiped shape with a vertically long overall shape and includes a housing 14.

[0022] The housing 14 constituting the main body portion 12 includes a front cover portion 14F and a rear cover portion 14B. The front cover portion 14F and the rear cover portion 14B are arranged to face each other in the front-rear direction and are each formed in a vertically long plate shape with the front-rear direction as the plate thickness direction.

[0023] Between the front cover part 14F and the rear cover part 14B, a right cover part 14R and a left cover part 14L are provided. The right cover part 14R constitutes the right half of the housing 14 and has a flat part extending in the front-rear direction and the up-down direction. Further, the right cover part 14R includes a part extending from the upper end of the flat part to the left side to connect the upper ends of the front cover part 14F and the rear cover part 14B, and a part extending from the lower end of the flat part to the left side to connect the lower ends of the front cover part 14F and the rear cover part 14B.

[0024] On the other hand, the left cover part 14L constitutes the left half of the housing 14 and has a flat part extending in the front-rear direction and the up-down direction. Further, the left cover part 14L includes a part extending from the upper end of the flat part to the right side to connect the upper ends of the front cover part 14F and the rear cover part 14B, and a part extending from the lower end of the flat part to the right side to connect the lower ends of the front cover part 14F and the rear cover part 14B. That is, the left cover part 14L and the right cover part 14R are in a bilaterally symmetric shape. In addition, in this embodiment, as an example, the right cover part 14R is fixed to the front cover part 14F and the rear cover part 14B. In contrast, the left cover part 14L is configured to be movable between a normal position in contact with the front cover part 14F and the rear cover part 14B and a deployed position on the left side of the front cover part 14F and the rear cover part 14B.

[0025] Here, legs 16 are provided on the right cover part 14R and the left cover part 14L, respectively. Specifically, one leg 16 is provided at the lower part of the right cover part 14R in the front and rear, and one leg 16 is provided at the lower part of the left cover part 14L in the front and rear. Therefore, the moving device 10 of this embodiment is provided with four legs 16.

[0026] Rollers 18 as moving mechanisms are provided on the four legs 16, respectively. Since the four rollers 18 are each configured to be rotatable in all directions, the moving device 10 can move in all directions without changing its posture.

[0027] The front cover portion 14F of the housing 14 is equipped with a microphone 20 as a sound collection unit, a speaker 21 as a sound output unit, and a camera 22 as a sensor. The microphone 20 is configured to collect sounds from the surrounding area of ​​the main unit 12. A directional microphone may be used as the microphone 20. In this case, it is possible to effectively collect sounds spoken from the front of the mobile device 10 in particular. Furthermore, although this embodiment describes a configuration with one microphone 20, a configuration with multiple microphones 20 may be adopted. For example, if directional microphones 20 are mounted on the front, back, left, and right of the main unit 12, the direction from which the speech was spoken can be identified.

[0028] The speaker 21 is mounted on the main unit 12 and configured to output sound to the surroundings. A directional speaker may be used as the speaker 21. In this case, notification can be sent only to people in front of the mobile device 10. Furthermore, although this embodiment describes a configuration with one speaker 21, a configuration with multiple speakers 21 may also be used. For example, if directional speakers 21 are mounted on the front, back, left, and right of the main unit 12, sound can be output in any direction without changing the orientation of the mobile device 10.

[0029] The camera 22 is configured to detect objects around the main unit 12. In addition to the camera 22, other sensors for surrounding detection, such as an ultrasonic sensor and LiDAR (Light Detection and Ranging, or Laser Imaging Detection and Ranging), may also be installed. It is particularly preferable to use sensors that can detect obstacles over a wide area around the main unit 12.

[0030] The roller 18, microphone 20, speaker 21, and camera 22 are controlled by a control unit 24 located inside the housing of the mobile device 10. The main unit 12 also contains a battery (not shown), and each function operates when power is supplied from this battery.

[0031] In this embodiment, the mobility device 10 is configured to be transformable from the normal mode shown in Figure 1 to the wheelchair mode shown in Figure 2, as an example.

[0032] As shown in Figure 2, in wheelchair mode, the left cover portion 14L of the housing 14 is moved to the deployed position. For example, with the left cover portion 14L unlocked, the rollers 18 provided on the left cover portion 14L rotate, causing the left cover portion 14L to move to the deployed position.

[0033] In wheelchair mode, the backrest 32 and seat 34 are stretched between the right cover 14R and the left cover 14L. The backrest 32 and seat 34 are made of, for example, fabric, and are stored folded inside the housing 14 in normal mode.

[0034] Furthermore, a rod (not shown) is provided at the upper end of the backrest portion 32, which is extendable and retractable with the left-right direction of the main body portion 12 as its longitudinal direction. One end of the rod is fixed to the right cover portion 14R, and the other end is fixed to the left cover portion 14L. Similar rods are also provided at the connection portion between the backrest portion 32 and the seat portion 34, and at the front end of the seat portion 34. Each rod is biased in the direction of extension, so when the left cover portion 14L is deployed, the rods extend, and the backrest portion 32 and the seat portion 34 are deployed.

[0035] Below the seat portion 34, a rod 36 is provided that connects the right cover portion 14R and the left cover portion 14L. In addition, a roughly crank-shaped right-side frame 38R is provided at the front of the right cover portion 14R, and the right-side footrest 40R is attached to the right-side frame 38R. Similarly, a roughly crank-shaped left-side frame 38L is provided at the front of the left cover portion 14L, and the left-side footrest 40L is attached to the left-side frame 38L.

[0036] The wheelchair mode of the mobility device 10 in this embodiment is configured as described above, and is set up so that the user can use it as a wheelchair while seated on the seat portion 34.

[0037] (Hardware configuration of mobile device 10)

[0038] Figure 3 is a block diagram showing the hardware configuration of the mobile device 10. As shown in Figure 3, the control unit 24 constituting the mobile device 10 includes a CPU (Central Processing Unit: processor) 40, ROM (Read Only Memory) 42, RAM (Random Access Memory) 44, storage 46, communication I / F (communication interface) 48, and input / output I / F (input / output interface) 50. Each component is connected to the others via a bus 52 so as to be able to communicate with each other.

[0039] The CPU 40 is a central processing unit that executes various programs and controls various components. Specifically, the CPU 40 reads programs from the ROM 42 or storage 46 and executes them using the RAM 44 as a working area. The CPU 40 controls each of the above components and performs various calculations according to the programs recorded in the ROM 42 or storage 46.

[0040] ROM 42 stores various programs and data. RAM 44 temporarily stores programs or data as a working area. Storage 46 is composed of an HDD (Hard Disk Drive) or SSD (Solid State Drive) and stores various programs, including the operating system, and various data. In this embodiment, ROM 42 or storage 46 stores programs and various data for performing sound collection processing and notification processing, etc.

[0041] Communication I / F48 is an interface for the mobile device 10 to communicate with external servers and other devices, and standards such as CAN (Controller Area Network), Ethernet (registered trademark), LTE (Long Term Evolution), FDDI (Fiber Distributed Data Interface), and Wi-Fi (registered trademark) are used.

[0042] The input / output interface 50 is electrically connected to the roller 18, microphone 20, speaker 21, camera 22, and fan 26, which acts as a blower. The fan 26 is located near the control unit 24 and cools the control unit 24 by blowing air onto it. For example, the control unit 24 activates the fan 26 when the temperature of the control unit 24, as detected by a temperature sensor (not shown), rises above a predetermined temperature. Conversely, the fan 26 is stopped when the temperature of the control unit 24 falls below the predetermined temperature.

[0043] (Functional configuration of the mobile device 10)

[0044] The mobile device 10 implements various functions using the hardware resources shown in Figure 3. The functional configuration implemented by the mobile device 10 will be explained with reference to Figure 4.

[0045] As shown in Figure 4, the mobile device 10 is configured as follows: a peripheral information acquisition unit 62, a mobile mechanism control unit 64, a person detection unit 66, a microphone control unit 68, a voice estimation unit 70, a notification unit 72, a mode switching unit 74, and a fan control unit 76. Each of these functional configurations is realized by the CPU 40 reading and executing a program stored in the ROM 42 or storage 46.

[0046] The peripheral information acquisition unit 62 acquires peripheral information of the mobile device 10. Specifically, the peripheral information acquisition unit 62 acquires peripheral information obtained by sensors such as the camera 22.

[0047] The movement mechanism control unit 64 controls the rollers 18, which act as the movement mechanism. By controlling the four rollers 18, the device can be moved in any direction. The main body 12 can also be rotated. Furthermore, if the noise level of the sound collected by the microphone 20 exceeds a predetermined value, the movement mechanism control unit 64 may move the device toward the person detected by the camera 22.

[0048] The person detection unit 66 determines whether or not a person is present around the mobile device 10 based on the surrounding information of the mobile device 10 acquired by the surrounding information acquisition unit 62. In addition, if a person to be notified has been set in advance, the person detection unit 66 may determine whether or not a person in the vicinity of the mobile device 10 is a person to be notified.

[0049] The microphone control unit 68, when certain conditions are met, uses the microphone 20 to collect sounds from the surrounding area of ​​the mobile device 10 and transmits them to the control unit 24. For example, the microphone control unit 68 may start collecting sounds when there are people around the mobile device 10. Alternatively, if the microphone 20 is directional, it may start collecting sounds when there are people in front of the mobile device 10. Furthermore, the microphone control unit 68 may stop collecting sounds with the microphone 20 when there are no people around the mobile device 10.

[0050] The speech estimation unit 70 estimates the content of what a person said from the sound collected by the microphone 20. For example, the speech estimation unit 70 may estimate what a person said by inputting the sound data collected by the microphone 20 into a machine learning model that has been trained based on the sound data and the meaning that the sound data represents. In this case, the machine learning model is stored in the storage 46 or an external server.

[0051] Furthermore, the voice estimation unit 70 analyzes the noise in the collected voice data. Specifically, the voice estimation unit 70 distinguishes between human speech and other noise within the voice data. The voice estimation unit 70 also determines whether the noise level is above a predetermined value.

[0052] The notification unit 72 notifies the target of notification via the speaker 21. For example, if the mobile device 10 receives instructions from a person nearby, it recognizes the person who gave the instructions as the target of notification. The notification unit 72 then notifies the target person by outputting a sound from the speaker 21 with the content of the response to the instructions. The notification content may be selected from patterns stored in storage 46 or the like in advance, corresponding to the instructions.

[0053] The mode switching unit 74 switches between normal mode and wheelchair mode. Specifically, the mode switching unit 74 switches from normal mode to wheelchair mode when it becomes necessary to switch to wheelchair mode. After moving the user to a designated location in wheelchair mode, if the user stands up from the wheelchair and moves away from the mobility device 10, the mode switching unit 74 switches back to normal mode.

[0054] The fan control unit 76 controls the operation and stopping of the fan 26. Specifically, the fan control unit 76 stops the fan 26 if the noise level of the sound picked up by the microphone 20 exceeds a predetermined value. The fan control unit 76 may also restart the fan 26 when it is no longer necessary to pick up sound.

[0055] (action) Next, the operation of this embodiment will be explained.

[0056] (Sound collection processing) An example of the sound collection process by the mobile device 10 will be explained using the flowchart shown in Figure 5. This process is achieved by the CPU 40 reading and executing a program stored in the ROM 42 or storage 46. In this embodiment, as an example, the sound collection process is performed at predetermined intervals.

[0057] In step S102, the CPU 40 acquires surrounding information. Specifically, the CPU 40 acquires surrounding information detected by the camera 22 and other sensors through the functions of the surrounding information acquisition unit 62.

[0058] In step S104, the CPU 40 determines whether or not a person has been recognized. Specifically, if the CPU 40 detects a person from the image information acquired in step S102 by the function of the person detection unit 66, it determines that a person is present around the mobile device 10 and proceeds to the processing in step S106. If the CPU 40 does not detect a person around the mobile device 10 in step S104, it terminates the sound collection process. In other words, in this embodiment, as an example, sound collection is not performed when there is no person around the mobile device 10.

[0059] The CPU 40 starts sound collection in step S106. Specifically, the CPU 40 turns on the microphone 20 using the function of the microphone control unit 68. If the microphone 20 is already turned on immediately after the mobile device 10 is started up, the microphone control unit 68 starts sound collection in step S106 by acquiring the audio data collected by the microphone 20.

[0060] In step S108, the CPU 40 determines whether the noise level is above a predetermined value. Specifically, the CPU 40 extracts noise from the sound data surrounding the mobile device 10 collected by the sound estimation unit 70 and determines whether the noise level is above a predetermined value.

[0061] If the CPU 40 determines in step S108 that the noise level is above a predetermined value, it proceeds to the process in step S110. If the CPU 40 determines in step S108 that the noise level is not above a predetermined value, that is, if the noise is below a predetermined value, it proceeds to the process in step S116. The processes from step S110 onwards will be described below, followed by the processes from step S116 onwards.

[0062] In step S110, the CPU 40 stops the fan 26. Specifically, if the fan 26 is operating due to the function of the fan control unit 76, the CPU 40 stops the fan 26. Furthermore, when the fan 26 is stopped, the fan control unit 76 maintains the stopped state of the fan 26. In other words, even if the temperature of the control unit 68 rises, the fan 26 will remain stopped without operating.

[0063] In step S112, the CPU 40 determines whether the noise level is above a predetermined value. Specifically, the CPU 40 extracts noise from the audio data surrounding the mobile device 10 collected by the audio estimation unit 70 and determines whether the noise level is above a predetermined value. The audio data from which noise is extracted is the audio data collected after the fan 26 was stopped in step S110.

[0064] If the CPU 40 determines that the noise level is above a predetermined value even when the fan 26 is stopped, the CPU 40 proceeds to step S114. In step S114, the CPU 40 moves the mobile device 10 toward the person. Specifically, the CPU 40 controls the rollers 18 by the function of the mobile mechanism control unit 64 to move toward the person speaking. At this time, the CPU 40 may move the mobile device 10 by a predetermined distance from its current position. Alternatively, the CPU 40 may move the mobile device 10 until the distance between it and the person is a predetermined distance.

[0065] On the other hand, if the CPU 40 determines in step S108 that the noise is smaller than a predetermined value, it proceeds to the process in step S116. Similarly, if the CPU 40 determines in step S112 that the noise is smaller than a predetermined value, it proceeds to the process in step S116. In step S116, the CPU 40 performs speech analysis. Specifically, the CPU 40 estimates the content of what a person said from the speech collected by the microphone 20 using the function of the speech estimation unit 70.

[0066] (Notification processing) Next, an example of the notification processing flow by the mobile device 10 will be explained using the flowchart shown in Figure 6. This processing is achieved by the CPU 40 reading and executing a program stored in the ROM 42 or storage 46. In this embodiment, as an example, the notification processing is executed when there is content to be notified.

[0067] In step S202, the CPU 40 acquires surrounding information. Specifically, the CPU 40 acquires surrounding information detected by the camera 22 and other sensors through the functions of the surrounding information acquisition unit 62.

[0068] In step S204, the CPU 40 determines whether or not it has recognized the target of notification. Specifically, if the CPU 40 detects the target of notification from the image information acquired in step S202 by the function of the person detection unit 66, it proceeds to the processing in step S206. If the CPU 40 does not detect the target of notification around the mobile device 10 in step S204, it terminates the notification process.

[0069] The CPU 40 starts sound collection in step S206. Specifically, the CPU 40 turns on the microphone 20 using the function of the microphone control unit 68. If the microphone 20 is already turned on immediately after the mobile device 10 is started up, the microphone control unit 68 starts sound collection in step S206 by acquiring the audio data collected by the microphone 20.

[0070] In step S208, the CPU 40 determines whether the noise level is above a predetermined value. Specifically, the CPU 40 extracts noise from the sound data surrounding the mobile device 10 collected by the sound estimation unit 70 and determines whether the noise level is above a predetermined value.

[0071] If the CPU 40 determines in step S208 that the noise level is above a predetermined value, it proceeds to the process in step S210. If the CPU 40 determines in step S208 that the noise level is not above a predetermined value, that is, if it determines that the noise is below a predetermined value, it proceeds to the process in step S212.

[0072] In step S210, the CPU 40 moves the mobile device 10 toward the person. Specifically, the CPU 40 controls the rollers 18 by the function of the movement mechanism control unit 64 to move toward the target of notification. At this time, the CPU 40 may move the mobile device 10 by a predetermined distance from its current position. Alternatively, the CPU 40 may move the mobile device 10 until the distance between it and the target of notification is a predetermined distance.

[0073] CPU 40 initiates notification in step S212. Specifically, CPU 40 uses the function of the notification unit 72 to notify the target via speaker 21.

[0074] As described above, the mobile device 10 according to this embodiment includes a main body 12 equipped with a roller 18 and a microphone 20, which are movement mechanisms. This allows it to move to any location, such as where a person is, and collect sound. In addition, the control unit 24 is cooled by airflow from the fan 26. This prevents the heat of the control unit 24 from rising even when the processing load on the control unit 24 increases.

[0075] Furthermore, in this embodiment, if the noise level of the sound collected by the microphone 20 exceeds a predetermined value, the control unit 24 stops the fan 26. This suppresses the generation of noise caused by the operation of the fan 26, allowing for effective sound collection.

[0076] Furthermore, in this embodiment, the microphone 20 collects sound only when the object detected by the sensor, such as the camera 22, is a person. This prevents sound collection in situations where there is no person present, thereby reducing the processing load on the control unit 24. In this embodiment, in particular, if the noise level exceeds a predetermined value, the sound can be collected more accurately by moving closer to the person speaking.

[0077] Furthermore, in this embodiment, interaction is possible by sending a predetermined notification to the notification target using the function of the notification unit 72. Here, if the speaker wishes to move, the system can be switched to wheelchair mode to support the speaker who has difficulty walking.

[0078] Furthermore, in this embodiment, the meaning of the speech is estimated using a machine learning model. This allows for more accurate estimation of the meaning of the speech than when the meaning is estimated based solely on pre-stored speech.

[0079] Although the mobility device 10 according to the embodiment has been described above, it goes without saying that it can be implemented in various forms without departing from the spirit of the present invention. For example, the mobility device 10 in the above embodiment had a structure that could be switched between a normal mode and a wheelchair mode, but it is not limited to this, and it may have a structure that cannot be switched to wheelchair mode.

[0080] Furthermore, although the mobile device 10 was formed in a substantially rectangular parallelepiped shape in the above embodiment, it is not limited to this and may take other shapes. For example, it may take the shape of a humanoid robot. Moreover, although a structure with four rollers 18 was described as the mobile mechanism in this embodiment, it is not limited to this. For example, it may be equipped with a mobile mechanism such as caterpillar tracks. It may also be equipped with a bipedal or quadrupedal mobile mechanism.

[0081] Furthermore, although the above embodiment is configured to store various data in the storage 46, it is not limited to this. For example, non-temporary recording media such as CDs (Compact Disks), DVDs (Digital Versatile Disks), and USB (Universal Serial Bus) memory may be used as the storage unit. In this case, various programs and data will be stored in these recording media.

[0082] Furthermore, the processing flow described in the above embodiment is merely an example, and unnecessary steps may be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose. [Explanation of symbols]

[0083] 10 Mobile device 12 Main body 18. Rollers (moving mechanism) 20. Microphone (sound collection unit) 21. Speaker (sound output section) 22 Cameras (Sensors) 24 Control Unit 26. Fan (Air Blower)

Claims

1. A main body equipped with a moving mechanism, The main body is provided with a sound collection unit capable of collecting ambient sounds, The control unit controls the aforementioned moving mechanism and the aforementioned sound collection unit, A blower that cools the control unit by blowing air, It has, A mobile device that distinguishes between human speech and other noise from the sound data collected by the sound collection unit, and if the noise exceeds a predetermined value, the control unit stops the blower.

2. The main body comprises a housing, The mobility device according to claim 1, configured to be transformable from a normal mode in which the backrest and seat are stored inside the housing in a folded state to a wheelchair mode in which the backrest and seat are extended and the user can use it as a wheelchair while seated on the seat.

3. The mobile device according to claim 1, wherein the meaning of a sound is estimated by inputting the sound data collected by the sound collection unit into a machine learning model that has been trained based on the sound data and the meaning that the sound data represents.

Images