Information processing device and information processing method

Abstract

To provide an information processing device and an information processing method capable of appropriately suppressing the influence of vibration generated in a vehicle caused by undulations of a road surface on the operation of an occupant while the vehicle is traveling.SOLUTION: An information processing device includes: an imaging information acquisition unit 21 for acquiring in-vehicle photographing information including an image of an occupant who has boarded a vehicle; an operation determination unit 26 for determining whether or not the occupant has started a predetermined preset operation based on the in-vehicle photographing information; a position information acquisition unit 22 for acquiring position information of the vehicle; a vehicle information acquisition unit 23 for acquiring vehicle information including at least traveling speed information of the vehicle; a vibration information acquisition unit 28 for acquiring information about vibration that may occur in the vehicle due to road undulations based on the position information and the vehicle information; a map information acquisition unit 24 for acquiring map information; and a speaker 16 or a display unit 17 for informing an occupant of a warning before the vehicle reaches an undulating position by referring to the acquired map information when it is determined that the occupant has started the operation.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing device and an information processing method. [Background technology]

[0002] Generally, when a vehicle is traveling on a road that is uneven, uneven, or has steps, the vehicle may vibrate due to the unevenness. Conventionally, a technology has been disclosed for controlling the suspension of a vehicle in accordance with the magnitude of the unevenness of the road on which the vehicle is traveling (see, for example, Patent Document 1). This type of technology can provide a vehicle that provides improved ride comfort for passengers. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2009-107584 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, if vibrations caused by unevenness of the road surface occur while a passenger is performing an action inside the vehicle that is easily affected by vibrations, such as applying makeup or eating and drinking, it is expected that the passenger will be unable to perform the action adequately due to the influence of the vibrations. For this reason, it is desirable to appropriately suppress the influence of vibrations that occur in the vehicle due to unevenness of the road surface while the vehicle is traveling on the actions of the passenger.

[0005] The present invention has been made in consideration of the above, and aims to provide an information processing device and an information processing method that can appropriately suppress the effect on the movements of occupants of vibrations that occur in the vehicle due to unevenness of the road surface while the vehicle is traveling. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems and achieve the object, the information processing device of the present invention comprises an imaging information acquisition unit that acquires in-vehicle image information including video of a passenger in the vehicle, an action determination unit that determines whether the passenger has started a predetermined action based on the in-vehicle image information, a position information acquisition unit that acquires vehicle position information, a vehicle information acquisition unit that acquires vehicle information including at least information about the vehicle's traveling speed, a vibration information acquisition unit that acquires vibration information that may occur in the vehicle due to road undulations based on the position information and the vehicle information, a map information acquisition unit that acquires map information, and a notification unit that, when it is determined that the passenger has started an action, refers to the acquired map information and issues a warning to the occupant before the vehicle reaches the location of the undulation.

[0007] In addition, the information processing method of the present invention has a computer execute the following steps: acquiring in-vehicle photographed information including video of a passenger in the vehicle; determining whether or not the passenger has started a predetermined action based on the in-vehicle photographed information; acquiring vehicle position information; acquiring vehicle information including at least information on the vehicle's traveling speed; acquiring vibration information that may occur in the vehicle due to road undulations based on the position information and vehicle information; acquiring map information; and, if it is determined that the passenger has started an action, referring to the acquired map information and issuing a warning to the passenger before the vehicle reaches the location of the undulation. Effect of the Invention

[0008] According to the present invention, it is possible to appropriately suppress the influence of vibrations generated in a vehicle due to unevenness of the road surface while the vehicle is traveling on the motion of a passenger. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of an information processing system having an information processing device according to this embodiment. [Diagram 2] FIG. 2 is a flowchart showing an example of a process flow for transmitting vibration information detected while the vehicle is traveling to the server device. [Diagram 3] FIG. 3 is a flowchart showing an example of issuing a warning to a passenger based on vibration information when vibration information caused by an undulating road is acquired after the passenger starts a predetermined action. [Figure 4] FIG. 4 is a flowchart showing another example of issuing a warning to a passenger based on vibration information when vibration information caused by an undulating road is acquired after the passenger starts a predetermined action. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] An information processing device and an information processing method according to the present embodiment will be described below with reference to the accompanying drawings. Note that the present invention is not limited to the following embodiment. FIG. 1 is a block diagram showing a configuration example of an information processing system 1 having an information processing device 10 according to the present embodiment. As shown in FIG. 1, the information processing system 1 includes the information processing device 10 and a server device 40. The information processing device 10 and the server device 40 are communicatively connected via a communication network 50 such as an Internet line. Although one information processing device 10 is shown in FIG. 1, this information processing device 10 is mounted on a vehicle, and actually, a plurality of information processing devices 10 are provided.

[0011] When the information processing device 10 acquires vibration information that may occur in the vehicle due to the undulation of the road while the occupant is performing a predetermined action that is easily affected by vibration in the vehicle, the information processing device 10 issues a warning based on the vibration information to the occupant before the vehicle reaches the location of the undulation. The predetermined action is, for example, an action that is easily affected by vibration, such as eating, drinking, putting on makeup, or writing. The information processing device 10 may be a portable device that can be used in the vehicle, in addition to a device that is installed in the vehicle. The information processing device 10 may also be realized by including the functions or configurations of a device that is installed in advance in the vehicle, a navigation device, and the like. The information processing device 10 has an in-vehicle camera (in-vehicle photographing unit) 11, a GPS (Global Positioning System) receiving unit 12, a CAN (Controller Area Network) interface unit (hereinafter referred to as "CANIF unit") 13, a memory unit 14, an acceleration sensor 15, a speaker 16, a display unit 17, a communication unit 18, and a control unit 20.

[0012] The in-vehicle camera 11 is disposed facing backward inside the vehicle and captures the interior of the vehicle, particularly the passengers. The passengers include not only the driver but also all passengers in the vehicle. The in-vehicle camera 11 captures a wide-angle image, for example, a horizontal image range of about 180°. In this embodiment, the in-vehicle camera 11 captures images at all times while the accessory power supply (ACC power supply) of the vehicle is on. The in-vehicle camera 11 outputs the captured image information to the image information acquisition unit 21 of the control unit 20. The image information is a moving image composed of images of, for example, 30 frames per second.

[0013] The GPS receiver 12 receives radio waves from a GPS satellite (not shown). The GPS receiver 12 outputs the received radio wave signal to the position information acquisition unit 22 of the control unit 20.

[0014] The CANIF unit 13 is an interface for acquiring various vehicle information via the CAN. The vehicle information includes status information indicating the status of the vehicle. The vehicle information includes, for example, vehicle type information, information on the tires mounted on the vehicle, and the like. The status information also includes, for example, the vehicle speed (traveling speed), the vehicle acceleration, the vehicle traveling direction (direction), the vehicle load weight (number of passengers), and the like.

[0015] The memory unit 14 stores map information registered in advance. This map information includes road information of the area where the vehicle travels. The road information is information about the road on which the vehicle travels, and includes road characteristics such as road width, number of lanes, and the position and size of undulations caused by bumps and steps. The memory unit 14 outputs the stored map information to the map information acquisition unit 24 of the control unit 20. The memory unit 14 is, for example, a semiconductor memory element such as a ROM (Read Only Memory) or a flash memory, or a memory unit such as a memory card. Alternatively, the memory unit 14 may be an external memory unit wirelessly connected via a communication device (not shown).

[0016] The acceleration sensor 15 is a sensor that detects acceleration occurring with respect to the vehicle. The acceleration sensor 15 outputs the detection result to the vibration detection unit 25 of the control unit 20. The acceleration sensor 15 is a sensor that detects acceleration in, for example, three axial directions, which are the forward / rearward direction, the left / right direction, and the up / down direction of the vehicle.

[0017] The speaker (notification unit) 16 outputs a sound indicating the notification content based on a sound signal output from the notification control unit 29 of the control unit 20. The speaker 16 is, for example, a sound output device specific to the information processing device 10, or a sound output device shared with other systems including a navigation system.

[0018] The display unit (notification unit) 17 displays an image showing the notification content based on a video signal output from the notification control unit 29 of the control unit 20. The display unit 17 is, for example, a display device specific to the information processing device 10, or a display device shared with other systems including a navigation system. The display unit 17 is, for example, a display including a liquid crystal display (LCD) or an organic EL (Electro-Luminescence) display. In this embodiment, the display unit 17 is disposed on a dashboard, an instrument panel, a center console, or the like in front of the driver of the vehicle.

[0019] The communication unit 18 is a communication unit. The communication unit 18 is connected so as to be able to communicate data from an external server device 40 via a communication network 50. The communication unit 18 may perform communication via any method, such as the Internet or a mobile phone line.

[0020] The control unit 20 is an arithmetic processing device constituted by, for example, a CPU (Central Processing Unit) and the like. When the control unit 20 acquires vibration information that may occur in the vehicle due to the undulation of the road while the occupant is performing a predetermined operation that is easily affected by vibration in the vehicle, the control unit 20 controls a series of operations to issue a warning based on the vibration information to the occupant before the vehicle reaches the location of the undulation. The control unit 20 includes an imaging information acquisition unit 21, a position information acquisition unit 22, a vehicle information acquisition unit 23, a map information acquisition unit 24, a vibration detection unit 25, an operation determination unit 26, a route guidance unit 27, a vibration information acquisition unit 28, a notification control unit 29, and a communication control unit 30.

[0021] The imaging information acquisition unit 21 acquires in-vehicle captured information including video of passengers in the vehicle captured by the in-vehicle camera 11. The imaging information acquisition unit 21 outputs the acquired in-vehicle captured information to the operation determination unit 26.

[0022] The position information acquisition unit 22 acquires position information indicating the current position of the vehicle. The position information acquisition unit 22 calculates the position information of the current position of the vehicle by a known method based on the radio waves received by the GPS receiving unit 12. In this embodiment, the position information acquisition unit 22 is capable of acquiring position information including lane positions when the vehicle is traveling on a multi-lane road.

[0023] The vehicle information acquisition unit 23 acquires the vehicle information acquired by the CANIF unit 13. More specifically, the vehicle information acquisition unit 23 acquires vehicle information such as vehicle type information, information on the installed tires, and other status information such as vehicle running speed information, vehicle traveling direction information, and vehicle load weight information.

[0024] The map information acquisition unit 24 acquires map information from the storage unit 14. Based on the location information acquired by the location information acquisition unit 22, the map information acquisition unit 24 acquires map information including the surrounding area of ​​the current location of the information processing device 10 (vehicle) from the storage unit 14. The map information includes road information related to the road on which the vehicle is traveling, such as road width, number of lanes, and the location and size of undulations caused by bumps and steps. The map information acquisition unit 24 outputs the acquired map information to the route guidance unit 27 and the notification control unit 29.

[0025] The vibration detection unit 25 detects the magnitude of vibration (vibration information) occurring in the vehicle based on the detection result of the acceleration sensor 15. When the vibration detection unit 25 detects vibration having a magnitude equal to or greater than a predetermined threshold, it transmits the detected vibration information to the server device 40 via the communication unit 18 together with the location information where the vibration occurred and the vehicle information.

[0026] The motion determination unit 26 determines whether or not the passenger has started a predetermined motion based on the in-vehicle photographed information including the video of the passenger acquired by the imaging information acquisition unit 21. Specifically, the motion determination unit 26 identifies the passengers by image recognition of the in-vehicle photographed information, and determines whether or not each identified passenger has started a predetermined motion. The predetermined motion is one that is easily affected by vibration, such as eating, drinking, applying makeup (lipstick), and writing. The motion determination unit 26 also learns in advance the time required for the predetermined motion for each passenger. That is, when one passenger (e.g., passenger A) applies makeup while driving, the motion determination unit 26 learns the time required for this makeup. The average value or the most frequent value of the time required for one passenger to apply makeup each time can be used as the required time.

[0027] The motion determination unit 26 also determines the degree of influence of vibration on the predetermined motion. That is, in the case of makeup as the predetermined motion, the influence of vibration is greater for the motion of applying lipstick or eyeliner than for the motion of applying primer or foundation. In the case of eating and drinking as the predetermined motion, the influence of vibration is greater for the motion of drinking a beverage by putting one's mouth to a cup or a plastic bottle than for the motion of drinking a beverage using a straw. The motion determination unit 26 outputs the determination result, the time required, and the magnitude of the influence to the route guidance unit 27 and the notification control unit 29.

[0028] When vibrations are likely to occur to the vehicle a predetermined threshold number of times or more within the time required for the above-mentioned predetermined operation while traveling along a route to the destination, the route guidance unit 27 guides the vehicle along an avoidance route that avoids the vibrations. When the vibration information acquired from the vibration information acquisition unit 28 is equal to or more than the predetermined threshold number of times, the route guidance unit 27 guides the vehicle along an avoidance route that avoids the area where the vibrations are generated, since it would be troublesome to issue a warning each time. This avoidance route may be a route along a new road that avoids the undulating position that generates vibrations from the road currently being traveled, but for example, when the road currently being traveled has multiple lanes, the vehicle may avoid the undulating position by moving to another lane.

[0029] The vibration information acquisition unit 28 acquires vibration information that may occur in the vehicle due to the undulation of the road from a vibration information database 41 of the server device 40 (described later) based on the position information and the vehicle information via the communication unit 18. The vibration information acquisition unit 28 periodically transmits the vehicle's position information and vehicle information, such as vehicle type information and traveling speed information, to the server device 40, and acquires vibration information that may occur due to the undulation when the vehicle travels on the undulation of the road from the server device 40. This vibration information is information about vibrations that are large enough to affect the predetermined actions of the passengers in the vehicle, and does not include small vibrations that do not affect the predetermined actions of the passengers. The vibration information includes information about the magnitude of the vibrations that occur in the vehicle and the time from the current position of the vehicle until the vibration occurs (the time until the vehicle reaches the undulation). When the vibration information acquisition unit 28 acquires the vibration information, it outputs the vibration information to the notification control unit 29.

[0030] The notification control unit 29 controls the sound and display contents output from the speaker 16 and the display unit 17. When the notification control unit 29 determines that the passenger has started a predetermined action and acquires vibration information that may occur in the vehicle due to the undulation of the road, it controls the output of the sound and display contents for notifying the passenger of a warning before the vehicle reaches the location of the undulation, for example, by referring to map information. Specifically, the notification control unit 29 issues a warning such as "Please refrain from applying lipstick because the vehicle will shake significantly due to vibration in 15 seconds" or "Please take your mouth off your drink because the vehicle will shake significantly due to vibration in 30 seconds" before the vehicle reaches the undulation that will cause vibration. In addition, the notification control unit 29 issues the above-mentioned warning until the required time for the predetermined action has elapsed. In addition, the notification control unit 29 changes the strength of the warning depending on the degree of the influence of the vibration. Specifically, in the case of actions such as applying lipstick or applying eyeliner, which are highly influenced by vibration, the strength of the warning is changed by repeatedly issuing a voice message or by enlarging the characters of the display message.

[0031] The communication control unit 30 controls the communication operation between the communication unit 18 and the server device 40. The communication control unit 30 periodically (every few seconds) transmits vehicle position information and vehicle information (vehicle type information and traveling speed information) to the server device 40, and when the vehicle approaches an undulating road that generates vibration, it receives vibration information generated when the vehicle passes over the undulating road. Furthermore, when the communication control unit 30 detects vibrations equal to or greater than a predetermined threshold while the vehicle is traveling, it transmits the vibration information, vehicle position information, and vehicle information to the server device 40.

[0032] The server device 40 is configured to be able to communicate with the information processing device 10 of each vehicle, and when each information processing device 10 detects vibration, it receives the position information, vehicle information, and vibration information. In addition, it periodically acquires position information and vehicle information from each information processing device 10, and if there is an undulation that causes vibration in the forward direction of the position information, it transmits vibration information corresponding to the position information and vehicle information. The server device 40 is configured by a computer installed in the cloud, and includes a vibration information database 41, a communication unit 42, and a control unit 43.

[0033] The vibration information database 41 receives the vibration information, position information, and vehicle information transmitted from each information processing device 10, and stores a data set of vibration magnitude, vehicle type, running speed, and position as a database. When the vehicle's position information indicates that the vehicle is approaching an undulating position that generates vibration, the vibration information database 41 transmits vibration information including the magnitude of vibration generated when the vehicle is traveling at an equivalent running speed for the same or a similar vehicle type.

[0034] The communication unit 42 constantly communicates with each information processing device 10. The control unit 43 controls the operation of the server device 40, in particular the operation of the vibration information database 41 and the communication unit 42.

[0035] Next, a process flow in the information processing device 10 will be described with reference to Fig. 2 to Fig. 4. Fig. 2 is a flowchart showing an example of a process flow for transmitting vibration information detected while the vehicle is traveling to the server device 40.

[0036] The control unit 20 acquires vehicle position information using the position information acquisition unit 22 (step S11). More specifically, the position information acquisition unit 22 acquires position information indicating the current position of the vehicle. The position information acquisition unit 22 calculates the position information of the current position of the vehicle using a known method based on the radio waves received by the GPS receiver 12. In this embodiment, the position information acquisition unit 22 is capable of acquiring position information including lane positions when the vehicle is traveling on a multi-lane road.

[0037] Next, the control unit 20 acquires vehicle information through the vehicle information acquisition unit 23 (step S12). More specifically, the vehicle information acquisition unit 23 acquires the vehicle information acquired by the CANIF unit 13. The vehicle information acquisition unit 23 acquires vehicle information such as vehicle type information, information on the installed tires, and other status information such as vehicle running speed information, vehicle running direction information, and vehicle load weight information.

[0038] Next, the control unit 20 determines whether or not the vibration detection unit 25 has detected a vibration having a magnitude equal to or greater than a predetermined threshold (step S13). In this embodiment, the vibration detection unit 25 detects the magnitude of vibration occurring in the vehicle based on the detection result of the acceleration sensor 15. In this determination, if the vibration detection unit 25 has not detected a vibration having a magnitude equal to or greater than the predetermined threshold (step S13; No), the process proceeds to step S15.

[0039] On the other hand, when the vibration detection unit 25 detects vibrations of a magnitude equal to or greater than a predetermined threshold (step S13; Yes), the control unit 20 transmits the detected vibration information together with the position information and vehicle information where the vibration occurred to the server device 40 set in advance via the communication unit 18 (step S14). As a result, the server device 40 receives vibration information, position information, and vehicle information transmitted from the information processing device 10 of each vehicle, and these can be used by storing them in a database. Next, the control unit 20 determines whether the accessory (ACC) power supply of the vehicle has been turned off (step S15). Specifically, it determines whether the driving operation of the vehicle has been stopped. In this determination, if the accessory power supply of the vehicle has not been turned off (step S15; No), the control unit 20 returns the process to step S11 and repeats steps S11 to S15. Also, if the accessory power supply of the vehicle has been turned off (step S15; Yes), the control unit 20 ends the process.

[0040] FIG. 3 is a flowchart showing an example of a process flow for notifying a passenger of a warning based on vibration information when vibration information caused by road undulations is acquired after the passenger starts a predetermined action.

[0041] The control unit 20 acquires in-vehicle captured information by the image capturing information acquisition unit 21 (step S21). The image capturing information acquisition unit 21 acquires in-vehicle captured information including video of passengers in the vehicle captured by the in-vehicle camera 11.

[0042] Next, the control unit 20 determines whether or not the passengers have started a predetermined motion using the motion determination unit 26 (step S22). The motion determination unit 26 determines whether or not the passengers have started a predetermined motion based on the in-vehicle captured image information including the video of the passengers acquired by the image information acquisition unit 21. Specifically, the motion determination unit 26 identifies the passengers by performing image recognition on the in-vehicle captured image information, and determines whether or not each identified passenger has started a predetermined motion. In this embodiment, the predetermined motion is, for example, the motion of applying lipstick in makeup.

[0043] If it is determined that the occupant has not started the predetermined motion (step S22; No), the control unit 20 transitions the process to step S28. On the other hand, if the occupant has started the predetermined motion (step S22; Yes), the control unit 20 determines the degree of influence of vibration on the predetermined motion (step S23) using the motion determination unit 26. Here, the motion of applying lipstick is relatively more influenced by vibration than other motions (for example, the motion of applying a base for makeup). Therefore, in the case of the motion of applying lipstick, the motion determination unit 26 determines that the degree of influence of vibration on the motion is large.

[0044] Next, the control unit 20 starts measuring the time required for a predetermined action by the action determination unit 26 (step S24). The action determination unit 26 has learned in advance the time required for a predetermined action for each passenger, and the time required for the target passenger to apply lipstick (e.g., 3 minutes) is known. Therefore, when the target passenger starts the action of applying lipstick, the action determination unit 26 starts measuring the time required.

[0045] Next, the control unit 20 periodically transmits the vehicle position information and vehicle information to the server device 40 via the communication unit 18 by the vibration information acquisition unit 28, and acquires vibration information ahead of the vehicle position in the traveling direction from the vibration information database 41 of the server device 40 (step S25). Specifically, the vibration information acquisition unit 28 periodically transmits the vehicle position information and vehicle information (vehicle type information and traveling speed information) to the server device 40. Then, when an undulating surface that generates vibration is close ahead of the vehicle in the traveling direction, the vibration information acquisition unit 28 acquires from the vibration information database 41 vibration information that is actually generated when a vehicle of the same or similar vehicle type travels over this undulating surface at an equivalent traveling speed. Here, the control unit 20 is assumed to have acquired map information including road information around the vehicle position information by the map information acquisition unit 24.

[0046] Next, the control unit 20, for example, refers to the acquired map information, and causes the notification control unit 29 to notify the passenger of a warning with a strength corresponding to the degree of influence of the vibration each time before the vehicle reaches an undulating position where vibration occurs (step S26). Specifically, the notification control unit 29 controls the sound and display contents output from the speaker 16 and the display unit 17 to output the sound and display contents for notifying the passenger of a warning. The notification control unit 29 sets the period from when it is determined that a predetermined operation has started until the required time has elapsed as the target period for notifying the warning. The notification control unit 29 refers to the map information, and issues a warning, for example, "Please refrain from wearing lipstick as the vehicle will shake significantly due to vibration in 15 seconds" each time before the vehicle reaches an undulating position where vibration occurs. The notification control unit 29 also changes the strength of the warning depending on the degree of influence of the vibration. Since the action of applying lipstick has a large effect on the vibration, the strength of the warning can be increased, for example, by repeatedly issuing a voice message or by displaying a message in large letters.

[0047] Next, the control unit 20 determines whether the required time has elapsed (step S27). That is, the control unit 20 determines whether the target period for issuing a warning has elapsed. In this determination, if the required time has not elapsed (step S27; No), the control unit 20 returns the process to step S25 and repeats the processes of steps S25 to S27. On the other hand, if the required time has elapsed (step S27; Yes), the control unit 20 determines whether the accessory (ACC) power supply of the vehicle has been turned off (step S28). Specifically, it determines whether the driving operation of the vehicle has stopped. In this determination, if the accessory power supply of the vehicle has not been turned off (step S28; No), the control unit 20 returns the process to step S21 and repeats steps S21 to S28. Also, if the accessory power supply of the vehicle has been turned off (step S28; Yes), the control unit 20 ends the process.

[0048] Next, another process flow will be described. Fig. 4 is a flow chart showing another example of notifying a passenger of a warning based on vibration information when the passenger acquires vibration information caused by undulations of a road after starting a predetermined action. In this flow chart, steps S31 to S35, S37, and S39 to S40 in Fig. 4 are equivalent to steps S21 to S28 in Fig. 3, and therefore description thereof will be omitted.

[0049] After acquiring vibration information ahead of the vehicle's position in the traveling direction from the vibration information database 41 of the server device 40 (step S35), the control unit 20 determines whether or not vibrations will occur a predetermined threshold number of times or more within a required time period using the vibration information acquisition unit 28 (step S36). Specifically, the vibration information acquisition unit 28 determines whether or not it is predicted that vibrations will occur to the vehicle a predetermined threshold number of times (e.g., three times) or more based on the acquired vibration information. With regard to the number of times, continuous vibrations for a predetermined time period (e.g., 10 seconds) or less are treated as one occurrence.

[0050] In this determination, if vibrations do not occur a predetermined threshold number of times or more within the required time (step S36; No), the control unit 20, for example, refers to the acquired map information and, as described above, causes the notification control unit 29 to issue a warning to the passenger with a strength corresponding to the influence of the vibration each time before the vehicle reaches the undulating position where vibration occurs (step S37). On the other hand, if vibrations occur a predetermined threshold number of times or more within the required time (step S36; Yes), the control unit 20 causes the route guidance unit 27 to guide an avoidance route that avoids the vibration. If the vibration information acquired from the vibration information acquisition unit 28 is a predetermined threshold number of times or more, it is troublesome to issue a warning each time, so the route guidance unit 27 searches for an avoidance route that avoids the area where the vibration occurs and guides the vehicle according to this avoidance route (step S38). This avoidance route may be a route that travels on a new road that avoids the undulating position where vibration occurs from the road currently being traveled, but for example, if the road currently being traveled has multiple lanes, the above-mentioned undulating position may be avoided by moving to another lane with reference to the map information.

[0051] As described above, the information processing device 10 according to the present embodiment includes an image information acquisition unit 21 that acquires in-vehicle photographed information including a video of a passenger who has boarded the vehicle, an action determination unit 26 that determines whether or not the passenger has started a predetermined action based on the in-vehicle photographed information, a position information acquisition unit 22 that acquires vehicle position information, a vehicle information acquisition unit 23 that acquires vehicle information including at least information on the vehicle's traveling speed, a vibration information acquisition unit 28 that acquires vibration information that may occur in the vehicle due to the undulations of the road based on the position information and the vehicle information, a map information acquisition unit 24 that acquires map information, and a speaker 16 or a display unit 17 that, when it is determined that the passenger has started a motion, refers to the acquired map information and issues a warning to the passenger before the vehicle reaches the location of the undulation. According to this configuration, a warning that vibration will occur is issued to the passenger by voice or display before the vibration occurs, so that the influence of vibration occurring in the vehicle due to the undulations of the road surface while the vehicle is traveling on the actions of the passenger can be appropriately suppressed.

[0052] In this information processing device 10, the motion determination unit 26 identifies the passenger based on the in-vehicle photographed information, and learns in advance the time required for a specific motion for each passenger, and the speaker 16 or the display unit 17 sets the period from when it is determined that the motion has started until the required time has elapsed as the target period for issuing a warning by voice or display. According to this configuration, a warning is issued by voice or display to the effect that vibration will occur in response to the motion of the passenger during the time required for the specific motion of the target passenger, so that the effect of vibration on the motion of the passenger can be appropriately suppressed.

[0053] In this information processing device 10, the motion determination unit 26 also determines the magnitude of the influence of vibration on motion, and the speaker 16 or the display unit 17 changes the strength of the warning according to the influence. With this configuration, the influence of vibration on the motion of the occupant can be more effectively suppressed.

[0054] In this information processing device 10, if vibrations are likely to occur to the vehicle a predetermined threshold number of times or more within a required time, a route guidance unit 27 is provided that guides the driver on an avoidance route to avoid the vibrations, thereby reducing bothersome warnings and appropriately suppressing the effects of vibrations on the passengers' actions.

[0055] Although the embodiment of the present invention has been described above, the embodiment is not limited to the contents of this embodiment. The above-mentioned components include those that a person skilled in the art can easily imagine, those that are substantially the same, and those that are within the so-called equivalent range. Furthermore, the above-mentioned components can be appropriately combined. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the gist of the above-mentioned embodiment.

[0056] In the present embodiment, the information processing device 10 is configured to transmit position information and vehicle information to the server device 40, and acquire vibration information corresponding to the position information and vehicle information from the server device 40. However, the information processing device 10 may be configured to include a vibration information calculation unit that calculates vibration information from position information, vehicle information, and map information including undulating positions, and acquire the calculated vibration information.

[0057] Moreover, each component of the control unit 20 according to this embodiment is functionally conceptual, and does not necessarily have to be physically configured as shown in the figure. That is, the specific form of each device is not limited to that shown in the figure, and all or part of each device may be functionally or physically distributed or integrated in any unit depending on the processing load and usage status of each device. The configuration of the control unit 20 is realized, for example, as software by a program loaded into a memory. In the above embodiment, these functional blocks have been described as being realized by the cooperation of these hardware or software. That is, these functional blocks can be realized in various forms by only hardware, only software, or a combination thereof. [Explanation of symbols]

[0058] 1. Information Processing Systems 10. Information processing device 11. In-car camera 14 Storage section 15 Acceleration Sensor 16 Speaker (notification section) 17 Display unit (notification unit) 20 Control section 21 Imaging information acquisition unit 22 Location information acquisition unit 23 Vehicle Information Acquisition Department 25 Vibration detection unit 26 Operation judgment section 27 Route Guidance Section 28 Vibration information acquisition unit 29 Notification control section 30 Communication control section 40 Server device 41 Vibration Information Database 50 Communication Network

Claims

1. an image capturing information acquiring unit that acquires in-vehicle captured image information including images of passengers in the vehicle; an action determination unit that determines whether or not the passenger has started a predetermined action based on the in-vehicle photographed information; a location information acquisition unit that acquires location information of the vehicle; a vehicle information acquisition unit that acquires vehicle information including at least traveling speed information of the vehicle; a vibration information acquisition unit that acquires vibration information that may be generated in the vehicle due to undulations of a road based on the position information and the vehicle information; A map information acquisition unit that acquires map information; a notification unit that, when it is determined that the occupant has started the motion, refers to the acquired map information and issues a warning to the occupant before the vehicle reaches a position of the undulation; An information processing device comprising:

2. The motion determination unit identifies the occupant based on the in-vehicle photographed information, and learns in advance a required time for each occupant to perform the motion, The information processing apparatus according to claim 1 , wherein the notification unit sets a period from when it is determined that the operation has started until when the required time has elapsed as a target period for notifying the warning.

3. The motion determination unit also determines the magnitude of the influence of the vibration on the motion, The information processing apparatus according to claim 1 , wherein the notification unit changes a strength of the warning in accordance with the degree of impact.

4. The information processing device according to claim 2 , further comprising a route guidance unit that, when vibrations are likely to occur to the vehicle a predetermined threshold number of times or more within the required time, provides guidance on an avoidance route for avoiding the vibrations.

5. A step of acquiring in-vehicle photographed information including images of passengers in the vehicle; determining whether the passenger has started a predetermined action based on the in-vehicle photographed information; acquiring location information of the vehicle; acquiring vehicle information including at least traveling speed information of the vehicle; acquiring vibration information that may be generated in the vehicle due to an undulation of a road based on the position information and the vehicle information; obtaining map information; when it is determined that the occupant has started the motion, referring to the acquired map information, and issuing a warning to the occupant before the vehicle reaches a position of the undulation; An information processing method for causing a computer to execute the above.

Images