Automobile, automobile program and gait recognition system

Abstract

To provide a car that can reliably detect family members or acquaintances walking around the nearest station during commuting hours or school hours while the car is running, and can persuade the detected family members or acquaintances to get in the car. [Solution] The system comprises one or more cameras that capture images of the outside of the vehicle, a memory unit that stores gait information of the target person, a gait discrimination means that determines from the image information captured by the camera whether a person with a gait that matches the gait of the target person stored in the memory unit has been detected among people present outside the vehicle, and an alarm means that alerts the detected person when the gait discrimination means determines that a person with a gait that matches the gait of the target person has been detected.

Description

[Technical Field]

[0001] The present invention relates to a vehicle, a program for a vehicle, and a gait recognition system. [Background technology]

[0002] The general way to use a car is to get in and out of a parked car of your own volition. Alternatively, if the car is a taxi, you can signal the driver by raising your hand, and the driver will stop the car near you so that you can get in and out.

[0003] Recently, self-driving cars that can drive autonomously even without a driver on board have also appeared, and it has been proposed that such self-driving cars be equipped with wireless communication functions so that passengers can make a request to the self-driving car from a mobile terminal such as a cell phone terminal, and have the car called to their desired location and board the vehicle (see, for example, Patent Document 1 (JP 2017-204867 A)). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-204867 Summary of the Invention [Problem to be solved by the invention]

[0005] Incidentally, if the driver and / or passengers, or the car itself, happen to detect a family member or acquaintance walking on the sidewalk ahead while the car is moving, the car can stop beside the family member or acquaintance and invite them to get in. It would be particularly beneficial if the family member or acquaintance is walking with a heavy load or is elderly, and the driver and / or passengers, or the car itself, could invite them to get in.

[0006] However, because drivers are mindful of driving safely, checking ahead and behind them for safety, it is difficult for them to detect such walking family members or acquaintances, and they often overlook them.

[0007] In view of the above, the object of this invention is to provide a vehicle that can reliably detect walking family members, acquaintances, etc. while the vehicle is moving, and can perform processes such as inviting the detected family members or acquaintances to board the vehicle. [Means for solving the problem]

[0008] In order to solve the above problem, the invention of claim 1 is as follows: One or more cameras that capture images of the outside of the vehicle; a storage unit that stores gait information of a target person; a gait discrimination means for determining whether or not a person whose gait matches the gait of the target person stored in the storage unit has been detected among people present outside the vehicle from the image information captured by the camera; a notification means for notifying the detected person when the gait determination means determines that a person having a gait matching the gait of the target person has been detected; To provide a vehicle equipped with the above.

[0009] As is well known, gait recognition is a technology that can authenticate a person from a distance of approximately 40 to 50 meters, and is attracting attention as a biometric for identifying individuals from a distance.

[0010] The automobile of the invention of claim 1 configured as described above has a memory unit storing gait information for detecting a target person to be subjected to gait recognition. In the automobile of this invention, a gait discrimination means recognizes the gait of a person present outside the vehicle from image information captured by a camera. The gait discrimination means further determines whether or not there is a person whose gait matches the gait of the target person stored in the memory unit among the people whose gaits have been recognized.

[0011] If the gait discrimination means determines that there is a person whose gait matches the gait of the target person, the detected person is notified. [Effects of the Invention]

[0012] According to the automobile of this invention, by using gait recognition (gait authentication) technology, when a target person who has been registered and stored in advance is walking in front of or to the side of the vehicle in the direction of travel, it is possible to detect this from images captured by a camera installed in the automobile and notify the detected person, making it possible to invite the detected person to get in the vehicle or not. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a block diagram showing an example of the configuration of an electronic control circuit unit of an example of an autonomous driving vehicle as an embodiment of the automobile according to the present invention. FIG. [Figure 2] FIG. 10 is a diagram illustrating a procedure for registering a target person who is a target of gait recognition in an autonomous driving vehicle according to an embodiment. [Figure 3] FIG. 10 is a diagram illustrating an example of registered target person information in an autonomous driving vehicle according to an embodiment. [Figure 4] FIG. 10 is a diagram illustrating a portion of a flowchart for explaining an example of a processing routine for presetting the gait recognition of a target person in an autonomous driving vehicle according to an embodiment. [Figure 5] FIG. 10 is a diagram illustrating a portion of a flowchart for explaining an example of a processing routine for presetting the gait recognition of a target person in an autonomous driving vehicle according to an embodiment. [Figure 6] FIG. 10 is a diagram illustrating a portion of a flowchart for explaining an example of a processing routine for detecting a target person through gait recognition in an autonomous driving vehicle according to an embodiment. [Figure 7] FIG. 10 is a diagram illustrating a portion of a flowchart for explaining an example of a processing routine for detecting a target person through gait recognition in an autonomous driving vehicle according to an embodiment. [Figure 8]FIG. 10 is a diagram illustrating a portion of a flowchart for explaining an example of a processing routine for detecting a target person through gait recognition in an autonomous driving vehicle according to an embodiment. [Figure 9] FIG. 10 is a diagram illustrating a portion of a flowchart for explaining an example of a processing routine for detecting a target person through gait recognition in an autonomous driving vehicle according to an embodiment. [Figure 10] FIG. 10 is a diagram illustrating a portion of a flowchart for explaining an example of a processing routine for detecting a target person through gait recognition in an autonomous driving vehicle according to an embodiment. [Figure 11] FIG. 10 is a diagram illustrating a portion of a flowchart for explaining an example of a processing routine for detecting a target person through gait recognition in an autonomous driving vehicle according to an embodiment. [Figure 12] FIG. 10 is a diagram illustrating a portion of a flowchart for explaining an example of a processing routine for detecting a target person through gait recognition in an autonomous driving vehicle according to an embodiment. [Figure 13] 10A and 10B are diagrams for explaining a specific example of how a target person is detected by gait recognition using an autonomous driving vehicle according to an embodiment. [Figure 14] FIG. 10 is a block diagram illustrating an example of the configuration of a system including an autonomously driven vehicle as another embodiment of the automobile according to the present invention. [Figure 15] 10A and 10B are diagrams for explaining a specific example of an aspect in which a target person is detected by gait recognition using an air-land vehicle as another embodiment of the automobile according to the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0014] An embodiment of a vehicle according to the present invention will be described below with reference to the drawings. The embodiment of the vehicle described below is an autonomous vehicle that has a manual driving mode in which the vehicle travels in response to manual driving operations by a driver, and an autonomous driving mode in which the vehicle travels autonomously without relying on driving operations by a driver.

[0015] The automobile according to the present invention is not limited to an autonomous vehicle having both a manual driving mode and an autonomous driving mode as in this embodiment, but can also be implemented in an automobile (fully autonomous vehicle) that has only an autonomous driving mode that allows autonomous driving without manual operation by the driver. The present invention can also be applied to an automobile that does not have an autonomous driving mode and only has a manual driving mode.

[0016] 1 is a block diagram showing an example of the hardware configuration of an electronic control circuit unit 10 of an autonomous vehicle 1 according to this embodiment. The autonomous vehicle 1 according to this embodiment is an example of an electric vehicle, and is equipped with a battery (not shown) as a driving source.

[0017] As described above, the self-driving vehicle 1 of this embodiment has a manual driving mode and an autonomous driving mode in which the vehicle travels autonomously in this embodiment. The manual driving mode is a mode in which the vehicle travels in response to the driver's accelerator pedal operation, brake pedal operation, shift lever operation, and steering (handle operation), just like a normal vehicle that is not a self-driving vehicle.

[0018] Furthermore, the autonomous driving mode is a mode in which the autonomous vehicle 1 automatically (autonomously) drives while avoiding obstacles without the driver having to operate the accelerator pedal, brake pedal, shift lever, or steering. In this autonomous driving mode, the vehicle can be automatically switched to the manual driving mode by a predetermined action by the driver. Here, the predetermined action by the driver includes, for example, driving operations such as operating the accelerator pedal or brake pedal by the driver, operation input by the driver via a touch panel described below, and switching instructions by voice input by the driver.

[0019] For example, the driver of the autonomous vehicle 1 can switch the autonomous vehicle 1, which is traveling in manual driving mode, to autonomous driving mode by performing a specific operation via the touch panel 112 described below, and the autonomous vehicle 1 is configured to automatically return to manual driving mode if the driver operates the accelerator pedal, brake pedal, shift lever, or steering while traveling in autonomous driving mode.

[0020] As will be described later, the autonomous vehicle of this embodiment is equipped with a forced autonomous driving mode that maintains the autonomous driving mode even if the driver performs an operation to return to the manual driving mode, such as operating the accelerator pedal, brake pedal, shift lever, or steering. Therefore, in the following description, the autonomous driving mode that switches to the manual driving mode when the driver performs an operation to return to the manual driving mode, such as operating the accelerator pedal, brake pedal, shift lever, or steering, will be referred to as the normal autonomous driving mode to distinguish it from the forced autonomous driving mode.

[0021] 1, in the electronic control circuit unit 10, a control unit 101 configured by mounting a computer is connected to a motor drive control unit 102, a steering drive control unit 103, a manual / automatic driving mode switching control unit 104, a brake drive control unit 105, a radar group 106, a camera group 107, a sensor group 108, a surrounding moving object recognition unit 109, a current position detection unit 110, a display unit 111, a touch panel 112, a car navigation (hereinafter abbreviated as "car navigation") function unit 113, a target person information registration unit 114, a gait recognition and discrimination unit 115, a gait recognition and discrimination result correspondence processing unit 116, a confirmation and authentication unit 117, a door opening and closing drive unit 118, a voice input and output unit 119, and a voice message generation unit 120 via a system bus 100. Gait recognition is also called gait authentication, but the terms are used interchangeably.

[0022] A motor drive unit 131 is connected to the motor drive control unit 102. A steering drive unit 132 is connected to the steering drive control unit 103. A manual operation detection unit 133 is connected to the manual / automatic driving mode switching control unit 104, and a brake drive unit 134 is connected to the brake drive control unit 105. A car navigation database 135 is connected to the car navigation function unit 113. A target person information storage unit 136 is connected to the target person information registration unit 114.

[0023] Furthermore, an in-vehicle speaker 137, an exterior speaker 138, an in-vehicle microphone 139, and an exterior microphone 140 are connected to the audio input / output unit 119. The in-vehicle speaker 137 is used to emit audio to the interior of the autonomously driven vehicle 1, and the exterior speaker 138 is used to emit audio to the exterior of the autonomously driven vehicle 1. The in-vehicle microphone 139 is used to pick up audio from inside the autonomously driven vehicle 1, and the exterior microphone 140 is used to pick up audio from outside the autonomously driven vehicle 1. Of course, the volume of the in-vehicle speaker 137 and the exterior speaker 138 and the sensitivity of the in-vehicle microphone 139 and the exterior microphone 140 can be varied and may be different inside and outside the vehicle. For example, when a target person is found, the volume of the exterior speaker 138 may be increased and the sensitivity of the exterior microphone 140 may be increased.

[0024] Under the control of the control unit 101, the motor drive control unit 102 controls the supply of drive signals to the motor drive unit 131 of the self-driving vehicle 1, which is an electric vehicle of this embodiment, to control the start of driving of the self-driving vehicle 1, driving speed control (including brake control and accelerator control), driving stop, etc.

[0025] Under the control of the control unit 101, the steering drive control unit 103 controls the supply of a drive control signal to the steering drive unit 132 of the autonomous vehicle 1 of this embodiment, thereby controlling the course change of the autonomous vehicle 1.

[0026] The manual / automatic driving mode switching control unit 104 controls the switching of the driving mode of the automatically driven vehicle 1 between a manual driving mode and a normal automatic driving mode in response to a driving mode selection operation input via the touch panel 112.

[0027] Note that the input that triggers manual / automatic driving mode switching control unit 104 to switch the driving mode of autonomous vehicle 1 between the manual driving mode and the normal automatic driving mode is not limited to a selection operation input via touch panel 112, but may also be a voice input of a switching instruction via in-vehicle microphone 139. In the case of a voice input, the voice input of the switching instruction via in-vehicle microphone 139 is recognized, and the recognition result is supplied to manual / automatic driving mode switching control unit 104. Manual / automatic driving mode switching control unit 104 switches the driving mode based on the received voice recognition result.

[0028] The manual operation detection unit 133 receives operation information such as accelerator pedal operation, brake pedal operation, shift lever operation, and steering operation by the driver, and supplies this manual driving operation information to the manual / automatic driving mode switching control unit 104. In the normal automatic driving mode, when the manual operation detection unit 133 detects a manual driving operation by the driver in the manual operation detection unit 134, the manual / automatic driving mode switching control unit 104 performs control to switch to the manual driving mode.

[0029] However, in this embodiment, in the forced automatic driving mode, even if a predetermined operation to return to the manual driving mode is performed, the manual / automatic driving mode switching control unit 104 invalidates the operation instruction and maintains the automatic driving mode (normal automatic driving mode). Then, in this embodiment, when there is an instruction from the control unit 101 to cancel the forced automatic driving mode, the manual / automatic driving mode switching control unit 104 switches from the forced automatic driving mode to the normal automatic driving mode.

[0030] When the autonomous vehicle 1 is in manual driving mode, the manual / autonomous driving mode switching control unit 104 supplies manual driving operation information from the manual operation detection unit 133 to the motor driving control unit 102 and the steering driving control unit 103, and controls the motor driving unit 131 and the steering driving unit 132 in accordance with the driver's pedal operation, shift lever operation, and steering operation (handle operation).

[0031] Furthermore, when the autonomous vehicle 1 is in normal autonomous driving mode or forced autonomous driving mode, the manual / autonomous driving mode switching control unit 104 supplies autonomous driving operation information generated by the control unit 101 based on the outputs of the radar group 106, the camera group 107, the sensor group 108, and the surrounding moving object recognition unit 109 to the motor drive control unit 102 and the steering drive control unit 103, as described below, and controls the motor drive unit 131 and the steering drive unit 132 to drive using the autonomous driving operation information, thereby enabling autonomous driving. Note that in normal autonomous driving mode, the car navigation function unit 113 searches for a route from the current position to a destination set by the driver or the like, and controls the vehicle to travel along the searched route. Also, in forced autonomous driving mode, as described below, the vehicle is controlled to stop at a location where it can call out to a target person detected by gait recognition.

[0032] In the case of a fully autonomous vehicle, since there is only an autonomous driving mode, there is no need for control to switch between manual driving mode and autonomous driving mode, and there is no manual / autonomous driving mode switching control unit 104 or manual operation detection unit 133. Furthermore, in the case of a fully autonomous vehicle, since the vehicle is always in autonomous driving mode, the concept of forced autonomous driving mode is also unnecessary.

[0033] Under the control of the control unit 101, the brake drive control unit 105 controls the supply of a drive control signal to the brake drive unit 134 of the autonomous vehicle 1 of this embodiment, thereby controlling the autonomous vehicle 1 to slow down or stop.

[0034] The radar group 106 is used to measure the distance to people and objects around the autonomous vehicle 1, and is made up of one or more laser radars (officially known as LIDAR (Light Detection and Ranging or Laser Imaging Detection and Ranging)) and millimeter-wave radars. The laser radars are embedded, for example, in the ceiling or near the bumper, and the millimeter-wave radars are installed, for example, at the front and rear of the vehicle. The vehicle may be equipped with both laser radar and millimeter-wave radar, or with only one of them. Other radars, such as microwave radar, may also be used. Furthermore, sonar (not shown) can be used for the same purpose as radar.

[0035] Camera group 107 includes one or more cameras that capture images of the interior of autonomously driven vehicle 1, and one or more cameras that capture images of the surroundings outside autonomously driven vehicle 1. The cameras that capture images of the interior of the vehicle are attached, for example, to a rearview mirror (rear-view mirror, room mirror) installed between the driver's seat and passenger seat or to the top of the front window, and include cameras that capture images of the behavior of the person sitting in the driver's seat (driver), as well as cameras that capture images of the behavior of passengers (fellow passengers) sitting in the passenger seat or in the back seat.

[0036] The cameras capturing images of the surroundings outside the autonomous vehicle 1 are intended to capture images of the visible areas outside the vehicle, such as in front, to the sides, and behind. Examples of such cameras include two cameras (stereo cameras) attached to the left and right sides of the rearview mirror and primarily capturing images of the left and right front of the autonomous vehicle 1, cameras attached to, for example, the door mirrors or fender mirrors of the autonomous vehicle 1 and capturing images of the left and right sides, and cameras capturing images behind the autonomous vehicle 1. Furthermore, an air vehicle such as a drone equipped with a camera capturing images of the surroundings outside the vehicle (not shown) may be installed on the autonomous vehicle 1, and may fly in the sky as needed in cooperation with the autonomous vehicle 1, performing gait recognition from images captured in the sky. In this case, the air vehicle such as the drone and the autonomous vehicle 1 can be connected via a wired or wireless connection, and images captured by the equipped cameras are transmitted to the autonomous vehicle 1 via a wired or wireless connection. These cameras may also include a night-vision camera or an infrared camera for performing gait recognition at night.

[0037] The sensor group 108 includes an open / close detection sensor that detects whether a door or window is open or closed, a sensor that detects whether a seat belt is fastened, a seating sensor (e.g., a weight sensor) that detects whether a passenger is seated in a seat such as the driver's seat or passenger seat, a touch sensor (e.g., a capacitance sensor) that detects whether a person touches the steering wheel of the driver's seat, a human presence sensor (e.g., an infrared sensor) that detects a person nearby outside the vehicle, and various other sensors for acquiring information that assists autonomous driving. The various sensors for acquiring information that assists autonomous driving include, for example, a vibration sensor that detects vibrations of the vehicle or tires, a rotation speed sensor that detects the number of tire rotations, a geomagnetic sensor that detects direction, an acceleration sensor that detects acceleration, and a gyro sensor (gyroscope) that detects angles and angular velocities. In this embodiment, the sensor group 108 also includes sensors that detect the illumination of right and left turn signals (direction indicators) and hazard lights (emergency flashers).

[0038] The surrounding moving object recognition unit 109 recognizes moving objects (including people) around the vehicle using images captured by the radar group 106, the sensor group 108, and the camera group 107. The surrounding moving object recognition unit 109 recognizes surrounding obstacles and moving objects by performing processing based on machine learning such as Bayes' theory or deep learning.

[0039] The current position detection unit 110 receives radio waves from GPS satellites to detect the current position of the vehicle. Because the accuracy of the position detected by radio waves from GPS satellites is poor, the current position detection unit 110 uses not only the information on the current position detected by receiving radio waves from GPS satellites, but also images captured by one or more sensors included in the sensor group 108, the radar group 106, and the camera group 107 (also using a navigation function), and performs processing based on machine learning such as Bayes' theorem or deep learning, thereby detecting and confirming the current position with higher accuracy.

[0040] In the autonomous driving mode (including normal autonomous driving mode and forced autonomous driving mode) of the autonomous vehicle 1, the current position detection unit 110 and the surrounding moving object grasping unit 109 process various information such as location information obtained by receiving radio waves from the radar group 106, the camera group 107, the sensor group 108, and GPS satellites, in other words, information corresponding to information obtained from the human eyes and ears, using machine learning such as Bayes' theory and deep learning, and based on this, the control unit 101 performs intelligent information processing (artificial intelligence) and control (artificial intelligence) such as changing the vehicle's course and avoiding obstacles, to generate autonomous driving operation information.

[0041] The display unit 111 is made up of, for example, an LCD (Liquid Crystal Display). The touch panel 112 is configured by superimposing a touch sensor on the display screen of the display unit 111, which is made up of an LCD, and which allows touch input with a finger, a touch pen, or the like. On the display screen of the display unit 111, a display image including software buttons (including character input buttons on a keyboard) is displayed under the control of the control unit 101. When the touch panel 112 detects a touch with a finger, a touch pen, or the like on a software button displayed on the display screen, it transmits the touch to the control unit 101. In response to this, the control unit 101 is configured to execute control processing corresponding to the software button. Of course, there may be multiple display units, and it is also possible to install a display unit facing outward on the body or window glass of the autonomously driven vehicle 1.

[0042] Domestic maps and route guidance data are stored in advance in car navigation database 135 connected to car navigation function unit 113. Car navigation function unit 113 is a functional unit that provides guidance to assist autonomous vehicle 1 in traveling to a specified destination based on the maps and route guidance data stored in car navigation database 135. In this embodiment, car navigation function unit 113 is configured to perform slightly different processes in manual driving mode and normal autonomous driving mode.

[0043] That is, in the manual driving mode, the car navigation function unit 113 displays an image on the display screen of the display unit 111 in which the vehicle position detected and confirmed by the current position detection unit 110 is superimposed on a map that clearly displays the route to the destination, and also moves the vehicle position (current position) on the map as the vehicle moves, and provides voice guidance at points where route guidance is necessary, such as intersections and branching points on the route. This is the same as the normal car navigation function.

[0044] On the other hand, in the normal autonomous driving mode, when the current position of the vehicle is away from the route to the destination, the car navigation function unit 113 notifies the control unit 101 of information on the direction and distance of the departure, and when the current position of the vehicle is on the route to the destination, the car navigation function unit 113 notifies the control unit 101 of information instructing the control unit 101 to change the course direction along the route before an intersection or branch point on the route as the vehicle moves. Based on the information notified from the car navigation function unit 113, the current position confirmation result by the current position detection unit 110, and the recognition result by the surrounding moving object recognition unit 109, the control unit 101 controls the motor drive unit 131 via the motor drive control unit 102 so that the vehicle moves along the route as instructed, and generates autonomous driving operation information for controlling the steering drive unit 132 via the steering drive control unit 103. Therefore, thanks to route guidance to the destination by the car navigation function unit 113 and the control unit 101 in the normal autonomous driving mode, autonomous vehicle 1 can travel to the destination even when there are no passengers.

[0045] The target person information registration unit 114 accepts registration input of information about target people whose gaits should be recognized, such as users of the autonomously driven vehicle 1 of this embodiment, such as family members or group members who share the autonomously driven vehicle 1, and stores and registers the information in the target person information storage unit 136. The target person information includes gait recognition information used for gait recognition discrimination, as well as identification information for the target person and information for enabling the target person to notice when called, and information for confirmation and authentication when the target person attempts to get into the vehicle.

[0046] In this embodiment, the information for gait recognition includes gait information and gait recognition auxiliary information of the target person. The target person information registration unit 114 extracts a walking silhouette image of the target person from an image captured by the camera and analyzes the walking silhouette image to generate gait information using the target person's stride length, limb swing, posture, walking cycle, left-right asymmetry of movement, and other feature quantities. The generated gait information is stored in the target person information storage unit 136 in association with the target person's identification information. The gait recognition auxiliary information includes, for example, the target person's height and head and skin texture information, and is information for improving the recognition rate of gait recognition. The target person information registration unit 114 obtains this gait recognition auxiliary information through input by the target person or from image information captured by the camera, and stores it in the target person information storage unit 136 in association with the target person's identification information.

[0047] It should be noted that gait recognition (gait authentication) using only gait information can authenticate individuals with an accuracy of 94 to 95%, but it has been reported that the authentication rate can be increased to 99% if auxiliary gait recognition information is used (for example, see "New exhibit at the Miraikan National Museum of Emerging Science and Innovation: More accurate than facial authentication!? What is gait authentication that can identify individuals by their "way of walking"; http: / / monoist.atmarkit.co.jp / mn / article / 1507 / 17 / news037.html"). The registration process and target person information in this target person information registration unit 114 will be described in detail later.

[0048] In this embodiment, the information used for verification and authentication when a target person attempts to get into the vehicle is the target person's facial image, spoken voice, fingerprint, vein, iris, or other biometric information. The verification and authentication information is not limited to biometric information such as a facial image or spoken voice, but may also be a telephone number, email address, password, or password. Furthermore, the key to the autonomously driven vehicle 1 may be used as the verification and authentication information, or a smartphone with a built-in contactless IC card may be held over a predetermined position on the autonomously driven vehicle 1, and the autonomously driven vehicle 1 may use the verification and authentication information recorded on the contactless IC card. Furthermore, a wristwatch-type terminal may be used instead of a smartphone.

[0049] In this embodiment, the gait recognition and discrimination unit 115 has a function of accepting a selection setting by the driver and / or passenger of target persons to be detected prior to gait recognition. That is, in the autonomously driven vehicle 1 of this embodiment, the driver and / or passenger can select and set one or more specific target persons from among registered target persons as target persons to be detected. Of course, it is also possible to select and set all registered target persons as target persons to be detected. Note that when the autonomously driven vehicle 1 of this embodiment autonomously drives in the normal autonomous driving mode in an unmanned state without a driver and / or passenger, all of the target persons stored and registered in the target person information storage unit 136 are automatically set as target persons to be detected.

[0050] Note that the autonomously driven vehicle 1 of this embodiment is configured so that when a target person gets in, the person is authenticated by the confirmation and authentication unit 117, so that target people who are already in the vehicle can be recognized. Therefore, target people who are already in the vehicle are configured to be excluded from target people to be detected by gait recognition. Of course, if a target person gets out of the vehicle, even if they were once excluded from target people to be detected by gait recognition, they will become a target person to be detected by gait recognition as soon as they get out of the vehicle.

[0051] In this example, the gait recognition and discrimination unit 115 extracts gait information from captured image information from one or more cameras in the camera group 107 that capture images of the field of view outside the vehicle, using feature quantities such as the stride length, swing of limbs, posture, walking cycle, and left-right asymmetry of movement of each person included in the captured image. The gait recognition and discrimination unit 115 compares the extracted gait information with the gait information of a target person who is set as a target person to be detected among registered target persons stored in the target person information storage unit 136 to perform gait recognition, and determines whether a person with a gait matching the gait of the target person to be detected has been detected. In this case, it goes without saying that the gait recognition and discrimination unit 115 of this embodiment also performs gait recognition using gait recognition auxiliary information. Furthermore, if a walking target person is walking toward the autonomously driven vehicle 1, face recognition may also be used.

[0052] When gait recognition and discrimination unit 115 detects, through the above-described gait recognition and discrimination, a person whose gait matches the gait of a registered target person stored in target person information storage unit 136, it transfers the discrimination result that the detected target person has been detected, including identification information of the target person having a gait that matches the gait of the detected person, to gait recognition and discrimination result response processing unit 116 via system bus 100.

[0053] Gait recognition discrimination result response processor 116 performs predetermined processing upon receiving the discrimination result from gait recognition discrimination unit 115. In this embodiment, gait recognition discrimination result response processor 116 has a function for receiving setting input for selecting a target person to be detected by the driver, passenger, or owner of the vehicle, as will be described later, and setting input for predetermined processing, and determines what kind of processing to perform as the predetermined processing based on the setting input.

[0054] The gait recognition discrimination result response processing unit 116 treats the above setting input as valid as is when the driving mode of the autonomous vehicle 1 is a manual driving mode, as well as when the driver and / or passengers are on board, even when the driving mode is a normal autonomous driving mode in which autonomous driving is performed.

[0055] However, when the vehicle is autonomously traveling in normal autonomous driving mode in an unmanned state without a driver and / or passenger, the gait recognition determination result response processing unit 116 validates the setting input for selecting a target person to be detected, but performs the predetermined processing appropriate for the unmanned normal autonomous driving mode. That is, in this embodiment, when a target person is detected by gait recognition in the unmanned normal autonomous driving mode, the predetermined processing is to stop near the detected person, call out to the detected person, and send a message inviting the person to board, and when the detected person replies that they will board, the confirmation authentication unit 117 confirms and authenticates that the person is the target person, and then drives and controls the door opening / closing drive unit 118 to open and close the doors and allow the person to board.

[0056] As will be described later, when the gait recognition determination result response processing unit 116 is set to stop the vehicle near the detected target person, it switches the driving mode to forced automatic driving mode and stops the vehicle regardless of whether the driving mode is manual driving mode or normal automatic driving mode.

[0057] It is also possible to control the vehicle to stop near a detected target person while remaining in normal automatic driving mode, without switching to forced automatic driving mode. In this case, however, the driver must manually switch to manual driving mode, allowing the vehicle to pass by the detected target person without stopping near them, and this configuration is also acceptable. This configuration is convenient when the vehicle is in a hurry, for example, while transporting a sick person to a hospital. It is also possible to easily determine whether a driver and / or passengers are present or unoccupied by monitoring images captured by a camera in the camera group 107 that captures the interior of the vehicle. It is also possible to monitor the opening and closing of the doors and passengers getting off and getting on.

[0058] The processing of setting input to the gait recognition determination result processing unit 116 and the predetermined processing when a target person is detected by gait recognition will be described in detail later.

[0059] The confirmation authentication unit 117 performs confirmation authentication when a target person detected by the gait recognition and discrimination unit 115 is about to get into the vehicle. In this embodiment, the target person information storage unit 136 stores a facial image and / or spoken voice of the registered target person as information for confirmation authentication. In this embodiment, the confirmation authentication unit 117 includes an image recognition function unit for facial image authentication and a voice recognition function unit for spoken voice authentication.

[0060] The image recognition function unit of confirmation and authentication unit 117 performs confirmation and authentication based on whether the facial image of the person about to get on, detected by gait recognition and discrimination unit 115 and photographed by a camera in camera group 107, matches or does not match the facial image of the target person stored in target person information storage unit 136. In addition, the voice recognition function unit of confirmation and authentication unit 117 performs confirmation and authentication based on whether the spoken voice of the person about to get on, detected by gait recognition and discrimination unit 115 and photographed by microphone 140, matches or does not match the spoken voice of the target person stored in target person information storage unit 136.

[0061] As described above, the authentication result in the confirmation authentication unit 117 is passed to the gait recognition discrimination result handling processing unit 116 and is used to determine whether or not to allow the person detected by the gait recognition discrimination unit 115 to board the vehicle.

[0062] Note that authentication can also be performed using the fingerprint of the person attempting to board the vehicle. In this case, the autonomously driven vehicle 1 is provided with a fingerprint reader, the fingerprint of the person is stored in the target person information storage unit 136, and the authentication unit 117 is configured with a fingerprint recognition function to perform authentication by determining whether the stored fingerprint matches the fingerprint acquired by the fingerprint reader. Vein, iris, voiceprint, and other biometric information can also be used for authentication, and these can be achieved by modifying the configuration in a similar manner. Naturally, a combination of biometric information such as facial images, voice, and fingerprints may also be stored and used for authentication.

[0063] Furthermore, confirmation and authentication can be performed by the person intending to board the vehicle holding a key to the autonomously driven vehicle 1. As mentioned above, confirmation and authentication can also be performed by holding a smartphone with a built-in contactless IC card over a predetermined position on the autonomously driven vehicle 1. Furthermore, a wristwatch-type terminal may be used instead of a smartphone.

[0064] The door opening / closing drive unit 118 is a drive unit for automatically opening and closing the doors of the vehicle. The door opening / closing drive unit 118 is controlled and driven in response to an opening / closing command from the control unit 101 in response to an operation of the touch panel 112 or other buttons, and also in response to an opening / closing command from the gait recognition determination result corresponding processing unit 116, as described above. The doors may be of a sliding type or a swing-open / close type.

[0065] The audio input / output unit 119 takes in audio picked up by the in-vehicle microphone 139 or the exterior microphone 140 and sends it to the system bus 100 for, for example, audio recognition processing. The audio input / output unit 119 also incorporates a voice synthesizer and a DA converter that receive audio message data to be emitted inside or outside the vehicle from the audio message generation unit 120 and convert the data into an analog audio signal. Then, under the control of the control unit 101, the audio input / output unit 119 supplies the converted analog audio signal to the in-vehicle speaker 137 or the exterior speaker 138, which then emits the audio.

[0066] The voice message generation unit 120, under the control of the control unit 101, generates voice message information to be emitted to the driver and / or passengers, and also generates a calling voice message to call out to a target person detected by gait recognition, and sends it to the voice input / output unit 119.

[0067] The target person information storage unit 136 stores, for example, a call name (nickname) as information that enables the target person to recognize that the call is directed to them, and the voice message generation unit 120 synthesizes this stored call name of the target person with a template voice message to generate the required voice message. Of course, the call name of the target person is not limited to a nickname, and may be a real name, first name, initials, or number.

[0068] For example, in the case of a voice message to notify the driver and / or passenger that a target person has been detected, message information such as "○○○○ has been detected" is prepared as a template voice message, and information on the address name of the detected target person is obtained from the target person information storage unit 136.If the address name is "Miko-chan," "Miko-chan" is inserted in the "○○○○" part to generate the voice message "Miko-chan has been detected."

[0069] The voice message generating unit 120 has prepared other template voice messages such as "Shall we stop at XXX and let you board?" for in-car messages and "XXX! Would you like a ride?" for out-of-car messages. For example, when the detected target person calls out to "Miko-chan," the message for in-car messages is "Shall we stop at Miko-chan and let you board?", and the message for out-of-car messages is "Miko-chan! Would you like a ride?".

[0070] The electronic control circuit unit 10 of the autonomous vehicle 1 is configured as described above. In the above description, the motor drive control unit 102, steering drive control unit 103, manual / autonomous driving mode switching control unit 104, surrounding moving object grasping unit 109, current position detection unit 110, car navigation function unit 113, target person information registration unit 114, gait recognition and discrimination unit 115, gait recognition and discrimination result response processing unit 116, confirmation and authentication unit 117, and voice message generation unit 120 shown in FIG. 1 can each be configured as a processing function unit using a software program, and by providing a program for executing each function in the control unit 101, each can be realized as software processing. It goes without saying that instead of configuring all of the above units as processing functions using programs, some can be configured as hardware components.

[0071] [Example of Target Person Information Registration Procedure in Target Person Information Registration Unit 114] FIG. 2 is a diagram showing an example of the flow of a target person information registration procedure in the target person information registration unit 114. As shown in FIG.

[0072] First, the target person or the registration assistant requests the self-driving vehicle 1 to register the target person information by selecting the target person information registration item from the function menu of the self-driving vehicle 1 displayed on the display unit 111 (Step S1).

[0073] In response to this request to register target person information, control unit 101 instructs the target person to perform a gesture for detecting the target person's gait information by using an instruction message displayed on the display screen of display unit 111 and / or an audible message emitted from speakers 137, 138 (step S2). Specifically, this instruction instructs the target person to walk so that a predetermined camera in camera group 107 can capture the target person's walking state. In this case, instructions may be given to the target person to walk on the left side of autonomous vehicle 1 from the rear to the front of the vehicle, or to walk in the opposite direction from the front to the rear of the vehicle, or to walk in a diagonal left direction or the opposite direction, a diagonal right direction or the opposite direction, crossing from left to right, or crossing from right to left. The instructions for the target person's walking style are not limited to these, and a wide variety of walking styles can be instructed. Furthermore, if the person is asked to walk in the opposite direction from the front to the rear of the vehicle, or if the person is walking diagonally or crossing left and right, facial images of the person can be captured at the same time, so the facial images can be registered along with the gait information.

[0074] It is also possible to register by dividing into sunny days, cloudy days, rainy days, and snowy days, and by dividing into daytime and nighttime. It is also possible to register by dividing into indoor and outdoor. By further expanding this, it is possible to register not only walking but also running states. When dividing into daytime and nighttime and by registering, it is also possible to take pictures with a normal camera during the day and with an infrared camera at night.

[0075] When the target person walks according to the instructions in step S2, control unit 101 causes a predetermined camera in camera group 107 to capture video of the target person walking, and sends the captured image information to target person information registration unit 114 (step S3). Then, control unit 101 determines whether sufficient information has been acquired to enable detection of the target person's gait information (step S4). Then, control unit 101 repeats steps S2 and S3 until sufficient information has been acquired to enable detection of the target person's gait information.

[0076] Next, in step S4, when it is determined that sufficient information has been acquired to enable detection of the target person's gait information, the control unit 101 instructs the target person information registration unit 114 to start the gait information extraction process, and causes the target person's feature quantities, such as stride length, limb swing, posture, walking cycle, and left-right asymmetry of movement, to be detected and stored as the target person's gait information (step S5).

[0077] Next, the control unit 101 prompts the target person to input information such as height and gender, which will serve as gait recognition auxiliary information, and acquires the input information. At the same time, the control unit 101 acquires the target person's head and skin texture as gait recognition auxiliary information from the image captured by the camera, and stores them in the target person information registration unit 114 (step S6).

[0078] Next, the control unit 101 prompts the target person to provide information for verification and authentication, and acquires the target person's face image and voice information in response, and stores them in the target person information registration unit 114 (step S7).

[0079] Furthermore, the control unit 101 prompts the target person to input a nickname (nickname) as an example of information to enable the target person to recognize that the call is directed at them, and acquires the information on the nickname (nickname) input by the target person in response, and stores it in the target person information registration unit 114 (step S8).

[0080] Finally, the control unit 101 assigns identification information to the target person, instructs the target person information registration unit 114 to store the target person's gait information, gait recognition auxiliary information, confirmation authentication information, and address name (nickname) information that it holds in the target person information storage unit 136 in association with the target person's identification information, and executes the storage (step S9).

[0081] This completes the registration of the target person information. For example, if the self-driving car 1 is for personal use, the registration of this target person information is performed for each family member. Furthermore, in a case where the self-driving car 1 is used jointly by a group, such as in car sharing, the registration of target person information is performed for each group member that makes up the group. Note that the task of registering the target person information does not have to be performed only by the self-driving car 1; family members or group members may also record their walking (or running) with a smartphone, video camera, or the like, and transmit this recorded information to the self-driving car 1. Note that the recording may also be performed by a drone (unmanned aerial vehicle) using a camera mounted on the drone as it flies and moves, following the target person.

[0082] An example of the contents of information stored in the target person information storage unit 136 in this manner is shown in Fig. 3. In the example of Fig. 3, family members are registered as target persons, and the target person information for each of four people, a father, a mother, a daughter, and an older brother, is registered and stored.

[0083] [Pre-setting of target person's gait recognition] The autonomous vehicle 1 of this embodiment is configured to accept pre-settings from the driver and / or passengers via a menu before starting gait recognition of a target person. The pre-setting items are:

[0084] (1) Selection of target person to be detected by gait recognition

[0085] (2) Setting the predetermined process to be executed when the target person is detected is.

[0086] Regarding (1), it is possible to set the detection target to be all of the target persons stored in the target person information storage unit 136, or to set it to be one or more selected target persons instead of all of them.

[0087] Regarding (2), when a target person is detected, it is possible to set whether or not the vehicle will always stop (unconditional stop setting), to stop only when there is room for passengers in the vehicle, to temporarily stop to leave a message, etc., to stop after asking the driver and / or passengers, or to notify the driver and / or passengers without stopping. Gait recognition auxiliary information such as the height of the target person can also be set.

[0088] This pre-setting acceptance function is activated when the driver and / or passengers get into the autonomous vehicle 1, by selecting it from the settings menu or operating the pre-setting acceptance button.

[0089] The function for accepting pre-settings may be preset as a default in the autonomous vehicle 1 so that it always automatically activates when the autonomous vehicle 1 detects that the driver and / or passengers have boarded the vehicle. This function for accepting pre-settings is not limited to being activated when the driver and / or passengers board the vehicle. The function may be configured to accept pre-settings by inquiring the driver and / or passengers about whether to make pre-settings when the driving mode is switched, such as when switching from manual driving mode to autonomous driving mode or when switching from autonomous driving mode to manual driving mode. Furthermore, the function may be configured to accept pre-settings when pre-settings have already been made, rather than accepting pre-settings when the driving mode is switched. The function may, of course, be configured to accept pre-settings without inquiring the driver and / or passengers about whether to make pre-settings or change the settings.

[0090] An example of a processing routine for presetting the gait recognition of a target person in this embodiment will be described below with reference to the flowcharts of Figures 4 and 5. The function of presetting the gait recognition of a target person is included in gait recognition discriminator 115, but in the following description of the example of Figures 4 and 5, the gait recognition discriminator 115 will be described as a functional block defined by a software program included in control unit 101, and the operation of each step will be described as being executed by control unit 101.

[0091] First, the control unit 101 displays a list of function menus on the display screen of the display unit 111 (step S11). Then, the control unit 101 determines whether or not a pre-setting item for gait recognition of a target person has been selected from the list display of this function menu (step S12).

[0092] In step S12, when it is determined that an item other than the pre-set item for gait recognition of the target person has been selected from the list display of the function menu, the control unit 101 proceeds to a processing routine for the selected other menu item and executes that processing routine.

[0093] When it is determined in step S12 that the item for presetting the gait recognition of a target person has been selected from the list display of the function menu, control unit 101 displays a list of target persons stored in target person information storage unit 136 on the display screen, and prompts the user to select and specify a target person as a detection target for gait recognition (step S14). When selecting and specifying a target person as a detection target for gait recognition, information on a facial image of the target person may be read from target person information storage unit 136, and the facial image of the target person may be displayed on the display screen of display unit 111 so that the target person can be referred to for selection and identification.

[0094] Then, the control unit 101 determines whether or not it has been selected to make all of the target persons stored in the target person information storage unit 136 the detection targets for gait recognition (step S15), and if it determines that it has been selected to make all of the target persons the detection targets for gait recognition, it sets all of the target persons to be the detection targets for gait recognition (step S16).

[0095] Furthermore, if it is determined in step S15 that not all persons have been selected as detection targets for gait recognition, the control unit 101 accepts individual selection of target persons to be detection targets for gait recognition (step S17), and sets the selected target persons to be detection targets for gait recognition (step S18). The number of selected persons is not limited to one, and may be multiple.

[0096] After the setting in step S16 is completed, or after the setting in step S18 is completed, the control unit 101 determines whether the setting "Always stop when a target person is detected by gait recognition" is selected (step S19), and if it determines that the setting "Always stop when a target person is detected by gait recognition" is selected, it sets an unconditional stop (step S20).

[0097] After step S20, the control unit 101 determines whether or not an instruction to perform gait recognition has been received (step S39 in FIG. 5), and if it determines that an instruction to perform gait recognition has been received, it starts a processing routine for detecting a target person using gait recognition (step S40), and ends the processing routine of FIGS. 4 and 5.

[0098] If it is determined in step S39 that an execution instruction has not been received, the control unit 101 determines whether or not an instruction input to stop the processing routine of this setting has been received (step S41), and if it is determined that an instruction input to stop has been received, the control unit 101 ends this processing routine. If it is determined in step S41 that an instruction input to stop has not been received, the control unit 101 returns the process to step S19 in Fig. 4 and repeats the processes from step S19 onwards.

[0099] If it is determined in step S19 that the setting "Always stop when a target person is detected by gait recognition" has not been selected, the control unit 101 determines whether or not the setting "Stop taking into consideration the passenger capacity of the vehicle" has been selected (step S31 in FIG. 5). If it is determined in step S31 that the setting "Stop taking into consideration the passenger capacity of the vehicle" has been selected, the control unit 101 further determines whether or not the setting "Inquiry to the driver and / or passengers is required" has been selected (step S32).

[0100] If it is determined in step S32 that the setting "inquiry to driver and / or passengers required" has been selected, the control unit 101 sets "stop conditional on passenger capacity, inquiry required" (step S33). Also, if it is determined in step S32 that the setting "inquiry to driver and / or passengers not required" has been selected, the control unit 101 sets "stop conditional on passenger capacity, inquiry not required" (step S34). Then, after step S33 or after step S34, the control unit 101 proceeds to step S39, and the above-mentioned processing from step S39 onwards is repeated.

[0101] Furthermore, when it is determined in step S31 that the setting "stop taking into consideration the passenger capacity of the vehicle" has not been selected, the control unit 101 determines whether the setting "decide whether to stop by inquiring of the driver and / or passengers" has been selected (step S35). When it is determined in step S35 that the setting "decide whether to stop by inquiring of the driver and / or passengers" has been selected, the control unit 101 sets "inquiry-conditional stop" (step S36). Then, after step S36, the process proceeds to step S39, and the processes from step S39 onwards are executed.

[0102] Then, when it is determined in step S35 that the setting "ask the driver and / or passengers to decide whether to stop or not" has not been selected, the control unit 101 determines whether the setting "when a target person is detected by gait recognition, only notify the detection" has been selected (step S37).

[0103] If it is determined in step S37 that the setting "When a target person is detected by gait recognition, only notify the detection" has been selected, the control unit 101 sets "Only notify detection of target person" (step S38). After step S38, the process proceeds to step S39, and the processes from step S39 onward are executed.

[0104] Furthermore, if it is determined in step S37 that the setting "When a target person is detected by gait recognition, only notify the detection" has not been selected, the control unit 101 proceeds to step S41, where it determines whether or not an instruction input to stop the processing routine for this setting has been received (step S41), and if it determines that an instruction input to stop has been received, it ends this processing routine. If it determines in step S41 that an instruction input to stop has not been received, the control unit 101 returns the processing to step S19 in Fig. 4, and repeats the processing from step S19 onwards.

[0105] [Detection of target person using gait recognition] Next, an example of the processing flow for detecting a target person by gait recognition, which is started in step S40 of Fig. 5, will be described with reference to the flowcharts shown in Fig. 6 to Fig. 10. This processing for detecting a target person by gait recognition is executed by gait recognition discrimination unit 115 following the settings explained above using Fig. 4 and Fig. 5, but in the following explanation, it is assumed that gait recognition discrimination unit 115 is configured as a functional block of a software program included in control unit 101, and the operation of each step will be explained as being executed by control unit 101.

[0106] Control unit 101 first determines whether the detection targets are all of the target persons stored in target person information storage unit 136 (step S51 in FIG. 6). If it is determined in step S51 that the detection targets are all of the target persons stored in target person information storage unit 136, control unit 101 starts detection by gait recognition, with all of the stored target persons as detection targets (step S52). On the other hand, if it is determined in step S51 that the detection targets are not all of the target persons stored in target person information storage unit 136, control unit 101 recognizes the selected and specified target person, and starts detection of the target person by gait recognition, with only the recognized target person as the detection target (step S53).

[0107] After step S52, or after step S53, the control unit 101 determines whether the current driving mode of the vehicle is a normal automatic driving mode or a manual driving mode. If it is determined in step S54 that the driving mode is the manual driving mode, it executes target person detection using gait recognition and determines whether the target person has been detected (step S55). If it is determined in step S55 that the target person has not been detected, the control unit 101 returns the process to step S54 and repeats the processes from step S54 onwards. Note that in step S55, if a facial image is registered along with the gait information, facial image recognition may also be used to detect the target person.

[0108] When it is determined in step S55 that a target person has been detected, the control unit 101 determines that a driver is present because the driving mode is the manual driving mode, and obtains the call name (nickname) of the detected target person from the target person information storage unit 136 and notifies the driver of who the detected target person is (step S56). In addition to notifying the driver by the call name using voice, the notification at this time may also be made by reading out information on a facial image of the detected target person from the target person information storage unit 136 and displaying the facial image of the target person on the display screen of the display unit 111.

[0109] Next, the control unit 101 determines whether or not an unconditional stop has been set according to a pre-setting (step S57). If it is determined in step S57 that an unconditional stop has been set, the control unit 101 notifies the driver that the driving mode will be switched to a forced automatic driving mode and the vehicle will be stopped (step S58), and then executes control to switch from the manual driving mode to the forced automatic driving mode and stop the vehicle near the detected target person (step S59).

[0110] Next, the control unit 101 determines whether the host vehicle has stopped (step S60), and if it determines that the host vehicle has not stopped, returns the process to step S59 and executes stop control. Then, if it determines in step S60 that the host vehicle has stopped, the control unit 101 cancels the forced automatic driving mode and switches the driving mode to the normal automatic driving mode (step S71 in FIG. 7).

[0111] Next, the control unit 101 uses the function of the voice message generation unit 120 to generate a voice message calling out to the detected target person, including the target person's name (nickname) as described above, and supplies the voice message calling out to the vehicle exterior speaker 138 via the voice input / output unit 119, thereby calling out to the detected target person and alerting the target person (step S72).

[0112] In this embodiment, the control unit 101 then controls the voice input / output unit 119 to enable a conversation between the driver and / or passenger and the detected external target person (step S73). This allows the driver and / or passenger to leave any necessary messages or, if the number of passengers on board is less than the vehicle capacity, to urge the detected target person to board. If the detected target person requests to board in response to the urge to board, the driver instructs the door to open and close, allowing the person to board. Furthermore, if the driver simply wants to leave a message for the detected target person or if the number of passengers on board is the same as the vehicle capacity, the driver starts the vehicle without urging the detected target person to board.

[0113] Therefore, the control unit 101 determines whether or not an instruction to open or close the door has been received (step S74), and when it determines that an instruction to open or close the door has been received, it controls the driving of the door opening / closing drive unit 118 to open or close the door, and also confirms that the target person has boarded the vehicle from images captured by the cameras in the camera group 107 that capture images of the outside and / or inside of the vehicle (step S75). The image recognition for this boarding confirmation is performed using an image recognition function provided in the confirmation authentication unit 117.

[0114] Then, when it is determined in step S74 that a door opening / closing command has not been received, or after step S75, the control unit 101 determines whether or not the driver has performed an operation to start driving the vehicle, that is, whether or not the accelerator pedal has been operated (step S76). If it is determined in step S76 that the driver has performed an accelerator pedal operation, the control unit 101 switches the driving mode from the normal automatic driving mode to the manual driving mode and starts driving (step S77). Then, after step S77, the control unit 101 returns the process to step S55 in Fig. 6, restarts the process of detecting a target person using gait recognition, and repeats the process from step S55 onwards.

[0115] In this embodiment, the control unit 101 of the autonomous vehicle is configured to automatically open and close the doors, and therefore performs the door opening and closing control described above, but it is of course also possible for the control unit 101 to only perform door locking control, and allow the person attempting to board or the person currently in the vehicle to open and close the doors. In this case, the control unit 101 will unlock the doors based on an instruction from the driver or the person currently in the vehicle, or, as will be described later, based on the result of confirmation authentication as to whether the person attempting to board is the target person.

[0116] Furthermore, if it is determined in step S76 that the driver has not operated the accelerator, the control unit 101 determines whether or not a command to start driving in the normal automatic driving mode has been received (step S78). If it is determined in step S78 that a command to start driving in the normal automatic driving mode has not been received, the control unit 101 returns the process to step S76 and repeats the processes from step S76 onwards.

[0117] If it is determined in step S78 that an instruction to start driving in the normal autonomous driving mode has been received, the control unit 101 controls the vehicle to start driving in the normal autonomous driving mode (step S79). After step S79, the control unit 101 proceeds to step S101 in Fig. 10, where it performs a process of detecting a target person by gait recognition in the normal autonomous driving mode, which will be described later.

[0118] Next, in step S57 of Fig. 6, when it is determined that the preset setting is not an unconditional stop, the control unit 101 determines whether or not the preset setting is "taking passenger capacity into consideration when stopping" (step S81 of Fig. 8). When it is determined in this step S81 that the preset setting is "taking passenger capacity into consideration when stopping", the control unit 101 checks the number of passengers from an image captured by a camera of the camera group 107 that captures the interior of the vehicle, and determines whether or not the number of passengers is less than the capacity (step S82).

[0119] If it is determined in step S82 that the number of passengers is not less than the capacity (the number of passengers is the capacity), the control unit 101 follows the driver's operation for the subsequent processing (step S89). In this case, the control unit 101 may generate a voice message in the voice message generation unit 120 saying, "The capacity will be exceeded, so the detected target person cannot be allowed to board. If you want to stop, please drive manually," and may output this voice message through the in-vehicle speaker 137.

[0120] The driver and / or passenger who receives this voice message cannot allow the detected target person to board the vehicle, but if it is necessary to leave a message, they can stop the vehicle by operating the vehicle (applying the brakes) in manual driving mode and take the necessary action, such as leaving a message to the detected target person. Furthermore, if the driver and / or passenger determines that there is no need to stop because there is no message to leave, they can also pass by the vehicle without stopping near the detected target person by operating the vehicle in manual driving mode. Even in autonomous driving mode, if the driver and / or passenger pre-sets the vehicle to stop temporarily to leave a message, the vehicle can stop temporarily near the detected target person, leave a message, and continue autonomous driving, regardless of whether the number of passengers is at capacity or less.

[0121] After step S89, the control unit 101 returns the process to step S55 in FIG. 6, and repeats the processes from step S55 onwards.

[0122] Then, when it is determined in step S81 that "stopping does not take passenger capacity into consideration" is preset, or when it is determined in step S82 that the number of passengers is less than the capacity, the control unit 101 determines whether or not "stopping after inquiring of the driver and / or passengers" is preset (step S83). When it is determined in step S83 that "stopping after inquiring of the driver and / or passengers" is preset, the control unit 101 generates a voice message inquiring of the driver and / or passengers as to whether or not to stop near the detected target person in the voice message generation unit 120, and executes the inquiry by emitting the generated voice message through the voice input / output unit 119 and the in-vehicle speaker 137 (step S84).

[0123] Next, the control unit 101 determines whether or not a response to this inquiry that the vehicle will be stopped has been received (step S85). The response from the driver and / or passenger to this inquiry is determined by recognizing the voice picked up by the in-vehicle microphone 139 using the voice recognition function of the confirmation and authentication unit 117, or by monitoring the operation input by the driver and / or passenger through the touch panel 112.

[0124] If the driver and / or passenger responds in step S85 that they will not stop, the control unit 101 proceeds to step S89, and the subsequent processing is performed in accordance with the operation of the driver and / or passenger. In this case, the control unit 101 may generate a voice message in the voice message generating unit 120 saying, "If you want to stop, please operate manually," and output the message through the in-vehicle speaker 137.

[0125] Then, when it is determined in step S83 that "querying the driver and / or passengers to stop the vehicle" is not pre-set, and when a response is received in step S85 from the driver and / or passengers that the vehicle will be stopped, the control unit 101 notifies the driver and / or passengers that the driving mode will be switched from manual driving mode to forced automatic driving mode and the vehicle will be stopped (step S86), and then executes control to switch from manual driving mode to forced automatic driving mode and stop the vehicle near the detected target person (step S87).

[0126] Next, the control unit 101 determines whether the host vehicle has stopped (step S88), and if it determines that the host vehicle has not stopped, returns the process to step S87 and executes stop control. Then, if it determines in step S88 that the host vehicle has stopped, the control unit 101 cancels the forced automatic driving mode and switches the driving mode to the normal automatic driving mode (step S91 in FIG. 9).

[0127] Next, the control unit 101 generates a calling voice message including the calling name (nickname) of the detected target person as described above in the voice message generating unit 120, and emits the calling voice message through the voice input / output unit 119 and the exterior speaker 138, thereby calling out to the detected target person and drawing their attention (step S92).

[0128] Then, the control unit 101 determines whether the number of passengers is less than the capacity (step S93), and if the number of passengers is equal to the capacity, the process proceeds to step S76 in Fig. 7, and the process from step S76 onwards is executed. At this time, the control unit 101 may generate a voice message in the voice message generation unit 120, such as "The capacity will be exceeded, so the detected target person cannot board. Please operate the vehicle to start or give a start signal after you have finished your business," and emit this voice message through the in-vehicle speaker 137. Note that the start operation is for starting in the manual driving mode, and the start signal is for starting in the normal automatic driving mode.

[0129] If it is determined in step S93 that the number of passengers is less than the capacity, the control unit 101 causes the voice message generation unit 120 to generate a voice message inquiring of the detected target person outside the vehicle as to whether or not they wish to board, and emits the generated voice message via the voice input / output unit 119 and speaker 138, thereby inquiring of the detected target person (step S94).

[0130] Next, the control unit 101 receives a response to this inquiry from the target person outside the vehicle and determines whether or not the target person wishes to board (step S95). The response from the target person to this inquiry is determined by recognizing the voice picked up by the exterior microphone 140 using the voice recognition function of the confirmation and authentication unit 117.

[0131] If it is determined in step S95 that the target person does not wish to board, the control unit 101 shifts the process to step S76 in FIG. 7, and executes the process from step S76 onwards.

[0132] Furthermore, when it is determined in step S95 that the target person wishes to board, the control unit 101 acquires information for verification and authentication from the target person and performs verification and authentication (step S96). That is, the control unit 101 acquires the verification and authentication information for the detected target person from the target person information storage unit 136. If the verification and authentication information is a facial image, the control unit 101 captures and acquires a facial image of the target person using an exterior camera in the camera group 107, and compares and refers to the acquired facial image and the facial image of the target person read from the target person information storage unit 136 to verify and authenticate the target person. At this time, the image recognition function of the verification and authentication unit 117 is used. Furthermore, if the verification and authentication information is a spoken voice, the control unit 101 collects the spoken voice of the target person using the exterior microphone 140, and compares and refers to the collected spoken voice and the spoken voice of the target person read from the target person information storage unit 136 to verify and authenticate the target person. At this time, the voice recognition function of the verification and authentication unit 117 is used.

[0133] Next, the control unit 101 determines whether the confirmation and authentication result is OK (step S97). If the control unit 101 determines that the confirmation and authentication result is OK, it drives and controls the door opening / closing drive unit 118 to open and close the door, and confirms that the target person has boarded the vehicle from images captured by the cameras in the camera group 107 that capture the vehicle exterior and / or interior (step S98). The image recognition for confirming boarding is performed using an image recognition function provided in the confirmation and authentication unit 117. If the control unit 101 determines that the confirmation and authentication result is NG rather than OK, it causes the voice message generation unit 120 to generate a voice message indicating that boarding is NG, and notifies the detected person of the generated voice message via the voice input / output unit 119 and the vehicle exterior speaker 138 (step S99).

[0134] Then, after step S98, or after step S99, the control unit 101 shifts the process to step S76 in FIG. 7, and executes the processes from step S76 onwards.

[0135] Next, when it is determined in step S54 of Fig. 6 that the driving mode is the normal autonomous driving mode, or when traveling is started in the normal autonomous driving mode in step S79 of Fig. 7, the control unit 101 executes detection of a target person by gait recognition and determines whether or not the target person has been detected (step S101 of Fig. 10). If it is determined in step S101 that the target person has not been detected, the control unit 101 returns the process to step S54 of Fig. 6 and repeats the processes from step S54 onwards.

[0136] When it is determined in step S101 that a target person has been detected, the control unit 101 determines whether a driver and / or passenger is present (step S102) since the driving mode is normal automatic driving mode, and when it is determined that a driver and / or passenger is present, it obtains the address name (nickname) of the detected target person from the target person information storage unit 136 and notifies the driver and / or passenger of the identity of the detected target person (step S103).

[0137] Next, the control unit 101 determines whether or not an unconditional stop has been set in advance (step S104). If it determines in step S104 that an unconditional stop has been set, the control unit 101 notifies the driver and / or passengers that the vehicle will be switched to forced automatic driving mode and stopped, and executes control to switch from manual driving mode to forced automatic driving mode and stop the vehicle near the detected target person (step S105).

[0138] Next, the control unit 101 determines whether the host vehicle has stopped (step S106), and if it determines that the host vehicle has not stopped, returns the process to step S105 and executes stop control. Then, if it determines in step S106 that the host vehicle has stopped, the control unit 101 proceeds to step S91 in Fig. 9, cancels the forced automatic driving mode, switches the driving mode to the normal automatic driving mode, and then executes the processes from step S91 onwards.

[0139] If it is determined in step S104 that the preset setting is not an unconditional stop, the control unit 101 determines whether or not it is preset to "consider the passenger capacity when stopping" (step S107). If it is determined in step S107 that it is preset to "consider the passenger capacity when stopping", the control unit 101 determines whether or not the number of passengers is less than the capacity (step S108). If it is determined in step S108 that the number of passengers is less than the capacity, or if it is determined in step S107 that it is not preset to "consider the passenger capacity when stopping", the control unit 101 shifts the process to step S83 in Fig. 8 and executes the processes from step S83 onwards.

[0140] Furthermore, when it is determined in step S108 that the number of passengers is not less than the capacity (the number of passengers is the capacity), the control unit 101 causes the voice message generation unit 120 to generate a voice message stating, "The capacity will be exceeded, so the detected target person cannot be allowed to board. If you wish to stop, please drive manually," and causes this message to be played through the in-vehicle speaker 137, while continuing the normal automatic driving mode (step S111 in Figure 11).

[0141] The driver and / or passenger who receives this voice message cannot allow the detected target person to board the vehicle, but if it is necessary to leave a message, they can stop the vehicle by operating the vehicle (applying the brakes) in manual driving mode and take the necessary action, such as leaving a message to the detected target person. Also, if the driver and / or passenger determines that there is no need to stop because there is no message to leave, they can pass by the vehicle without stopping near the detected target person by operating the vehicle in manual driving mode. Note that even in autonomous driving mode, if the vehicle is preset to stop temporarily to leave a message, it can stop temporarily near the detected target person, leave a message, and continue autonomous driving, regardless of whether the number of passengers is at capacity or less.

[0142] Next, the control unit 101 determines whether or not a switching operation to the manual driving mode has been detected (step S112). If it determines that a switching operation to the manual driving mode has not been detected, the control unit 101 returns to step S101 in Fig. 10 and repeats the processing from step S101 onwards. If it determines in step S112 that a switching operation to the manual driving mode has been detected, the control unit 101 switches the driving mode from the normal automatic driving mode to the manual driving mode (step S113), and then performs processing in accordance with the driver's operation (step S114). Then, after step S114, the control unit 101 returns to step S55 in Fig. 6 and executes the processing from step S55 onwards.

[0143] Next, when it is determined in step S102 of Fig. 10 that there is no driver and / or passenger and the vehicle is being driven unmanned, the control unit 101 executes control to stop the vehicle near the detected target person while remaining in normal autonomous driving mode (step S121 of Fig. 12). In other words, during unmanned driving in normal autonomous driving mode, the vehicle is always stopped near the detected target person, even if unconditional stopping is not set. In this case, however, the mode may be switched to forced autonomous driving mode, but since the vehicle is being driven unmanned, there is no need to switch to forced autonomous driving mode and the mode may remain normal autonomous driving mode.

[0144] Next, the control unit 101 determines whether the vehicle has stopped (step S122), and if it determines that the vehicle has not stopped, returns the process to step S121 and executes stop control. Then, if it determines in step S122 that the vehicle has stopped, the control unit 101 causes the voice message generation unit 120 to generate a calling voice message including the calling name (nickname) of the detected target person as described above, and emits the calling voice message through the voice input / output unit 119 and the exterior speaker 138, thereby calling out to the detected target person and calling their attention (step S123).

[0145] Next, the control unit 101 uses the voice message generation unit 120 to generate a voice message inquiring of the detected target person outside the vehicle as to whether or not they wish to ride, and emits the generated voice message via the voice input / output unit 119 and the exterior speaker 138, thereby inquiring of the detected target person as to whether or not they wish to ride (step S124).

[0146] Next, the control unit 101 receives a response to this inquiry from the target person outside the vehicle and determines whether or not the target person wishes to board (step S125). The response from the target person to this inquiry is determined by recognizing the voice picked up by the exterior microphone 140 using the voice recognition function of the confirmation and authentication unit 117.

[0147] If it is determined in step S125 that the target person does not wish to ride, the control unit 101 starts driving in normal automatic driving mode (step S130), and then transitions the processing to step S101 in Figure 10 and executes the processing from step S101 onwards.

[0148] Furthermore, when it is determined in step S125 that the target person wishes to board, the control unit 101 acquires information such as biometric information for identification and authentication from the target person and performs identification and authentication (step S126). That is, the control unit 101 acquires identification and authentication information for the detected target person from the target person information storage unit 136. If the identification and authentication information is a facial image, the control unit 101 captures and acquires a facial image of the target person using an exterior camera in the camera group 107, and compares and refers to the acquired facial image with the facial image of the target person read from the target person information storage unit 136 to identify and authenticate the target person. At this time, the image recognition function of the identification and authentication unit 117 is used. Furthermore, if the identification and authentication information is a spoken voice, the control unit 101 collects the spoken voice of the target person using the exterior microphone 140, and compares and refers to the collected spoken voice with the spoken voice of the target person read from the target person information storage unit 136 to identify and authenticate the target person. At this time, the voice recognition function provided in the confirmation authentication unit 117 is used.

[0149] Next, the control unit 101 determines whether the confirmation and authentication result is OK (step S127). If the control unit 101 determines that the confirmation and authentication result is OK, it drives and controls the door opening / closing drive unit 118 to open and close the door, and confirms that the target person has boarded the vehicle from images captured by the cameras in the camera group 107 that capture the vehicle exterior and / or interior (step S128). The image recognition for confirming boarding is performed using an image recognition function provided in the confirmation and authentication unit 117. If the control unit 101 determines that the confirmation and authentication result is NG rather than OK, it causes the voice message generation unit 120 to generate a voice message indicating that boarding is NG, and notifies the detected person of the generated voice message via the voice input / output unit 119 and the vehicle exterior speaker 138 (step S129).

[0150] Then, after step S128 and after step S129, the control unit 101 starts driving in normal automatic driving mode (step S130), and then transitions the processing to step S101 in Figure 10 and executes the processing from this step S101 onwards.

[0151] [One aspect of the target person detection processing operation by gait recognition in an autonomous driving vehicle according to an embodiment] Next, an example of a target person detection process operation using gait recognition by the autonomously driven vehicle 1 of this embodiment will be described with reference to Fig. 13. In this example, when a target person is found, the autonomously driven vehicle 1 stops nearby and inquires about getting on.

[0152] As shown in FIG. 13(A), assume that while the autonomous vehicle 1 of this embodiment is traveling, person 2, who is one of the target persons stored in the target person information storage unit 136, is walking in front of the autonomous vehicle 1. At this time, if the autonomous vehicle 1 has set person 2 as a detection target for gait recognition and is executing a detection process using gait recognition, when the autonomous vehicle 1 approaches person 2 to a distance L of, for example, about 40 to 50 meters as shown in FIG. 13(A), the control unit 101 of the autonomous vehicle 1 detects person 2 as a person to be detected by gait recognition. Of course, the distance may be longer or shorter than about 40 to 50 meters, and may be variable depending on the traveling speed of the autonomous vehicle. In other words, when the traveling speed is fast, gait recognition begins from a greater distance.

[0153] At this time, if a driver and / or passenger is present in autonomous vehicle 1, control unit 101 notifies the driver and / or passenger of the detection of person 2, as described above. Then, when an unconditional stop is set, or when the driver and / or passenger instructs autonomous vehicle 1 to stop in response to an inquiry, autonomous vehicle 1 approaches near person 2, as indicated by the solid arrow in Figure 13(A), and stops as shown in Figure 13(B).

[0154] Then, the control unit 101 of the autonomous vehicle 1 calls out to the person 2 to alert them to the presence of the stopped autonomous vehicle 1. Furthermore, the control unit 101 of the autonomous vehicle 1 inquires of the person 2 whether or not they would like to get in the vehicle, and when the control unit 101 receives a response from the person 2 that they would like to get in the vehicle, the control unit 101 allows the person 2 to get in the autonomous vehicle 1.

[0155] As described above, according to the self-driving vehicle 1 of this embodiment, by using gait recognition technology, it is possible to detect, from images captured by a camera equipped on the vehicle, when a target person who has been registered and stored in advance is walking ahead in the direction of travel of the vehicle, and, depending on the detection results, it is possible to stop the vehicle near the detected target person and encourage them to get on, which is extremely convenient.

[0156] [Other embodiments] In the above-described embodiment, the self-driving vehicle 1 is provided with the function of detecting and identifying target persons through gait recognition, but the self-driving vehicle 1 may also be configured to detect and identify target persons through gait recognition by providing the function of detecting and identifying target persons through gait recognition to a server device via a communication network such as the Internet, and to detect and identify target persons through gait recognition by working in cooperation with the server device.

[0157] Fig. 14 is a diagram illustrating the main parts of the configuration of a system configured in this way, which is made up of automatically-driven vehicle 1A and server device 3. As shown in Fig. 14, in this example, automatically-driven vehicle 1A and server device 3 are configured to be able to connect to each other wirelessly via the Internet 4, which is an example of a communications network.

[0158] 14, electronic control circuit unit 10A of autonomously driven vehicle 1A of this example has a wireless communication unit 121 for wirelessly connecting with server device 3 via the Internet 4, a wireless transmission information generation unit 122, and a wireless reception information processing unit 123 connected to control unit 101A via system bus 100. Although not shown in FIG. 14, electronic control circuit unit 10A of autonomously driven vehicle 1A of this example does not have target person information registration unit 114, gait recognition / discrimination unit 115, confirmation authentication unit 117, and target person information storage unit 136 that are included in electronic control circuit unit 10 of autonomously driven vehicle 1 of the above-mentioned embodiment.

[0159] Instead, in server device 3, a target person information registration unit 32, a gait recognition and discrimination unit 33, and a confirmation and authentication unit 34 are connected to control unit 31 via system bus 30, and a target person information storage unit 35 is provided and connected to target person information registration unit 32, gait recognition and discrimination unit 33, and confirmation and authentication unit 34. Furthermore, a wireless communication unit 36 ​​is connected to system bus 30 for connecting to automatically driven vehicle 1A via Internet 4.

[0160] The target person information storage unit 35 of the server device 3 stores information similar to the information stored in the target person information storage unit 136 shown in Fig. 3. However, in this example, each of the autonomously driven vehicles 1A is assigned identification information, and in the target person information storage unit 35 of the server device 3, the target person information for each autonomously driven vehicle 1A is stored in a storage area distinguished by the identification information based on the identification information of the autonomously driven vehicle 1A.

[0161] That is, in this example, when registering target person information, the image information of the target person captured by the camera of the camera group 107 of the self-driving vehicle 1A is converted into wireless transmission information including the vehicle's identification information by the wireless transmission information generation unit 122, and sent to the server device 3 via the wireless communication unit 121 and the Internet 4.

[0162] In server device 3, wireless communication unit 36 ​​receives the information transmitted from autonomously driven vehicle 1A, and target person information registration unit 32 extracts gait information of the target person. The extracted gait information is then registered as target person information in a storage area corresponding to the identification information of autonomously driven vehicle 1A, in a manner similar to that shown in Fig. 3. Furthermore, verification and authentication information for the target person is also wirelessly transmitted from autonomously driven vehicle 1A to server device 3, and stored by target person information registration unit 32 in a storage area of ​​target person information storage unit 35 corresponding to the identification information of autonomously driven vehicle 1A, in a manner similar to that shown in Fig. 3.

[0163] In this example, when autonomous vehicle 1A is specified to perform a target person detection process using gait recognition while driving, image information captured by the external camera in camera group 107 is converted into wireless transmission information including the vehicle's own identification information by wireless transmission information generation unit 122 and sent to server device 3 via wireless communication unit 121 and Internet 4.

[0164] In the server device 3, the wireless communication unit 36 ​​receives the transmitted information from the autonomous vehicle 1A, the gait recognition and discrimination unit 33 recognizes the gait and discriminates the target person, and the discrimination results are transmitted to the autonomous vehicle 1A via the wireless communication unit 36 ​​and the Internet 4.

[0165] Self-driving vehicle 1A receives information from server device 3 via wireless communication unit 121, and analyzes the discrimination result of gait recognition discrimination unit 33 via wireless reception information processing unit 123 to determine whether a target person has been detected. Then, gait recognition discrimination result response processing unit 116 (control unit 101A) of self-driving vehicle 1A executes processing similar to that of the above-described embodiment in accordance with the discrimination result of wireless reception information processing unit 123.

[0166] The same applies to the confirmation and authentication process, where confirmation and authentication unit 34 of server device 3 performs confirmation and authentication using the confirmation and authentication information wirelessly transmitted from autonomous vehicle 1A. Server device 3g then transmits the authentication result to autonomous vehicle 1A. In autonomous vehicle 1A, wireless reception information processing unit 123 analyzes the confirmation and authentication result, and based on the analysis result, control unit 101A performs processing according to the above-mentioned confirmation and authentication result. The other configurations in this example are the same as those in the above-mentioned embodiment.

[0167] In this example, the automatically driven vehicle 1A is configured such that the processing related to gait recognition is performed by the server device 3, which reduces the load on the control unit 101A, allowing it to concentrate on driving control such as automatic driving.

[0168] [Other embodiments or modifications] In the above-described embodiment, when a detected target person is allowed to board, the target person is confirmed and authenticated. However, since the recognition accuracy of gait recognition is high, and in this embodiment, gait recognition auxiliary information is also used, it has been reported that the recognition rate is 99%. Therefore, this confirmation and authentication may be omitted.

[0169] In the above-described embodiment, when a detected target person is alerted, a voice call including the target person's nickname (nickname) is sent. However, instead of a voice call, the vehicle may also be alerted by honking a horn or emitting a warning sound. Instead of sending a voice call, the vehicle may store the target person's mobile phone number or email address, call the phone number, and send a voice message via mobile phone saying, "We're parked nearby to give you a ride." Alternatively, the vehicle may send an email to the email address saying, "We're parked nearby to give you a ride." GPS information about the vehicle's location may also be sent. Of course, this is not limited to email, and a communication app such as LINE may also be used. In these cases, the vehicle may be equipped with a mobile phone function so that the driver and / or passengers can make calls using the mobile phone function. The vehicle may also be configured with, for example, an AI (artificial intelligence) function or a function for making calls to the detected target person using an automated voice. In such a configuration, a person outside the vehicle, such as a family member or a group member, may be contacted and set as a target person for gait recognition. In this case, the current location is also known, so if the user goes to the surrounding area, thorough gait recognition can be performed.

[0170] In the above-described embodiment, it is assumed that a target person is accidentally detected by incidentally recognizing gait while the vehicle is traveling. However, a separate mode (search mode or gaze mode) for carefully and thoroughly searching for the target person may be provided. For example, if the search mode or gaze mode is set when the vehicle is traveling around a station during the time when family members are returning home or leaving work, the probability of detecting the family member as the target person can be increased. In this case, the search mode or gaze mode involves traveling around the same location, such as around a station. Furthermore, the target person's schedule may be registered in advance, and the self-driving vehicle may be made to travel around the scheduled location at a scheduled time to make it easier to find the target person. For example, the self-driving vehicle may be made to travel around the nearest station during the target person's commuting time or school commute time.

[0171] In addition, in the above embodiment, the case of an electric vehicle has been described, but the vehicle of the present invention is not limited to an electric vehicle, and may be, for example, a gasoline engine vehicle, a diesel engine vehicle, etc. Furthermore, the present invention is not limited to automobiles, and may also be applied to two-wheeled vehicles such as motorcycles, electric bicycles, and electric wheelchairs.

[0172] Furthermore, although the above embodiment has been described with reference to an autonomous vehicle, the present invention can also be applied to a normal vehicle that does not have an autonomous driving mode and only has a manual driving mode, and it goes without saying that the present invention can also be applied to an autonomous vehicle that only has an autonomous driving mode.

[0173] Furthermore, the automobile according to the present invention can also be applied to an air-land amphibious vehicle (flying automobile) that can move (navigate) not only on land but also in the air. Figure 15 is a diagram illustrating an example of the appearance of an air-land amphibious vehicle 1AL as an embodiment of the automobile according to the present invention, and an example of how the air-land amphibious vehicle 1AL of this embodiment detects a target person 2 through gait recognition, in particular, an example of how the air-land amphibious vehicle 1AL of this embodiment detects a target person 2 through gait recognition while moving (navigating) in the air.

[0174] The air-land amphibious vehicle 1AL of this embodiment, like the autonomous vehicle 1 of the above-mentioned embodiment, is equipped with one or more cameras that capture images of the field of view outside the vehicle, as well as a function for registering target persons to be subjected to gait recognition, a gait recognition and discrimination function that performs gait recognition based on images captured by the cameras and detects target persons, and a gait recognition and discrimination result processing function.

[0175] The air-land amphibious vehicle 1AL of this embodiment has a land mode for traveling on land and an air mode for traveling (navigating) in the air, and is configured so that the registration for gait recognition, gait recognition discrimination, and recognition discrimination result response processing differ depending on the mode.

[0176] That is, the registration of the target person's gait information for the land mode is performed by photographing the target person with a camera that photographs the area ahead in the direction of travel of the vehicle, extracting the target person's gait information from the photographed information, and storing it in a memory unit, just as in the case of the autonomous vehicle 1 of the above-described embodiment. On the other hand, the registration of the target person's gait information for the aerial mode is performed by photographing the target person walking with a camera that photographs an area diagonally below while the air-land amphibian 1AL is in the air, and extracting the target person's gait information from the photographed information and storing it.

[0177] In this embodiment, in land mode, the air-land amphibian 1AL recognizes and discriminates the gait of the target person from images captured using a camera positioned ahead in the vehicle's traveling direction and from gait information for the land mode, as in the above-described embodiments. In air mode, the air-land amphibian 1AL recognizes and discriminates the gait of the target person from images captured using a camera that captures images diagonally below the airborne air-land amphibian 1AL and from gait information for the airborne navigation mode.

[0178] In land mode, when the air-land amphibian 1AL of this embodiment detects a target person through gait recognition, it performs the same gait recognition determination result response processing as the autonomous vehicle 1 of the above-described embodiment. In contrast, in air mode, it may land and switch to land mode to perform gait recognition determination result response processing, but if the detected target person is inquired about and refused to board, switching to land mode would result in the gait recognition determination result response processing in land mode being wasted, which would be inefficient.

[0179] Therefore, in this example, the air-land amphibious vehicle 1AL in the air uses the target person's mobile phone number, which is previously stored and included in the registration information, to create a wireless communication path with the target person's mobile phone terminal and inquire as to whether or not the target person will board.Based on the results of this inquiry, the air-land amphibious vehicle 1AL decides whether to land and switch to land mode to allow the target person to board, or to continue aerial movement (navigation) in air mode.

[0180] That is, if the target person replies that they will board as a result of the inquiry via mobile phone, the air-land amphibian 1AL lands its own vehicle, switches to land mode, stops near the detected target person, performs confirmation and authentication, and allows the target person to board. Also, if the target person refuses to board or there is no need to allow the target person to board as a result of the inquiry via mobile phone, the air-land amphibian 1AL keeps its own vehicle in air mode and continues moving (navigating) in the air.

[0181] In addition, the wireless communication path for inquiring about the target person is not limited to a mobile phone terminal, but may be any other wireless communication terminal, in which case the connection information for creating a wireless communication path between the wireless communication terminal and the air-land hybrid vehicle 1AL is included in the target person's registration information and stored.

[0182] If the mobile phone terminal or wireless communication terminal carried by the target person is capable of communicating with the air-land amphibian 1AL via a wireless communication path, not only can the air-land amphibian 1AL make an inquiry, but the target person can also wirelessly send a boarding request to the air-land amphibian 1AL using the mobile phone terminal or wireless communication terminal to request gait recognition. In this case, the air-land amphibian 1AL stores gait information in association with the target person's identification information, just as in the above-described embodiment.

[0183] In this case, the ride request wirelessly transmitted from the target person includes the target person's identification information and the target person's current location information, as well as information input and specified by the target person about where to go from the current location (information on intermediate points and destination). If the mobile phone terminal or wireless communication terminal carried by the target person is equipped with a GPS (Global Positioning System) function, the GPS information can be used as the current location information.

[0184] Then, the air-land amphibian 1AL that has received the boarding request identifies the target person from the identification information included in the boarding request, estimates the position where the vehicle will encounter the target person while taking into account the target person's travel route from information on the current location and information on where to go from the current location (information on intermediate points and destination), and when the vehicle arrives near the estimated position, focuses on gait recognition near that position. Note that the target person's travel route can be detected and understood from information on the current location and information on where to go from the current location (information on intermediate points and destination) using the route search function of the car navigation function unit equipped in the air-land amphibian 1AL.

[0185] It should be noted that this mode of wirelessly transmitting a request to board from the target person is not limited to the air-land vehicle 1AL, but can also be configured in a similar manner for automobiles such as the self-driving car of the above-mentioned embodiment, motorcycles, and other automobiles. [Explanation of symbols]

[0186] 1...Autonomous driving vehicle, 3...Server device, 4...Internet, 10...Electronic control circuit unit, 101...Control unit, 114...Target person information registration unit, 115...Gait recognition and discrimination unit, 116...Gait recognition and discrimination result response processing unit, 117...Confirmation and authentication unit, 118...Door opening and closing drive unit, 119...Voice input / output unit, 120...Voice message generation unit, 121...Wireless communication unit

Claims

1. One or more cameras that capture images of the outside of the vehicle; a storage unit that stores gait information of a target person; a gait discrimination means for determining whether or not a person present outside the vehicle has been detected whose gait matches the gait of the target person stored in the storage unit from the image information captured by the camera; a notification means for notifying the detected person when the gait determination means determines that a person having a gait matching the gait of the target person has been detected; A vehicle equipped with:

2. The notification is a voice call.

2. The vehicle according to claim 1.

3. The notification is a voice message stating that boarding will be refused.

3. The vehicle according to claim 1 or claim 2.

4. The notification is a horn sound or a warning sound. The automobile according to any one of claims 1 to 3.

5. The notification is made by communication via telephone, email, or LINE. The automobile according to any one of claims 1 to 4.

6. The notification is made by transmitting GPS information of the vehicle's stopping position. The automobile according to any one of claims 1 to 5.

7. a registration request receiving means for receiving a registration request for the target person; a registration means for, when the registration request receiving means receives a registration request for the target person, capturing an image of the target person walking with the camera, detecting the gait information from the captured image information of the target person, and storing the detected gait information in the storage unit; The automobile according to any one of claims 1 to 6, characterized in that it comprises:

8. a registration request receiving means for receiving a registration request for the target person; a wireless communication unit; a requesting means for, when the registration request receiving means receives a registration request for the target person, capturing an image of the target person walking with the camera, transmitting captured image information of the target person via the wireless communication unit to a gait detecting device that detects gait information of the target person from the captured image information of the person, and requesting detection of gait information; a registration means for receiving gait information transmitted from the gait detection device via the wireless communication unit based on a request for detection of the gait information and storing the gait information in the storage unit; The automobile according to any one of claims 1 to 6, characterized in that it comprises:

9. the storage unit stores the gait information in association with information for identifying the target person, The notification includes information for identifying the target person. The automobile according to any one of claims 1 to 8.

10. It is an autonomous vehicle equipped with an autonomous driving mode that allows autonomous driving, The predetermined process is a driving control process that, when it is determined that the target person has been detected by the gait determination means, stops the vehicle in the vicinity of the detected target person in the automatic driving mode. The automobile according to any one of claims 1 to 9.

11. When the vehicle is traveling in the manual driving mode, the driving mode is switched to the automatic driving mode, and a driving control process is performed to stop the vehicle near the detected target person.

11. The vehicle according to claim 10.

12. Equipped with a speaker to project sound outside the vehicle, When the vehicle stops near the target person, it emits a voice message to the target person through the speaker.

12. A motor vehicle according to claim 10 or claim 11.

13. the storage unit stores information for enabling the target person to recognize that the call is directed to the target person, The call voice includes a voice based on the information for making the voice recognizable.

13. The vehicle according to claim 12.

14. A confirmation and authentication means is provided for confirming and authenticating the target person when the vehicle stops near the target person. The automobile according to any one of claims 10 to 13.

15. a face image storage unit that stores face image information of the target person; The verification and authentication means verifies the target person by comparing the facial image information of the detected target person obtained from the image information photographed by the camera and the facial image information stored in the facial image storage unit, and determining whether the target person is the person whose facial image information is stored in the facial image storage unit.

15. The vehicle of claim 14.

16. a voice storage unit that stores voice information of the target person and a microphone that collects voices outside the vehicle; The verification and authentication means verifies the target person based on the voice information of the detected target person picked up by the microphone and the voice information stored in the voice storage unit, based on whether the target person is the person whose voice information is stored in the voice storage unit.

16. A motor vehicle according to claim 14 or claim 15.

17. When the target person confirmed by the confirmation authentication means gets into the vehicle, the target person who is in the vehicle is excluded from the target persons to be detected by the gait recognition. The automobile according to any one of claims 14 to 16.

18. An air-land hybrid vehicle having a land mode for traveling on land and an air mode for traveling in the air, The storage unit stores gait information for detecting a target person who is a target of gait recognition, the gait information being for the land mode and the gait information being for the aerial mode. The automobile according to any one of claims 1 to 17.

19. An air-land hybrid vehicle having a land mode for traveling on land and an air mode for traveling in the air, A camera for the land mode and a camera for the aerial mode are provided to perform the gait recognition. The automobile according to any one of claims 1 to 18.

20. a wireless communication means for wirelessly communicating with a mobile communication terminal; In the aerial mode, when the gait determination means determines that there is a person whose gait matches the gait of the target person, the wireless communication means wirelessly communicates with the mobile communication terminal carried by the detected target person to inquire whether the target person wishes to ride in the vehicle.

20. A motor vehicle according to claim 18 or claim 19.

21. When the processing means receives a response to the inquiry indicating a desire to ride, the processing means switches from the air mode to the land mode and stops the vehicle near the detected target person.

21. The vehicle of claim 20.

22. If the reply to the inquiry is that there is no desire to board, the aerial mode is continued.

22. A motor vehicle according to claim 20 or claim 21.

23. a wireless communication means for wirelessly communicating with a mobile communication terminal; When a boarding request including information on the current location of the target person is received from the target person via the wireless communication means, the gait determination means determines whether or not there is a person whose gait matches that of the target person in the vicinity of a position detected based on the information on the current location of the target person. The automobile according to any one of claims 1 to 17.

24. means for presenting a list of the target persons stored in the storage unit; a selection setting receiving means for receiving a selection setting of the target person from the presented list; Equipped with The gait determination means determines whether or not there is a person whose gait matches the gait of the selected target person among the people whose gaits have been recognized. The automobile according to any one of claims 1 to 23.

25. the selection setting receiving means is capable of receiving a selection setting of a predetermined process to be executed by the vehicle when the target person is detected, The processing means performs the predetermined processing accepted by the processing setting accepting means.

25. The vehicle of claim 24.

26. The storage unit stores gait information of the target person divided into daytime and nighttime gait information. The automobile according to any one of claims 1 to 25.

27. A vehicle capable of communicating with a server device via the Internet, the server device including a storage unit that stores gait information of a target person and a gait discrimination means that determines whether a person having a gait that matches the gait of the target person stored in the storage unit has been detected, One or more cameras that capture images of the outside of the vehicle; a transmitting means for transmitting image information captured by the camera to the server device; a receiving means for receiving from the server device a determination result as to whether a person having a gait matching the gait of the target person has been detected; a notification means for notifying the detected person when it is determined that the receiving means has detected a person whose gait matches the gait of the target person; A vehicle equipped with:

28. A system comprising: an automobile having a first wireless communication means; and a gait discrimination device having a second wireless communication means that wirelessly communicates with the first wireless communication means, The vehicle is One or more cameras that capture images of the outside of the vehicle; a gait discrimination requesting means for transmitting a gait discrimination request, which includes identification information of the vehicle itself and image information captured by the camera, to the gait discrimination device via the first wireless communication means; a first receiving means for receiving, from the gait discrimination device, a determination result indicating whether the target person has been detected or not via the first wireless communication means; a notification means for notifying the detected person when the determination result received by the first receiving means indicates that the target person has been detected; Equipped with The gait discrimination device includes: a storage unit that stores gait information of a target person in association with identification information of the vehicle; a second receiving means for receiving, via the second wireless communication means, the identification information of the vehicle from the vehicle and the image information captured by the camera; a gait discrimination means for determining whether or not there is a person present outside the vehicle, based on the image information received by the second receiving means, whose gait matches the gait of the target person stored in the storage unit in association with the identification information of the vehicle; a transmitting means for transmitting the determination result of the gait determination means to the vehicle that has sent the gait determination request via the second wireless communication means; A gait recognition system comprising:

29. The vehicle is a means for storing a list of the target persons stored in the storage unit of the gait discrimination device in association with identification information of the vehicle, and for presenting the list; a selection setting receiving means for receiving a selection setting of the target person from the presented list; the gait discrimination request includes identification information of the selected and set target person in addition to identification information of the vehicle itself and image information captured by the camera, and is transmitted to the gait discrimination device via the first wireless communication means; The gait discrimination means of the gait discrimination device determines whether or not there is a person whose gait matches the gait of the selected target person identified by identification information among the people whose gaits have been recognized.

29. The gait recognition system of claim 28.

30. A computer provided in an automobile having one or more cameras for capturing images of the exterior of the automobile and a storage unit for storing gait information of a target person, a gait discrimination means for determining whether or not a person whose gait matches the gait of the target person stored in the storage unit has been detected among people present outside the vehicle from image information captured by the camera; a notification means for notifying the detected person when it is determined by the gait determination means that a person having a gait matching the gait of the target person has been detected; Automotive program to function as.

31. A vehicle that can be communicatively connected via the Internet to a server device that includes a storage unit that stores gait information of a target person and gait discrimination means that determines whether a person with a gait that matches the gait of the target person stored in the storage unit has been detected, and that is equipped with one or more cameras that capture images of the outside of the vehicle, a transmitting means for transmitting image information captured by the camera to the server device; a receiving means for receiving from the server device a determination result as to whether a person having a gait matching the gait of the target person has been detected; a notification means for notifying the detected person when it is determined by the receiving means that a person having a gait matching the gait of the target person has been detected; Automotive program to function as.

Images