Automobile, program for automobiles, and gait recognition system

Abstract

To provide an automobile that reliably detects a family member or acquaintance who is walking around the nearest station during a time zone of commuting to work or school, and enables predetermined processing such as asking the detected family or acquaintance whether he / she boards the automobile.SOLUTION: An automobile includes one or plural cameras that image an outside of an own vehicle, a memory unit that stores gait information for use in detecting an object person who is an object of gait recognition, gait recognition determination means that recognizes the gait of a person, who exists in a prospect area outside of the own vehicle, from image information obtained by the cameras, and determines whether a person exhibiting a gait consistent with the gait of the object person stored in the memory unit is detected among persons whose gaits are recognized, and processing means that, when the gait recognition determination means determines that the person exhibiting the gait consistent with the gait of the object person has been detected, performs predetermined processing.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a vehicle, a program for a vehicle, and a gait recognition system. [Background technology]

[0002] The general way of using a car is to get in and out of a parked car of one's own volition. In addition, when the car is a taxi, a passenger may signal the driver by raising their hand, so that the driver will stop the car near the passenger and the passenger can get in and out.

[0003] Recently, self-driving cars that are capable of autonomous driving even when there is no driver on board have also appeared. In the case of such self-driving cars, it has been proposed to equip them with wireless communication capabilities so that a request to call the self-driving car can be made from a mobile terminal such as a mobile phone terminal, and passengers wishing to board the car can be called to their desired location and allowed to board the car (see, for example, Patent Document 1 (JP 2017-204867 A)). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2017-204867 A Summary of the Invention [Problem to be solved by the invention]

[0005] Incidentally, if the driver and / or passenger, or the car itself, happens to detect a family member or acquaintance walking on the sidewalk ahead while the car is moving, the car can stop beside the family member or acquaintance and invite them to get in. It would be particularly beneficial if the family member or acquaintance is walking with a heavy load or is elderly, and the driver and / or passenger, or the car itself, could invite them to get in.

[0006] However, because drivers are mindful of driving safely, checking the road ahead and behind them, it is often difficult for them to detect such walking family members or acquaintances, and they end up overlooking them.

[0007] In view of the above, the object of the present invention is to provide a car that can reliably detect walking family members, acquaintances, etc. while the car is moving, and can perform processes such as inviting the detected family members or acquaintances to get in the car. [Means for solving the problem]

[0008] In order to solve the above problem, the invention of claim 1 is as follows: One or more cameras for capturing images of the outside of the vehicle; A storage unit that stores gait information for detecting a target person who is a target of gait recognition; a gait recognition and discrimination means for recognizing the gait of a person present outside the vehicle from image information captured by the camera, and discriminating whether or not a person whose gait matches the gait of the target person stored in the storage unit has been detected among the people whose gait has been recognized; a processing means for performing a predetermined process when it is determined that the gait recognition and discrimination means has detected a person having a gait matching the gait of the target person during a predetermined time period and / or at a predetermined location; The present invention provides a vehicle equipped with the above-mentioned.

[0009] As is well known, gait recognition is a technology that can authenticate humans from a distance of approximately 40 to 50 meters, and is attracting attention as a biometric for identifying individuals from a distance.

[0010] The automobile of the invention of claim 1 having the above-mentioned configuration has a storage unit in which gait information for detecting a target person to be subjected to gait recognition is stored. In the automobile of this invention, the gait recognition and discrimination means recognizes the gait of a person present outside the vehicle from image information captured by the camera. The gait recognition and discrimination means further discriminates whether or not there is a person whose gait matches the gait of the target person stored in the storage unit among the people whose gaits have been recognized.

[0011] If the gait recognition and discrimination means determines that there is a person with a gait matching that of the target person during a specified time period and / or at a specified location, the means will perform specified processing, such as notifying the driver or passengers that the target person has been detected, or stopping to invite them in for a ride. Effect of the Invention

[0012] The automobile of this invention employs gait recognition (gait authentication) technology, which has the remarkable effect of being able to detect when a target person who has been registered and stored in advance is walking in front of or to the side of the vehicle in the direction of travel from images captured by the vehicle's camera, and to carry out predetermined processing in accordance with the detection results. [Brief description of the drawings]

[0013] [Figure 1] 1 is a block diagram showing an example of the configuration of an electronic control circuit unit of an example of an autonomous vehicle as an embodiment of the automobile according to the present invention. FIG. [Diagram 2] 1 is a diagram for explaining a procedure for registering a target person to be subjected to gait recognition in an autonomous driving vehicle according to an embodiment. FIG. [Diagram 3] 1 is a diagram for explaining an example of registered target person information in an autonomous driving vehicle of an embodiment. FIG. [Figure 4] FIG. 11 is a diagram illustrating a portion of a flowchart for explaining an example of a processing routine for presetting a target person's gait recognition in an autonomous driving vehicle according to an embodiment. [Diagram 5]FIG. 11 is a diagram illustrating a portion of a flowchart for explaining an example of a processing routine for presetting a target person's gait recognition in an autonomous driving vehicle according to an embodiment. [Figure 6] FIG. 11 is a diagram illustrating a portion of a flowchart for explaining an example of a processing routine for detecting a target person through gait recognition in an autonomous driving vehicle of an embodiment. [Figure 7] FIG. 11 is a diagram illustrating a portion of a flowchart for explaining an example of a processing routine for detecting a target person through gait recognition in an autonomous driving vehicle of an embodiment. [Figure 8] FIG. 11 is a diagram illustrating a portion of a flowchart for explaining an example of a processing routine for detecting a target person through gait recognition in an autonomous driving vehicle of an embodiment. [Figure 9] FIG. 11 is a diagram illustrating a portion of a flowchart for explaining an example of a processing routine for detecting a target person through gait recognition in an autonomous driving vehicle of an embodiment. [Figure 10] FIG. 11 is a diagram illustrating a portion of a flowchart for explaining an example of a processing routine for detecting a target person through gait recognition in an autonomous driving vehicle of an embodiment. [Figure 11] FIG. 11 is a diagram illustrating a portion of a flowchart for explaining an example of a processing routine for detecting a target person through gait recognition in an autonomous driving vehicle of an embodiment. [Figure 12] FIG. 11 is a diagram illustrating a portion of a flowchart for explaining an example of a processing routine for detecting a target person through gait recognition in an autonomous driving vehicle of an embodiment. [Figure 13] 1 is a diagram for explaining a specific example of a manner in which a target person is detected by gait recognition using an autonomous driving vehicle according to an embodiment. [Figure 14] FIG. 11 is a block diagram for explaining an example of the configuration of a system including an autonomous vehicle as another embodiment of the automobile according to the present invention. [Figure 15] 11 is a diagram for explaining a specific example of a manner in which a target person is detected by gait recognition using an air-land hybrid vehicle as another embodiment of the automobile according to the present invention. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, an embodiment of the automobile according to the present invention will be described with reference to the drawings. The embodiment of the automobile described below is an autonomous vehicle equipped with a manual driving mode in which the vehicle travels in response to manual driving operations by a driver, and an autonomous driving mode in which the vehicle travels autonomously without being driven by a driver.

[0015] The automobile according to the present invention is not limited to an autonomous vehicle having a manual driving mode and an autonomous driving mode as in this embodiment, but can be implemented even in an automobile (fully autonomous vehicle) that has only an autonomous driving mode that runs autonomously without manual operation by the driver. The present invention can also be applied to an automobile that does not have an autonomous driving mode and has only a manual driving mode.

[0016] 1 is a block diagram showing an example of the hardware configuration of an electronic control circuit unit 10 of an autonomous vehicle 1 according to this embodiment. Note that the autonomous vehicle 1 according to this embodiment is an example of an electric vehicle, and is equipped with a battery (not shown) as a drive source.

[0017] As described above, the self-driving car 1 of this embodiment has a manual driving mode and an autonomous driving mode in which the car travels autonomously in this embodiment. The manual driving mode is a mode in which the car can travel according to the driver's accelerator pedal operation, brake pedal operation, shift lever operation, and steering operation (handle operation) in the same way as a normal car that is not a self-driving car.

[0018] The autonomous driving mode is a mode in which the autonomous vehicle 1 automatically (autonomously) drives while avoiding obstacles without the driver operating the accelerator pedal, the brake pedal, the shift lever, or the steering wheel. In this autonomous driving mode, the mode can be automatically switched to the manual driving mode by a predetermined action by the driver. Here, the predetermined action by the driver includes, for example, driving operations such as the accelerator pedal operation or the brake pedal operation by the driver, operation input by the driver through a touch panel described later, and switching instructions by voice input by the driver.

[0019] For example, the driver of the autonomous vehicle 1 can switch the autonomous vehicle 1, which is traveling in manual driving mode, to autonomous driving mode by performing a specific operation via the touch panel 112 described below.The autonomous vehicle 1 is also configured to automatically return to manual driving mode if the driver operates the accelerator pedal, brake pedal, shift lever, steering wheel, or the like while traveling in autonomous driving mode.

[0020] As described below, the autonomous vehicle of this embodiment is equipped with a forced autonomous driving mode that maintains the autonomous driving mode even if the driver performs an operation to return to the manual driving mode, such as operating the accelerator pedal, the brake pedal, the shift lever, or the steering wheel. Therefore, in the following description, the autonomous driving mode that switches to the manual driving mode when the driver performs an operation to return to the manual driving mode, such as operating the accelerator pedal, the brake pedal, the shift lever, or the steering wheel, will be referred to as the normal autonomous driving mode to distinguish it from the forced autonomous driving mode.

[0021] 1, in the electronic control circuit unit 10, a motor drive control unit 102, a steering drive control unit 103, a manual / automatic driving mode switching control unit 104, a brake drive control unit 105, a radar group 106, a camera group 107, a sensor group 108, a surrounding moving object grasping unit 109, a current position detection unit 110, a display unit 111, a touch panel 112, a car navigation (hereinafter abbreviated as car navigation) function unit 113, a target person information registration unit 114, a gait recognition discrimination unit 115, a gait recognition discrimination result corresponding processing unit 116, a confirmation authentication unit 117, a door opening / closing drive unit 118, a voice input / output unit 119, and a voice message generation unit 120 are connected to a control unit 101 configured by mounting a computer via a system bus 100. Gait recognition is also called gait authentication, but the terms are used synonymously.

[0022] A motor drive unit 131 is connected to the motor drive control unit 102. A steering drive unit 132 is connected to the steering drive control unit 103. A manual operation detection unit 133 is connected to the manual / automatic driving mode switching control unit 104, and a brake drive unit 134 is connected to the brake drive control unit 105. A car navigation database 135 is connected to the car navigation function unit 113. A target person information storage unit 136 is connected to the target person information registration unit 114.

[0023] Furthermore, an in-vehicle speaker 137, an exterior speaker 138, an in-vehicle microphone 139, and an exterior microphone 140 are connected to the audio input / output unit 119. The in-vehicle speaker 137 is for emitting audio to the interior of the autonomous vehicle 1, and the exterior speaker 138 is for emitting audio to the exterior of the autonomous vehicle 1. The in-vehicle microphone 139 is for picking up audio inside the autonomous vehicle 1, and the exterior microphone 140 is for picking up audio outside the autonomous vehicle 1. Of course, the volume of the in-vehicle speaker 137 and the exterior speaker 138 and the sensitivity of the in-vehicle microphone 139 and the exterior microphone 140 can be varied and may be different inside and outside the vehicle. For example, when a target person is found, the volume of the exterior speaker 138 may be increased and the sensitivity of the exterior microphone 140 may be increased.

[0024] Under the control of the control unit 101, the motor drive control unit 102 controls the supply of a drive signal to the motor drive unit 131 of the autonomous vehicle 1, which is an electric vehicle of this embodiment, so as to control the start of driving of the autonomous vehicle 1, driving speed control (including brake control and accelerator control), stopping of driving, etc.

[0025] Under the control of the control unit 101, the steering drive control unit 103 controls the supply of a drive control signal to the steering drive unit 132 of the autonomous vehicle 1 of this embodiment, thereby controlling the course change of the autonomous vehicle 1.

[0026] The manual / automatic driving mode switching control unit 104 performs control to switch the driving mode of the autonomous vehicle 1 between a manual driving mode and a normal autonomous driving mode in response to a driving mode selection operation input via the touch panel 112.

[0027] Note that the input that triggers the manual / automatic driving mode switching control unit 104 to perform control to switch the driving mode of the autonomous vehicle 1 between the manual driving mode and the normal autonomous driving mode is not limited to a selection operation input via the touch panel 112, but may also be a voice input of a switching instruction via the in-vehicle microphone 139. In the case of a voice input, the voice input of the switching instruction via the in-vehicle microphone 139 is subjected to voice recognition, and the recognition result is supplied to the manual / automatic driving mode switching control unit 104. The manual / automatic driving mode switching control unit 104 switches the driving mode based on the received voice recognition result.

[0028] Then, the manual operation detection unit 133 receives operation information of the accelerator pedal operation, brake pedal operation, shift lever operation, and steering operation by the driver, and supplies the manual driving operation information to the manual / automatic driving mode switching control unit 104. In the normal automatic driving mode, when the manual operation detection unit 133 detects a manual driving operation by the driver at the manual operation detection unit 134, the manual / automatic driving mode switching control unit 104 performs control to switch to the manual driving mode.

[0029] However, in this embodiment, in the forced automatic driving mode, even if there is a predetermined operation to return to the manual driving mode, the manual / automatic driving mode switching control unit 104 invalidates the operation instruction and maintains the automatic driving mode (normal automatic driving mode). Then, when there is an instruction from the control unit 101 to cancel the forced automatic driving mode, in this embodiment, the manual / automatic driving mode switching control unit 104 switches from the forced automatic driving mode to the normal automatic driving mode.

[0030] When the autonomous vehicle 1 is in manual driving mode, the manual / autonomous driving mode switching control unit 104 supplies manual driving operation information from the manual operation detection unit 133 to the motor drive control unit 102 and the steering drive control unit 103, and controls the motor drive unit 131 and the steering drive unit 132 in accordance with the driver's pedal operation, shift lever operation, and steering operation (handle operation).

[0031] Furthermore, when the autonomous vehicle 1 is in the normal autonomous driving mode and the forced autonomous driving mode, the manual / automatic driving mode switching control unit 104 supplies the autonomous driving operation information generated by the control unit 101 based on the outputs of the radar group 106, the camera group 107, the sensor group 108, and the surrounding moving object grasping unit 109 to the motor drive control unit 102 and the steering drive control unit 103, as described below, and controls the motor drive unit 131 and the steering drive unit 132 to drive by the autonomous driving operation information, so as to perform autonomous driving. In the normal autonomous driving mode, the car navigation function unit 113 searches for a route from the current position to a destination (destination) set by the driver or the like, and controls the vehicle to travel along the searched route. In the forced autonomous driving mode, as described below, the vehicle is controlled to stop at a location where it is possible to call out to a target person detected by gait recognition.

[0032] In the case of a fully autonomous vehicle, since there is only an autonomous driving mode, there is no need for switching control between the manual driving mode and the autonomous driving mode, and there is no manual / autonomous driving mode switching control unit 104 or manual operation detection unit 133. In addition, since a fully autonomous vehicle is always in autonomous driving, the concept of a forced autonomous driving mode is also unnecessary.

[0033] The brake drive control unit 105, under the control of the control unit 101, controls the supply of a drive control signal to the brake drive unit 134 of the autonomous vehicle 1 of this embodiment, thereby controlling the autonomous vehicle 1 to slow down or stop.

[0034] The radar group 106 is used to measure the distance to people and objects around the autonomous vehicle 1, and consists of one or more laser radars (formally known as LIDAR (Light Detection and Ranging or Laser Imaging Detection and Ranging)) and millimeter wave radars. The laser radar is embedded, for example, near the ceiling or bumper, and the millimeter wave radar is provided, for example, at the front and rear of the vehicle. The vehicle may be equipped with both laser radar and millimeter wave radar, or only one of them. Other radars, such as microwave radar, may also be used. In addition, sonar (not shown) can be used for the same purpose as radar.

[0035] Camera group 107 includes one or more cameras that capture images of the interior of autonomous vehicle 1, and one or more cameras that capture images of the surroundings outside autonomous vehicle 1. The cameras that capture images of the interior of the vehicle are attached, for example, to a rearview mirror (rear-view mirror, room mirror) installed between the driver's seat and the passenger seat or to the top of the front window, and include cameras that capture images of the behavior of a person sitting in the driver's seat (driver), as well as cameras for capturing images of the behavior of passengers (fellow passengers) sitting in the passenger seat or in the back seat.

[0036] The camera that captures the surroundings outside the self-driving car 1 is for capturing the view area in front, sides, rear, etc. of the self-driving car, and includes, for example, two cameras (stereo cameras) attached to the left and right sides of the rearview mirror and mainly capturing the left and right front of the self-driving car 1, cameras attached to, for example, the door mirror or fender mirror of the self-driving car 1 and capturing the left and right sides, and a camera capturing the rear of the self-driving car 1. Furthermore, an air vehicle such as a drone equipped with a camera that captures the surroundings outside the vehicle may be installed (not shown) on the self-driving car 1, and may fly in the sky as necessary in cooperation with the self-driving car 1, and gait recognition may be performed from images captured in the sky. In this case, the air vehicle such as the drone and the self-driving car 1 can be connected by wire or wirelessly, and the images captured by the equipped cameras are transmitted to the self-driving car 1 by wire or wirelessly. Note that these cameras may include a night vision camera or an infrared camera in order to perform gait recognition at night.

[0037] The sensor group 108 includes an opening / closing detection sensor for detecting the opening / closing of a door or window, a sensor for detecting the fastening of a seat belt, a seating sensor (e.g., a weight sensor) for detecting that a passenger is seated in a seat such as the driver's seat or the passenger seat, a touch sensor (e.g., a capacitance sensor) for detecting that a person touches the steering wheel of the driver's seat, a human presence sensor (e.g., an infrared sensor) for detecting a person nearby outside the vehicle, and various sensors for acquiring information that assists in automatic driving. The various sensors for acquiring information that assists in automatic driving include, for example, a vibration sensor for detecting vibrations of the vehicle or tires, a rotation speed sensor for detecting the number of rotations of the tires, a geomagnetic sensor for detecting a direction, an acceleration sensor for detecting acceleration, a gyro sensor (gyroscope) for detecting an angle or an angular velocity, and the like. In this embodiment, the sensor group 108 also includes a sensor for detecting the lighting of a right turn signal, a left turn signal (direction indicator), and a hazard lamp (emergency flashing lamp).

[0038] The surrounding moving object grasping unit 109 grasps moving objects (including people) around the vehicle using images captured by the radar group 106, the sensor group 108, and the camera group 107. The surrounding moving object grasping unit 109 grasps surrounding obstacles and moving objects by performing processing based on machine learning such as Bayes' theory and deep learning.

[0039] The current position detection unit 110 receives radio waves from GPS satellites to detect the current position of the vehicle. Since the accuracy of the position detected by radio waves from GPS satellites is poor, the current position detection unit 110 uses not only the information on the current position detected by receiving radio waves from GPS satellites, but also one or more sensors included in the sensor group 108, the radar group 106, and the captured images (also using a navigation function) of the camera group 107, and performs processing based on machine learning such as Bayes' theory and deep learning, thereby detecting and confirming the current position with higher accuracy.

[0040] In autonomous driving mode (including normal autonomous driving mode and forced autonomous driving mode), the autonomous vehicle 1 uses the current position detection unit 110 and surrounding moving object grasping unit 109 to process various information such as position information obtained by receiving radio waves from the radar group 106, the camera group 107, the sensor group 108, and GPS satellites, in other words, information corresponding to information obtained by the human eyes and ears, using machine learning such as Bayes' theory and deep learning, and based on this, the control unit 101 performs intelligent information processing (artificial intelligence) and control (artificial intelligence) such as changing the vehicle's course and avoiding obstacles, to generate autonomous driving operation information.

[0041] The display unit 111 is, for example, an LCD (Liquid Crystal Display). The touch panel 112 is arranged by superimposing a touch sensor that allows touch input with a finger, a touch pen, or the like on the display screen of the display unit 111 made of an LCD. On the display screen of the display unit 111, a display image including software buttons (including character input buttons of a keyboard) is displayed based on the control of the control unit 101. When the touch panel 112 detects a touch with a finger, a touch pen, or the like on a software button displayed on the display screen, it transmits the touch to the control unit 101. The control unit 101 that receives this is configured to execute a control process corresponding to the software button. Of course, there may be multiple display units, and it is possible to install a display unit facing the outside on the body or window glass of the autonomous vehicle 1.

[0042] Domestic maps and route guidance data are stored in advance in car navigation database 135 connected to car navigation function unit 113. Car navigation function unit 113 is a functional unit that provides guidance to assist autonomous vehicle 1 in moving to a specified destination, based on the maps and route guidance data stored in car navigation database 135. In this embodiment, car navigation function unit 113 is configured to perform slightly different processes between manual driving mode and normal autonomous driving mode.

[0043] That is, in the manual driving mode, the car navigation function unit 113 displays an image on the display screen of the display unit 111 in which the vehicle position detected and confirmed by the current position detection unit 110 is superimposed on a map that clearly displays the route to the destination, and also moves the vehicle position (current position) on the map as the vehicle moves, and provides voice guidance at points where route guidance is necessary, such as intersections and branching points on the route. This is the same as the normal car navigation function.

[0044] On the other hand, in the normal autonomous driving mode, when the current position of the vehicle is away from the route to the destination, the car navigation function unit 113 notifies the control unit 101 of information on the direction of departure and distance, and when the current position of the vehicle is on the route to the destination, the car navigation function unit 113 notifies the control unit 101 of information instructing the change of the course direction along the route before an intersection or a branch point on the route as the vehicle moves. Based on the information notified from the car navigation function unit 113, the current position confirmation result of the current position detection unit 110 and the recognition result of the surrounding moving object recognition unit 109, the control unit 101 controls the motor drive unit 131 via the motor drive control unit 102 so that the vehicle moves along the route as instructed, and generates autonomous driving operation information for controlling the steering drive unit 132 via the steering drive control unit 103. Therefore, the route guidance to the destination by the car navigation function unit 113 and the control unit 101 in the normal autonomous driving mode allows the autonomous vehicle 1 to move to the destination even when there are no passengers.

[0045] The target person information registration unit 114 accepts registration input of information on a target person whose gait is to be recognized, such as a user of the autonomously driven vehicle 1 of this embodiment, for example a family member or a group member who shares the autonomously driven vehicle 1, and stores and registers the information in the target person information storage unit 136. The target person information includes, in addition to gait recognition information used for gait recognition discrimination, identification information of the target person, information for enabling the target person to notice when called, and information for confirmation and authentication when the target person attempts to get into the vehicle.

[0046] In this embodiment, the information for gait recognition includes gait information and gait recognition auxiliary information of the target person. The target person information registration unit 114 extracts a walking silhouette image of the target person from an image captured by the camera, and analyzes the walking silhouette image to generate gait information with the target person's stride length, swing of limbs, posture, walking cycle, left-right asymmetry of movement, and the like as feature quantities, and stores the information in the target person information storage unit 136 in association with the identification information of the target person. The gait recognition auxiliary information is, for example, the height of the target person, head and skin texture information, and the like, and is information for improving the recognition rate of gait recognition. The target person information registration unit 114 acquires the gait recognition auxiliary information by input by the target person or from the captured image information of the camera, and stores the information in the target person information storage unit 136 in association with the identification information of the target person.

[0047] It should be noted that gait recognition (gait authentication) using only gait information can authenticate an individual with a 94 to 95% accuracy, but it has been reported that the authentication rate can be increased to 99% if auxiliary gait recognition information is used (for example, see the new exhibit at the Miraikan National Museum of Emerging Science and Innovation: More accurate than face authentication!? What is "gait authentication" that can identify an individual by "how they walk"; http: / / monoist.atmarkit.co.jp / mn / article / 1507 / 17 / news037.html). The registration process and the target individual information in the target individual information registration unit 114 will be described in detail later.

[0048] In this embodiment, the information for verification and authentication when a target person attempts to get into the vehicle is the target person's facial image, spoken voice, fingerprint, vein, iris, or other biometric information. The information for verification and authentication is not limited to biometric information such as a facial image or spoken voice, but may be a telephone number, an email address, a password, or a password. The key to the autonomous vehicle 1 may be used as the information for verification and authentication, or a smartphone with a built-in non-contact IC card may be held up to a predetermined position on the autonomous vehicle 1, and the autonomous vehicle 1 may use the information for verification and authentication recorded on the non-contact IC card. Furthermore, a wristwatch-type terminal may be used instead of a smartphone.

[0049] In this embodiment, the gait recognition discrimination unit 115 has a function of accepting a selection setting of a target person to be detected by the driver and / or passenger prior to gait recognition. That is, in the autonomous vehicle 1 of this embodiment, the driver and / or passenger can select and set a specific one or more target persons among the registered target persons as the target persons to be detected. It is of course possible to select and set all of the registered target persons as the target persons to be detected. Note that, when the autonomous vehicle 1 of this embodiment autonomously travels in the normal autonomous driving mode in an unmanned state without the driver and / or passenger, all of the target persons stored and registered in the target person information storage unit 136 are automatically set as the target persons to be detected.

[0050] In addition, the autonomous vehicle 1 of this embodiment is configured to authenticate a target person using the confirmation authentication unit 117 when the target person gets in, so that target people who are already in the vehicle can be recognized. Therefore, the autonomous vehicle 1 is configured to exclude target people who are already in the vehicle from target people who should be detected by gait recognition. Of course, when a target person gets out of the vehicle, even if they were once excluded from target people who should be detected by gait recognition, they become a target person who should be detected by gait recognition at the same time as they get out of the vehicle.

[0051] In this example, the gait recognition discrimination unit 115 extracts gait information from captured image information from one or more cameras that capture the view area outside the vehicle among the camera group 107, with the stride length, swing of limbs, posture, walking cycle, left-right asymmetry of movement, and the like of each person included in the captured image as a feature, and performs gait recognition by comparing the gait information of a target person set as a target person to be detected among registered target persons stored in the target person information storage unit 136, and determines whether a person with a gait matching the gait of the target person to be detected has been detected. In this case, it goes without saying that the gait recognition discrimination unit 115 of this embodiment also performs gait recognition using gait recognition auxiliary information. In addition, when a walking target person is walking toward the autonomously driven vehicle 1, face recognition may be used in combination.

[0052] When gait recognition discrimination unit 115 detects a person having a gait matching the gait of a registered target person stored in target person information storage unit 136 through the above-mentioned gait recognition discrimination, it transfers the discrimination result that the detected target person has been detected, including identification information of the target person having a gait matching the gait of the detected person, to gait recognition discrimination result response processing unit 116 via system bus 100.

[0053] The gait recognition discrimination result response processing unit 116 performs predetermined processing upon receiving the discrimination result from the gait recognition discrimination unit 115. In this embodiment, the gait recognition discrimination result response processing unit 116 has a function of receiving a setting input for selecting a target person to be detected by the driver, passenger, or owner of the vehicle, and a setting input for predetermined processing, as will be described later, and determines what kind of processing to perform as the predetermined processing based on the setting input.

[0054] The gait recognition discrimination result response processing unit 116 treats the above setting input as valid not only when the driving mode of the autonomous vehicle 1 is a manual driving mode, but also when the autonomous vehicle 1 is in a normal autonomous driving mode in which the vehicle travels autonomously, provided that a driver and / or passengers are present on board.

[0055] However, when there is no driver and / or passenger and the vehicle is autonomously traveling in the normal autonomous driving mode in an unmanned state, the gait recognition determination result response processing unit 116 validates the setting input of the selection of the target person to be detected, but performs the predetermined processing suitable for the unmanned normal autonomous driving mode. That is, in this embodiment, when a target person is detected by gait recognition in the unmanned normal autonomous driving mode, the predetermined processing is to stop near the detected person, call out to the detected person, and send a message inviting the person to board, and when the detected person responds that he or she will board, the confirmation authentication unit 117 confirms and authenticates that the person is the target person, and then drives and controls the door opening / closing drive unit 118 to open and close the door, thereby allowing the person to board.

[0056] As described later, when the gait recognition discrimination result response processing unit 116 is set to stop the vehicle in the vicinity of a detected target person, the processing unit 116 switches the driving mode to a forced automatic driving mode to stop the vehicle, regardless of whether the driving mode is a manual driving mode or a normal automatic driving mode.

[0057] In addition, the vehicle may be controlled to stop in the vicinity of the detected target person while remaining in the normal automatic driving mode, without switching to the forced automatic driving mode. In that case, however, the driver may manually operate the vehicle to switch to the manual driving mode, and the vehicle may pass by the detected target person without stopping near the target person, and such a configuration may be adopted. Such a configuration is convenient when the vehicle is in a hurry, for example, when a sick person is being transported to a hospital. It is possible to easily determine whether the driver and / or passengers are present or not by monitoring the images captured by the camera that captures the interior of the vehicle, among the camera group 107. It is also possible to monitor the opening and closing of the doors and the passengers getting off and getting on.

[0058] The process of inputting settings to the gait recognition determination result processing unit 116 and the predetermined process when a target person is detected by gait recognition will be described in detail later.

[0059] The confirmation authentication unit 117 performs confirmation authentication when a target person detected by the gait recognition and discrimination unit 115 is about to get into the vehicle. In this embodiment, the target person information storage unit 136 stores a face image and / or a voice of the registered target person as information for confirmation authentication. In this embodiment, the confirmation authentication unit 117 includes an image recognition function unit for face image authentication and a voice recognition function unit for voice authentication.

[0060] The image recognition function unit of the confirmation authentication unit 117 performs confirmation authentication based on whether a face image of the person who is about to get on and detected by the gait recognition and discrimination unit 115, which is captured by a camera in the camera group 107, matches or does not match with a face image of the target person stored in the target person information storage unit 136. In addition, the voice recognition function unit of the confirmation authentication unit 117 performs confirmation authentication based on whether a voice uttered by the person who is about to get on and detected by the gait recognition and discrimination unit 115, which is captured by the microphone 140, matches or does not match with the voice uttered by the target person stored in the target person information storage unit 136.

[0061] As described above, the authentication result in the confirmation authentication unit 117 is passed to the gait recognition discrimination result handling processing unit 116 and is used to determine whether or not to permit the person detected in the gait recognition discrimination unit 115 to board the vehicle.

[0062] It should be noted that authentication can also be performed using the fingerprint of a person who wishes to board the vehicle. In this case, the autonomous vehicle 1 is provided with a fingerprint reader, the fingerprint of the person is stored in the target person information storage unit 136, and the authentication unit 117 is configured to have a fingerprint recognition function, and performs authentication by determining whether the stored fingerprint matches the fingerprint acquired by the fingerprint reader. For authentication, veins, irises, voiceprints, and other biometric information can be used, and in these cases, similar configuration changes can be made to make this possible. Of course, biometric information such as face images, voices, and fingerprints may be stored in combination and used for authentication.

[0063] Furthermore, confirmation and authentication can be performed by the person who wishes to get in holding a key to autonomous vehicle 1. Also, as described above, confirmation and authentication can be performed by holding a smartphone with a built-in contactless IC card over a predetermined position on autonomous vehicle 1. Furthermore, instead of a smartphone, a wristwatch-type terminal may be used.

[0064] The door opening / closing drive unit 118 is a drive unit for automatically opening and closing the doors of the vehicle. The door opening / closing drive unit 118 is driven and controlled in response to an opening / closing command from the control unit 101 in response to an operation of the touch panel 112 or other buttons, and is also driven and controlled in response to an opening / closing command from the gait recognition discrimination result corresponding processing unit 116, as described above. The doors may be of a sliding type or of a swinging type.

[0065] The voice input / output unit 119 takes in voice picked up by the in-vehicle microphone 139 or the outside-vehicle microphone 140 and sends it to the system bus 100 for voice recognition processing, for example. The voice input / output unit 119 also incorporates a voice synthesizer and a DA converter that receive voice message data to be emitted inside or outside the vehicle from the voice message generating unit 120 and converts the data into an analog voice signal. Then, under the control of the control unit 101, the voice input / output unit 119 supplies the converted analog voice signal to the in-vehicle speaker 137 or the outside-vehicle speaker 138 to be emitted as voice.

[0066] The voice message generation unit 120, under the control of the control unit 101, generates voice message information to be emitted to the driver and / or passengers, and also generates a calling voice message to call out to a target person detected by gait recognition, and sends it to the voice input / output unit 119.

[0067] The target person information storage unit 136 stores, for example, a call name (nickname) as information for enabling the target person to recognize that the call is directed to him / her, and the voice message generating unit 120 synthesizes the stored call name of the target person with a template voice message to generate a required voice message. Of course, the call name of the target person is not limited to a nickname, and may be a real name, first name, initials, or number.

[0068] For example, in the case of a voice message to inform the driver and / or passengers that a target person has been detected, message information such as "○○○○ has been detected" is prepared as a template voice message, and information on the address name of the detected target person is obtained from the target person information storage unit 136.When the address name is "Miko-chan," "Miko-chan" is inserted into the "○○○○" part to generate the voice message "Miko-chan has been detected."

[0069] The voice message generating unit 120 has prepared other template voice messages such as "Shall we stop at XXX and let you board?" for in-car use and "XXX! Would you like to get in?" for outside the vehicle. For example, when the detected target person calls out to "Miko-chan," the in-car message is "Shall we stop at Miko-chan and let you board?" and the outside message is "Miko-chan! Would you like to get in?".

[0070] The electronic control circuit unit 10 of the autonomous vehicle 1 is configured as described above. In the above description, the motor drive control unit 102, the steering drive control unit 103, the manual / autonomous driving mode switching control unit 104, the surrounding moving object grasping unit 109, the current position detection unit 110, the car navigation function unit 113, the target person information registration unit 114, the gait recognition and discrimination unit 115, the gait recognition and discrimination result corresponding processing unit 116, the confirmation authentication unit 117, and the voice message generation unit 120 shown in Fig. 1 can be configured as processing function units using software programs, and each unit can be realized as software processing by providing the control unit 101 with a program that executes each function. It goes without saying that not all of the above units are configured as processing functions using programs, but some of them may be configured as hardware components.

[0071] [Example of Target Person Information Registration Procedure in Target Person Information Registration Unit 114] FIG. 2 is a diagram showing an example of the flow of a target individual information registration procedure in the target individual information registration unit 114. As shown in FIG.

[0072] First, the target person or the registration assistant makes a request to the self-driving vehicle 1 to register target person information by selecting the target person information registration item from the function menu of the self-driving vehicle 1 displayed on the display unit 111 (Step S1).

[0073] In response to the request to register the target person information, the control unit 101 instructs the target person to perform a gesture for detecting the target person's gait information by an instruction message displayed on the display screen of the display unit 111 and / or by an audible message from the speakers 137, 138 (step S2). Specifically, this instruction is an instruction to have the target person walk so that the walking state of the target person is photographed by a predetermined camera in the camera group 107. In this case, in addition to a scene in which the target person walks on the left side of the self-driving car 1 from the rear to the front of the vehicle, the target person is instructed to walk in the opposite direction from the front to the rear of the vehicle, and further, a plurality of walking patterns such as a left diagonal direction or the opposite direction, a right diagonal direction or the opposite direction, a direction crossing from left to right, and a direction crossing from right to left, etc. The instructions for the walking patterns of the target person are not limited to these, and it is needless to say that a wide variety of walking patterns can be instructed. In addition, if the subject is asked to walk in the opposite direction from the front to the rear of the vehicle, or if the subject is walking diagonally or crossing left and right, facial images of the subject can be captured at the same time, so that the facial images can be registered along with the gait information.

[0074] It is also possible to classify and register sunny days, cloudy days, rainy days, and snowy days, and it is also possible to classify and register daytime and nighttime. It is also possible to classify and register indoors and outdoors. By expanding this further, it is also possible to register not only walking but also running conditions. When classifying and registering daytime and nighttime, it is also possible to take pictures with a normal camera during the day and with an infrared camera at night.

[0075] When the target person walks according to the instructions in step S2, the control unit 101 causes a predetermined camera in the camera group 107 to capture a video of the target person walking, and transmits the captured image information to the target person information registration unit 114 (step S3). Then, the control unit 101 determines whether or not sufficient information has been acquired to enable detection of the target person's gait information (step S4). Then, the control unit 101 repeats steps S2 and S3 until sufficient information has been acquired to enable detection of the target person's gait information.

[0076] Next, when it is determined in step S4 that sufficient information has been acquired to enable detection of the gait information of the target person, the control unit 101 instructs the target person information registration unit 114 to begin extraction processing of gait information, and causes feature quantities such as the target person's stride length, limb swing, posture, walking cycle, and left-right asymmetry of movement to be detected and stored as the gait information of the target person (step S5).

[0077] Next, the control unit 101 prompts the target person to input information such as height and gender, which serve as gait recognition auxiliary information, and acquires the input information. At the same time, the control unit 101 acquires the target person's head and skin texture as gait recognition auxiliary information from the image captured by the camera, and stores them in the target person information registration unit 114 (step S6).

[0078] Next, the control unit 101 prompts the target person to provide information for verification and authentication, and in response, acquires the target person's face image and voice information, and stores them in the target person information registration unit 114 (step S7).

[0079] Furthermore, the control unit 101 prompts the target person to input a nickname (nickname) as an example of information to enable the target person to recognize that the call is directed at him / her, and acquires the information of the nickname (nickname) input by the target person in response, and stores it in the target person information registration unit 114 (step S8).

[0080] Finally, the control unit 101 assigns identification information to the target person, instructs the target person information registration unit 114 to store the target person's gait information, gait recognition assistance information, confirmation authentication information, and address name (nickname) information in the target person information storage unit 136 in association with the target person's identification information, and executes the storage (step S9).

[0081] This completes the registration of the target person information. For example, when the autonomous vehicle 1 is for personal use, the registration of the target person information is performed for each family member. Furthermore, in a mode in which the autonomous vehicle 1 is used jointly by a group, such as car sharing, the registration of the target person information is performed for each group member that constitutes the group. Note that the registration of the target person information is not limited to being performed by the autonomous vehicle 1, but the family members or group members may record their walking (or running) with a smartphone, video camera, or the like, and transmit the recorded information to the autonomous vehicle 1. Note that the recording of the images may be performed by a drone (unmanned aerial vehicle) using a camera mounted on the drone while flying and moving in pursuit of the target person.

[0082] An example of the stored information contents in the target person information storage unit 136 in which the target person information is stored in this manner is shown in Fig. 3. In the example of Fig. 3, family members are registered as target persons, and the target person information for each of four people, a father, a mother, a daughter, and an older brother, is registered and stored.

[0083] [Presetting the target person's gait recognition] In the self-driving car 1 of this embodiment, it is configured to accept pre-settings by the driver and / or passengers from a menu before starting gait recognition of a target person. The pre-setting items are as follows:

[0084] (1) Selection of target persons to be detected by gait recognition

[0085] (2) Setting the predetermined action to be taken when a target person is detected It is.

[0086] Regarding (1), it is possible to set the detection targets to be all of the target persons stored in the target person information storage unit 136, or to set the detection targets to be one or more selected target persons rather than all of the target persons.

[0087] As for (2), when a target person is detected, it is possible to set whether or not the vehicle will always stop (unconditional stop setting), to stop only when there is room for passengers in the vehicle, to temporarily stop to leave a message, etc., to inquire of the driver and / or passengers before stopping, or to notify the driver and / or passengers without stopping. Gait recognition auxiliary information, such as the height of the target person, can also be set.

[0088] This pre-setting acceptance function is activated when the driver and / or passengers get into the autonomous vehicle 1, by selecting it from the settings menu or by operating the pre-setting acceptance button.

[0089] The function for accepting the presetting may be preset as a default in the autonomous vehicle 1 so as to always be automatically activated when the autonomous vehicle 1 detects that the driver and / or passenger has boarded the vehicle. The function for accepting the presetting is not limited to being activated when the driver and / or passenger boards the vehicle, but may be configured to accept the presetting by inquiring the driver and / or passenger as to whether or not to make a presetting when the driving mode is switched, such as when the driving mode is switched from the manual driving mode to the autonomous driving mode, or when the driving mode is switched from the autonomous driving mode to the manual driving mode. Furthermore, the function may be configured to accept a change in the setting when the presetting has already been set, not limited to accepting the presetting when the driving mode is switched. The function may of course be configured to accept the presetting without inquiring the driver and / or passenger as to whether or not to make a presetting or whether or not to change the setting.

[0090] An example of a processing routine for presetting the gait recognition of a target person in this embodiment will be described below with reference to the flowcharts of Figures 4 and 5. The function of presetting the gait recognition of a target person is included in gait recognition discrimination unit 115, but in the following description of the example of Figures 4 and 5, the gait recognition discrimination unit 115 will be described as a functional block based on a software program included in control unit 101, and the operation of each step will be described as being executed by control unit 101.

[0091] First, the control unit 101 displays a list of a function menu on the display screen of the display unit 111 (step S11). Then, the control unit 101 determines whether or not a pre-setting item for gait recognition of a target person has been selected from the list display of this function menu (step S12).

[0092] In this step S12, when it is determined that an item other than the pre-set item for gait recognition of the target person has been selected from the list display of the function menu, the control unit 101 transitions to a processing routine for the selected other menu item and executes that processing routine.

[0093] When it is determined in step S12 that the item for presetting the gait recognition of a target person has been selected from the list display of the function menu, the control unit 101 displays a list of target persons stored in the target person information storage unit 136 on the display screen, and prompts the selection and designation of a target person to be a detection target for gait recognition (step S14). When selecting and designating a target person to be a detection target for gait recognition, information on a facial image of the target person may be read from the target person information storage unit 136, and the facial image of the target person may be displayed on the display screen of the display unit 111, so that the target person can be referred to for selection and identification.

[0094] Then, the control unit 101 determines whether or not it has been selected to make all of the target persons stored in the target person information storage unit 136 the detection targets for gait recognition (step S15), and if it is determined that it has been selected to make all of the target persons the detection targets for gait recognition, it sets all of the target persons to be the detection targets for gait recognition (step S16).

[0095] Furthermore, when it is determined in step S15 that not all people have been selected as detection targets for gait recognition, the control unit 101 accepts individual selection of target people to be detection targets for gait recognition (step S17), and sets the selected target people to be detection targets for gait recognition (step S18). The number of selected people is not limited to one, and may be multiple.

[0096] When the setting in step S16 is completed, and when the setting in step S18 is completed, the control unit 101 determines whether the setting "Always stop when a target person is detected by gait recognition" has been selected (step S19), and when it determines that the setting "Always stop when a target person is detected by gait recognition" has been selected, it sets an unconditional stop (step S20).

[0097] After step S20, the control unit 101 determines whether or not an instruction to perform gait recognition has been received (step S39 in FIG. 5). If it is determined that the instruction has been received, the control unit 101 starts a processing routine for detecting a target person using gait recognition (step S40), and ends the processing routine of FIGS. 4 and 5.

[0098] If it is determined in step S39 that the execution instruction has not been received, the control unit 101 determines whether or not an instruction input to stop the processing routine of this setting has been received (step S41), and if it is determined that an instruction input to stop has been received, the control unit 101 ends this processing routine. If it is determined in step S41 that an instruction input to stop has not been received, the control unit 101 returns the process to step S19 in Fig. 4, and repeats the processes from step S19 onwards.

[0099] Then, when it is determined in step S19 that the setting "Always stop when a target person is detected by gait recognition" has not been selected, the control unit 101 determines whether or not the setting "Stop taking into consideration the passenger capacity of the vehicle" has been selected (step S31 in FIG. 5). When it is determined in this step S31 that the setting "Stop taking into consideration the passenger capacity of the vehicle" has been selected, the control unit 101 further determines whether or not the setting "Inquiry to the driver and / or passengers is required" has been selected (step S32).

[0100] In this step S32, when it is determined that the setting "inquiry to driver and / or passengers is necessary" has been selected, the control unit 101 sets "stop conditional on passenger capacity, inquiry required" (step S33). Also, when it is determined that the setting "inquiry to driver and / or passengers is not necessary" has been selected in step S32, the control unit 101 sets "stop conditional on passenger capacity, inquiry not required" (step S34). Then, after this step S33, or after step S34, the control unit 101 proceeds to step S39, and the above-mentioned processing from step S39 onwards is repeated.

[0101] Furthermore, when it is determined in step S31 that the setting "Stop taking into consideration the passenger capacity of the vehicle" has not been selected, the control unit 101 determines whether or not the setting "Deciding whether to stop by inquiring of the driver and / or passengers" has been selected (step S35). When it is determined in this step S35 that the setting "Deciding whether to stop by inquiring of the driver and / or passengers" has been selected, the control unit 101 sets "inquiry conditional stop" (step S36). Then, after this step S36, the process proceeds to step S39, and the processes from step S39 onwards are executed.

[0102] Then, when it is determined in step S35 that the setting "ask the driver and / or passengers to decide whether or not to stop" has not been selected, the control unit 101 determines whether the setting "when a target person is detected by gait recognition, only notify the detection" has been selected (step S37).

[0103] Then, in step S37, if it is determined that the setting "when a target person is detected by gait recognition, only notify the detection" has been selected, the control unit 101 sets "only notify detection of target person" (step S38). Then, after step S38, the process proceeds to step S39, and the processes from step S39 onwards are executed.

[0104] Also, when it is determined in step S37 that the setting "when a target person is detected by gait recognition, the detection is only notified" has not been selected, the control unit 101 advances the process to step S41, where it determines whether or not an instruction input to stop the processing routine of this setting has been received (step S41), and when it is determined that an instruction input to stop has been received, it ends this processing routine. Then, when it is determined in step S41 that an instruction input to stop has not been received, the control unit 101 returns the process to step S19 in Fig. 4, and repeats the processes from step S19 onwards.

[0105] [Detection of target person using gait recognition] Next, an example of the process flow of detecting a target person by gait recognition activated in step S40 in Fig. 5 will be described with reference to the flowcharts shown in Fig. 6 to Fig. 10. This process of detecting a target person by gait recognition is executed by gait recognition discrimination unit 115 following the settings described above with reference to Fig. 4 and Fig. 5, but in the following description, it is assumed that gait recognition discrimination unit 115 is configured as a functional block of a software program included in control unit 101, and the operation of each step is executed by control unit 101.

[0106] The control unit 101 first determines whether the detection targets are all of the target persons stored in the target person information storage unit 136 (step S51 in FIG. 6). If it is determined in step S51 that the detection targets are all of the target persons stored in the target person information storage unit 136, the control unit 101 starts detection by gait recognition with all of the stored target persons as detection targets (step S52). If it is determined in step S51 that the detection targets are not all of the target persons stored in the target person information storage unit 136, the control unit 101 recognizes the selected and specified target person, and starts detection of the target person by gait recognition with only the recognized target person as the detection target (step S53).

[0107] After step S52 and after step S53, the control unit 101 determines whether the current driving mode of the vehicle is a normal automatic driving mode or a manual driving mode. If it is determined in step S54 that the driving mode is the manual driving mode, it executes detection of a target person by gait recognition and determines whether the target person has been detected (step S55). If it is determined in step S55 that the target person has not been detected, the control unit 101 returns the process to step S54 and repeats the processes from step S54 onwards. Note that in step S55, if a face image is registered together with the gait information, face image recognition may also be used to detect the target person.

[0108] When it is determined in step S55 that a target person has been detected, the control unit 101 determines that a driver is present since the driving mode is the manual driving mode, obtains the call name (nickname) of the detected target person from the target person information storage unit 136, and notifies the driver of who the detected target person is (step S56). In addition to the call name by voice, the notification at this time may be made by reading out information on a face image of the detected target person from the target person information storage unit 136 and displaying the face image of the target person on the display screen of the display unit 111.

[0109] Next, the control unit 101 determines whether or not an unconditional stop is set according to a pre-setting (step S57). If it is determined in step S57 that an unconditional stop is set, the control unit 101 notifies the driver that the driving mode will be switched to a forced automatic driving mode and the vehicle will be stopped (step S58), and then executes control to switch from the manual driving mode to the forced automatic driving mode and to stop the vehicle near the detected target person (step S59).

[0110] Next, the control unit 101 determines whether the host vehicle has stopped (step S60), and if it determines that the host vehicle has not stopped, returns the process to step S59 and executes stop control. Then, if it determines in step S60 that the host vehicle has stopped, the control unit 101 cancels the forced automatic driving mode and switches the driving mode to the normal automatic driving mode (step S71 in FIG. 7).

[0111] Next, the control unit 101, using the function of the voice message generation unit 120, generates a voice message calling out to the detected target person including the call name (nickname) of the detected target person as described above, and supplies the voice message calling out to the vehicle exterior speaker 138 via the voice input / output unit 119, thereby calling out to the detected target person and alerting the target person (step S72).

[0112] In this embodiment, the control unit 101 controls the voice input / output unit 119 so as to enable a conversation between the driver and / or passenger and the detected external target person (step S73). This allows the driver and / or passenger to have a conversation to leave a necessary message or to encourage the detected target person to get on when the number of passengers on board is less than the passenger capacity. If the detected target person requests to get on in response to the prompting, the driver instructs the door to open and close, allowing the person to get on. If the driver simply wants to leave a message for the detected target person or the number of passengers on board is the same as the passenger capacity, the driver starts the vehicle without encouraging the detected target person to get on.

[0113] Therefore, the control unit 101 determines whether or not an instruction to open or close the door has been received (step S74), and when it determines that the instruction to open or close the door has been received, it controls the driving of the door opening / closing drive unit 118 to open or close the door, and confirms that the target person has boarded the vehicle from images captured by the cameras in the camera group 107 that capture the outside and / or inside of the vehicle (step S75). The image recognition for confirming the boarding of the target person is performed using an image recognition function provided in the confirmation authentication unit 117.

[0114] Then, when it is determined in step S74 that a door opening / closing command has not been received, and after step S75, the control unit 101 determines whether or not the driver has operated the vehicle to start traveling, that is, operated the accelerator (step S76). If it is determined in step S76 that the driver has operated the accelerator, the control unit 101 switches the driving mode from the normal automatic driving mode to the manual driving mode and starts traveling (step S77). Then, after step S77, the control unit 101 returns the process to step S55 in Fig. 6, restarts the process of detecting a target person by gait recognition, and repeats the process from step S55 onwards.

[0115] In this embodiment, the control unit 101 of the autonomous vehicle is configured to automatically open and close the doors, so the door opening and closing control is performed as described above, but the control unit 101 may of course only perform door lock control and allow the person who is about to board or the person currently in the vehicle to open and close the doors. In that case, the control unit 101 unlocks the doors based on an instruction from the driver or the person currently in the vehicle, or, as described below, based on the result of confirmation authentication as to whether the person who is about to board is a target person.

[0116] Furthermore, when it is determined in step S76 that the driver has not operated the accelerator, the control unit 101 determines whether or not a command to start driving in the normal automatic driving mode has been received (step S78). When it is determined in step S78 that a command to start driving in the normal automatic driving mode has not been received, the control unit 101 returns the process to step S76 and repeats the processes from step S76 onward.

[0117] Then, when it is determined in step S78 that a command to start driving in the normal autonomous driving mode has been received, the control unit 101 controls the vehicle to start driving in the normal autonomous driving mode (step S79). After step S79, the control unit 101 proceeds to step S101 in Fig. 10, where it performs a process of detecting a target person by gait recognition in the normal autonomous driving mode, which will be described later.

[0118] Next, in step S57 of Fig. 6, when it is determined that the preset is not an unconditional stop, the control unit 101 determines whether or not the preset is set to "consider the passenger capacity when stopping" (step S81 of Fig. 8). In this step S81, when it is determined that the preset is set to "consider the passenger capacity when stopping", the control unit 101 checks the number of passengers from an image captured by a camera that captures the interior of the vehicle among the camera group 107, and determines whether or not the number of passengers is less than the capacity (step S82).

[0119] If it is determined in step S82 that the number of passengers is not less than the capacity (the number of passengers is the capacity), the control unit 101 follows the driver's operation for the subsequent processing (step S89). In this case, the control unit 101 may generate a voice message in the voice message generating unit 120 saying, "The capacity will be exceeded, so the detected target person cannot be allowed to board. If you want to stop, please drive manually," and emit the message through the in-car speaker 137.

[0120] The driver and / or passenger who receives this voice message cannot allow the detected target person to board the vehicle, but if it is necessary to leave a message, the driver and / or passenger can stop the vehicle by operating the vehicle (brake) in the manual driving mode and perform the necessary action such as leaving a message to the detected target person. Also, if the driver and / or passenger judges that there is no message to leave and there is no need to stop, the driver and / or passenger can pass by the detected target person without stopping by operating the vehicle in the manual driving mode. Even in the automatic driving mode, if the driver and / or passenger pre-sets the vehicle to stop temporarily to leave a message, the driver and / or passenger can stop temporarily near the detected target person, leave a message, and continue automatic driving, regardless of whether the number of passengers is at capacity or less.

[0121] After step S89, the control unit 101 returns the process to step S55 in FIG. 6, and repeats the processes from step S55 onwards.

[0122] Then, when it is determined in step S81 that "stopping does not take into account the passenger capacity" is preset, or when it is determined in step S82 that the number of passengers is less than the capacity, the control unit 101 determines whether or not "stopping after inquiring of the driver and / or passengers" is preset (step S83). When it is determined in this step S83 that "stopping after inquiring of the driver and / or passengers" is preset, the control unit 101 generates a voice message inquiring of the driver and / or passengers whether or not to stop near the detected target person in the voice message generating unit 120, and executes the inquiry by emitting the generated voice message through the voice input / output unit 119 and the in-vehicle speaker 137 (step S84).

[0123] Next, the control unit 101 determines whether or not a response to the inquiry that the vehicle will be stopped has been received (step S85). The response from the driver and / or passenger to the inquiry is determined by recognizing the voice picked up by the in-vehicle microphone 139 using the voice recognition function of the confirmation authentication unit 117, or by monitoring the operation input by the driver and / or passenger through the touch panel 112.

[0124] In step S85, if the driver and / or passenger responds that they will not stop, the control unit 101 shifts the process to step S89, and the subsequent processes are performed according to the operation of the driver and / or passenger. In this case, the control unit 101 may generate a voice message "If you want to stop the vehicle, please operate the vehicle manually" in the voice message generating unit 120, and emit the message through the in-vehicle speaker 137.

[0125] Then, when it is determined in step S83 that "querying the driver and / or passengers to stop the vehicle" is not pre-set, and when a response to stop is obtained from the driver and / or passengers in step S85, the control unit 101 notifies the driver and / or passengers that the driving mode will be switched from manual driving mode to forced automatic driving mode and the vehicle will be stopped (step S86), and then executes control to switch from manual driving mode to forced automatic driving mode and stop the vehicle near the detected target person (step S87).

[0126] Next, the control unit 101 determines whether the host vehicle has stopped (step S88), and if it determines that the host vehicle has not stopped, returns the process to step S87 and executes stop control. Then, if it determines in step S88 that the host vehicle has stopped, the control unit 101 cancels the forced automatic driving mode and switches the driving mode to the normal automatic driving mode (step S91 in FIG. 9).

[0127] Next, the control unit 101 generates a voice message including a call name (nickname) of the detected target person as described above in the voice message generation unit 120, and emits the call voice message through the voice input / output unit 119 and the exterior speaker 138, thereby calling out to the detected target person and alerting them (step S92).

[0128] Then, the control unit 101 determines whether the number of passengers is less than the capacity (step S93), and when the number of passengers is the capacity, the process proceeds to step S76 in Fig. 7, and the process from step S76 onwards is executed. At this time, the control unit 101 may generate a voice message in the voice message generating unit 120, for example, "The capacity will be exceeded, so the detected target person cannot board. Please operate the vehicle or give a start signal after you have completed your business," and emit the message through the in-vehicle speaker 137. The start operation is for starting in the manual driving mode, and the start signal is for starting in the normal automatic driving mode.

[0129] If it is determined in step S93 that the number of passengers is less than the capacity, the control unit 101 causes the voice message generation unit 120 to generate a voice message inquiring of the detected target person outside the vehicle as to whether or not they wish to board the vehicle, and emits the generated voice message through the voice input / output unit 119 and the speaker 138, thereby inquiring of the detected target person (step S94).

[0130] Next, the control unit 101 receives a response to this inquiry from the target person outside the vehicle and determines whether or not the target person wishes to board the vehicle (step S95). The response from the target person to this inquiry is determined by performing voice recognition on the voice picked up by the exterior microphone 140 using the voice recognition function of the confirmation authentication unit 117.

[0131] If it is determined in step S95 that the target person does not wish to board the vehicle, the control unit 101 shifts the process to step S76 in FIG. 7, and executes the processes from step S76 onward described above.

[0132] Also, when it is determined in step S95 that the target person wishes to board, the control unit 101 acquires information for confirmation authentication from the target person and executes confirmation authentication (step S96). That is, the control unit 101 acquires information for confirmation authentication about the detected target person from the target person information storage unit 136, and if the information for confirmation authentication is a face image, the control unit 101 captures and acquires a face image of the target person using the outside-vehicle camera in the camera group 107, and performs confirmation authentication of the target person by comparing and referring to the acquired face image and the face image of the target person read from the target person information storage unit 136. At this time, the image recognition function of the confirmation authentication unit 117 is used. Also, if the information for confirmation authentication is a voice, the control unit 101 collects the voice of the target person using the outside-vehicle microphone 140, and compares and refers to the collected voice and the voice of the target person read from the target person information storage unit 136, and performs confirmation authentication of the target person. At this time, the voice recognition function of the confirmation authentication unit 117 is used.

[0133] Next, the control unit 101 determines whether the authentication is OK or not based on the result of the confirmation authentication (step S97). When the control unit 101 determines that the authentication is OK, it drives and controls the door opening / closing drive unit 118 to open and close the door, and confirms the boarding of the target person from the images captured by the cameras of the camera group 107 that capture the outside and / or inside of the vehicle (step S98). The image recognition for confirming the boarding is performed using an image recognition function provided in the confirmation authentication unit 117. When the control unit 101 determines that the authentication is NG rather than OK, it generates a voice message in the voice message generation unit 120 to the effect that the boarding is rejected since the authentication is NG, and notifies the detected person of the generated voice message through the voice input / output unit 119 and the vehicle exterior speaker 138 (step S99).

[0134] Then, after step S98 or after step S99, the control unit 101 shifts the process to step S76 in FIG. 7, and executes the processes from step S76 onward described above.

[0135] Next, when it is determined in step S54 of Fig. 6 that the driving mode is the normal automatic driving mode, or when traveling is started in the normal automatic driving mode in step S79 of Fig. 7, the control unit 101 executes detection of a target person by gait recognition and determines whether or not the target person has been detected (step S101 of Fig. 10). If it is determined in this step S101 that the target person has not been detected, the control unit 101 returns the process to step S54 of Fig. 6, and repeats the processes from step S54 onwards.

[0136] When it is determined in step S101 that a target person has been detected, the control unit 101 determines whether a driver and / or passenger is present (step S102) since the driving mode is normal automatic driving mode, and when it is determined that a driver and / or passenger is present, it obtains the address name (nickname) of the detected target person from the target person information storage unit 136 and notifies the driver and / or passenger of who the detected target person is (step S103).

[0137] Next, the control unit 101 determines whether or not an unconditional stop is set by a presetting (step S104). If it is determined in step S104 that an unconditional stop is set, the control unit 101 notifies the driver and / or passengers that the vehicle will be switched to a forced automatic driving mode and stopped, and executes control to switch from the manual driving mode to the forced automatic driving mode and to stop the vehicle near the detected target person (step S105).

[0138] Next, the control unit 101 determines whether the vehicle has stopped (step S106), and if it determines that the vehicle has not stopped, returns the process to step S105 and executes stop control. Then, if it determines in step S106 that the vehicle has stopped, the control unit 101 transitions the process to step S91 in Fig. 9, cancels the forced automatic driving mode, switches the driving mode to the normal automatic driving mode, and then executes the processes from step S91 onward.

[0139] Then, when it is determined in step S104 that the preset is not an unconditional stop, the control unit 101 determines whether or not it is preset to "consider the passenger capacity when stopping" (step S107). When it is determined in this step S107 that it is preset to "consider the passenger capacity when stopping", the control unit 101 determines whether or not the number of passengers is less than the capacity (step S108). When it is determined in this step S108 that the number of passengers is less than the capacity, or when it is determined in step S107 that it is not preset to "consider the passenger capacity when stopping", the control unit 101 shifts the process to step S83 in FIG. 8 and executes the process from step S83 onwards.

[0140] Furthermore, when it is determined in step S108 that the number of passengers is not below capacity (the number of passengers is at capacity), the control unit 101 causes the voice message generation unit 120 to generate a voice message stating, "The capacity will be exceeded, so the detected target person cannot be allowed to board. If you wish to stop the vehicle, please drive manually," and causes this message to be played through the in-vehicle speaker 137, while also causing the normal automatic driving mode to continue (step S111 in FIG. 11).

[0141] The driver and / or passenger who receives this voice message cannot allow the detected target person to board the vehicle, but if it is necessary to leave a message, the driver and / or passenger can stop the vehicle by operating the vehicle (brake) in manual driving mode and perform the necessary action such as leaving a message to the detected target person. Also, if the driver and / or passenger judges that there is no message to leave and there is no need to stop, the driver and / or passenger can pass by without stopping near the detected target person by operating the vehicle in manual driving mode. Even in automatic driving mode, if the driver and / or passenger pre-sets the vehicle to stop temporarily to leave a message, the driver and / or passenger can stop temporarily near the detected target person, leave a message, and continue automatic driving, regardless of whether the number of passengers is at capacity or less.

[0142] Next, the control unit 101 determines whether or not a switching operation to the manual driving mode has been detected (step S112), and if it determines that a switching operation to the manual driving mode has not been detected, returns the process to step S101 in Fig. 10, and repeats the processes from step S101 onward. Also, if it determines in step S112 that a switching operation to the manual driving mode has been detected, the control unit 101 switches the driving mode from the normal automatic driving mode to the manual driving mode (step S113), and thereafter performs a process according to the driver's operation (step S114). Then, after step S114, the control unit 101 returns the process to step S55 in Fig. 6, and executes the processes from step S55 onward.

[0143] Next, when it is determined in step S102 of Fig. 10 that there is no driver and / or passenger and the vehicle is being driven unmanned, the control unit 101 executes control to stop the vehicle in the vicinity of the detected target person while remaining in the normal automatic driving mode (step S121 of Fig. 12). In other words, during unmanned driving in the normal automatic driving mode, the vehicle is always stopped in the vicinity of the detected target person even if unconditional stopping is not set. In this case, however, the vehicle may be switched to the forced automatic driving mode, but since the vehicle is being driven unmanned, there is no need to switch to the forced automatic driving mode and the vehicle may remain in the normal automatic driving mode.

[0144] Next, the control unit 101 determines whether the vehicle has stopped (step S122), and if it has not stopped, returns the process to step S121 and executes stop control. Then, if it has been determined in step S122 that the vehicle has stopped, the control unit 101 causes the voice message generating unit 120 to generate a calling voice message including the calling name (nickname) of the detected target person as described above, and emits the calling voice message through the voice input / output unit 119 and the exterior speaker 138, thereby calling the detected target person and calling their attention (step S123).

[0145] Next, the control unit 101 generates a voice message in the voice message generation unit 120 to inquire of the detected target person outside the vehicle as to whether or not they wish to ride, and emits the generated voice message through the voice input / output unit 119 and the exterior speaker 138, thereby inquiring of the detected target person as to whether or not they wish to ride (step S124).

[0146] Next, the control unit 101 receives a response to this inquiry from the target person outside the vehicle and determines whether or not the target person wishes to board the vehicle (step S125). The response from the target person to this inquiry is determined by performing voice recognition on the voice picked up by the exterior microphone 140 using the voice recognition function of the confirmation authentication unit 117.

[0147] If it is determined in step S125 that the target person does not wish to ride, the control unit 101 starts driving in normal automatic driving mode (step S130), and then transitions the processing to step S101 in FIG. 10 and executes the processing from step S101 onward.

[0148] Also, when it is determined in step S125 that the target person wishes to board, the control unit 101 acquires information such as biometric information for identification and authentication from the target person and executes identification and authentication (step S126). That is, the control unit 101 acquires identification and authentication information for the detected target person from the target person information storage unit 136, and if the identification and authentication information is a face image, the control unit 101 captures and acquires a face image of the target person using the outside-vehicle camera in the camera group 107, and compares and refers to the acquired face image and the face image of the target person read from the target person information storage unit 136 to identify and authenticate the target person. At this time, the image recognition function of the identification and authentication unit 117 is used. Also, if the identification and authentication information is a voice, the control unit 101 collects the voice of the target person using the outside-vehicle microphone 140, and compares and refers to the collected voice and the voice of the target person read from the target person information storage unit 136 to identify and authenticate the target person. At this time, the voice recognition function provided in the confirmation authentication unit 117 is used.

[0149] Next, the control unit 101 determines whether the authentication is OK or not based on the result of the confirmation authentication (step S127). When the control unit 101 determines that the authentication is OK, it drives and controls the door opening / closing drive unit 118 to open and close the door, and confirms the boarding of the target person from the images captured by the cameras of the camera group 107 that capture the outside and / or inside of the vehicle (step S128). The image recognition for confirming the boarding is performed using an image recognition function provided in the confirmation authentication unit 117. When the control unit 101 determines that the authentication is NG rather than OK, it generates a voice message in the voice message generation unit 120 to the effect that the boarding is rejected since the authentication is NG, and notifies the detected person of the generated voice message through the voice input / output unit 119 and the vehicle exterior speaker 138 (step S129).

[0150] Then, after step S128 and after step S129, the control unit 101 starts driving in normal automatic driving mode (step S130), and then transitions the processing to step S101 in FIG. 10, and executes the processing from step S101 onwards.

[0151] [One aspect of the target person detection processing operation by gait recognition in an autonomous driving vehicle according to an embodiment] Next, an example of a detection process operation of a target person through gait recognition by the autonomous vehicle 1 of this embodiment will be described with reference to Fig. 13. In this example, when a target person is found, the autonomous vehicle 1 stops in the vicinity of the target person and inquires about getting on the vehicle.

[0152] As shown in FIG. 13(A), assume that a person 2, who is one of the target persons stored in the target person information storage unit 136, is walking in front of the autonomous vehicle 1 while the autonomous vehicle 1 of this embodiment is traveling. In this case, if the person 2 is set as a detection target for gait recognition in the autonomous vehicle 1 and a detection process using gait recognition is being executed, when the autonomous vehicle 1 approaches the person 2 at a distance L of, for example, about 40 to 50 meters as shown in FIG. 13(A), the control unit 101 of the autonomous vehicle 1 detects the person 2 as a person to be detected by gait recognition. Of course, the distance may be longer or shorter than about 40 to 50 meters, and may be variable depending on the traveling speed of the autonomous vehicle. In other words, when the traveling speed is fast, gait recognition is started from a greater distance.

[0153] At this time, if a driver and / or passenger is present in autonomous vehicle 1, control unit 101 notifies the driver and / or passenger of the detection of person 2, as described above. Then, when an unconditional stop is set or when the driver and / or passenger instructs autonomous vehicle 1 to stop in response to an inquiry, autonomous vehicle 1 approaches person 2, as indicated by the solid arrow in Figure 13(A), and stops as shown in Figure 13(B).

[0154] Then, the control unit 101 of the autonomous vehicle 1 calls out to the person 2 to alert them to the presence of the stopped autonomous vehicle 1. Furthermore, the control unit 101 of the autonomous vehicle 1 inquires of the person 2 whether or not they would like to get in the vehicle, and when the control unit 101 receives a response from the person 2 in response to the inquiry that they will get in the vehicle, the control unit 101 allows the person 2 to get in the autonomous vehicle 1.

[0155] As described above, according to the self-driving vehicle 1 of this embodiment, by using gait recognition technology, it is possible to detect, from images captured by a camera equipped in the vehicle, when a target person who has been registered and stored in advance is walking ahead in the direction in which the vehicle is traveling, and, depending on the detection results, it is possible to stop the vehicle near the detected target person and encourage the person to get on, which is extremely convenient.

[0156] [Other embodiments] In the autonomous vehicle 1 of the above-described embodiment, the autonomous vehicle is equipped with the function of detecting and identifying a target person through gait recognition. However, the autonomous vehicle may also be configured to detect and identify a target person through gait recognition by providing the function of detecting and identifying a target person through gait recognition in a server device connected via a communication network such as the Internet, and to detect and identify a target person through gait recognition by working in cooperation with the server device.

[0157] Fig. 14 is a diagram for explaining the main parts of the configuration of a system configured in this way, which is made up of autonomously driven vehicle 1A and server device 3. As shown in Fig. 14, in this example, autonomously driven vehicle 1A and server device 3 are configured to be able to wirelessly connect to each other via the Internet 4, which is an example of a communications network.

[0158] 14, in the electronic control circuit unit 10A of the autonomously driven vehicle 1A of this example, a wireless communication unit 121 for wirelessly connecting to the server device 3 via the Internet 4, a wireless transmission information generation unit 122, and a wireless reception information processing unit 123 are connected to the control unit 101A via a system bus 100. Although not shown in Fig. 14, the electronic control circuit unit 10A of the autonomously driven vehicle 1A of this example does not include the target person information registration unit 114, gait recognition determination unit 115, confirmation authentication unit 117, and target person information storage unit 136 in the electronic control circuit unit 10 of the autonomously driven vehicle 1 of the embodiment described above.

[0159] Instead, in server device 3, a target person information registration unit 32, a gait recognition discrimination unit 33, and a confirmation authentication unit 34 are connected to control unit 31 via system bus 30, and a target person information storage unit 35 is provided so as to be connected to target person information registration unit 32, gait recognition discrimination unit 33, and confirmation authentication unit 34. Furthermore, a wireless communication unit 36 ​​for connecting to autonomous vehicle 1A via the Internet 4 is connected to system bus 30.

[0160] The target person information storage unit 35 of the server device 3 stores information similar to the information stored in the target person information storage unit 136 shown in Fig. 3. However, in this example, identification information is given to each of the autonomously driven vehicles 1A, and in the target person information storage unit 35 of the server device 3, the target person information for each autonomously driven vehicle 1A is stored in a storage area distinguished by the identification information based on the identification information of the autonomously driven vehicle 1A.

[0161] That is, in this example, when registering target person information, the captured image information of the target person captured by the camera of the camera group 107 of the autonomous vehicle 1A is converted into wireless transmission information including the vehicle's identification information by the wireless communication unit 122 and sent to the server device 3 via the Internet 4 through the wireless communication unit 121.

[0162] In server device 3, wireless communication unit 36 ​​receives the information transmitted from autonomous vehicle 1A, and target person information registration unit 32 extracts gait information of the target person. The extracted gait information is then registered as target person information in a storage area corresponding to the identification information of autonomous vehicle 1A, in a manner similar to that shown in Fig. 3. Furthermore, verification and authentication information for the target person is also wirelessly transmitted from autonomous vehicle 1A to server device 3, and stored by target person information registration unit 32 in a storage area in target person information storage unit 35 corresponding to the identification information of autonomous vehicle 1A, in a manner similar to that shown in Fig. 3.

[0163] In this example, when autonomous vehicle 1A is specified to perform target person detection processing using gait recognition while driving, image information captured by the external camera in camera group 107 is converted into wireless transmission information including the vehicle's identification information by wireless transmission information generation unit 122 and sent to server device 3 via wireless communication unit 121 and Internet 4.

[0164] In the server device 3, the wireless communication unit 36 ​​receives the transmitted information from the autonomous vehicle 1A, and the gait recognition and discrimination unit 33 recognizes the gait and discriminates the target person.The discrimination results are then transmitted to the autonomous vehicle 1A via the wireless communication unit 36 ​​and the Internet 4.

[0165] Self-driving vehicle 1A receives information from server device 3 via wireless communication unit 121, and analyzes the discrimination result of gait recognition discrimination unit 33 via wireless reception information processing unit 123 to determine whether a target person has been detected. Then, gait recognition discrimination result response processing unit 116 (control unit 101A) of self-driving vehicle 1A executes processing similar to that of the above-described embodiment according to the discrimination result of wireless reception information processing unit 123.

[0166] The same applies to the confirmation and authentication process, where confirmation and authentication unit 34 of server device 3 performs confirmation and authentication using the confirmation and authentication information wirelessly transmitted from autonomous vehicle 1A. Server device 3g then transmits the authentication result to autonomous vehicle 1A. In autonomous vehicle 1A, wireless received information processing unit 123 analyzes the confirmation and authentication result, and based on the analysis result, control unit 101A performs processing according to the above-mentioned confirmation and authentication result. The other configurations in this example are similar to those in the above-mentioned embodiment.

[0167] In this example, the autonomous vehicle 1A is configured such that processing related to gait recognition is performed by the server device 3. This reduces the load on the control unit 101A, allowing it to concentrate on driving control such as autonomous driving.

[0168] [Other embodiments or modifications] In the above-described embodiment, when a detected target person is allowed to board the vehicle, the target person is subjected to confirmation and authentication. However, since the recognition accuracy of gait recognition is high, and in this embodiment, gait recognition auxiliary information is also used, and it has been reported that the recognition rate is 99%, this confirmation and authentication may be omitted.

[0169] In the above embodiment, when the detected target person is called for attention, a call voice including the call name (nickname) of the target person is sent. However, instead of the call voice, attention may be called by sounding a horn or emitting an attention-calling sound. Also, instead of sending a call voice, the telephone number or email address of the target person's mobile phone terminal may be stored, and the telephone number may be called to notify the target person by mobile phone of a voice message saying "We are stopping nearby to give you a ride", or an email may be sent to the email address saying "We are stopping nearby to give you a ride", and GPS information of the stopping position may also be sent. Of course, this is not limited to email, and a communication app such as LINE may also be used. In these cases, the automobile may be equipped with a mobile phone function so that the driver and / or passenger can make a call using the mobile phone function, or the automobile may be configured to have, for example, an AI (Artificial Intelligence) function or a function to make a call to the target person detected by an automatic voice. In such a configuration, the target person for gait recognition may be set by a contact from a person outside the automobile, such as a family member or a group member. In this case, the current location is also known, so if the user goes into the surrounding area, thorough gait recognition can be performed.

[0170] In the above embodiment, it is assumed that the target person is accidentally detected by recognizing the gait while driving the car. However, a mode (search mode or gaze mode) for carefully and thoroughly searching for the target person may be provided separately. For example, if the search mode or gaze mode is set when driving around a station during the time when the family members are returning home or leaving work, the probability of detecting the family members as the target person can be increased. In this case, the search mode or gaze mode drives the vehicle around the same place, such as around the station. In addition, the target person's schedule may be registered in advance, and the self-driving car may be made to go around the scheduled place at the scheduled time, making it easier to find the target person. For example, the self-driving car may be made to go around the nearest station during the target person's commuting time or school time.

[0171] In the above embodiment, the case of an electric vehicle has been described, but the vehicle of the present invention is not limited to an electric vehicle, and may be, for example, a gasoline engine vehicle, a diesel engine vehicle, etc. Furthermore, the present invention is not limited to automobiles, and may also be applied to two-wheeled vehicles such as motorcycles, electric bicycles, and electric wheelchairs.

[0172] In addition, although the above embodiment has been described with reference to an autonomous vehicle, the present invention can also be applied to a normal vehicle that does not have an autonomous driving mode and has only a manual driving mode, and it goes without saying that the present invention can also be applied to an autonomous vehicle that has only an autonomous driving mode.

[0173] The automobile according to the present invention can also be applied to an air-land amphibious vehicle (flying automobile) that can move (navigate) not only on land but also in the air. Fig. 15 is a diagram for explaining an example of the appearance of an air-land amphibious vehicle 1AL as an embodiment of the automobile according to the present invention, and an example of a manner in which the air-land amphibious vehicle 1AL of this embodiment detects a target person 2 by gait recognition, in particular, an example of a manner in which the air-land amphibious vehicle 1AL of the embodiment detects a target person 2 by gait recognition while moving (navigating) in the air.

[0174] The air-land amphibious vehicle 1AL of this embodiment, like the autonomous vehicle 1 of the embodiment described above, is equipped with one or more cameras that capture images of the visible area outside the vehicle, as well as a function for registering target persons to be subjected to gait recognition, a gait recognition discrimination function for performing gait recognition based on images captured by the cameras and detecting target persons, and a gait recognition discrimination result processing function.

[0175] The land-and-air amphibious vehicle 1AL of this embodiment has a land mode for running on land and an aerial mode for moving (navigating) in the air, and is configured so that registration for gait recognition, gait recognition discrimination, and processing for responding to the recognition discrimination results are different depending on the mode.

[0176] That is, the target person's gait information for the land mode is registered by photographing the target person with a camera that photographs the area ahead in the traveling direction of the vehicle, extracting the target person's gait information from the photographed information, and storing it in the storage unit, as in the case of the autonomous vehicle 1 of the above-mentioned embodiment. On the other hand, the target person's gait information for the aerial mode is registered by photographing the target person's walking with a camera that photographs an area diagonally below while the air-land hybrid vehicle 1AL is in the air, and extracting the target person's gait information from the photographed information and storing it.

[0177] In the land mode, the air-land hybrid vehicle 1AL of this embodiment recognizes and discriminates the gait of the target person from an image captured by a camera in the forward direction of the vehicle's travel and gait information for the land mode, as in the above-mentioned embodiment, and in the air mode, recognizes the gait of the target person from an image captured by a camera that captures an image diagonally below the air-land hybrid vehicle 1AL in the air and gait information for the air navigation mode.

[0178] In the land mode, when the air-land hybrid vehicle 1AL of this embodiment detects a target person through gait recognition, it performs the same gait recognition determination result response processing as the autonomous vehicle 1 of the above-mentioned embodiment. In contrast, in the air mode, it may land and switch to the land mode to perform the gait recognition determination result response processing, but if the detected target person is inquired about and refused boarding, it is inefficient to switch to the land mode and perform the gait recognition determination result response processing in the land mode in vain.

[0179] Therefore, in this example, the air-land amphibious vehicle 1AL in the air uses the target person's mobile phone number, which is previously included and stored in the registration information, to create a wireless communication path with the target person's mobile phone terminal and inquire as to whether or not the target person will board.Based on the result of the inquiry, it decides whether to land and switch to land mode to allow the target person to board, or to continue aerial movement (navigation) in aerial mode.

[0180] That is, when the target person replies that they will board the vehicle as a result of the inquiry by mobile phone, the air-land amphibious vehicle 1AL lands its own vehicle, switches to land mode, stops near the detected target person, performs confirmation authentication, and allows the target person to board. Also, when the target person refuses to board or there is no need to allow the target person to board as a result of the inquiry by mobile phone, the air-land amphibious vehicle 1AL keeps its own vehicle in air mode and continues airborne movement (navigation).

[0181] In addition, the wireless communication path for inquiring about the target person is not limited to a mobile phone terminal, but may be other wireless communication terminals, in which case connection information for generating a wireless communication path between the wireless communication terminal and the air-land hybrid vehicle 1AL is included in the registration information of the target person and stored.

[0182] When the mobile phone terminal or wireless communication terminal carried by the target person is capable of communicating with the air-land hybrid vehicle 1AL via a wireless communication path, not only can the air-land hybrid vehicle 1AL send an inquiry to the target person, but also the target person can wirelessly transmit a boarding request to the air-land hybrid vehicle 1AL using the mobile phone terminal or wireless communication terminal to request gait recognition. In this case, the air-land hybrid vehicle 1AL stores gait information in association with the target person's identification information, as in the above-mentioned embodiment.

[0183] In this case, the ride request wirelessly transmitted from the target person includes the target person's identification information and the target person's current location information, as well as information input by the target person about where to go from the current location (information on intermediate locations and destination). If the mobile phone terminal or wireless communication terminal carried by the target person is equipped with a GPS (Global Positioning System) function, the GPS information can be used for the current location information.

[0184] The air-land amphibious vehicle 1AL, which has received the boarding request, identifies the target person from the identification information included in the boarding request, estimates the position where the vehicle will encounter the target person while taking into consideration the target person's movement route from the current position information and information on where to go from the current position (information on intermediate points and destination), and when the vehicle arrives near the estimated position, performs gait recognition in the vicinity of that position. Note that the target person's movement route can be detected and understood from the current position information and information on where to go from the current position (information on intermediate points and destination) by the route search function of the car navigation function unit provided in the air-land amphibious vehicle 1AL.

[0185] It should be noted that the mode in which the target person wirelessly transmits a request to board is not limited to the air-land vehicle 1AL, but can be configured similarly for automobiles such as the self-driving car of the above-mentioned embodiment, motorcycles, and other automobiles. [Explanation of symbols]

[0186] 1...Autonomous driving vehicle, 3...Server device, 4...Internet, 10...Electronic control circuit unit, 101...Control unit, 114...Target person information registration unit, 115...Gait recognition and discrimination unit, 116...Gait recognition and discrimination result response processing unit, 117...Confirmation and authentication unit, 118...Door opening and closing drive unit, 119...Voice input / output unit, 120...Voice message generation unit, 121...Wireless communication unit

Claims

1. One or more cameras for capturing images of the outside of the vehicle; A storage unit that stores gait information for detecting a target person who is a target of gait recognition; a gait recognition and discrimination means for recognizing the gait of a person present outside the vehicle from image information captured by the camera, and discriminating whether or not a person whose gait matches the gait of the target person stored in the storage unit has been detected among the people whose gait has been recognized; a processing means for performing a predetermined process when it is determined that the gait recognition and discrimination means has detected a person having a gait matching the gait of the target person during a predetermined time period and / or at a predetermined location; A vehicle equipped with:

2. The predetermined time period is the target person's commuting time or school commuting time, and the predetermined location is a neighborhood of a nearest station used by the target person.

2. The vehicle according to claim 1 .

3. The predetermined time period is the time when the target person returns home or leaves work, and the predetermined location is the area around the nearest station used by the target person.

3. A vehicle according to claim 1 or 2.

4. Circling the specified location The automobile according to any one of claims 1 to 3.

5. A schedule of the target person's activities is registered; Based on the action schedule, the vehicle travels around the predetermined location during the predetermined time period. The automobile according to any one of claims 1 to 4.

6. A registration request receiving means for receiving a registration request for the target person; a registration means for, when the registration request receiving means receives a registration request for the target person, capturing an image of the target person walking with the camera, detecting the gait information from the captured image of the target person, and storing the gait information in the storage unit; The automobile according to any one of claims 1 to 5, further comprising:

7. A registration request receiving means for receiving a registration request for the target person; A wireless communication unit; a requesting means for, when the registration request receiving means receives a registration request for the target person, capturing an image of the target person walking with the camera, transmitting captured image information of the target person via the wireless communication unit to a gait detection device that detects gait information of the target person from the captured image information of the person, and requesting detection of the gait information; a registration means for receiving gait information transmitted from the gait detection device through the wireless communication unit based on a request for detection of the gait information, and storing the gait information in the storage unit; The automobile according to any one of claims 1 to 5, further comprising:

8. The predetermined process is a notification process for notifying a driver and / or a passenger that the target person has been detected. The automobile according to any one of claims 1 to 7,

9. The storage unit stores the gait information in association with information for identifying the target person, The notification by the notification means includes information for identifying the target person.

9. The vehicle according to claim 8.

10. It is an autonomous vehicle equipped with an autonomous driving mode that enables autonomous driving, The predetermined process is a driving control process for stopping the vehicle in the vicinity of the detected target person in the automatic driving mode when the gait recognition and discrimination means determines that the target person has been detected. The automobile according to any one of claims 1 to 9.

11. a passenger number determination means for determining whether the number of passengers in the vehicle is less than the passenger capacity of the vehicle; When the number of passengers determining means determines that the number of passengers is less than the passenger capacity, a driving control process is performed in the automatic driving mode to stop the vehicle near the detected target person.

11. The vehicle according to claim 10.

12. The predetermined process is a process of notifying the driver and / or passenger that the target person has been detected and asking whether to stop the vehicle near the detected target person.

12. A motor vehicle according to claim 10 or 11.

13. When the driver and / or the passenger responds to the inquiry that the vehicle will be stopped, the vehicle control process is performed in the automatic driving mode to stop the vehicle near the detected target person.

13. The vehicle according to claim 12.

14. A preset determination means is provided for determining whether or not the vehicle is preset to stop when the target person is detected, and when the preset determination means determines that the vehicle is preset to stop when the target person is detected, the vehicle performs a driving control process in the automatic driving mode to stop the vehicle near the detected target person without making the inquiry.

13. The vehicle according to claim 12.

15. When performing the predetermined process, during driving in a manual driving mode, the driving control process is performed to switch to the automatic driving mode and stop the vehicle in the vicinity of the detected target person. The automobile according to any one of claims 10 to 14.

16. The automatic driving mode to which the manual driving mode can be switched is a forced automatic driving mode in which the automatic driving mode is continued even if an operation to return to the manual driving mode is performed.

16. The vehicle according to claim 15.

17. Equipped with a speaker to project sound outside the vehicle, The predetermined process includes a process of emitting a voice message to the target person through the speaker when the vehicle stops near the target person. The vehicle according to any one of claims 10 to 16.

18. The storage unit stores information for enabling the target person to recognize that the call is directed to the target person, The call voice includes a voice based on the information for making the voice recognizable.

18. A vehicle as claimed in claim 17.

19. The voice message includes a voice message inviting the target person to get in the vehicle.

19. A motor vehicle according to claim 17 or 18.

20. A confirmation and authentication means for confirming and authenticating the target person when the vehicle stops near the target person through the predetermined process is provided. The vehicle according to any one of claims 17 to 19.

21. a face image storage unit for storing face image information of the target person, The verification and authentication means verifies the target person based on the facial image information of the detected target person obtained from the image information photographed by the camera and the facial image information stored in the facial image storage unit, depending on whether the target person is a person whose facial image information is stored in the facial image storage unit.

21. The vehicle according to claim 20.

22. A voice storage unit is provided for storing voice information of the target person, and a microphone is provided for collecting voices outside the vehicle, The identification and authentication means identifies the target person based on the voice information of the detected target person picked up by the microphone and the voice information stored in the voice storage unit, depending on whether the target person is the person whose voice information is stored in the voice storage unit.

22. A motor vehicle as claimed in claim 20 or 21.

23. When the target person confirmed by the confirmation authentication means gets in the vehicle, the target person who is in the vehicle is excluded from the target persons to be detected by the gait recognition. The vehicle according to any one of claims 20 to 22.

24. The target person who has got off the vehicle is determined to be a target person to be detected by the gait recognition.

24. The vehicle according to claim 23.

25. An air-land hybrid vehicle having a land mode for traveling on land and an air mode for traveling in the air, The storage unit stores gait information for detecting a target person who is a target of gait recognition, the gait information being for the land mode and the aerial mode, A camera for obtaining image information for performing the gait recognition is switched between the land mode and the aerial mode. The vehicle according to any one of claims 1 to 24.

26. A wireless communication means for wirelessly communicating with a mobile communication terminal, When the gait recognition and discrimination means determines in the aerial mode that a person having a gait matching the gait of the target person is present, the processing means wirelessly communicates with the mobile communication terminal carried by the detected target person in the wireless communication means to inquire whether the target person wishes to board the vehicle.

26. The vehicle according to claim 25.

27. When the processing means receives a response to the inquiry indicating that the person wishes to board the vehicle, the processing means switches from the air mode to the land mode and stops the vehicle near the detected target person.

27. The vehicle of claim 26.

28. When the processing means receives a response to the inquiry that the passenger does not wish to board the vehicle, the processing means continues the aerial mode.

28. A motor vehicle as claimed in claim 26 or 27.

29. When a boarding request including information on the current location of the target person is received from the target person via the wireless communication means, the gait recognition and discrimination means discriminates whether or not a person whose gait matches that of the target person is present in the vicinity of a location detected based on the information on the current location of the target person. The vehicle according to any one of claims 26 to 28.

30. A wireless communication means for wirelessly communicating with a mobile communication terminal, When a boarding request including information on the current location of the target person is received from the target person via the wireless communication means, the gait recognition and discrimination means discriminates whether or not a person whose gait matches that of the target person is present in the vicinity of a location detected based on the information on the current location of the target person. The vehicle according to any one of claims 1 to 24.

31. The boarding request includes information on the current location as well as information on a direction from the current location, and the vicinity of the location is determined by grasping the moving route of the target person from the information on the current location and the information on the direction from the current location, and the gait recognition and discrimination means discriminates whether or not a person whose gait matches that of the target person is present in the vicinity of the location.

31. A motor vehicle as claimed in claim 29 or claim 30.

32. A means for presenting a list of the target persons stored in the storage unit; a selection setting receiving means for receiving a selection setting of the target person from the presented list; Equipped with The gait recognition and discrimination means determines whether or not there is a person whose gait matches the gait of the selected target person among the people whose gaits have been recognized. A vehicle according to any one of claims 1 to 31.

33. the selection setting receiving means is capable of receiving a selection setting of a predetermined process to be executed by the vehicle when the target person is detected, The processing means performs the predetermined processing accepted by the processing setting accepting means. A vehicle according to any one of claims 1 to 32.

34. In the selection setting receiving means, the selection setting of the process to be executed by the vehicle is any one of the following selection settings: a setting of whether or not to always stop the vehicle when a target person is detected (unconditional stop setting), a setting of stopping only when there is room for passengers in the vehicle, a setting of temporarily stopping to leave a message, etc., a setting of stopping after inquiring of the driver and / or passengers, or a setting of notifying the driver and / or passengers without stopping.

34. The vehicle of claim 33.

35. The selection setting receiving means is activated when the driver and / or the passenger gets into the vehicle.

35. A motor vehicle as claimed in claim 33 or claim 34.

36. The storage unit stores gait information of the target person divided into daytime and nighttime. A vehicle according to any one of claims 1 to 35.

37. A system including an automobile having a first wireless communication means, and a gait recognition and discrimination device having a second wireless communication means that wirelessly communicates with the first wireless communication means, The vehicle is One or more cameras for capturing images of the outside of the vehicle; a gait recognition discrimination requesting means for transmitting, via the first wireless communication means, identification information of the vehicle itself and image information captured by the camera to a gait recognition discrimination device having a function of recognizing the gait of a person present outside the vehicle itself, and for transmitting a gait recognition discrimination request; a first receiving means for receiving, from the gait recognition and discrimination device, a determination result as to whether or not the target person has been detected, via the first wireless communication means; a processing means for performing a predetermined process when the discrimination result received by the first receiving means indicates that the target person has been detected during a predetermined time period and / or at a predetermined location; Equipped with The gait recognition and discrimination device includes: a storage unit that stores gait information for detecting a target person who is a target of gait recognition in association with identification information of the vehicle; a second receiving means for receiving, via the second wireless communication means, the identification information of the vehicle from the vehicle and the image information captured by the camera; a gait recognition and discrimination means for recognizing the gait of a person present outside the vehicle from the image information captured by the camera and received by the second receiving means, and for discriminating whether or not there is a person whose gait matches the gait of the target person stored in the storage unit in association with the identification information of the vehicle, among the people whose gait has been recognized; a transmitting means for transmitting a determination result of the gait recognition determination means to the vehicle that has transmitted the gait recognition determination request via the second wireless communication means; A gait recognition system comprising:

38. The vehicle is a means for storing a list of the target persons stored in the storage unit of the gait recognition and discrimination device in association with identification information of the vehicle, and presenting the list; a selection setting receiving means for receiving a selection setting of the target person from the presented list; the gait recognition and discrimination request includes identification information of the selected and set target person in addition to identification information of the vehicle itself and image information captured by the camera, and is transmitted to the gait recognition and discrimination device via the first wireless communication means; The gait recognition and discrimination means of the gait recognition and discrimination device discriminates whether or not there is a person whose gait matches the gait of the selected and set target person specified by the identification information among the people whose gaits have been recognized.

38. The gait recognition system of claim 37.

39. A computer provided in an automobile having one or more cameras for capturing images of the outside of the automobile and a storage unit for storing gait information for detecting a target person to be subjected to gait recognition, a gait recognition and discrimination means for recognizing the gait of a person present outside the vehicle from image information captured by the camera, and discriminating whether or not a person whose gait matches the gait of the target person stored in the storage unit has been detected among the people whose gait has been recognized; a processing means for performing a predetermined process when it is determined that the gait recognition and discrimination means has detected a person having a gait matching the gait of the target person during a predetermined time period and / or at a predetermined location; A program for automobiles to function as.

Images