Device for detecting passenger accompanied by guide dog and vehicle control system

Abstract

To provide a device for detecting a passenger accompanied by a guide dog, which can detect a passenger moving to an entrance door of a vehicle together with a guide dog before the passenger gets on the vehicle.SOLUTION: A device 50 for detecting a passenger accompanied by a guide dog comprises: a feature amount calculation unit 11 which divides image data captured by an outside camera 3 into a plurality of blocks and calculates a feature quantity indicating a direction of movement and an amount of movement with respect to each block; a moving body block extraction unit 13 which extracts blocks in which amounts of movement toward an entrance door 2a of a bus 2 are equal to or larger than a specified amount, as moving body blocks on the basis of the feature amounts; a grouping unit 15 which determines moving bodies moving toward the entrance door 2a by grouping adjacent moving body blocks; a determination unit 16 which determines whether both of persons and dogs are included in the moving bodies or not; a paring determination unit 17 which determines whether both of a person and a dog belong to a same moving body group or not; and a paring continuation determination unit 18 which determines whether both of the person and the dog continuously belong to the same moving body group for a prescribed time.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a detection device for passengers accompanied by guide dogs and a vehicle control system. [Background technology]

[0002] For example, Patent Document 1 describes a technology in which a multi-beam sensor is used to detect shading information of passengers passing through the entrance and exit of a car, and based on the detection results from this multi-beam sensor, a passenger image representing the characteristics of the passenger passing through the entrance and exit is generated, and based on that passenger image, it is determined whether the passenger getting into the car is a guide dog user accompanied by a guide dog. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-3097 Summary of the Invention [Problem to be solved by the invention]

[0004] In the above-described conventional technology, it is determined whether a passenger is a guide dog user accompanied by a guide dog based on a passenger image showing the characteristics of the passenger passing through the entrance / exit of the elevator. Such determination by image processing can also be performed using machine learning, which is a type of artificial intelligence (AI). However, with the above-described conventional technology, it is not possible to determine whether a passenger getting into an elevator is a guide dog user until the passenger passes through the entrance / exit of the elevator. For this reason, when the above-described conventional technology is applied to passenger transport vehicles such as buses, it is not possible to detect a passenger (guide dog user) who is accompanied by a guide dog and moving toward the entrance / exit of the vehicle before boarding the vehicle.

[0005] An object of the present invention is to provide a detection device for passengers with guide dogs and a vehicle control system that can detect passengers who are walking toward the entrance or exit of a vehicle with a guide dog before they board the vehicle. [Means for solving the problem]

[0006] (1) One aspect of the present invention is a detection device for passengers accompanied by guide dogs that detects passengers accompanied by guide dogs when they board a vehicle having an entrance / exit, the device comprising: an object detection unit that detects objects present in an area outside the vehicle that includes the entrance / exit; a feature calculation unit that divides the detection data of the object detection unit into a plurality of blocks and calculates feature values ​​representing the direction and amount of movement for each block; an extraction unit that extracts, as moving object blocks, blocks that have movement toward the entrance / exit that is greater than a specified amount based on the feature values ​​calculated for each block by the feature calculation unit; a moving object determination unit that determines moving objects toward the entrance / exit by grouping adjacent moving object blocks among the moving object blocks extracted by the extraction unit; and a moving object determined by the moving object determination unit that includes a person and a dog. a pairing determination unit that, when it is determined by the determination unit that the moving object includes a person and a dog, determines whether both the person and the dog belong to the same moving object group in order to recognize the person and the dog as a pair; a pairing continuation determination unit that, when it is determined by the pairing determination unit that both the person and the dog belong to the same moving object group, determines whether both the person and the dog have belonged to the same moving object group for a specified continuous time; and a detection unit that, when it is determined by the pairing continuation determination unit that both the person and the dog have belonged to the same moving object group for a specified continuous time, recognizes the person and the dog as a pair, detects that the dog is a guide dog, and detects that the person is a passenger accompanied by a guide dog.

[0007] (2) In (1) above, the feature calculation unit may calculate a motion vector as a feature for each block, and the extraction unit may convert the magnitude of the motion vector into a scalar value and extract blocks whose scalar value is equal to or greater than a predetermined threshold as moving body blocks.

[0008] (3) In (1) or (2) above, the object detection unit is a camera that captures an area outside the vehicle including an entrance and exit, the feature calculation unit generates image frames by dividing the camera's image data into multiple blocks, and the pairing continuation determination unit, when it is determined that both the person and the dog belong to the same moving object group, determines whether the number of image frames in which both the person and the dog belong to the same moving object group is equal to or greater than a predetermined specified number based on the image frames generated in chronological order by the feature calculation unit, and the detection unit may recognize that the person and the dog are a pair when it is determined that the number of image frames in which both the person and the dog belong to the same moving object group is equal to or greater than the specified number.

[0009] (4) A vehicle control system according to another aspect of the present invention includes a passenger-accompanied-guide-dog detection device according to any one of (1) to (3) above, and a boarding assistance control unit that performs control to assist a passenger accompanied by a guide dog in boarding when the detection unit detects that the person is a passenger accompanied by a guide dog.

[0010] (5) The vehicle control system may further include a guide dog control unit that controls the dog so as not to stimulate it when the detection unit detects that the dog is a guide dog, and an emergency situation control unit that controls the dog so as not to stimulate it when the detection unit detects that the dog is not a guide dog. [Effects of the Invention]

[0011] According to the present invention, a passenger who is walking toward the entrance or exit of a vehicle with a guide dog can be detected before boarding the vehicle. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a block diagram showing the configuration of a vehicle control system equipped with a detection device for detecting a passenger accompanied by a guide dog according to an embodiment of the present invention; [Figure 2] FIG. 2 is a perspective view of a bus equipped with the detection device for passengers accompanied by guide dogs shown in FIG. 1. [Figure 3]2 is a flowchart showing the procedure of a detection and control process executed by the ECU shown in FIG. 1. [Figure 4] FIG. 10 is a diagram showing how a moving object heading toward a bus entrance / exit is determined in an image frame. [Figure 5] FIG. 10 is an enlarged view showing how a moving object block is extracted from an image frame. [Figure 6] FIG. 10 is a diagram showing a time series of image frames in which both a person and a dog belong to the same moving object group. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0014] Fig. 1 is a block diagram showing the configuration of a vehicle control system equipped with a detection device for detecting passengers accompanied by guide dogs according to one embodiment of the present invention. In Fig. 1, vehicle control system 1 of this embodiment includes an exterior camera 3, an interior camera 4, a communication device 5, a door driver 6, a ramp driver 7, an announcement device 8, a signage device 9, and an ECU (Electronic Control Unit) 10 mounted on a bus 2 (see Fig. 2), which is a vehicle for transporting passengers, as well as a monitoring control device 31 and a communication device 32 in a remote center 30.

[0015] As shown in Figure 2, a boarding / alighting door 2a (entrance / exit) for passengers to get on and off is located in the center of the left side of the bus 2. The bus 2 has a door 25 for opening and closing the boarding / alighting door 2a, and a door ramp (not shown) for passenger Q accompanied by a guide dog P to easily get on and off the bus. Passenger Q accompanied by a guide dog P is the owner of the guide dog P.

[0016] The exterior camera 3 is located on the left side of the roof of the bus 2. The exterior camera 3 is a camera that captures an image of an area outside the bus 2, including the boarding / alighting entrance 2a, and detects objects present in that area. For example, a bird's-eye view camera with RGB functionality is used as the exterior camera 3. The exterior camera 3 constitutes an object detection unit that detects objects present in an area outside the bus 2, including the boarding / alighting entrance 2a. Objects include people, dogs, etc.

[0017] The interior camera 4 is disposed on the ceiling inside the bus 2. The interior camera 4 is a camera that captures images of the entire interior of the bus 2 and detects the situation inside the bus 2.

[0018] The communication device 5 communicates information relating to the monitoring and control of the bus 2 with the remote center 30 via wireless communication.

[0019] As described above, the remote center 30 has the monitoring control device 31 and the communication device 32. The monitoring control device 31 monitors the status of the bus 2 and provides driving support for the bus 2. The communication device 32 communicates wirelessly with the communication device 5 of the bus 2.

[0020] The door driving unit 6 is a driving unit that opens and closes the door 25. The slope driving unit 7 is a driving unit that deploys and retracts the door slope (not shown). The announcement unit 8 notifies passengers of information regarding the status of the bus 2 by voice guidance. The signage unit 9 displays the status of the bus 2, advertisements, etc.

[0021] The ECU 10 is configured with a CPU, RAM, ROM, an input / output interface, etc. The ECU 10 loads a program recorded in the ROM, for example, into the RAM, and executes the program loaded into the RAM with the CPU.

[0022] The ECU 10 has a feature calculation unit 11, a feature conversion unit 12, a moving body block extraction unit 13, a noise removal unit 14, a grouping unit 15, a discrimination unit 16, a pairing determination unit 17, a pairing continuation determination unit 18, a detection unit 19, a riding assistance control unit 20, a guide dog control unit 21, and an emergency situation control unit 22.

[0023] The feature amount calculation unit 11 generates an image frame F by dividing the image data (detection data) of the outside camera 3 into a plurality of blocks B, and calculates a feature amount representing the direction and amount of movement for each block B (see FIG. 4). The feature amount calculation unit 11 calculates a motion vector V as a feature amount for each block B. The motion vector V has a magnitude and a direction.

[0024] The feature conversion unit 12 converts the magnitude of the motion vector V calculated for each block B by the feature calculation unit 11 into a scalar value (see FIG. 5). The feature conversion unit 12 converts the magnitude of the motion vector V for each block B into a scalar value.

[0025] The moving body block extraction unit 13 extracts, as a moving body block Bm, a block B that has a movement of a specified amount or more toward the boarding / alighting entrance 2a of the bus 2, based on the scalar value converted for each block B by the feature conversion unit 12. The moving body block extraction unit 13 extracts, as a moving body block Bm, a block B whose scalar value is equal to or greater than a predetermined threshold (see FIG. 5).

[0026] The moving body block extraction unit 13 cooperates with the feature conversion unit 12 to form an extraction unit that extracts a block B that has movement greater than a specified amount toward the boarding and alighting entrance 2a of the bus 2 as a moving body block Bm based on the feature calculated for each block B by the feature calculation unit 11.

[0027] The noise removal unit 14 removes isolated moving body blocks Bm from the moving body blocks Bm extracted by the moving body block extraction unit 13 as noise.

[0028] The grouping unit 15 determines the moving object moving toward the boarding and alighting gate 2a of the bus 2 by grouping adjacent moving object blocks Bm that contain the same moving object from among the moving object blocks Bm extracted by the moving object block extraction unit 13 and removed by the noise removal unit 14 (see Figure 4(b)).

[0029] The grouping unit 15 cooperates with the noise removal unit 14 to form a moving body determination unit that determines moving bodies heading towards the boarding and alighting gate 2a of the bus 2 by grouping adjacent moving body blocks Bm among the moving body blocks Bm extracted by the extraction unit.

[0030] The determination unit 16 determines whether the moving objects determined by the grouping unit 15 include a person and a dog.

[0031] When the discrimination unit 16 determines that the moving objects include a person and a dog, the pairing determination unit 17 determines whether both the person and the dog belong to the same moving object group Gm for recognizing the person and the dog as a pair (see Figure 6(a)).

[0032] When the pairing determination unit 17 determines that both the person and the dog belong to the same moving body group Gm, the pairing continuation determination unit 18 determines whether both the person and the dog have continued to belong to the same moving body group Gm for a specified time.

[0033] When the pairing determination unit 17 determines that both the person and the dog belong to the same moving object group Gm, the pairing continuation determination unit 18 determines, based on the image frames F generated in time series by the feature calculation unit 11, whether the number of image frames F in which both the person and the dog belong to the same moving object group Gm is greater than or equal to a predetermined specified number (see Figures 6(a) to 6(c)).

[0034] When the pairing continuation determination unit 18 determines that both the person and the dog have belonged to the same moving object group Gm for a specified period of time, the detection unit 19 recognizes that the person and the dog are a pair, detects that the dog is a guide dog P, and detects that the person is a passenger Q accompanied by the guide dog P.

[0035] When the detection unit 19 detects that a person is a passenger Q accompanied by a guide dog P, the boarding assistance control unit 20 controls the door driving unit 6, the slope driving unit 7, and the announcement unit 8 to assist the passenger Q accompanied by the guide dog P in boarding.

[0036] When the detection unit 19 detects that the dog is a guide dog P, the guide dog control unit 21 controls the signage unit 9 so as not to stimulate the guide dog P.

[0037] When the detection unit 19 detects that the dog is not a guide dog P, the emergency situation control unit 22 controls the announcement unit 8 in an emergency situation.

[0038] Here, the exterior camera 3, interior camera 4, door driver 6, slope driver 7, announcement unit 8, signage unit 9, feature calculation unit 11 of ECU 10, feature conversion unit 12, moving object block extraction unit 13, noise removal unit 14, grouping unit 15, discrimination unit 16, pairing determination unit 17, pairing continuation determination unit 18, and detection unit 19 constitute a guide dog passenger detection device 50. The guide dog passenger detection device 50 is a device that detects a passenger Q accompanied by a guide dog P when the passenger boards a bus 2 having a boarding / alighting entrance 2a.

[0039] 3 is a flowchart showing the procedure of the detection and control process executed by the ECU 10. This process is executed, for example, when the bus 2 stops at a boarding / disembarking position such as a bus stop.

[0040] 3, the ECU 10 first acquires image data from the outside camera 3 (step S101). Then, as shown in FIG. 4(a), the ECU 10 generates image frames F by dividing the image data from the outside camera 3 into a plurality of blocks B (step S102).

[0041] 4A, the ECU 10 calculates a motion vector V for each block B in the image frame F (step S103). Specifically, the ECU 10 calculates the motion vector V for each block B based on the image frame F obtained at the current time t and the image frame F obtained at the previous time t-1, using, for example, deep learning or dense optical flow.

[0042] In this case, the blocks B are set mainly in the ground area R1 located outside the boarding / alighting door 2a of the bus 2 in the image data. The ground area R1 is captured between the bus ceiling area R2 and the sky area R3 in the image data. In each block B, the direction and amount of movement are expressed by the direction and magnitude of the motion vector V. Note that in the image frame F shown in Figure 4(a), the direction and magnitude of the motion vector V are expressed only for four blocks B (see the upper left corner of the figure), but the direction and magnitude of the motion vector V are expressed for all blocks B.

[0043] Next, the ECU 10 extracts a motion vector V directed toward the entrance 2a of the bus 2 for each block B (step S104). The motion vector V directed toward the entrance 2a of the bus 2 is a motion vector V with a direction of ±α degrees toward the entrance. α is a variable that takes error into account. In the image frame F shown in FIG. 4(a), the motion vectors V of the four blocks B located in the upper left corner are not extracted because they do not direct toward the entrance 2a of the bus 2. Next, the ECU 10 converts the magnitude of the motion vector V directed toward the entrance 2a of the bus 2 for each block B into a scalar value (step S105), as shown in FIG. 5.

[0044] Next, the ECU 10 determines moving object blocks Bm by comparing the scalar value with a predetermined threshold for each block B (step S106). In the image frame F shown in Fig. 5, the threshold is 7, and blocks B with a scalar value of 7 or more are determined to be moving object blocks Bm. Each moving object block Bm contains part of a moving object.

[0045] Next, the ECU 10 removes the isolated moving object block Bm as noise (step S107). In the image frame F shown in Fig. 4(b), the single moving object block Bm1 has a motion vector V pointing toward the entrance 2a of the bus 2, but is an isolated moving object block Bm that is not adjacent to other moving object blocks Bm, and is therefore removed as noise.

[0046] Next, the ECU 10 determines moving objects heading toward the entrance 2a of the bus 2 by grouping adjacent moving object blocks Bm that include the same moving object (step S108). In the image frame F shown in FIG. 4(b), two moving objects m1 and m2 heading toward the entrance 2a of the bus 2 are determined. The moving objects m1 and m2 are included in moving object detection frames W1 and W2, each of which is made up of a plurality of moving object blocks Bm.

[0047] Next, the ECU 10 uses an existing image recognition method to determine whether the moving objects heading toward the boarding / alighting entrance 2a of the bus 2 include both a person and a dog (step S109). For example, in the image frame F shown in FIG. 4(b), the moving object m1 included in the moving object detection frame W1 is determined to be a person, and the moving object m2 included in the moving object detection frame W2 is determined to be a dog.

[0048] When ECU10 determines that the moving objects heading toward the boarding / alighting entrance 2a of the bus 2 include both a person and a dog, it determines whether both the person and the dog belong to the same moving object group Gm, as shown in Figure 6 (step S110).

[0049] The moving object group Gm is a group for recognizing a person and a dog as a pair of a guide dog and an owner. Specifically, the moving object group Gm is a group that includes a moving object detection frame W1 in which a person is detected and a moving object detection frame W2 in which a dog is detected, and in which the moving object detection frame W1 and the moving object detection frame W2 are close to each other.

[0050] In image frame F shown in Fig. 6, moving object detection frame W1 and moving object detection frame W2 are close to each other, so the person and dog belong to the same moving object group Gm, and the person and dog are recognized as a pair of a guide dog and its owner. On the other hand, in image frame F shown in Fig. 4(b), moving object detection frame W1 and moving object detection frame W2 are far apart, so the person and dog do not belong to the same moving object group Gm, and the person and dog are not recognized as a pair of a guide dog and its owner.

[0051] When it is determined that both the person and the dog belong to the same moving object group Gm, the ECU 10 determines whether the number of image frames F in which both the person and the dog belong to the same moving object group Gm is equal to or greater than a threshold value, based on the image frames F generated in time series (step S111). The threshold value is a predetermined number. Specifically, as shown in FIGS. 6(a) to 6(c), the ECU 10 determines whether the number of image frames F in which both the person and the dog consecutively belong to the same moving object group Gm is equal to or greater than a threshold value (x+1) from time (tx) to time t.

[0052] When ECU10 determines that the number of image frames F in which both a person and a dog belong to the same moving object group Gm is greater than or equal to a threshold value, it determines that the dog is a guide dog P and that the person is the owner of the guide dog P (step S112).

[0053] Then, the ECU 10 controls the door driving unit 6, the slope driving unit 7, and the announcement unit 8 to assist the passenger Q accompanied by the guide dog P in boarding, and transmits to the remote center 30 via the communication device 5 a command to control the vehicle to assist the passenger Q accompanied by the guide dog P in boarding (step S113).

[0054] Specifically, ECU 10 controls door drive unit 6 to open and close doors 25 slowly, and controls slope drive unit 7 to deploy a door slope (not shown). ECU 10 also recognizes vacant seats inside bus 2 based on image data from in-vehicle camera 4, and controls announcement unit 8 to guide passenger Q accompanied by guide dog P to the vacant seat using audio guidance, and to notify the driver of bus 2 and passengers inside the bus that passenger Q accompanied by guide dog P will be boarding.

[0055] Furthermore, the ECU 10 controls the signage unit 9 so as not to stimulate the guide dog P, and transmits a control to the remote center 30 via the communication device 5 to the effect that control not to stimulate the guide dog P will be performed (step S114). Specifically, the ECU 10 controls the signage unit 9 to turn off the signage advertisement. In addition to such control, the ECU 10 may also turn off the background music in the vehicle or brighten the lights in the vehicle.

[0056] When ECU10 determines in step S110 that both the person and the dog do not belong to the same moving object group Gm, or when ECU10 determines in step S111 that the number of image frames F in which both the person and the dog belong to the same moving object group Gm is not greater than a threshold, it determines that the dog is not a guide dog P (step S115).

[0057] Then, the ECU 10 controls the announcing unit 8 to provide a service for an emergency situation, and transmits a notice of an emergency situation to the remote center 30 via the communication device 5 (step S116). For example, the ECU 10 controls the announcing unit 8 to notify the driver of the bus 2, passengers inside the bus 2, and people around the bus 2 that there is a possibility that a stray dog ​​may enter the bus 2.

[0058] If the ECU 10 determines in step S109 that the condition that the moving objects heading toward the entrance 2a of the bus 2 include both a person and a dog is not met, the ECU 10 does not execute the above-described steps S110 and thereafter.

[0059] Here, the feature calculation unit 11 executes steps S101 to S103. The feature conversion unit 12 executes steps S104 and S105. The moving object block extraction unit 13 executes step S106. The noise removal unit 14 executes step S107. The grouping unit 15 executes step S108. The discrimination unit 16 executes step S109. The pairing determination unit 17 executes step S110. The pairing continuation determination unit 18 executes step S111. The detection unit 19 executes steps S112 and S115. The riding assistance control unit 20 executes step S113. The guide dog control unit 21 executes step S114. The emergency situation control unit 22 executes step S116.

[0060] In the vehicle control system 1 described above, as shown in Fig. 6(a), if a person and a dog heading toward the bus 2 entrance 2a both belong to the same moving object group Gm, the person and the dog are determined to be a pair of a guide dog and its owner. However, if the pair of a guide dog and its owner is determined based on only a single image frame F, erroneous detection of the person and the dog or erroneous determination of the pair of a guide dog and its owner may occur, resulting in a lack of stability.

[0061] 6(a) to 6(c), if the number of image frames F in which a person and a dog heading toward the bus 2 entrance 2a both belong to the same moving object group Gm during the time series (tx) to t is equal to or greater than a threshold, the person and dog are determined to be a pair of a guide dog and its owner. This improves the stability of the accuracy of determining the guide dog and its owner.

[0062] As a result, a service that takes into consideration people with mobility disabilities is provided to passenger Q who is accompanied by guide dog P. In other words, control is performed to assist passenger Q who is accompanied by guide dog P in boarding, and control is also performed to avoid irritating guide dog P.

[0063] On the other hand, if the dog is not a guide dog but a stray dog, an emergency service response is implemented to prevent the stray dog ​​from entering the bus 2. For example, the driver and passengers of the bus 2 or people around the bus 2 who are notified by the announcement unit 8 may chase away the stray dog, or a security officer who is contacted by the remote center 30 may rush to the bus 2 and chase away the stray dog.

[0064] As described above, in this embodiment, the exterior camera 3 detects objects present in an area outside the bus 2 that includes the boarding / alighting entrance 2a. The image data from the exterior camera 3 is then divided into multiple blocks B, and feature values ​​representing the direction and amount of movement are calculated for each block B. Based on the feature values ​​calculated for each block B, blocks B that show movement toward the boarding / alighting entrance 2a of the bus 2 by a specified amount or more are extracted as moving object blocks Bm. Moving objects heading toward the boarding / alighting entrance 2a of the bus 2 are then determined by grouping adjacent moving object blocks Bm among the moving object blocks Bm. It is then determined whether the moving objects include a person and a dog. If it is determined that the moving objects include a person and a dog, it is determined whether both the person and the dog belong to the same moving object group Gm. The moving object group Gm is a group for identifying a person and a dog as a pair. If it is determined that both the person and the dog belong to the same moving object group Gm, it is determined whether both the person and the dog have continuously belonged to the same moving object group Gm for a specified period of time. When it is determined that both the person and the dog belong to the same moving object group Gm continuously for a specified time, the person and dog are recognized as a pair, the dog is detected as guide dog P, and the person is detected as passenger Q accompanied by guide dog P. In this way, when both the person and the dog belong to the same moving object group Gm continuously over time, the person is detected as passenger Q accompanied by guide dog P. In this way, passenger Q, who is moving toward the boarding and alighting gate 2a of bus 2 with guide dog P, is detected before boarding bus 2.

[0065] In this embodiment, a motion vector V is calculated for each block B, the magnitude of the motion vector V is converted to a scalar value, and blocks B whose scalar value is equal to or greater than a threshold value are extracted as moving body blocks Bm. Therefore, by simple calculation processing, moving body blocks Bm that are moving toward the bus 2 entrance 2a by a specified amount or more are extracted.

[0066] In addition, in this embodiment, by using an exterior camera 3 that captures images of the area outside the bus 2 including the boarding and alighting entrance 2a, people and dogs present in the area outside the bus 2 including the boarding and alighting entrance 2a can be accurately detected based on the image data from the exterior camera 3.

[0067] Furthermore, in this embodiment, image frames F are generated by dividing the image data from the exterior camera 3 into multiple blocks B, and when it is determined that both the person and the dog belong to the same moving object group Gm, it is determined based on the image frames F generated in chronological order whether the number of image frames F in which both the person and the dog belong to the same moving object group Gm is equal to or greater than a specified number, and when it is determined that the number of image frames F in which both the person and the dog belong to the same moving object group Gm is equal to or greater than the specified number, the person and the dog are recognized as a pair. Therefore, it is easy to determine whether both the person and the dog have belonged to the same moving object group Gm for a specified period of time.

[0068] Furthermore, in this embodiment, since isolated moving object blocks Bm are removed, moving objects heading towards the entrance 2a of the bus 2 are determined with high accuracy.

[0069] Furthermore, in this embodiment, as described above, passenger Q who is moving toward the boarding / alighting entrance 2a of the bus 2 with his / her guide dog P is detected before boarding the bus 2. Then, control is performed to assist passenger Q accompanied by guide dog P in boarding. Therefore, when passenger Q accompanied by guide dog P boards the bus 2, appropriate support service can be provided to passenger Q accompanied by guide dog P.

[0070] Furthermore, in this embodiment, when a dog is detected as a guide dog P, control is performed so as not to stimulate the guide dog P, thereby reducing the influence of the guide dog P on the interior of the bus 2. When a dog is detected as not a guide dog P, control for an emergency is performed, thereby preventing stray dogs and the like from entering the interior of the bus 2.

[0071] The present invention is not limited to the above embodiment. For example, in the above embodiment, an object present in the area including the boarding / alighting entrance 2a outside the bus 2 is detected by capturing an image of the area including the boarding / alighting entrance 2a outside the bus 2 using the exterior camera 3, but the present invention is not particularly limited to such an embodiment. For example, an object present in the area including the boarding / alighting entrance 2a outside the bus 2 may be detected using a lens sensor such as LiDAR or radar.

[0072] Furthermore, in the above embodiment, the image data of the outside vehicle camera 3 is divided into multiple blocks B, and a motion vector V is calculated for each block B, but the calculation is not limited to the motion vector V as long as a feature representing the direction and amount of motion is calculated for each block B.

[0073] In addition, in the above embodiment, passenger Q accompanied by a guide dog P is detected when the passenger boards bus 2, but the present invention is not limited to bus 2 in particular, and can also be applied to a device that detects passenger Q accompanied by a guide dog P when the passenger boards a vehicle such as a commercial vehicle that has an entrance and exit. [Explanation of symbols]

[0074] 1...vehicle control system, 2...bus (vehicle), 2a...boarding / exiting door (entrance / exit), 3...exterior camera (object detection unit), 11...feature calculation unit, 12...feature conversion unit (extraction unit), 13...moving object block extraction unit (extraction unit), 14...noise removal unit (moving object determination unit), 15...grouping unit (moving object determination unit), 16...discrimination unit, 17...pairing determination unit, 18...pairing continuation determination unit, 19...detection unit, 20...boarding assistance control unit, 21...guide dog control unit, 22...abnormal situation control unit, 50...guide dog accompanying passenger detection device, B...block, Bm...moving object block, F...image frame, Gm...moving object group, P...guide dog (dog), Q...passenger (person), V...motion vector (feature).

Claims

1. A passenger detection device for detecting a passenger accompanied by a guide dog when a passenger gets on a vehicle having an entrance and exit, an object detection unit that detects an object present in an area outside the vehicle that includes the entrance; a feature amount calculation unit that divides the detection data of the object detection unit into a plurality of blocks and calculates feature amounts that represent a direction and amount of movement for each of the blocks; an extraction unit that extracts, as a moving object block, a block in which there is a movement of a specified amount or more toward the entrance / exit based on the feature amount calculated for each block by the feature amount calculation unit; a moving object determination unit that determines a moving object heading toward the entrance / exit by grouping adjacent moving object blocks among the moving object blocks extracted by the extraction unit; a determination unit that determines whether the moving objects determined by the moving object determination unit include a person and a dog; a pairing determination unit that, when the determination unit determines that the moving object includes a person and a dog, determines whether both the person and the dog belong to the same moving object group to recognize the person and the dog as a pair; a pairing continuation determination unit that, when the pairing determination unit determines that both the person and the dog belong to the same moving object group, determines whether both the person and the dog have belonged to the same moving object group for a specified period of time; A guide dog-accompanied passenger detection device comprising a detection unit that, when the pairing continuation determination unit determines that both the person and the dog have continuously belonged to the same moving object group for the specified time, recognizes the person and dog as a pair, detects the dog as the guide dog, and detects the person as a passenger accompanied by the guide dog.

2. the feature amount calculation unit calculates a motion vector as the feature amount for each of the blocks; 2. The detection device for passengers accompanied by guide dogs according to claim 1, wherein the extraction unit converts the magnitude of the motion vector into a scalar value, and extracts, as the moving object block, a block whose scalar value is equal to or greater than a predetermined threshold value.

3. the object detection unit is a camera that captures an image of an area outside the vehicle that includes the entrance / exit, the feature amount calculation unit generates image frames by dividing the image data of the camera into the plurality of blocks; when it is determined that both the person and the dog belong to the same moving object group, the pairing continuation determination unit determines, based on the image frames generated in time series by the feature amount calculation unit, whether or not the number of image frames in which both the person and the dog belong to the same moving object group is equal to or greater than a predetermined specified number; A passenger detection device for guide dogs as described in claim 1, wherein the detection unit recognizes that the person and the dog are a pair when it determines that the number of image frames in which both the person and the dog belong to the same moving object group is equal to or greater than the specified number.

4. The detection device for passengers accompanied by guide dogs according to claim 1; A vehicle control system comprising: a boarding assistance control unit that performs control to assist the passenger accompanied by the guide dog in boarding when the detection unit detects that the person is a passenger accompanied by the guide dog.

5. a guide dog control unit that performs control not to stimulate the guide dog when the detection unit detects that the dog is a guide dog; 5. The vehicle control system according to claim 4, further comprising an emergency situation control unit that performs control for an emergency situation when the detection unit detects that the dog is not a guide dog.

Images