Door control device, vehicle control system, door control method, and door control program

Abstract

To provide more precise control over the opening and closing of vehicle doors. [Solution] The vehicle includes a control unit that controls the opening and closing of a door provided in an opening based on imaging data of a predetermined imaging range of an external area with respect to the opening of the vehicle. The control unit detects movement from the time-series imaging data in a predetermined area included in the imaging range and extracts the object of the movement as at least one moving object moving toward the opening. The control unit also performs a process of changing the opening and closing timing of the door based on the extracted at least one moving object. In a first area included in the predetermined area, the control unit extracts the object as the moving object if the direction of the movement is included in a predetermined first range. In a second area included in the predetermined area, the control unit extracts the area included in a predetermined second range where the direction of the movement is included as the moving object.

Description

Technical Field

[0001] The present disclosure relates to a vehicle door control device, a vehicle control system, a vehicle door control method, and a door control program.

Background Art

[0002] Patent Document 1 describes an elevator boarding detection system that estimates the presence or absence of a user's boarding intention based on the time-series change in the position of an elevator user and the speed of the user's movement toward the door, and controls the opening and closing operation of the door based on the estimation result.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] It is conceivable to estimate the presence or absence of the boarding intention of a person existing on the road facing the vehicle door in order to control the opening and closing operation of the vehicle door. However, even when a person on the road is moving in the direction approaching the vehicle, that person does not necessarily approach the vehicle and board it, and may just pass by while approaching the vehicle. Therefore, different from the determination of the presence or absence of the boarding intention of a person existing in front of an elevator, it is difficult to determine the presence or absence of the boarding intention of a person moving in the direction approaching the vehicle. It is required to appropriately control the opening and closing operation of the door.

[0005] An object of the present disclosure is to more appropriately control the opening and closing of a vehicle door.

Means for Solving the Problems

[0006] A door control device according to one embodiment of the present disclosure includes a control unit that controls the opening and closing of a door provided in an opening of a vehicle based on imaging data of a predetermined imaging range of an external area with respect to the opening of the vehicle, the control unit performs the following processes: detecting movement from the time-series imaging data in a predetermined area included in the imaging range and extracting the object of the movement as at least one moving object moving toward the opening; and changing the opening and closing timing of the door based on the extracted at least one moving object, the control unit extracts the object as the moving object in a first area included in the predetermined area if the direction of the movement is included in a predetermined first range, and extracts the object as the moving object in a second area included in the predetermined area if the direction of the movement is included in a predetermined second range.

[0007] A vehicle control system according to one embodiment of the present disclosure comprises the door control device, the vehicle, and an imaging unit that captures the imaging range.

[0008] A door control method according to one embodiment of the present disclosure is a door control method for controlling the opening and closing of a door provided in an opening of a vehicle based on imaging data of a predetermined imaging range of an external area with respect to the opening, the method comprising: a control unit detecting movement from the time-series imaging data in a predetermined area included in the imaging range, extracting the object of the movement as at least one moving body moving toward the opening, and changing the opening and closing timing of the door based on the extracted at least one moving body, wherein in a first area included in the predetermined area, the control unit extracts the object as the moving body when the direction of the movement is included in a predetermined first range, and in a second area included in the predetermined area, the control unit extracts the object as the moving body when the direction of the movement is included in a predetermined second range.

[0009] A door control program according to one embodiment of the present disclosure is a door control program that controls the opening and closing of a door provided in an opening of a vehicle based on imaging data of a predetermined imaging range of an external area with respect to the opening, and causes a processor to perform the following actions: detect movement from the time-series imaging data in a predetermined area included in the imaging range, extract a moving object as at least one object moving toward the opening, and change the opening and closing timing of the door based on the extracted at least one moving object, wherein in a first area included in the predetermined area, the object is extracted as the moving object when the direction of the movement is included in a predetermined first range, and in a second area included in the predetermined area, the object is extracted as the moving object when the direction of the movement is included in a predetermined second range. [Effects of the Invention]

[0010] According to one embodiment of the present disclosure, the opening and closing of vehicle doors can be controlled more effectively. [Brief explanation of the drawing]

[0011] [Figure 1] This is a schematic diagram illustrating an example of the configuration of a vehicle equipped with a controllable door as described in this disclosure. [Figure 2] This is a block diagram showing an example configuration of a vehicle control system related to this disclosure. [Figure 3] This figure shows an example of an image captured by a camera. [Figure 4] This is a schematic diagram explaining the determination of moving block. [Figure 5] This is a side view showing an example of the imaging range where a person is present. [Figure 6] This is a schematic diagram showing the captured image corresponding to Figure 5. [Figure 7] This flowchart shows an example procedure for the door control method related to this disclosure. [Figure 8] Figure 7 is a flowchart illustrating an example of the conversion process procedure. [Figure 9A]It is a diagram showing an example of a predetermined area divided into a plurality of partial areas. [Figure 9B] It is a diagram showing an example of the range of a motion vector. [Figure 9C] It is a diagram showing an example of the range of a motion vector. [Figure 10] It is a diagram showing an example of a predetermined area divided into a plurality of partial areas. [Figure 11A] It is a diagram showing an example of the range of a motion vector. [Figure 11B] It is a diagram showing an example of the range of a motion vector. [Figure 12] It is a diagram showing an example of the range of a motion vector. [Figure 13] It is a diagram showing an example of a predetermined area divided into a plurality of partial areas.

Mode for Carrying Out the Invention

[0012] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings. In each drawing, parts having the same configuration or function are denoted by the same reference numerals. In the description of this embodiment, redundant descriptions of the same parts may be omitted or simplified as appropriate.

[0013] FIG. 1 is a schematic diagram showing a configuration example of a vehicle 300 including a door 72 to be controlled in the present disclosure. As shown in FIG. 1, the vehicle 300 includes a door 72 at an opening. A passenger of the vehicle 300 gets on the vehicle 300 from the outside of the vehicle 300 through the door 72. The door 72 may be replaced with various other configurations that can be opened and closed.

[0014] A vehicle control system 1 (see FIG. 2) images an external area of the vehicle 300 around the door 72 by a camera 10 as an imaging unit. The vehicle control system 1 includes a bird's-eye camera with a wide angle of view as the camera 10, so that an imaging image of a wide imaging range A can be obtained even while the imaging direction of the camera 10 is fixed. The vehicle control system 1 detects a passenger existing within the imaging range 10A as a moving object based on the imaging image, and controls the opening and closing of the door ...

[0015] Passengers may rush towards vehicle 300. Around vehicle 300, there are not only passengers who intend to rush towards vehicle 300, but also persons other than passengers such as pedestrians. In order to improve the accuracy of detecting the rush towards vehicle 300, in other words, to reduce the false detection of the rush, it is necessary to appropriately determine the boarding intention of the person.

[0016] The vehicle control system 1 determines whether a moving object is a person with a boarding intention based on the positional relationship between the direction of movement of a predetermined area including the detected moving object and an opening such as the door 72 of vehicle 300 from the time-series data of the captured images. Specifically, the vehicle control system 1 determines that a passenger has an intention to board vehicle 300 when the direction of movement of the predetermined area including the moving object is within a certain range. The range of the direction of movement may be determined based on, for example, the positional relationship with the opening of vehicle 300.

[0017] The vehicle control system 1 determines whether a moving object is a passenger with a boarding intention based on different ranges of directions according to the positional relationship between the predetermined area and the door 72 of vehicle 300. Therefore, even if the captured image includes distortion due to, for example, the camera 10 being a bird's-eye view camera, the vehicle control system 1 can accurately determine the boarding intention of the moving object. Therefore, according to the vehicle control system 1, it is possible to accurately determine the boarding intention of the moving object and more appropriately control the opening and closing of the door 72 of vehicle 300.

[0018] Hereinafter, an example of an embodiment of the vehicle control system 1 according to the present disclosure will be described.

[0019] (Configuration example of vehicle control system 1) FIG. 2 is a block diagram showing a configuration example of the vehicle control system 1 according to the present disclosure. As shown in FIG. 2, the vehicle control system 1 according to the present disclosure includes a driving control device 100 and an object recognition device 16.

[0020] <Driving control device 100> The driving control device 100 includes an external environment recognition unit 110, a vehicle position recognition unit 120, an operation detection unit 130, a driving control unit 140, a contact possibility determination unit 150, a driver state recognition unit 160, a driving support control unit 170, a motion vector calculator 180, a motion vector corrector 182, and a door control unit 190.

[0021] The external environment recognition unit 110 recognizes the external environment of the vehicle 300 based on the recognition results of the object recognition device 16, which will be described later. The vehicle position recognition unit 120 recognizes the position of the vehicle 300 based on satellite positioning data, etc. The operation detection unit 130 detects the driving operations of the vehicle 300. The driving control unit 140 controls the driving of the vehicle 300. The driving control unit 140 may control the vehicle 300 to drive automatically. The contact possibility determination unit 150 determines the possibility that the vehicle 300 will come into contact with an object in its vicinity based on information from the camera 10, etc. The driver state recognition unit 160 recognizes the state of the driver of the vehicle 300 based on data detected by the in-vehicle camera 50 or the biometric information detection sensor 60, which will be described later.

[0022] The driver assistance control unit 170 assists in driving the vehicle 300 based on the recognition result of the state of the vehicle 300 or the state of the driver of the vehicle 300. The driver assistance control unit 170 includes a braking force control unit 172 and a stop control unit 174. The braking force control unit 172 automatically applies the brakes to the vehicle 300 as needed. The stop control unit 174 automatically stops the vehicle 300 from moving as needed.

[0023] The motion vector calculator 180 calculates motion vectors representing the direction of movement of moving objects detected around the vehicle 300. The motion vector corrector 182 corrects the motion vectors. The door control unit 190 controls the opening and closing of the doors 72 of the vehicle 300.

[0024] The operation control device 100 may be configured to include one or more processors or dedicated circuits in order to realize the functions of each component. In this embodiment, the processor is a general-purpose processor or a dedicated processor specialized for a specific process, but is not limited to these. The dedicated circuit may include, for example, an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit).

[0025] The operation control device 100 may include a storage unit. The storage unit may include, but is not limited to, semiconductor memory, magnetic memory, or optical memory. The storage unit may function as, for example, a main memory, auxiliary memory, or cache memory. The storage unit may include an electromagnetic storage medium such as a magnetic disk. The storage unit may include a non-temporary computer-readable medium. The storage unit stores any information or programs used for the operation of the operation control device 100. The storage unit may store, for example, a system program or an application program. The storage unit may be included in a processor or dedicated circuit.

[0026] The driving control device 100 may be configured to include an interface for communicating information or data with other components of the vehicle control system 1 or with external devices.

[0027] The interface may include a communication module configured to communicate with other components or external devices via a network. The communication module may support mobile communication standards such as 4G (4th Generation) or 5G (5th Generation). The communication module may also support communication standards such as LAN (Local Area Network). The communication module may support wired or wireless communication standards. The communication module is not limited to these and may support various communication standards. The interface may be configured to connect to the communication module.

[0028] The interface may include terminals that conform to standards such as RS-232C or RS-485, allowing for direct connection to other components or external devices.

[0029] The driving control device 100 may be configured to include an input device that receives information or data from a user of the vehicle control system 1. The input device may include, for example, a touch panel or touch sensor, or a pointing device such as a mouse. The input device may also include physical keys. The input device may also include an audio input device such as a microphone. The driving control device 100 may be configured to be connectable to an external input device. The driving control device 100 may be configured to be able to acquire information or data input to an external input device from that external input device.

[0030] The operation control device 100 may be configured to include an output device that outputs information or data to the user. The output device may include, for example, a display device that outputs visual information such as images, characters, or graphics. The display device may include, for example, an LCD (Liquid Crystal Display), an organic EL (Electro-Luminescence) display, an inorganic EL display, or a PDP (Plasma Display Panel). The display device is not limited to these displays and may include various other types of displays. The display device may include a light-emitting device such as an LED (Light Emitting Diode) or an LD (Laser Diode). The display device may include various other devices. The output device may include, for example, an audio output device such as a speaker that outputs auditory information such as voice. The output device is not limited to these examples and may include various other devices. The operation control device 100 may be configured to be connectable to an external output device. The operation control device 100 may be configured to output information or data to an external output device.

[0031] The driving control device 100 may be configured to include one or a plurality of server devices that can communicate with each other. The driving control device 100 may be implemented as a cloud server. The driving control device 100 may be mounted on the vehicle 300. The driving control device 100 does not have to be mounted on the vehicle 300. At least a part of the components of the driving control device 100 may be mounted on the vehicle 300. At least a part of the components of the driving control device 100 does not have to be mounted on the vehicle 300.

[0032] <Object recognition device 16> The object recognition device 16 acquires data from the camera 10, radar 12, and finder 14 (described later) that detect objects around the vehicle 300, and recognizes people present around the vehicle 300.

[0033] The object recognition device 16 may be configured to include one or more processors or dedicated circuits in order to realize the functions of each component. The processor or dedicated circuit may be configured in the same way as the processor or dedicated circuit of the driving control device 100. The object recognition device 16 may include a storage unit. The storage unit may be configured in the same way as the storage unit of the driving control device 100. The driving control device 100 may be configured to include an interface for communicating information or data with other components of the vehicle control system 1 or with external devices. The interface may be configured in the same way as the interface of the driving control device 100. The object recognition device 16 may be configured as part of the driving control device 100.

[0034] The object recognition device 16 may be configured to include one or a plurality of server devices that can communicate with each other. The object recognition device 16 may be implemented as a cloud server. The object recognition device 16 may be mounted on the vehicle 300. The object recognition device 16 may not be mounted on the vehicle 300. At least a portion of the components of the object recognition device 16 may be mounted on the vehicle 300. At least a portion of the components of the object recognition device 16 may not be mounted on the vehicle 300.

[0035] <Vehicle 300> The vehicle control system 1 may further include various components of the vehicle 300. The vehicle 300 includes a camera 10. The vehicle 300 may also include, although not required, a radar 12, a finder 14, a communication device 20, an HMI (Human Machine Interface) 30, a vehicle sensor 40, an in-cabin camera 50, a biometric information detection sensor 60, a driver control unit 80, a driving control device 230, and a boarding / alighting control device 70.

[0036] Camera 10 captures images of the area around the vehicle 300. The area captured by camera 10 is also called the imaging range 10A (see Figure 1). Camera 10 may be a bird's-eye view camera or other bird's-eye view camera with a wide field of view. Radar 12 and finder 14 may detect objects around the vehicle 300 in a manner other than images, such as point cloud data. The information about the area around the vehicle 300 detected by camera 10, radar 12 and finder 14 is output to the object recognition device 16.

[0037] The communication device 20 connects the remote center 2, which monitors the status of the vehicle 300, and the driving control device 100 in a communicative manner. The HMI 30 functions as an interface between the driver of the vehicle 300 and the driving control device 100.

[0038] The vehicle sensor 40 detects various states of the vehicle 300 and outputs the detection results to the driving control device 100. The vehicle sensor 40 may be controlled by the driving control device 100.

[0039] The in-vehicle camera 50 captures images of the interior of the vehicle 300's passenger compartment. The in-vehicle camera 50 may capture images of the driver or passengers of the vehicle 300. The in-vehicle camera 50 outputs the captured images to the driving control device 100. The in-vehicle camera 50 may be controlled by the driving control device 100.

[0040] The biometric information detection sensor 60 detects biometric information of the driver of the vehicle 300. The biometric information may include, for example, heart rate or blood pressure. The biometric information may also include the driver's state of alertness.

[0041] The driver control unit 80 includes an accelerator pedal 82, a brake pedal 84, and a steering wheel 86. The driver control unit 80 is operated by the driver of the vehicle 300. If the vehicle 300 is controlled by automated driving, it does not need to be equipped with the driver control unit 80.

[0042] The driving control device 230 includes a driving force output device 200, a brake device 210, and a steering device 220. The driving control device 230 may operate in response to the operation of the driver control device 80. The driving control device 230 may also operate in response to instructions from the driving control unit 140 or the driver assistance control unit 170 of the driver control device 100.

[0043] The passenger boarding / alighting control device 70 is equipped with doors 72 of the vehicle 300. The opening and closing of the doors 72 are controlled by the door control unit 190. The passenger boarding / alighting control device 70 is also equipped with a display unit 74 and a speaker 76, although these are not mandatory. The display unit 74 is installed outside the vehicle 300 and displays destination information or departure time of the vehicle 300 to passengers outside the vehicle 300. The speaker 76 is installed outside the vehicle 300 and outputs destination information or departure time of the vehicle 300 as audio information to passengers outside the vehicle 300.

[0044] (Example of operation of vehicle control system 1) The vehicle control system 1 according to this embodiment detects passengers who intend to rush into the vehicle 300 and controls the opening and closing of the door 72. The driving control device 100 or object recognition device 16 of the vehicle control system 1 is a device used to control the opening and closing of the door 72 and is also called a door control device. An example of the operation of the vehicle control system 1 will be described below.

[0045] The object recognition device 16 acquires an image 900 of the imaging range 10A from the camera 10. Figure 3 shows an example of an image 900 acquired by the camera 10.

[0046] As illustrated in Figure 3, the captured image 900 includes images of objects located within the imaging range 10A surrounding the vehicle 300. In the example in Figure 3, the captured image 900 includes not only the area 400 near the vehicle 300, but also the area occupied by the vehicle 300 (including the doors 72 and entrances / exits (openings)), as well as the distant area 500.

[0047] In this example, since camera 10 is, for example, a bird's-eye view camera with a wide field of view, the captured image 900 contains distortion. For example, in the example in Figure 3, the boundary between the area occupied by vehicle 300 and the nearby area 400, and the boundary between the nearby area 400 and the distant area 500, have an arc shape. In addition, the captured image 900 includes not only the side of vehicle 300, but also the areas near the front and rear of vehicle 300.

[0048] The captured image 900 shows two moving objects 3 and 4 in the vicinity of vehicle 300. Moving object 3 is a person who intends to board vehicle 300. Moving object 4 is a person who does not intend to board vehicle 300.

[0049] The object recognition device 16 acquires images captured by the camera 10 at each of multiple time points. Specifically, the camera 10 captures the imaging range 10A at both the first and second time points. The second time point is a predetermined time elapsed from the first time point. The predetermined time may be, for example, the frame rate when the camera 10 captures a video. The image captured at the first time point is also called the first captured image. The image captured at the second time point is also called the second captured image.

[0050] The object recognition device 16 recognizes a person, such as a moving object 3 or 4, in the captured image 900, tracks the person, and calculates the person's movement. The object recognition device 16 outputs the recognition result of the person, such as a moving object 3 or 4, in the captured image 900, and the calculation result of their movement to the operation control device 100.

[0051] The object recognition device 16 may determine that any person it detects is a pedestrian. A pedestrian is defined as a person who does not intend to board the vehicle 300. The object recognition device 16 may determine, based on the time-series trajectory of the detected person, whether the person is not heading towards the door 72 of the vehicle 300 based on the range of motion vector direction, and may identify any person not heading towards the door 72 as a pedestrian. Specific examples of the range of motion vector direction will be described later with reference to Figures 9A to 13.

[0052] The motion vector calculator 180 of the driving control device 100 calculates motion vectors as feature quantities of an object, such as a moving object 3 or 4, captured in the captured image 900. As preparation for calculating motion vectors, the motion vector calculator 180 divides a predetermined region 700 in the captured image 900 into multiple blocks 5. The predetermined region 700 is the region where a person running into the road is detected.

[0053] In this example, the predetermined region 700 is divided into a plurality of blocks 5 arranged in a grid. The manner in which the blocks 5 are divided is not limited to this. The motion vector calculator 180 calculates the motion vector in each of the divided blocks 5.

[0054] The motion vector of an object represents the direction and distance the object moved from its position in the first image to its position in the second image. The value obtained by dividing the distance the object moved by the difference (a predetermined time) between the acquisition times of the first and second images corresponds to the speed of the object. The motion vector calculator 180 may calculate the motion vector in each block 5 using a model that outputs a motion vector for block 5 units when the first and second images are input. The model may be, for example, a model based on Dense Opticalflow or Deep Learning.

[0055] Figure 4 is a schematic diagram illustrating the determination of a moving object block. For example, as shown in Figure 4, the motion vector calculator 180 calculates a motion vector 5V representing the direction and distance of movement of the parts of the moving object 3 in each block 5 in which the moving object 3 (a person) is depicted.

[0056] The motion vector corrector 182 of the driving control device 100 converts the motion vectors of objects in each block 5, such as moving bodies 3 and 4, calculated by the motion vector calculator 180, into scalar values ​​of the object's motion. The scalar value of an object's motion represents the magnitude of the object's motion, regardless of the direction of its motion. The scalar value of an object's motion is calculated as the length, i.e., the absolute value, of the object's motion vector. In the example in Figure 4, the motion vector corrector 182 calculates a scalar value 5S converted from vector 5V, which is the motion vector of the part of the moving body 3, in each block 5 that includes the area occupied by the moving body 3.

[0057] The motion vector corrector 182 may extract motion vectors in the captured image 900 in Figures 3 and 4 whose direction relative to the upward direction falls within a predetermined range, and convert the extracted motion vectors into scalar values ​​of the object's motion. Specific examples of the range of motion vector directions will be described later with reference to Figures 9A to 13. By limiting the block 5 that performs the conversion from motion vector to scalar value based on the predetermined range of motion vector directions, objects moving in directions other than toward the door 72 of the vehicle 300, i.e., objects unrelated to the opening and closing control of the door 72, are excluded from recognition. As a result, the vehicle control system 1 can determine with high accuracy whether a person near the vehicle 300 intends to board and appropriately control the opening and closing operation of the door 72.

[0058] The motion vector corrector 182 determines that blocks 5 whose converted scalar value is greater than or equal to a threshold value are dynamic blocks. In this example, the threshold value for the scalar value is 7, but the threshold value can be set arbitrarily. In this case, in the example in Figure 4, the motion vector corrector 182 determines that the seven blocks 5 whose scalar value 5S is 7 or greater are dynamic blocks 6. The seven blocks 5 determined to be dynamic blocks 6 are enclosed by thick lines in the example in Figure 4. On the other hand, blocks 5 whose scalar value 5S is 6 are not included in dynamic blocks 6.

[0059] The motion vector corrector 182 removes motion blocks that are considered noise from the determined motion blocks. For example, the motion vector corrector 182 may remove isolated motion blocks 6 as noise from among the motion blocks 6 in multiple blocks 5 obtained by dividing a predetermined area 700 of the captured image 900. For example, even if multiple motion blocks 6 are adjacent, the motion vector corrector 182 may remove multiple adjacent motion blocks 6 as noise if the area of ​​the region connecting the adjacent motion blocks 6 is smaller than the area expected when a person is captured.

[0060] The motion vector corrector 182 may exclude the motion block 6 from the motion block 6 by setting the magnitude of the motion vector of the motion block 6 to 0 to remove it as noise.

[0061] The motion vector corrector 182 determines the motion block 6 corresponding to the moving object from the motion block 6 remaining after noise reduction. Specifically, the motion vector corrector 182 groups adjacent motion block 6 together and determines the grouped motion block 6 as the motion block 6 corresponding to the moving object. In the example in Figure 4, seven motion block 6 are grouped together as the motion block 6 corresponding to the moving object 3.

[0062] Here, objects located near camera 10 and objects located far from camera 10 appear to be of different sizes in the captured image 900, even if their actual size is the same. Specifically, objects located far from camera 10 appear smaller in the captured image 900 than objects located near camera 10.

[0063] Figure 5 is a side view showing an example of the imaging range in which a person is present. Figure 6 is a schematic diagram showing the captured image 900 corresponding to Figure 5. As illustrated in Figure 3, in the captured image 900, horizontally extending grids 7 are set at equal intervals in the vertical direction. The spacing of these grids 7 widens as the distance from the camera 10 in the actual imaging range 10A increases, as shown in Figure 5. As shown in Figure 5, even if moving objects 3 and 4 are people of the same height, as shown in Figure 6, in the captured image 900, moving object 4, which is located farther from the camera 10, will appear smaller than moving object 3, which is located closer to the camera 10. The grids 7 in Figures 5 and 6 are corresponding by the labels P0 to P5.

[0064] From the above, the magnitude of motion of an object that is positioned higher in the captured image 900 appears smaller than the magnitude of motion of an object that is positioned lower in the captured image 900. Therefore, the motion vector corrector 182 corrects the scalar value according to the position in the captured image 900 in order to reduce the influence of the determination based on the distance from the camera 10 to the object.

[0065] The motion vector corrector 182 determines the grounding position of the moving body as preparation for correcting the scalar value. Specifically, for each group of moving body blocks 6, the motion vector corrector 182 identifies the moving body block 6 located at the bottom edge of the captured image 900. As illustrated in Figure 5, when the imaging range 10A by the camera 10 is viewed from the side, the grounding position 3A of the moving body 3 closest to the camera 10 and the grounding position 4A of the moving body 4 closest to the camera 10 are identified. Grounding position 3A is the moving body block 6 located at the bottom edge of the group of moving body blocks 6 corresponding to the moving body 3. Grounding position 4A is the moving body block 6 located at the bottom edge of the group of moving body blocks 6 corresponding to the moving body 4.

[0066] The motion vector corrector 182 corrects the scalar value according to the ground contact position of the moving object. The object recognition device 16 corrects the scalar value by increasing it as the closer the ground contact position of the moving object is to the top edge of the captured image 900, the further away the moving object is from the camera 10 and the smaller it appears in the image. The motion vector corrector 182 may correct the scalar value by multiplying it by a scalar value correction coefficient listed in the correction table shown in Table 1, for example.

[0067] [Table 1]

[0068] In the correction table in Table 1, B0 to B7 are codes corresponding to each row when a predetermined region 700 contained in the captured image 900 is divided vertically into eight blocks 5. B0 is the region located at the bottom edge of the predetermined region 700 and is the region closest to the camera 10. B7 is the region located at the top edge of the predetermined region 700 and is the region furthest from the camera 10. P0 to P3 correspond to the grid 7 illustrated in Figures 3, 5, and 6.

[0069] Specifically, the motion vector corrector 182 corrects the scalar values ​​using the correction table in Table 1 as described below. If the bottom edge of a group of motion blocks 6 is located on the grid represented by P0, the motion vector corrector 182 multiplies the scalar value of the motion block 6 in row B0, which is at the same height as P0, by 1, multiplies the scalar value of the motion block 6 in row B1 by 1.1, multiplies the scalar value of the motion block in any row from B2 to B6 by 1.3, and multiplies the scalar value of the motion block 6 in row B7 by 1.5. If the bottom edge of the group of motion blocks 6 is located on the grid represented by P1, the motion vector corrector 182 multiplies the scalar value of the motion block 6 in row B1, which is at the same height as P1, by 2, multiplies the scalar value of the motion block in any row from B2 to B6 by 2.2, and multiplies the scalar value of the motion block 6 in row B7 by 2.6. If the bottom edge of the group of motion blocks 6 is located on the grid represented by P2, the motion vector corrector 182 multiplies the scalar value of the motion block 6 in any row from B2 to B6, which is at the same height as P2, by 3, multiplies the scalar value of the motion block in any row from B2 to B6 by 3, and multiplies the scalar value of the motion block 6 in row B7 by 3.4. If the bottom edge of a group of motion blocks 6 is located on the grid represented by P3, the motion vector corrector 182 multiplies the scalar value of the motion block 6 in row B7 that is at the same height as P3 by 4.

[0070] The correction table in Table 1 above is just one example. The motion vector corrector 182 may create a correction table as appropriate based on the relationship between the size and position of objects captured in a predetermined area 700 of the camera 10 and the actual size and position of objects in the imaging range 10A. The motion vector corrector 182 may also adjust the correction coefficient in the column (left-right) direction according to the distortion of the captured image 900 caused by the camera 10 being a bird's-eye view camera.

[0071] The motion vector corrector 182 outputs the corrected scalar value, as described above, to the door control unit 190 of the driving control device 100. The door control unit 190 integrates (sums) the corrected scalar values ​​of each moving block 6 included in each group of moving block 6. The integrated corrected scalar value represents the amount of movement of the object corresponding to the group of moving block 6. For example, the door control unit 190 can calculate the amount of movement of the moving body 3 by correcting the scalar values ​​of the seven moving block 6 corresponding to the moving body 3 shown in Figure 4, and then integrating the corrected scalar values.

[0072] The door control unit 190 determines that an object, i.e., a person, intends to rush onto the vehicle 300 if the amount of movement of the object exceeds a threshold. If the door control unit 190 determines that one or more people captured in the image 900 intend to rush onto the vehicle 300, it detects the rush onto the vehicle 300.

[0073] The door control unit 190 keeps the door 72 open when it detects someone rushing onto the vehicle 300. It closes the door 72 when the length of time during which no rushing onto the vehicle 300 is detected exceeds a threshold.

[0074] The door control unit 190 may leave the door 72 open if it does not detect someone rushing onto the vehicle 300. The door control unit 190 may close the door 72 if it detects someone rushing onto the vehicle 300.

[0075] Figure 7 is a flowchart showing an example of the procedure for a door control method according to this disclosure. In this embodiment, the vehicle control system 1 may execute a door control method including the procedure of the flowchart illustrated in Figure 7 in order to determine whether a moving object present around the vehicle 300 intends to board the vehicle 300 and to control the opening and closing of the door 72. The door control method may be implemented as a door control program to be executed by a processor such as the driving control device 100 or object recognition device 16 provided in the vehicle control system 1. The door control program may be stored in a non-temporary computer-readable medium.

[0076] The object recognition device 16 receives an image input from the camera 10 capturing the imaging range 10A around the vehicle 300 (S1). The object recognition device 16 may also receive input of information about objects around the vehicle 300 from the radar 12 or the finder 14.

[0077] The object recognition device 16 calculates feature quantities of objects captured in a predetermined region 700 of the captured image 900 (S2). The object recognition device 16 divides the predetermined region 700 into multiple blocks and calculates the motion vector of the objects in each block as a feature quantity. Based on a comparison of the captured image 900 at the first time step and the captured image 900 at the second time step, which is a predetermined time after the first time step, the object recognition device 16 calculates the direction in which the object moved and the distance the object moved between the first and second time steps. The motion vector of the object represents the direction and distance the object moved. The value obtained by dividing the distance the object moved by the predetermined time step corresponds to the speed of the object.

[0078] The object recognition device 16 performs a conversion process to convert the motion vector of an object in each block into a scalar value of the object's motion (S3). The scalar value of an object's motion does not include a component of the direction of the object's motion, but only includes a component of the magnitude of the motion, i.e., the speed of the motion. The object recognition device 16 may extract motion vectors whose direction is within a predetermined range on a block-by-block basis, and convert the extracted motion vectors into scalar values ​​of the object's motion. Details of the conversion process will be described later with reference to Figures 8 to 13.

[0079] The object recognition device 16 determines the moving blocks (S4). The object recognition device 16 determines blocks whose scalar value is greater than or equal to a threshold as moving blocks. The threshold is a value that is set as appropriate.

[0080] The object recognition device 16 removes motion noise (S5). The object recognition device 16 may exclude motion blocks that are isolated from other motion blocks from the motion blocks.

[0081] The object recognition device 16 determines the moving object (S6). The object recognition device 16 groups adjacent moving object blocks together.

[0082] The object recognition device 16 determines the ground contact position of the moving object (S7). The object recognition device 16 identifies the block located at the bottom edge of the image in groups of moving object blocks. The object recognition device 16 determines the grid number of the identified block located at the bottom edge.

[0083] The object recognition device 16 corrects the scalar value based on the ground contact position (S8). The object recognition device 16 obtains the scalar value correction coefficient for each moving block included in the group of moving blocks from the correction table shown in Table 1 above, etc., based on the grid number. The object recognition device 16 corrects the scalar value for each moving block by multiplying the scalar value by the scalar value correction coefficient.

[0084] The object recognition device 16 calculates the amount of movement for each moving object (S9). The object recognition device 16 accumulates the corrected scalar values ​​of the moving object blocks in groups. The scalar values ​​accumulated in groups represent the amount of movement of the moving object corresponding to the group of moving object blocks.

[0085] The driving control device 100 determines whether it has detected a moving body rushing onto the vehicle 300 (S10). The driving control device 100 determines that a moving body corresponding to a group of moving body blocks whose amount of movement is greater than or equal to a threshold has the intention to board the vehicle 300, and detects the moving body rushing onto the vehicle 300. In other words, the driving control device 100 determines that it has detected a moving body rushing onto the vehicle 300 if the amount of movement in one or more groups of moving body blocks is greater than or equal to a threshold.

[0086] If the driving control device 100 does not detect a moving object rushing onto the vehicle 300 (S10: NO), it controls the door control unit 190 to close the door 72 (S11). If the driving control device 100 detects a moving object rushing onto the vehicle 300 (S10: YES), it controls the door control unit 190 not to close the door 72 (S12). After executing the procedure in S11 or S12, the vehicle control system 1 finishes executing the procedure in the flowchart of Figure 7.

[0087] Next, the details of the conversion process performed in step S3 of Figure 7 will be explained with reference to Figure 8. Figure 8 is a flowchart showing an example of the steps of the conversion process in Figure 7.

[0088] The object recognition device 16 extracts motion vectors whose orientation falls within a first range from each block included in the first region included in the predetermined region 700 (S21). The object recognition device 16 extracts motion vectors whose orientation falls within a second range from each block included in the second region included in the predetermined region 700 (S22).

[0089] Figure 9A shows an example of a predetermined region 700 divided into multiple subregions. Figures 9B and 9C show examples of the range of motion vectors.

[0090] In Figure 9A, region 520, as the first region, is closer to the opening than region 510, as the second region. Figure 9B shows the range of directions (second range) for the motion vector to be converted to a scalar value in region 510. In the example of Figure 9B, when the direction of the motion vector is specified clockwise with respect to the top of the paper, the range of directions for the motion vector to be converted to a scalar value is 120 to 240 degrees. Figure 9C shows the range of directions (first range) for the motion vector to be converted to a scalar value in region 520. In the example of Figure 9C, when the direction of the motion vector is specified clockwise with respect to the top of the paper, the range of directions for the motion vector to be converted to a scalar value is 80 to 280 degrees.

[0091] In the examples in Figures 9A to 9C, the first region (region 520) indicates a location closer to the opening of the vehicle 300 than the second region (region 510). The first range (Figure 9C) specifies a wider range of orientations than the second range (Figure 9B). Thus, the vehicle control system 1 may convert motion vectors in a wider range of orientations into scalar values ​​in regions closer to the opening. Therefore, even if the captured image 900 contains distortion due to the characteristics of the camera 10, as in the example of the captured image 900 in Figure 3, the vehicle control system 1 can determine the intention to board the moving object with high accuracy.

[0092] The method of dividing the predetermined region 700 as exemplified in Figure 9A, and the range of motion vector directions in each sub-region as exemplified in Figures 9B and 9C, are examples only and can be arbitrarily set according to the characteristics of the camera 10 and the behavior of the person.

[0093] For example, Figure 10 shows an example of a predetermined region 700 divided into three subregions. Figures 11A and 11B show examples of the range of motion vectors.

[0094] In Figure 10, region 531 indicates a location located in front of the vehicle 300 beyond region 532. Region 510 indicates a location further away from the opening than regions 531 and 532. Figure 11A shows the range of directions for motion vectors to be converted to scalar values ​​in region 531. In the example of Figure 11A, when the direction of the motion vector is specified clockwise with respect to the top of the paper, the range of directions for motion vectors to be converted to scalar values ​​is 80 to 240 degrees. Figure 11B shows the range of directions for motion vectors to be converted to scalar values ​​in region 532. In the example of Figure 11B, when the direction of the motion vector is specified clockwise with respect to the top of the paper, the range of directions for motion vectors to be converted to scalar values ​​is 120 to 280 degrees. Note that the range of directions for motion vectors to be converted to scalar values ​​in region 510 may be shown in Figure 9B, similar to Figure 9A.

[0095] In the examples in Figures 10, 11A, and 11B, region 531 indicates a location located in front of the vehicle 300 compared to region 532. The directional range in region 531 (Figure 11A) specifies a wider directional range towards the rear of the vehicle 300 than the directional range in region 532 (Figure 11B). The directional range in region 532 (Figure 11B) specifies a wider directional range towards the front of the vehicle 300 than the directional range in region 531 (Figure 11A). Note that the directional range in region 510 (Figure 9B), which is located further from the opening than regions 531 and 532, specifies a narrower range than regions 531 and 532. Therefore, even when a wide-angle imaging device is used as the camera 10 to image a wide area, the vehicle control system 1 can accurately determine whether the moving object is moving towards the opening and can determine the moving object's intention to board with high accuracy.

[0096] Furthermore, the vehicle control system 1 may specify a range of directions for extracting motion vectors more precisely, depending on the relative position of block 5 in a predetermined region 700. Figure 12 shows an example of a range of motion vectors. For example, for block 5 included in region 531, the extraction of motion vectors may be determined based on a range of directions obtained by linearly interpolating the ranges of directions A, B, E, F in Figure 12, based on the relative position of block 5 with respect to points 561, 562, 564, 565. For example, for block 5 included in region 532, the extraction of motion vectors may be determined based on a range of directions obtained by linearly interpolating the ranges of directions C, D, F, G in Figure 12, based on the relative position of block 5 with respect to points 562, 563, 565, 566. For example, for block 5 included in region 510, the extraction of motion vectors may be determined based on the range of directions shown in Figure 9B. With such a configuration, the vehicle control system 1 can accurately determine whether a moving body is moving toward an opening and can determine the intention of the moving body to board with high accuracy.

[0097] Furthermore, the vehicle control system 1 may further subdivide the predetermined region 700 based on the positional relationship between the camera 10 that captures the imaging range and the aperture, and specify the range of directions for extracting motion vectors. Figure 13 shows an example of a predetermined region 700 divided into multiple sub-regions. For example, as shown in Figure 13, the predetermined region 700 may be divided into seven regions 510, 541-546, and the range of directions for extracting motion vectors may be specified for each region. With such a configuration, the vehicle control system 1 can determine with high accuracy whether a moving object intends to board the vehicle, depending on the positional relationship between the camera 10 and the aperture.

[0098] Returning to the explanation of Figure 8, the object recognition device 16 obtains the scalar values ​​of each motion vector extracted in S21 and S22 (S23). Specifically, it extracts the magnitude of the motion (velocity component) indicated by the motion vector from each motion vector. After completing the process in S23, the object recognition device 16 finishes the conversion process and proceeds to S4b in Figure 7.

[0099] This disclosure is not limited to the embodiments described above. For example, multiple blocks shown in the block diagram may be combined, or a single block may be divided. Multiple steps shown in the flowchart may be performed in parallel or in a different order, depending on the processing capacity of the device performing each step, or as necessary, instead of being performed in chronological order as described. Other modifications are possible without departing from the spirit of this disclosure.

[0100] Some embodiments of the present disclosure are described below. However, it should be noted that the embodiments of the present disclosure are not limited to these. [Note 1] The system includes a control unit that controls the opening and closing of a door provided in an opening of a vehicle based on imaging data of a predetermined imaging range of the external area relative to the opening of the vehicle. The control unit, A process of detecting motion from the time-series imaging data in a predetermined region included in the imaging range, and extracting the object of the motion as at least one moving object moving toward the opening, A process to change the opening and closing timing of the door based on at least one of the extracted moving bodies, Execute, The control unit, In the first region included in the predetermined region, the object is extracted as the moving body when the direction of movement falls within a predetermined first range. In the second region included in the predetermined region, the object is extracted as the moving body when the direction of movement falls within a predetermined second range. Door control device. [Note 2] The first region is a region closer to the opening than the second region. The first range is wider than the second range. The door control device described in [Note 1]. [Note 3] The first region indicates a location located further forward in the longitudinal direction of the vehicle than the second region. The first range includes the rearward side in the front-rear direction, The door control device described in [Note 1]. [Note 4] The door control device according to [Appendix 3], wherein the second range includes the front side of the vehicle in the longitudinal direction, more than the first range. [Note 5] The door control device according to any one of [Appendix 1] to [Appendix 4], wherein the first range and the second range are set based on the positional relationship between the imaging unit that images the imaging range and the opening. [Note 6] The control unit, In the time-series imaging data, a vector including the direction of motion and the magnitude of motion is calculated in each of the multiple blocks obtained by dividing the predetermined region. Blocks whose vectors satisfy predetermined conditions are determined to be the moving body blocks corresponding to the moving body. A door control device as described in any of [Appendix 1] to [Appendix 5]. [Note 7] The control unit groups adjacent blocks among the moving body blocks and detects the group of moving blocks as the moving body, as described in [Appendix 6]. [Note 8] A vehicle control system comprising a door control device described in any of [Appendix 1] to [Appendix 7], the vehicle, and an imaging unit for imaging the imaging range. [Note 9] A door control method for controlling the opening and closing of a door provided in an opening of a vehicle, based on imaging data of a predetermined imaging range of an external area relative to the opening of the vehicle, The control unit, In a predetermined region included in the imaging range, motion is detected from the time-series imaging data, and the object of the motion is extracted as at least one moving object moving toward the opening. The opening and closing timing of the door is changed based on at least one of the extracted moving bodies, Includes, The control unit, In the first region included in the predetermined region, the object is extracted as the moving body when the direction of movement falls within a predetermined first range. In the second region included in the predetermined region, the object is extracted as the moving body when the direction of movement falls within a predetermined second range. Door control method. [Note 10] The first region is a region closer to the opening than the second region. The first range is wider than the second range. The door control method described in [Appendix 9]. [Note 11] The first region indicates a location located further forward in the longitudinal direction of the vehicle than the second region. The first range includes the rearward side in the front-rear direction, The door control method described in [Appendix 9]. [Note 12] The door control method according to [Appendix 11], wherein the second range includes the front side of the vehicle in the longitudinal direction, more than the first range. [Note 13] The door control method according to any one of [Appendix 9] to [Appendix 12], wherein the first range and the second range are set based on the positional relationship between the imaging unit that images the imaging range and the opening. [Note 14] The control unit, In the time-series imaging data, a vector including the direction of motion and the magnitude of motion is calculated in each of the multiple blocks obtained by dividing the predetermined region. Blocks whose vectors satisfy predetermined conditions are determined to be the moving body blocks corresponding to the moving body. A door control method described in any one of the items from [Appendix 9] to [Appendix 13]. [Note 15] A door control program that controls the opening and closing of a door provided in an opening of a vehicle based on imaging data of a predetermined imaging range of an external area relative to the opening of the vehicle, In a predetermined region included in the imaging range, motion is detected from the time-series imaging data, and the object of the motion is extracted as at least one moving object moving toward the opening. The opening and closing timing of the door is changed based on at least one of the extracted moving bodies, Make the processor execute it, In the first region included in the predetermined region, the object is extracted as the moving body when the direction of movement falls within a predetermined first range. In the second region included in the predetermined region, the object is extracted as the moving body when the direction of movement falls within a predetermined second range. Door control program. [Note 16] The first region is a region closer to the opening than the second region. The first range is wider than the second range. The door control program described in [Note 15]. [Note 17] The first region indicates a location located further forward in the longitudinal direction of the vehicle than the second region. The first range includes the rearward side in the front-rear direction, The door control program described in [Note 15]. [Note 18] The door control program described in [Appendix 17], wherein the second range includes the front side of the vehicle in the longitudinal direction, more than the first range. [Note 19] The first range and the second range are set based on the positional relationship between the imaging unit that images the imaging range and the opening, according to the door control program described in any of [Appendix 15] to [Appendix 18]. [Note 20] The control unit, In the time-series imaging data, a vector including the direction of motion and the magnitude of motion is calculated in each of the multiple blocks obtained by dividing the predetermined region. Blocks whose vectors satisfy predetermined conditions are determined to be the moving body blocks corresponding to the moving body. The door control program described in any of the following [Appendix 15] to [Appendix 19]. [Explanation of Symbols]

[0101] 1. Vehicle control system 2 Remote Center 3, 4 Moving object (3A, 4A: ground position) 5 Blocks (5S: Scalar value representation, 5V: Vector quantity representation) 6. Moving Blocks 7 grid 8 passerby 9. Pedestrian block 16. Object recognition device (162: Human detection unit, 164: Human detection ReID unit, 166: Human area detection unit) 100 Driving control device (110: external environment recognition unit, 120: vehicle position recognition unit, 130: operation detection unit, 140: driving control unit, 150: contact possibility determination unit, 160: driver state recognition unit, 170: driving support control unit, 172: braking force control unit, 174: cancellation control unit, 180: motion vector calculator, 182: motion vector corrector, 190: door control unit) 300 Vehicle (10: Camera, 10A: Imaging range, 12: Radar, 14: Finder, 20: Communication device, 30: HMI, 40: Vehicle sensor, 50: In-cabin camera, 60: Biometric information detection sensor, 70: Boarding / Alighting control device, 72: Door, 74: Display unit, 76: Speaker, 80: Driver's controls, 82: Accelerator pedal, 84: Brake pedal, 86: Steering wheel, 200: Driving force output device, 210: Brake device, 220: Steering device, 230: Driving control device) 400, 500 area 700 predetermined range 900 captured images

Claims

1. The system includes a control unit that controls the opening and closing of a door provided in an opening of a vehicle based on imaging data of a predetermined imaging range of the external area relative to the opening of the vehicle. The control unit, A process of detecting motion from the time-series imaging data in a predetermined region included in the imaging range, and extracting the object of the motion as at least one moving object moving toward the opening, A process to change the opening and closing timing of the door based on at least one of the extracted moving bodies, Execute, The control unit, In the first region included in the predetermined region, the object is extracted as the moving body when the direction of movement falls within a predetermined first range. In the second region included in the predetermined region, the object is extracted as the moving body when the direction of movement falls within a predetermined second range. Door control device.

2. The first region is a region closer to the opening than the second region. The first range is wider than the second range. The door control device according to claim 1.

3. The first region indicates a location located further forward in the longitudinal direction of the vehicle than the second region. The first range includes the rearward side in the front-rear direction more than the second range. The door control device according to claim 1.

4. The door control device according to claim 3, wherein the second range includes the front side of the vehicle in the longitudinal direction, more than the first range.

5. The door control device according to claim 1, wherein the first range and the second range are set based on the positional relationship between the imaging unit that images the imaging range and the opening.

6. The control unit, In the time-series imaging data, a vector including the direction of motion and the magnitude of motion is calculated in each of the multiple blocks obtained by dividing the predetermined region. Blocks whose vectors satisfy predetermined conditions are determined to be the moving body blocks corresponding to the moving body. The door control device according to claim 1.

7. The control unit groups adjacent blocks among the moving body blocks and detects the group of moving blocks as the moving body, according to claim 6.

8. A vehicle control system comprising a door control device according to any one of claims 1 to 7, the vehicle, and an imaging unit for imaging the imaging range.

9. A door control method for controlling the opening and closing of a door provided in an opening of a vehicle, based on imaging data of a predetermined imaging range of an external area relative to the opening of the vehicle, The control unit, In a predetermined region included in the imaging range, motion is detected from the time-series imaging data, and the object of the motion is extracted as at least one moving object moving toward the opening. The opening and closing timing of the door is changed based on at least one of the extracted moving bodies, Includes, The control unit, In the first region included in the predetermined region, the object is extracted as the moving body when the direction of movement falls within a predetermined first range. In the second region included in the predetermined region, the object is extracted as the moving body when the direction of movement falls within a predetermined second range. Door control method.

10. The first region is a region closer to the opening than the second region. The first range is wider than the second range. The door control method according to claim 9.

11. The first region indicates a location located further forward in the longitudinal direction of the vehicle than the second region. The first range includes the rearward side in the front-rear direction more than the second range. The door control method according to claim 9.

12. The door control method according to claim 11, wherein the second range includes the front side of the vehicle in the longitudinal direction, more than the first range.

13. The door control method according to claim 9, wherein the first range and the second range are set based on the positional relationship between the imaging unit that captures the imaging range and the opening.

14. The control unit, In the time-series imaging data, a vector including the direction of motion and the magnitude of motion is calculated in each of the multiple blocks obtained by dividing the predetermined region. Blocks whose vectors satisfy predetermined conditions are determined to be the moving body blocks corresponding to the moving body. The door control method according to any one of claims 9 to 13.

15. A door control program that controls the opening and closing of a door provided in an opening of a vehicle based on imaging data of a predetermined imaging range of an external area relative to the opening of the vehicle, In a predetermined region included in the imaging range, motion is detected from the time-series imaging data, and the object of the motion is extracted as at least one moving object moving toward the opening. The opening and closing timing of the door is changed based on at least one of the extracted moving bodies, Make the processor execute it, In the first region included in the predetermined region, the object is extracted as the moving body when the direction of movement falls within a predetermined first range. In the second region included in the predetermined region, the object is extracted as the moving body when the direction of movement falls within a predetermined second range. Door control program.

16. The first region is a region closer to the opening than the second region. The first range is wider than the second range. The door control program according to claim 15.

17. The first region indicates a location located further forward in the longitudinal direction of the vehicle than the second region. The first range includes the rearward side in the front-rear direction more than the second range. The door control program according to claim 15.

18. The door control program according to claim 17, wherein the second range includes the front side of the vehicle in the longitudinal direction, more than the first range.

19. The door control program according to claim 15, wherein the first range and the second range are set based on the positional relationship between the imaging unit that captures the imaging range and the opening.

20. In the time-series imaging data, a vector including the direction of motion and the magnitude of motion is calculated in each of the multiple blocks obtained by dividing the predetermined region. Blocks whose vectors satisfy predetermined conditions are determined to be the moving body blocks corresponding to the moving body. A door control program according to any one of claims 15 to 19.

Images