Door control device and program

Abstract

To provide a door control device and a program capable of preventing delay while executing opening / closing control based on the operation of a user.SOLUTION: The door 6 provided in the opening 7 is provided with a control part 11 for executing opening / closing control of the door based on imaging data obtained by imaging a predetermined imaging range of an external region with respect to the opening, and the control part analyzes the imaging data according to a time series, and extracts a moving object R moving toward the opening from a predetermined region included in the imaging range. The door control device 10 evaluates a movement amount of a moving object moving toward an opening, restricts a closing operation of a door when it is determined that there is a moving object moving by a predetermined amount or more based on an evaluation result of the movement amount, and executes the closing operation of the door at a predetermined timing when it is determined that there is no moving object moving by the predetermined amount or more and there is no moving object that arrives at the opening in time for the predetermined timing set in advance based on the evaluation result of the movement amount.SELECTED DRAWING: Figure 11

Description

[Technical Field]

[0001] The present invention relates to a door control device and a program for automatically controlling the opening and closing of a door. [Background technology]

[0002] A door control device is known that controls the opening and closing of elevator doors based on the user's movements captured in a captured image (see, for example, Patent Document 1). This door control device is configured to analyze the captured image data in a time series, estimate whether the user intends to board the elevator based on the time series changes in the user's position in the image and the speed at which the user moves toward the door, and control the door opening and closing operation based on the estimation result. The door control device described in Patent Document 1 determines that the user intends to board the elevator when the user moves toward the door at a speed equal to or greater than a threshold. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-124899 Summary of the Invention [Problem to be solved by the invention]

[0004] According to the door control device described in Patent Document 1, if a user's intention to board is determined while the door is being closed, the door closing operation is stopped and the door is kept open, which could cause unnecessary delays in operation when applied to a moving object. Also, this door control device determines the user's intention to board and forcibly closes the door if a preset allowable time has elapsed, which could result in a person intending to board coming into contact with the door.

[0005] An object of the present invention is to provide a door control device and program that can prevent delays while performing opening and closing control based on the user's actions. [Means for solving the problem]

[0006] One aspect of the present invention is a door control device that includes a control unit that controls the opening and closing of a door installed at an opening based on image data obtained by capturing an image of a predetermined image range of an external area relative to the opening, and that analyzes the image data according to a time series, extracts a moving object moving toward the opening from within a predetermined area included in the image range, evaluates the amount of movement of the moving object toward the opening, and if it is determined based on the evaluation result of the amount of movement that there is a moving object moving more than a predetermined amount, it restricts the closing operation of the door, and if it is determined based on the evaluation result of the amount of movement that there is no moving object moving more than the predetermined amount and that there is no moving object that will arrive in time for the moving object to reach the opening at a predetermined timing, it executes the closing operation of the door at the predetermined timing. [Effects of the Invention]

[0007] According to the present invention, it is possible to prevent delays while executing opening and closing control based on the user's actions. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram showing a configuration of a door control device according to an embodiment; [Figure 2] FIG. 2 is a diagram illustrating an imaging range of an external camera. [Figure 3] FIG. 10 is a diagram showing a plurality of blocks set in a predetermined area included in an imaging range. [Figure 4] FIG. 10 is a diagram illustrating a method for calculating a movement vector and a scalar component. [Figure 5] FIG. 2 is a diagram showing the actual positional relationship between a moving object and a reference position. [Figure 6] FIG. 2 is a diagram showing the positional relationship between a moving object and a reference position in a captured image. [Figure 7] FIG. 10 is a diagram illustrating a method for correcting a scalar component. [Figure 8] FIG. 10 is a diagram showing a map of correction coefficients for scalar components. [Figure 9] FIG. 10 is a diagram showing a first threshold map. [Figure 10] FIG. 10 is a diagram showing a second threshold map. [Figure 11] 3 is a flowchart showing a process flow of a door control method executed in the door control device. DETAILED DESCRIPTION OF THE INVENTION

[0009] As shown in Figures 1 and 2, vehicle 1 is equipped with a door device 4 that can be opened and closed automatically. Vehicle 1 is equipped with a door control device 10 that controls the door device 4. Vehicle 1 is equipped with a detection unit 2 that detects a detection value for controlling the door device 4. Vehicle 1 is, for example, a bus vehicle that transports passengers. Vehicle 1 may be an autonomous vehicle or a manually driven vehicle. The detection unit 2, door device 4, and door control device 10 constitute a door system.

[0010] The detection unit 2 is configured, for example, by an external camera 2A that captures an image of the outside of the vehicle and generates image data. The external camera 2A outputs the image data to the door control device 10. The external camera 2A captures a plurality of frame images per unit time based on a predetermined frame rate (for example, 30 fps) and generates image data that will become a video. The external camera 2A outputs the image data to the door control device 10. The detection unit 2 may be configured not only by the external camera 2A but also by a lidar device or a radar device, as long as it is capable of detecting the external environment of the vehicle 1.

[0011] The external camera 2A has, for example, a wide-angle lens and captures an image of the outside of the vehicle 1. The external camera 2A captures an image of a predetermined range from an area adjacent to the door device 4 outside the vehicle 1. The external camera 2A is provided, for example, at a position outside the vehicle 1 that is spaced apart from the door device 4 and can capture an image of a predetermined range.

[0012] The door device 4 includes, for example, a door 6 that can be opened and closed freely. The door 6 opens and closes based on, for example, a sliding opening and closing mechanism. The door 6 may also be opened and closed based on a folding door or swing door opening and closing mechanism. The door 6 is driven to open and close by a drive unit 5. The drive unit 5 is an actuator having a mechanism for opening and closing the door 6. The drive unit 5 is controlled by the door control device 10. The drive unit 5 is configured to, for example, slide the door 6 to open and close. The drive unit 5 may also be configured to open and close the door 6 in a swing door or folding door manner. The drive unit 5 may be configured using any mechanism as long as it can drive the door 6 to open and close.

[0013] The door control device 10 includes a control unit 11 that controls the opening and closing of the door 6 based on the detection value detected by the detection unit 2. The control unit 11 is configured with at least one hardware processor such as a CPU (Central Processing Unit). The door control device 10 includes a storage unit 12 that stores data and programs. The storage unit 12 is configured with a non-transitory storage medium such as a hard disk drive (HDD) or a solid state disk (SSD). The storage unit 12 stores computer programs and data required for controlling the opening and closing of the door 6.

[0014] The control unit 11 is configured to be able to recognize image data using, for example, artificial intelligence (AI). The control unit 11 is configured to be able to recognize the content of the image data by, for example, previously executing machine learning based on deep learning or optical flow using the image data as training data. The control unit 11 is configured to be able to recognize the environment near the door 6 and a user present near the door 6 based on the image data captured by the detection unit 2. The control unit 11 is configured to control the opening and closing of the door 6 when it recognizes that a user is approaching the door 6.

[0015] FIG. 3 shows a frame image F1 included in the imaging data captured by the external camera 2A. The imaging data is video data having a plurality of frame images within a predetermined unit time based on a predetermined frame rate. The frame image F1 includes a predetermined imaging range outside the vehicle 1. For example, the frame image F1 captures an image of the opening 7, the door 6 provided in the opening 7, and the external environment of the vehicle 1 including the door 6. The frame image F1 is, for example, image data captured over a wide-angle imaging range.

[0016] The frame image F1 includes a predetermined imaging range of the external area relative to the opening 7. In the illustrated example, the position is farther away from the door 6 as it moves upward in the frame image F1, and the position is closer to the door 6 as it moves toward the door 6 in the frame image F1.

[0017] The control unit 11 controls the opening and closing of the door 6 based on the imaging data. The control unit 11 analyzes multiple frame images F1 included in the imaging data in chronological order. The control unit 11 sets a rectangular predetermined area Q within the imaging range of the frame image F1. The predetermined area Q may have any shape other than a rectangle. The control unit 11 divides the predetermined area Q into multiple blocks Qmn (m and n are natural numbers corresponding to coordinates) of image components arranged in a matrix.

[0018] The control unit 11 extracts one or more moving objects R moving toward the opening 7 from within a predetermined area Q included in the imaging range. The moving objects R are mainly users who plan to board the vehicle 1. The extraction process of the moving objects R by the control unit 11 is as follows.

[0019] 4, the control unit 11, for example, compares the frame image F1 at time t with the frame image F1 at time t+Δt, and calculates a motion vector Cmn of the image component that moves in time series in each block Qmn. The control unit 11 performs dense optical flow estimation, for example, by calculating the motion trajectories of the coordinates of all pixels in each block Qmn. The control unit 11 averages the motion trajectories of the coordinates of all pixels to calculate the motion vector Cmn in each block Qmn.

[0020] The control unit 11 converts the calculated movement vector Cmn, that is, the movement vector toward the door 6 (opening 7), into a scalar component Smn. The control unit 11 compares the scalar component Smn of each block Qmn with a threshold. The control unit 11 extracts scalar components Smn equal to or greater than the threshold as moving object blocks. In the illustrated example, the threshold is set to 7. The control unit 11 groups adjacent blocks Qmn whose scalar components are equal to or greater than the threshold to generate a cluster Qt. The control unit 11 recognizes the cluster Qt as a moving object R. The control unit 11 determines isolated blocks Qmn from the blocks Qmn whose scalar components have been calculated to be moving object noise and removes them.

[0021] FIG. 5 shows, in a side view, the relationship between the actual position of the moving object R and a predetermined area Q included in the frame image F1 captured by the external camera 2A. In the predetermined area Q, a plurality of reference positions P0 to P5 are set. The reference positions P0 to P5 are set so that the subscripts increase from closer to the door 6 to farther away. The number of reference positions P0 to P5 is just an example and may be increased or decreased. Each of the reference positions P0 to P5 is set in accordance with the actual distance from the door 6.

[0022] FIG. 6 shows the relationship between the predetermined region Q and the reference positions P0 to P5 in the frame image F1. In the frame image F1, for example, the lowest position of the moving object R1 (for example, the position of the toes) is located at the reference position P1. The lowest position of the moving object R1 is treated as one of the closest relative positions to the door 6 (see FIG. 5). In the frame image F1, for example, the lowest position of the moving object R2 (for example, the position of the toes) is located at the reference position P3. The lowest position of the moving object R2 is treated as one of the closest relative positions to the door 6 (see FIG. 5). The control unit 11 calculates the position of the moving object based on the positional relationship between the lowest position of the moving object R2 in the frame image F1 and the reference positions P0 to P5.

[0023] In the frame image F1, the moving object R1 located closer to the door 6 is displayed larger than the moving object R2 located farther from the door 6 than the moving object R1. The scalar component Smn (see FIG. 4) of the block Qmn calculated by the control unit 11 appears smaller the further away from the door 6 in the predetermined region Q of the frame image F1. Therefore, the control unit 11 corrects the scalar component Smn of the block Qmn in the predetermined region Q so that it increases the further away from the door 6.

[0024] 7 shows row numbers set for a plurality of row-unit blocks Qmn and reference positions P0 to P5 in the predetermined area Q. The subscripts of the row numbers B0 to B7 are set to increase as the distance from the door 6 increases. The number of row numbers is an example, and may be changed as appropriate depending on the number of blocks Qmn set in the predetermined area Q.

[0025] 8 shows the reference positions P0 to P5 of the lower limit position of the assembly Qt and a map M1 of correction coefficients applied to the scalar component Smn of each block Qmn included in the assembly Qt. Once the reference positions P0 to P5 of the lower limit position of the assembly Qt have been determined, the scalar component Smn of each block Qmn included in the assembly Qt is multiplied by the correction coefficient for the reference position in map M1. The correction coefficient is set so that it increases as the reference positions P0 to P5 of the lower limit position of the assembly Qt are farther away from the door 6, and as each block Qmn is farther away from the door 6.

[0026] In the example of Figure 7, the lower limit position of the aggregate Qt is the reference position P1, and the scalar component Smn of the bottom block Qmn of the aggregate Qt is multiplied by a correction coefficient of row number B2 for the reference position P1 based on map M1, and the scalar component Smn of the top block Qmn of the aggregate Qt is multiplied by a correction coefficient of row number B5 for the reference position P1.

[0027] Based on the map M1, the control unit 11 calculates a corrected scalar component by correcting the scalar component in accordance with the relationship between the position of the block Qmn included in the aggregate Qt and the lower limit position of the aggregate Qt. The control unit 11 averages the corrected scalar components included in the aggregate Qt to calculate a movement amount evaluation parameter for evaluating the movement amount of the moving object R. The movement amount evaluation parameter is an evaluation parameter corresponding to the movement amount of the moving object R between frame images captured at different times (i.e., between the time differences in the capture timing). The movement amount evaluated by the movement amount evaluation parameter is the movement amount per time (e.g., the time interval between the capture timings of consecutive frame images). In other words, the movement amount evaluation parameter can be considered to correspond to the movement speed of the moving object R. The movement amount evaluation parameter can be used to evaluate the movement speed of the moving object R.

[0028] When the control unit 11 recognizes one or more moving objects R, it executes the following process for each moving object R. Based on the position of the moving object R and the result of evaluating the amount of movement using the movement amount evaluation parameter, the control unit 11 determines whether the moving object R will arrive at the opening 7 in time for a predetermined timing that has been set in advance. The predetermined timing is, for example, the time at which the door closing operation starts. The predetermined timing is set, for example, to coincide with the departure time of the vehicle 1. The control unit 11 starts the closing operation of the door 6 from a predetermined timing that is a predetermined time before the departure time (for example, several seconds before), and ends the closing operation of the door 6 when the predetermined time has elapsed, bringing the door 6 into a closed state. The predetermined time may be adjusted as appropriate depending on the environment in which the vehicle 1 is present.

[0029] If the control unit 11 determines, based on the analysis result of the frame image F1, that there is no moving object R that will arrive in time, the control unit 11 executes the closing operation of the door 6 at the predetermined timing. If the control unit 11 determines, based on the analysis result of the frame image F1, that there is a moving object R that will arrive in time, the control unit 11 restricts the closing operation of the door 6. If the moving speed of the moving object R, such as a passenger running to the exit, is too strong, there is a concern about the magnitude of the impact when the door 6 comes into contact with the moving object R. Based on the first comparison result of the movement amount evaluation parameter and the first threshold, the control unit 11 determines whether there is a moving object that has moved a predetermined distance or more to extract a moving object that is moving too strongly. For example, the control unit 11 determines whether there is a moving object that is moving too strongly by determining whether the movement amount evaluation parameter is equal to or greater than the first threshold. The method of comparison with the first threshold is just an example, and other comparison methods may be used as long as they can determine the possibility of contact between the moving object R and the door 6.

[0030] FIG. 9 shows a first threshold map T1 in which the first threshold is set. The first threshold is set to extract moving objects with excessively strong movement momentum. The first threshold is used to determine whether or not there is a moving object moving a predetermined distance or more. A moving object moving a predetermined distance or more has such a strong momentum that, when it comes into contact with the door, it may receive a greater impact than a moving object moving less than the predetermined distance. The first threshold is set, for example, based on the relationship between the movement amount of the moving object R, the reference position of the moving object R, and the opening degree of the door 6. For example, at the same reference position, the first threshold is set to be lower as the opening degree of the door 6 increases and higher as the opening degree of the door 6 decreases.

[0031] For example, for the same opening degree of the door 6, the first threshold is set to be higher as the reference position is farther from the door 6. For example, the first threshold is set to be lower as the opening degree of the door 6 increases and the reference position of the moving object R approaches the door 6, and the first threshold is set to be higher as the opening degree of the door 6 decreases and the reference position of the moving object R is farther from the door 6.

[0032] Based on the first determination result, if the movement amount evaluation parameter is equal to or greater than the first threshold, the control unit 11 evaluates that the momentum of the moving object R is too strong. Based on the evaluation result, the control unit 11 stops the closing operation of the door 6 and executes the opening operation to reduce the possibility of contact between the moving object R and the door 6. The control unit 11 opens the door 6 and allows the moving object R (user) to pass through the opening 7.

[0033] 10 shows a second threshold map T2 in which the second threshold is set. The second threshold is used to determine whether the moving object R will arrive in time for a predetermined timing. The second threshold is set to a value lower than the first threshold. The second threshold is set, for example, based on the relationship between the moving speed at which the moving object R may come into contact with the door 6, the reference position of the moving object R, and the opening degree of the door 6. For example, at the same reference position, the second threshold is set to be lower as the opening degree of the door 6 increases and higher as the opening degree of the door 6 decreases.

[0034] For example, for the same opening degree of the door 6, the second threshold is set to be higher as the reference position is farther away from the door 6. For example, the second threshold is set to be lower as the opening degree of the door 6 increases and the reference position of the moving object R approaches the door 6, and the second threshold is set to be higher as the opening degree of the door 6 decreases and the reference position of the moving object R is farther away from the door 6. The region TB where the second threshold exceeds 100 is used to determine that the moving object R will not arrive in time for the predetermined timing.

[0035] The control unit 11 determines whether the moving object R will arrive in time for the predetermined timing based on the second comparison result of the movement amount evaluation parameter and the second threshold value. If the movement amount evaluation parameter is equal to or greater than the second threshold value, the control unit 11 determines that the moving object R will arrive in time for the predetermined timing, stops the closing operation of the door 6 during a stopping time according to the position of the moving object R, and maintains the state of the door 6 until the predetermined timing. At this time, the control unit 11 may output predetermined notification content based on audio, image, etc. from an alarm unit (not shown) provided in the vehicle 1 to notify the driver and passengers of the vehicle 1 that the vehicle 1 should wait.

[0036] The control unit 11 stops the closing operation of the door 6 and allows the moving object R (user) to pass through the opening 7. When the control unit 11 determines that the opening degree of the door 6 is low and that simply stopping the opening operation of the door 6 will cause contact between the moving object R and the door 6 or that it will be difficult for the moving object R to pass through the opening 7, the control unit 11 may stop the closing operation of the door 6 and also execute the opening operation.

[0037] 11 shows the processing flow of a door control method executed in the door control device 10. The door control method is executed based on a computer program installed in a computer mounted on the door control device 10. The computer program causes the door control device 10 to execute the following processes.

[0038] The control unit 11 acquires imaging data of a predetermined imaging range of an external area relative to the opening, captured by the external camera 2A (step S100). The control unit 11 divides a predetermined area included in the frame image F1 into image components of a plurality of blocks Qmn, and calculates a motion vector Cmn of the image components moving in the blocks Qmn (step S102). The control unit 11 converts the motion vector Cmn in the direction toward the door 6 into a scalar component Smn (step S104). The control unit 11 recognizes a block whose scalar component Smn is equal to or greater than a threshold as a moving object block (step S106).

[0039] The control unit 11 recognizes a group of adjacent moving object blocks as a moving object R (step S108). The control unit 11 removes isolated blocks whose scalar components are equal to or greater than a threshold as noise (step S110). The control unit 11 determines the reference position of the block to which the lowest position of the group (the position closest to the door 6) belongs (step S112). The control unit 11 calculates a corrected scalar component by correcting the scalar component according to the relationship between the positions of the blocks included in the group and the lowest position of the group (step S114). The control unit 11 averages the corrected scalar components included in the group to calculate a movement amount evaluation parameter of the moving object R (step S116).

[0040] Based on the result of the first comparison between the movement amount evaluation parameter and the first threshold (see FIG. 9), the control unit 11 determines whether or not there is a moving object whose movement amount is equal to or greater than a predetermined amount, which is a moving object whose movement momentum is too strong (step S118). If the control unit 11 determines that there is a moving object whose movement amount is equal to or greater than the predetermined amount, the control unit 11 stops the closing operation of the door 6 and executes the opening operation, because there is a concern that the momentum of the moving object is too strong and that the impact when it comes into contact with the door will be large (step S120). If the control unit 11 determines that there is no moving object whose movement amount is equal to or greater than the predetermined amount, based on the result of the second comparison between the movement amount evaluation parameter and the second threshold (see FIG. 10), it determines whether or not the moving object will arrive in time for the predetermined timing (step S122).

[0041] If the control unit 11 determines that the moving object R will arrive in time for the predetermined timing, it stops the closing operation of the door 6 (step S124). If the control unit 11 determines that the moving object R will not arrive in time for the predetermined timing, it executes the closing operation of the door 6 and puts the door 6 in the closed state (step S126).

[0042] As described above, the door control device 10 can prevent delays of the vehicle 1 while executing opening / closing control based on the user's actions captured in the image data. The door control device 10 can appropriately evaluate the movement state of the user captured in the image data by correcting the movement state according to the user's position in the image data. The door control device 10 can simplify the device configuration by recognizing the user's movement based on the image data without using any special sensors other than cameras.

[0043] In the above-described embodiment, the computer program executed by each component of the door control device 10 may be provided in a form recorded on a computer-readable, portable, non-transitory recording medium such as a semiconductor memory, a magnetic recording medium, or an optical recording medium. The present invention is not limited to the above-described embodiment and may be modified as appropriate without departing from the spirit of the present invention. For example, the door control device 10 may be applied to control doors of not only the vehicle 1 but also doors of vehicles other than the vehicle 1. [Explanation of symbols]

[0044] 1 vehicle, 2 detection unit, 2A external camera, 4 door device, 5 drive unit, 6 door, 7 opening, 10 door control device, 11 control unit, 12 memory unit, Cmn movement vector, F1 frame image, M1 map, Q specified area, Qmn block, Qt aggregate, R moving object, T1 first threshold map, T2 second threshold map, T3 threshold map

Claims

1. a control unit that controls opening and closing of a door provided at an opening based on image data obtained by capturing an image of a predetermined image range of an external area relative to the opening, The control unit Analyzing the imaging data according to a time series; extracting a moving object moving toward the opening from within a predetermined area included in the imaging range; evaluating the amount of movement of the moving object toward the opening; If it is determined based on the evaluation result of the amount of movement that the moving object is present and has moved by a predetermined amount or more, the closing operation of the door is restricted; When it is determined based on the evaluation result of the movement amount that there is no moving object that moves by a predetermined amount or more and that there is no moving object that will arrive in time for a predetermined timing before the moving object reaches the opening, a closing operation of the door is performed at the predetermined timing. Door control device.

2. The control unit restricting the closing operation of the door when it is determined based on the evaluation result of the movement amount that the moving object exists and will arrive in time for the predetermined timing; The door control device according to claim 1 .

3. The control unit Dividing the predetermined area included in the imaging data into a plurality of blocks of image components; Calculating a motion vector of the image component that moves in time series in the block; converting the movement vector in a direction toward the door into a scalar component; A group of adjacent blocks whose scalar component is equal to or greater than a threshold is recognized as the moving object; calculating a corrected scalar component by correcting the scalar component in accordance with a relationship between the position of the block included in the aggregate and a lower limit position of the aggregate; averaging the correction scalar components included in the aggregate to calculate a movement amount evaluation parameter of the moving object; determining whether or not there is a moving object that has moved by a predetermined amount or more based on a first comparison result between the movement amount evaluation parameter and a first threshold value; determining whether the moving object will arrive in time for the predetermined timing based on a second comparison result of the movement amount evaluation parameter and a second threshold value; The door control device according to claim 1 .

4. The control unit When it is determined based on the first comparison result that the moving object moving by a predetermined amount or more is present, the closing operation of the door is stopped and an opening operation is performed. The door control device according to claim 3 .

5. The control unit If it is determined based on the second comparison result that the moving object will arrive in time for the predetermined timing, the closing operation of the door is stopped. The door control device according to claim 3 .

6. A program installed in a computer of a door control device that executes opening and closing control of a door provided at an opening based on imaging data obtained by imaging a predetermined imaging range of an external area relative to the opening, Analyzing the imaging data according to a time series; extracting a moving object moving toward the opening from within a predetermined area included in the imaging range; evaluating the amount of movement of the moving object toward the opening; If it is determined based on the evaluation result of the amount of movement that the moving object is present and has moved by a predetermined amount or more, the closing operation of the door is restricted; When it is determined based on the evaluation result of the movement amount that there is no moving object that moves by a predetermined amount or more and that there is no moving object that will arrive in time for a predetermined timing before the moving object reaches the opening, a closing operation of the door is performed at the predetermined timing. causing the computer to execute a process; program.

Images