Door control device and program

Abstract

To provide a door control device and a program capable of preventing delay while executing opening / closing control based on the operation of a user.SOLUTION: The door 6 provided in the opening 7 includes a control unit 11 that executes opening / closing control of the door based on imaging data obtained by imaging a predetermined imaging range of an external region with respect to the opening, and the control unit analyzes the imaging data according to a time series and extracts a moving object moving toward the opening from a predetermined region included in the imaging range. The door control device 10 determines whether or not at least one extracted moving object is in time for a predetermined timing set in advance before reaching the opening, and executes the closing operation of the door when it is determined that there is no moving object in time for the predetermined timing.SELECTED DRAWING: Figure 10

Description

[Technical Field]

[0001] The present invention relates to a door control device and a program for automatically controlling the opening and closing of a door. [Background technology]

[0002] A door control device is known that controls the opening and closing of elevator doors based on the user's movements captured in a captured image (see, for example, Patent Document 1). This door control device is configured to analyze the captured image data in a time series, estimate whether the user intends to board the elevator based on the time series changes in the user's position in the image and the speed at which the user moves toward the door, and control the door opening and closing operation based on the estimation result. The door control device described in Patent Document 1 determines that the user intends to board the elevator when the user moves toward the door at a speed equal to or greater than a threshold. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-124899 Summary of the Invention [Problem to be solved by the invention]

[0004] According to the door control device described in Patent Document 1, if a user's intention to board is determined while the door is being closed, the door closing operation is stopped and the door is kept open, which could cause unnecessary delays in operation when applied to a moving object. Also, this door control device determines the user's intention to board and forcibly closes the door if a preset allowable time has elapsed, which could result in a person intending to board coming into contact with the door.

[0005] An object of the present invention is to provide a door control device and program that can prevent delays while performing opening and closing control based on the user's actions. [Means for solving the problem]

[0006] One aspect of the present invention is a door control device that includes a control unit that controls the opening and closing of a door installed at an opening based on image data obtained by capturing an image of a predetermined image range of an external area relative to the opening, and the control unit analyzes the image data according to a time series, extracts moving objects moving toward the opening from within a predetermined area included in the image range, determines whether at least one of the extracted moving objects will arrive at the opening in time for a predetermined timing, and if it determines that no moving object will arrive in time for the predetermined timing, executes a closing operation of the door. [Effects of the Invention]

[0007] According to the present invention, it is possible to prevent delays while executing opening and closing control based on the user's actions. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram showing a configuration of a door control device according to an embodiment; [Figure 2] FIG. 2 is a diagram illustrating an imaging range of an external camera. [Figure 3] FIG. 10 is a diagram showing a plurality of blocks set in a predetermined area included in an imaging range. [Figure 4] FIG. 10 is a diagram illustrating a method for calculating a movement vector and a scalar component. [Figure 5] FIG. 2 is a diagram showing the actual positional relationship between a moving object and a reference position. [Figure 6] FIG. 2 is a diagram showing the positional relationship between a moving object and a reference position in a captured image. [Figure 7] FIG. 10 is a diagram illustrating a method for correcting a scalar component. [Figure 8] FIG. 10 is a diagram showing a map of correction coefficients for scalar components. [Figure 9] FIG. 10 is a diagram showing a threshold map. [Figure 10] 3 is a flowchart showing a process flow of a door control method executed in the door control device. DETAILED DESCRIPTION OF THE INVENTION

[0009] As shown in Figures 1 and 2, vehicle 1 is equipped with a door device 4 that can be opened and closed automatically. Vehicle 1 is equipped with a door control device 10 that controls the door device 4. Vehicle 1 is equipped with a detection unit 2 that detects a detection value for controlling the door device 4. Vehicle 1 is, for example, a bus vehicle that transports passengers. Vehicle 1 may be an autonomous vehicle or a manually driven vehicle. The detection unit 2, door device 4, and door control device 10 constitute a door system.

[0010] The detection unit 2 is configured, for example, by an external camera 2A that captures an image of the outside of the vehicle and generates image data. The external camera 2A outputs the image data to the door control device 10. The external camera 2A captures a plurality of frame images per unit time based on a predetermined frame rate (for example, 30 fps) and generates image data that will become a video. The external camera 2A outputs the image data to the door control device 10. The detection unit 2 may be configured not only by the external camera 2A but also by a lidar device or a radar device, as long as it is capable of detecting the external environment of the vehicle 1.

[0011] The external camera 2A has, for example, a wide-angle lens and captures an image of the outside of the vehicle 1. The external camera 2A captures an image of a predetermined range from an area adjacent to the door device 4 outside the vehicle 1. The external camera 2A is provided, for example, at a position outside the vehicle 1 that is spaced apart from the door device 4 and can capture an image of a predetermined range.

[0012] The door device 4 includes, for example, a door 6 that can be opened and closed freely. The door 6 opens and closes based on, for example, a sliding opening and closing mechanism. The door 6 may also be opened and closed based on a folding door or swing door opening and closing mechanism. The door 6 is driven to open and close by a drive unit 5. The drive unit 5 is an actuator having a mechanism for opening and closing the door 6. The drive unit 5 is controlled by the door control device 10. The drive unit 5 is configured to, for example, slide the door 6 to open and close. The drive unit 5 may also be configured to open and close the door 6 in a swing door or folding door manner. The drive unit 5 may be configured using any mechanism as long as it can drive the door 6 to open and close.

[0013] The door control device 10 includes a control unit 11 that controls the opening and closing of the door 6 based on the detection value detected by the detection unit 2. The control unit 11 is configured with at least one hardware processor such as a CPU (Central Processing Unit). The door control device 10 includes a storage unit 12 that stores data and programs. The storage unit 12 is configured with a non-transitory storage medium such as a hard disk drive (HDD) or a solid state disk (SSD). The storage unit 12 stores computer programs and data required for controlling the opening and closing of the door 6.

[0014] The control unit 11 is configured to be able to recognize image data using, for example, artificial intelligence (AI). The control unit 11 is configured to be able to recognize the content of the image data by, for example, previously executing machine learning based on deep learning or optical flow using the image data as training data. The control unit 11 is configured to be able to recognize the environment near the door 6 and a user present near the door 6 based on the image data captured by the detection unit 2. The control unit 11 is configured to control the opening and closing of the door 6 when it recognizes that a user is approaching the door 6.

[0015] FIG. 3 shows a frame image F1 included in the imaging data captured by the external camera 2A. The imaging data is video data having a plurality of frame images within a predetermined unit time based on a predetermined frame rate. The frame image F1 includes a predetermined imaging range outside the vehicle 1. For example, the frame image F1 captures an image of the opening 7, the door 6 provided in the opening 7, and the external environment of the vehicle 1 including the door 6. The frame image F1 is, for example, image data captured over a wide-angle imaging range.

[0016] The frame image F1 includes a predetermined imaging range of the external area relative to the opening 7. In the illustrated example, the position is farther away from the door 6 as it moves upward in the frame image F1, and the position is closer to the door 6 as it moves toward the door 6 in the frame image F1.

[0017] The control unit 11 controls the opening and closing of the door 6 based on the imaging data. The control unit 11 analyzes multiple frame images F1 included in the imaging data in chronological order. The control unit 11 sets a rectangular predetermined area Q within the imaging range of the frame image F1. The predetermined area Q may have any shape other than a rectangle. The control unit 11 divides the predetermined area Q into multiple blocks Qmn (m and n are natural numbers corresponding to coordinates) of image components arranged in a matrix.

[0018] The control unit 11 extracts one or more moving objects R moving toward the opening 7 from within a predetermined area Q included in the imaging range. The moving objects R are mainly users who plan to board the vehicle 1. The process of extracting the moving objects R by the control unit 11 is as follows.

[0019] 4, the control unit 11, for example, compares the frame image F1 at time t with the frame image F1 at time t+Δt, and calculates a motion vector Cmn of the image component that moves in time series in each block Qmn. The control unit 11 performs dense optical flow estimation, for example, by calculating the motion trajectories of the coordinates of all pixels in each block Qmn. The control unit 11 averages the motion trajectories of the coordinates of all pixels to calculate the motion vector Cmn in each block Qmn.

[0020] The control unit 11 converts the calculated movement vector Cmn, that is, the movement vector toward the door 6 (opening 7), into a scalar component Smn. The control unit 11 compares the scalar component Smn of each block Qmn with a threshold. The control unit 11 extracts scalar components Smn equal to or greater than the threshold as moving object blocks. In the illustrated example, the threshold is set to 7. The control unit 11 groups adjacent blocks Qmn whose scalar components are equal to or greater than the threshold to generate a cluster Qt. The control unit 11 recognizes the cluster Qt as a moving object R. The control unit 11 determines isolated blocks Qmn from the blocks Qmn whose scalar components have been calculated to be moving object noise and removes them.

[0021] FIG. 5 shows, in a side view, the relationship between the actual position of the moving object R and a predetermined area Q included in the frame image F1 captured by the external camera 2A. In the predetermined area Q, a plurality of reference positions P0 to P5 are set. The reference positions P0 to P5 are set so that the subscripts increase from closer to the door 6 to farther away. The number of reference positions P0 to P5 is just an example and may be increased or decreased. Each of the reference positions P0 to P5 is set in accordance with the actual distance from the door 6.

[0022] FIG. 6 shows the relationship between the predetermined region Q and the reference positions P0 to P5 in the frame image F1. In the frame image F1, for example, the lowest position of the moving object R1 (for example, the position of the toes) is located at the reference position P1. The lowest position of the moving object R1 is treated as one of the closest relative positions to the door 6 (see FIG. 5). In the frame image F1, for example, the lowest position of the moving object R2 (for example, the position of the toes) is located at the reference position P3. The lowest position of the moving object R2 is treated as one of the closest relative positions to the door 6 (see FIG. 5). The control unit 11 calculates the position of the moving object based on the positional relationship between the lowest position of the moving object R2 in the frame image F1 and the reference positions P0 to P5.

[0023] In the frame image F1, the moving object R1 located closer to the door 6 is displayed larger than the moving object R2 located farther from the door 6 than the moving object R1. The scalar component Smn (see FIG. 4) of the block Qmn calculated by the control unit 11 appears smaller the further away from the door 6 in the predetermined region Q of the frame image F1. Therefore, the control unit 11 corrects the scalar component Smn of the block Qmn in the predetermined region Q so that it increases the further away from the door 6.

[0024] 7 shows row numbers set for a plurality of row-unit blocks Qmn and reference positions P0 to P5 in the predetermined area Q. The subscripts of the row numbers B0 to B7 are set to increase as the distance from the door 6 increases. The number of row numbers is an example, and may be changed as appropriate depending on the number of blocks Qmn set in the predetermined area Q.

[0025] 8 shows the reference positions P0 to P5 of the lower limit position of the assembly Qt and a map M1 of correction coefficients applied to the scalar component Smn of each block Qmn included in the assembly Qt. Once the reference positions P0 to P5 of the lower limit position of the assembly Qt have been determined, the scalar component Smn of each block Qmn included in the assembly Qt is multiplied by the correction coefficient for the reference position in map M1. The correction coefficient is set so that it increases as the reference positions P0 to P5 of the lower limit position of the assembly Qt are farther away from the door 6, and as each block Qmn is farther away from the door 6.

[0026] In the example of Figure 7, the lower limit position of the aggregate Qt is the reference position P1, and the scalar component Smn of the bottom block Qmn of the aggregate Qt is multiplied by a correction coefficient of row number B2 for the reference position P1 based on map M1, and the scalar component Smn of the top block Qmn of the aggregate Qt is multiplied by a correction coefficient of row number B5 for the reference position P1.

[0027] The control unit 11 calculates a corrected scalar component by correcting the scalar component based on the map M1 in accordance with the relationship between the position of the block Qmn included in the aggregate Qt and the lower limit position of the aggregate Qt. The control unit 11 integrates the corrected scalar components included in the aggregate Qt and calculates a movement amount evaluation parameter for evaluating the movement amount of the moving object R.

[0028] In the frame image, a moving object R1 located closer to the door 6 is displayed larger than a moving object R2 located further from the door 6, and therefore the number of blocks Qmn included in a moving object R located closer to the door 6 (i.e., the aggregate Qt) is greater than the number of moving objects R2 located further from the door 6. The movement amount evaluation parameter is calculated by accumulating the correction scalar components of each block Qmn included in the moving object R (i.e., the aggregate Qt), and therefore, assuming that the correction scalar components of each block Qmn are the same, the movement amount evaluation parameter becomes larger for a moving object R located closer to the door 6 (i.e., the greater the number of blocks Qmn included in the aggregate Qt). The movement amount evaluation parameter is an evaluation parameter equivalent to the movement amount weighted according to the position of the moving object R.

[0029] The movement amount evaluation parameter is an evaluation parameter that reflects the movement amount weighted according to the position of the moving object R between frame images captured at different times (i.e., between the time differences in the capture times). In other words, the movement amount evaluation parameter can be considered to correspond to the movement speed weighted according to the position of the moving object R.

[0030] When the control unit 11 recognizes one or more moving objects R, it executes the following process for each moving object R. Based on the position of the moving object R and the movement amount evaluation parameter, the control unit 11 determines whether the moving object R will arrive at the opening 7 in time for a predetermined timing that has been set in advance. The predetermined timing is, for example, the time at which the door closing operation starts. The predetermined timing is set, for example, to coincide with the departure time of the vehicle 1. The control unit 11 starts the closing operation of the door 6 from a predetermined timing that is a predetermined time before the departure time (for example, several seconds before), and ends the closing operation of the door 6 when the predetermined time has elapsed, bringing the door 6 into a closed state. The predetermined time may be adjusted as appropriate depending on the environment in which the vehicle 1 is present.

[0031] If the control unit 11 determines, based on the analysis result of the frame image F1, that there is no moving object R that will arrive in time for the predetermined timing, the control unit 11 executes the closing operation of the door 6 at the predetermined timing. If the control unit 11 determines, based on the analysis result of the frame image F1, that there is a moving object R that will arrive in time for the predetermined timing, the control unit 11 does not execute the closing operation of the door 6. The control unit 11 determines, based on the comparison result between the movement amount evaluation parameter and a threshold value, whether there is a moving object R that will arrive in time for the predetermined timing. For example, the control unit 11 determines whether the movement amount evaluation parameter is equal to or greater than a threshold value, and determines whether the moving object R will arrive in time for the predetermined timing. The method of comparison with the threshold value is one example, and other comparison methods may be used as long as they can determine whether the moving object R will arrive in time for the predetermined timing.

[0032] 9 shows a threshold map T in which thresholds are set. The thresholds are used to determine the possibility of contact between the door 6 and the moving object R. The thresholds are set, for example, based on the relationship between the moving speed of the moving object R, the reference position of the moving object R, and the opening degree of the door 6. For example, at the same reference position, the thresholds are set so that the higher the opening degree of the door 6, the lower the thresholds are, and the lower the opening degree of the door 6, the higher the thresholds are.

[0033] For example, for the same opening degree of the door 6, the threshold value is set to be higher as the reference position is farther away from the door 6. For example, the first threshold value is set to be lower as the opening degree of the door 6 is higher and the reference position of the moving object R is closer to the door 6, and the threshold value is set to be higher as the opening degree of the door 6 is lower and the reference position of the moving object R is farther away from the door 6. A region TA where the threshold value exceeds 100 is used to determine that the moving object R will not arrive in time for the predetermined timing. If the movement amount evaluation parameter is equal to or greater than the threshold based on the first determination result, the control unit 11 determines that there is a high possibility that the door 6 and the moving object R will come into contact with each other, and stops the closing operation of the door 6 and executes the opening operation.

[0034] The control unit 11 opens the door 6 and allows the moving object R (user) to pass through the opening 7. If it determines that the moving object R will arrive in time, it stops the execution of the closing operation of the door 6 during a stopping time according to the position of the moving object R and maintains the state of the door 6 until the predetermined timing. At this time, the control unit 11 may output predetermined notification content, based on audio, image, etc., from a notification unit (not shown) provided in the vehicle 1, to notify the driver or passengers of the vehicle 1 that the vehicle 1 must wait. If the control unit 11 determines that the opening degree of the door 6 is low and that simply stopping the opening operation of the door 6 will cause contact between the moving object R and the door 6 or that it will be difficult for the moving object R to pass through the opening 7, it may stop the closing operation of the door 6 and execute the opening operation.

[0035] 10 shows the processing flow of a door control method executed in the door control device 10. The door control method is executed based on a computer program installed in a computer mounted on the door control device 10. The computer program causes the door control device 10 to execute the following processes.

[0036] The control unit 11 acquires imaging data of a predetermined imaging range of an external area relative to the opening, captured by the external camera 2A (step S100). The control unit 11 divides a predetermined area included in the frame image F1 into image components of a plurality of blocks Qmn, and calculates a motion vector Cmn of the image components moving in the blocks Qmn (step S102). The control unit 11 converts the motion vector Cmn in the direction toward the door 6 into a scalar component (step S104). The control unit 11 recognizes a block whose scalar component is equal to or greater than a threshold as a moving object block (step S106).

[0037] The control unit 11 recognizes a group of adjacent moving object blocks as a moving object R (step S108). The control unit 11 removes isolated blocks whose scalar components are equal to or greater than a threshold as noise (step S110). The control unit 11 determines the reference position of the block to which the lowest position of the group (the position closest to the door 6) belongs (step S112). The control unit 11 calculates a corrected scalar component by correcting the scalar component according to the relationship between the position of the block included in the group and the lowest position of the group (step S114). The control unit 11 integrates the corrected scalar components included in the group to calculate a movement amount evaluation parameter P of the moving object R (step S116).

[0038] The control unit 11 determines whether the moving object R will arrive at the predetermined timing based on the comparison result between the movement amount evaluation parameter P and the threshold value (step S118). If the movement amount evaluation parameter P is equal to or greater than the threshold value, the control unit 11 determines that the moving object R will arrive at the predetermined timing (step S120). The control unit 11 stops or does not execute the closing operation of the door 6 (step S122). If the movement amount evaluation parameter P is less than the threshold value, the control unit 11 determines that the moving object R will not arrive at the predetermined timing (step S124). The control unit 11 executes the closing operation of the door 6 and closes the door 6 (step S126).

[0039] As described above, the door control device 10 can prevent delays to the vehicle 1 while performing opening / closing control based on the actions of the user captured in the imaging data. The door control device 10 can prevent users who will not be able to make the predetermined timing for executing closing control of the door 6, which is set based on the time the vehicle 1 departs, from continuing to wait, thereby preventing delays in the departure timing of the vehicle 1. The door control device 10 can prevent users from being unable to board the vehicle by closing the door 6 even though there are users who will be able to make the predetermined timing. The door control device 10 can appropriately determine whether the moving object R will make it to the predetermined timing based on the position of the moving object R by correcting an error based on the perspective difference of the moving object R for each block at the position of the frame image F1 and then evaluating the amount of movement weighted according to the position of the moving object R.

[0040] In the above-described embodiment, the computer program executed by each component of the door control device 10 may be provided in a form recorded on a computer-readable, portable, non-transitory recording medium such as a semiconductor memory, a magnetic recording medium, or an optical recording medium. The present invention is not limited to the above-described embodiment and may be modified as appropriate without departing from the spirit of the present invention. For example, the door control device 10 may be applied to control doors of not only the vehicle 1 but also doors of vehicles other than the vehicle 1. [Explanation of symbols]

[0041] 1 vehicle, 2 detection unit, 2A external camera, 4 door device, 5 drive unit, 6 door, 7 opening, 10 door control device, 11 control unit, 12 memory unit, Cmn movement vector, F1 frame image, M1 map, Q specified area, Qmn block, Qt aggregate, R moving object, T threshold map

Claims

1. a control unit that controls opening and closing of a door provided at an opening based on image data obtained by capturing an image of a predetermined image range of an external area relative to the opening, The control unit Analyzing the imaging data according to a time series; extracting a moving object moving toward the opening from within a predetermined area included in the imaging range; determining whether or not at least one of the extracted moving objects will arrive at the opening within a predetermined time; When it is determined that the moving object does not exist in time for the predetermined timing, the closing operation of the door is executed. Door control device.

2. The control unit When it is determined that the moving object exists and will arrive in time for the predetermined timing, the closing operation of the door is not executed. The door control device according to claim 1 .

3. The control unit Dividing the predetermined area included in the imaging data into a plurality of blocks of image components; Calculating a motion vector of the image component that moves in time series in the block; converting the movement vector in a direction toward the door into a scalar component; A group of adjacent blocks whose scalar component is equal to or greater than a threshold is recognized as the moving object; calculating a corrected scalar component by correcting the scalar component in accordance with a relationship between the position of the block included in the aggregate and a lower limit position of the aggregate; calculating a movement amount evaluation parameter of the moving object by integrating the correction scalar components included in the aggregate; determining whether or not there is at least one moving object that will be able to start the closing operation of the door in time for the predetermined timing based on a determination result of whether or not the movement amount evaluation parameter is equal to or greater than a threshold value; The door control device according to claim 1 .

4. A program installed in a computer mounted on a door control device that controls opening and closing of a door provided at an opening based on image data obtained by capturing an image of a predetermined image range of an external area relative to the opening, Analyzing the imaging data according to a time series; extracting a moving object moving toward the opening from within a predetermined area included in the imaging range; determining whether or not at least one of the extracted moving objects will arrive at the opening within a predetermined time; When it is determined that the moving object does not exist in time for the predetermined timing, the computer executes a process of closing the door. program.

Images