Obstacle detection system, obstacle detection method, and moving device

Abstract

To smoothly open and close a door by more accurately detecting an obstacle and preventing the door from colliding with the obstacle when opening and closing the door of a vehicle.SOLUTION: An obstacle detection system for detecting an obstacle in the vicinity of an opening / closing body included in a moving device includes a ranging data acquisition unit that acquires ranging data by an imaging device installed such that a surface of the opening / closing body in a driving direction and at least a part of a driving range of the opening / closing body fall within a ranging data acquisition range, and a distance detection unit that detects a distance from an installation position of the ranging data acquisition unit to an end of the opening / closing body based on an output of the ranging data acquisition unit. An obstacle detection unit configured to detect an obstacle based on whether or not a distance value less than a predetermined distance exists in a driving range of the opening / closing body in the distance map, wherein SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an obstacle detection system, an obstacle detection method, and a mobile device, and in particular to an obstacle detection system, an obstacle detection method, and a mobile device that are suitable for use in applications such as detecting an obstacle and preventing a collision between the door and the obstacle when opening or closing a vehicle door. [Background technology]

[0002] In recent years, automatic doors that open and close automatically when operated by the driver have become popular. Although these automatic doors are convenient because they do not require drivers to operate them, they require caution when opening and closing the doors to avoid hitting any objects in the vicinity.

[0003] Therefore, a technique has been proposed for detecting an obstacle within the driving range of a vehicle door when the door is opened or closed.

[0004] For example, Patent Document 1 discloses a technology that uses an ultrasonic sensor to measure distance, captures an image of the area within the detection range of the ultrasonic sensor with a camera, and controls the opening and closing of a door based on the detection results of the ultrasonic sensor and the results of image processing. Also, Patent Document 2 discloses a means for detecting an obstacle based on an image captured by an imaging device by positioning the imaging device so that the driving range of the back door is captured (Figs. 1, 3, and 15). [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Publication No. 2022-123787 [Patent Document 2] Japanese Patent Application Publication No. 2022-168699 Summary of the Invention [Problem to be solved by the invention]

[0006] The above-mentioned conventional technologies all have in common that they are means for avoiding collision with an obstacle by detecting an obstacle near the door using an image captured by an imaging device.

[0007] However, the obstacle detection device disclosed in Patent Document 1 uses an ultrasonic sensor, so it may not be able to accurately detect the position of an obstacle if the obstacle has a complex shape. Also, the imaging system disclosed in Patent Document 2 is fixed to the vehicle body, so even if the back door is opened or closed, a blind spot is created by the opening or closing of the back door, and it may not be possible to detect an obstacle in the blind spot.

[0008] An object of the present invention is to provide an obstacle detection system that can more accurately detect obstacles when opening and closing a vehicle door, prevent collisions between the door and the obstacle, and enable smooth opening and closing of the door. [Means for solving the problem]

[0009] The obstacle detection system of the present invention is preferably configured as an obstacle detection system that detects obstacles near an opening / closing body provided on a mobile device, and includes a ranging data acquisition unit that acquires ranging data using an imaging device that is installed so that the surface in the driving direction of the opening / closing body and at least a portion of the driving range of the opening / closing body fall within the ranging data acquisition range, a ranging processing unit that detects the distance from the installation position of the imaging device to the end of the opening / closing body based on the output of the ranging data acquisition unit and generates a distance map, and an obstacle detection unit that detects obstacles based on whether there is a distance value less than a predetermined distance within the driving range of the opening / closing body in the distance map. [Effects of the Invention]

[0010] According to the present invention, an obstacle detection system can be provided that can more accurately detect obstacles when opening and closing a vehicle door, prevent collisions between the door and the obstacle, and enable smooth opening and closing of the door. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 2 is a diagram showing the positional relationship of each door of a vehicle. [Figure 2] 1 is a functional configuration diagram of an obstacle detection system according to a first embodiment. [Figure 3] FIG. 2 is a hardware configuration diagram of a vehicle. [Figure 4A] FIG. 1 is a cross-sectional view of the imaging element taken along the line lm. [Figure 4B] FIG. 2 is a schematic diagram of a cross section II′ of the pixel group. [Figure 4C] FIG. 2 is a diagram showing a first light flux received by a first photoelectric conversion unit and a second light flux received by a second photoelectric conversion unit. [Figure 5A] FIG. 1 is a side view of the rear of a conventional vehicle when there is a pole as an obstacle. [Figure 5B] FIG. 1 is a side view of the rear of a conventional vehicle when there is a wall as an obstacle. [Figure 5C] FIG. 1 is a side view of the rear of a conventional vehicle when there is a wall as an obstacle. [Figure 6A] FIG. 1 is a first side view of the rear of a vehicle for explaining the installation position of an imaging device in the first embodiment. [Figure 6B] FIG. 2 is a side view of the rear of the vehicle for explaining the installation position of the imaging device in the first embodiment (part 2). [Figure 7A] 4 is a flowchart (part 1) showing the processing of the obstacle detection system according to the first embodiment of the present invention. [Figure 7B] 6 is a flowchart showing the processing of the obstacle detection system according to the first embodiment of the present invention (part 2). [Figure 8A] FIG. 10 is a diagram showing the state of a distance map (part 1). [Figure 8B] FIG. 10 is a diagram showing the state of a distance map (part 2). [Figure 9] FIG. 10 is a diagram illustrating the installation position of the imaging device in the second embodiment, as viewed from above the vehicle. [Figure 10A]FIG. 11 is a diagram (part 1) seen from above the vehicle to explain the installation position of the imaging device in the third embodiment. [Figure 10B] FIG. 11 is a diagram (part 2) seen from above the vehicle to explain the installation position of the imaging device in the third embodiment. [Figure 11] FIG. 10 is a view seen from above the vehicle to explain the installation position of the imaging device in the fourth embodiment. [Figure 12] FIG. 10 is a functional configuration diagram of an obstacle detection system according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, each embodiment of the present invention will be described with reference to FIGS.

[0013] [Embodiment 1]

[0014] Hereinafter, a first embodiment of the present invention will be described with reference to FIGS. 1 to 8B. First, the configuration of an obstacle detection system according to a first embodiment of the present invention will be described with reference to FIGS. FIG. 1 is a diagram showing the positional relationship of the doors of a vehicle. FIG. 2 is a functional configuration diagram of the obstacle detection system according to the first embodiment. FIG. 3 is a diagram showing the hardware configuration of the vehicle.

[0015] The obstacle detection system of this embodiment is a system that detects an obstacle when an opening / closing body attached to a mobile device is opened or closed, and prevents a collision between the opening / closing body and the obstacle. In the following embodiment, an automobile is used as an example of the mobile device. The automobile may be a vehicle that runs on a reciprocating engine using gasoline or diesel as fuel, or may be a so-called electric vehicle that obtains driving power by rotating a motor using a battery. It may also be a so-called hybrid vehicle in which driving by fuel is assisted by electricity from a battery. In this embodiment, the opening / closing body is described as a back door.

[0016] In this specification, as shown in FIG. 1 , the x-axis represents the left-right direction of the vehicle 1, the y-axis represents the front-rear direction, and the z-axis represents the up-down direction. The +x direction represents the rightward direction, the -x direction represents the leftward direction, the +y direction represents the rearward direction, the -y direction represents the forward direction, the +z direction represents the upward direction, and the -z direction represents the downward direction. The doors equipped on the vehicle 1 are a tailgate 10 at the rear of the vehicle 1, front doors 11r and 11l on the left and right sides attached to the front, and rear doors 12r and 12l attached to the rear. The tailgate 10 is a hinged door that can be opened by being driven in the +y and +z directions. The front door 11r is a hinged door that can be opened by being driven in the +x direction, and the front door 11l is a hinged door that can be opened by being driven in the -x direction, with the -y direction being a common direction for both doors. On the other hand, the rear door 12r is a sliding door that is driven in the +x direction, and the rear door 12l is a sliding door that is driven in the -x direction and then driven in the +y direction to open. Note that a system for detecting obstacles in the case of the front door and the rear door as opening and closing bodies will be described in a later embodiment.

[0017] Next, the functional configuration of the obstacle detection system will be described with reference to FIG.

[0018] As shown in Figure 2, the obstacle detection system 100 comprises an imaging unit 101, a ranging data acquisition unit 102, an image processing unit 110, a ranging processing unit 111, an image recognition unit 112, an obstacle detection unit 113, a warning unit 114, an opening / closing body control unit 115, and a memory unit 120.

[0019] The imaging unit 101 is a functional unit realized by an imaging device such as a camera that captures external images. The distance measurement data acquisition unit 102 is a functional unit that captures an image and acquires data for measuring the distance to an object from the image. The image processing unit 110 is a functional unit that processes image data from an image signal. The distance measurement processing unit 111 is a functional unit that generates a distance map based on the data acquired by the distance measurement data acquisition unit 102 and calculates the distance between an obstacle and the door (i.e., performs distance measurement processing). The image recognition unit 112 is a functional unit that performs image recognition processing on the image data. The obstacle detection unit 113 is a functional unit that detects the presence of an obstacle based on the output of the distance measurement processing unit 111. The warning unit 114 is a functional unit that notifies the user or notifies each unit of the vehicle to issue a warning when there is a possibility of a collision with an obstacle. The warning unit 114 notifies the user, for example, by displaying a message on a display, turning on or flashing a warning lamp, or emitting an audio warning sound or audio warning guidance. The opening / closing body control unit 115 is a functional unit that controls the opening and closing of the door based on the output result of the obstacle detection unit 113. The storage unit 120 is a functional unit that stores data used in the obstacle detection system 100.

[0020] The storage unit 120 stores distance measurement data 121, image data 122, distance map data 123, and opening / closing body control data 124.

[0021] Distance measurement data 121 is data for measuring distance acquired by distance measurement data acquisition unit 102. Image data 122 is image data captured by imaging unit 101. Distance map data 123 is data of a distance map (details will be described later) created by distance measurement processing unit 111. Opening / closing body control data 124 is data related to the timing and speed for controlling the opening and closing of the door.

[0022] Next, the hardware configuration of the vehicle will be explained using FIG. 3, while also including the relationship with the functional configuration of the obstacle detection system in FIG. 2 as appropriate.

[0023] As shown in FIG. 3, the vehicle 1 includes an information processing device 200, an imaging device 210, an ECU (Electronic Control Unit) 220, an opening / closing body drive system device 230, a user interface device 240, an operation device 250, and a drive system device 260.

[0024] The information processing device 200 is a device that executes various computer programs that operate on the vehicle 1. In this embodiment, the information processing device 200 executes programs that implement the functions of the image processing unit 110, the distance measurement processing unit 111, the image recognition unit 112, the obstacle detection unit 113, the warning unit 114, and the opening / closing body control unit 115, in particular.

[0025] The information processing device 200 includes a CPU (Central Processing Unit) 201, a main memory 202, a SOC (System On Chip) / FPGA (Field Programmable Gate Array) 203, and a non-volatile memory 204.

[0026] The CPU 201 is a processor that executes various functions by executing programs loaded onto the main memory 202. In this embodiment, the CPU 201 particularly executes the above-mentioned functions related to the obstacle detection system 100. The CPU 201 also performs settings and control for the imaging device 210 and the SOC / FPGA 203. The main memory 202 is a volatile semiconductor storage device that stores programs and work data referenced by the CPU 201. The SOC / FPGA 203 is a non-volatile semiconductor storage device in which programs executed by the vehicle are written. In this embodiment, in particular, the non-volatile memory 204 stores code for programs that realize the functions of the obstacle detection system (programs that realize the functions of the image processing unit 110, distance measurement processing unit 111, image recognition unit 112, obstacle detection unit 113, warning unit 114, and opening / closing body control unit 115). The non-volatile memory 204 is a non-volatile semiconductor storage device such as a flash memory that stores setting data and the like that is retained even when the power is turned off.

[0027] The imaging device 210 is a camera device that captures images and acquires distance measurement data. The imaging device 210 includes an imaging optical system 211, an imaging element 212, and a digital data processing circuit 213.

[0028] The imaging optical system 211 is an optical element such as a photographic lens, and has the function of forming an image of a subject on the image sensor 212. The imaging optical system 211 is usually made up of multiple lens groups (not shown), and has an exit pupil at a predetermined distance from the image sensor 212. The exit pupil is an image of the aperture stop formed by an optical system on the image side of the aperture stop.

[0029] In this specification, the l, m, and n axes in the imaging optical system 211 are defined such that the n axis is parallel to the optical axis of the imaging optical system 211, and the l axis and m axis are perpendicular to each other and perpendicular to the optical axis.

[0030] The image sensor 212 is configured with a CMOS (complementary metal oxide semiconductor) or a CCD (charge-coupled device) and has a distance measurement function using an imaging surface phase difference distance measurement method. The subject image formed on the image sensor 212 via the imaging optical system 211 is photoelectrically converted by the image sensor 212 to generate an image signal based on the subject image.

[0031] The digital data processing circuit 213 converts the image signal from the image sensor 212 into digital data and transmits it to the information processing device 200. In the information processing device 200, the digital image data is stored in the non-volatile memory 204. Furthermore, the captured image may be displayed on the user interface device 240 based on the digital image data.

[0032] The ECU 220 is a unit incorporating a computer and memory for comprehensively performing drive control, direction control, etc. of the vehicle 1, and is interconnected with the information processing device 200 so as to be able to exchange information. The ECU 220 outputs vehicle control signals to the information processing device 200, the user interface device 240, the drive system device 260, and the opening / closing body drive device 230. The vehicle control signals output by the ECU 220 include signals that transmit information relating to the running (moving state) of the vehicle, such as the running speed, running direction, state of the shift lever, shift gear, and turn signal, and the direction of the vehicle as determined by a geomagnetic sensor, etc.

[0033] The opening / closing body driving device 230 is a device that drives the opening and closing operation of each door of the vehicle 1, and is a device such as a stepping motor, a belt, a pulley, a gear, and a door lock mechanism for opening and closing the doors.

[0034] The user interface device 240 is a device that displays information about the state of the vehicle 1 and the situation around the vehicle 1, and is an instrument panel such as a speedometer and a fuel gauge on the dashboard, or a panel that displays information about the opening and closing of doors and information about obstacles. The user interface device 240 may also be provided with warning lamps for information about the opening and closing of doors and information about obstacles. The user interface device 240 may also be an audio output device that notifies the user of the information about the state of the vehicle 1 and the situation around the vehicle 1, information about the opening and closing of doors, and information about obstacles by means of a warning sound, or that provides guidance.

[0035] The operating device 250 is a device that allows the driver or passengers of the vehicle 1 to operate the vehicle, and is, for example, a driving handle, a shift lever, a door handle, or the like.

[0036] The drive system 260 is a drive device for driving the vehicle 1, and includes, for example, an internal combustion engine, a motor, a transmission, a drive shaft, and the like.

[0037] Next, the configuration and optical operation of the image sensor will be described with reference to Figures 4A to 4C, particularly the fact that the image sensor can acquire image data and distance measurement data. FIG. 4A is a cross-sectional view of the imaging element taken along line lm. FIG. 4B is a schematic diagram of a cross section II' of the pixel group. FIG. 4C is a diagram showing a first light flux received by a first photoelectric conversion unit and a second light flux received by a second photoelectric conversion unit.

[0038] 4A, the image sensor 212 is configured with a plurality of pixel groups 321 arranged in 2 rows and 2 columns. In the pixel group 321, a green pixel 321G1 (1st row, 1st column) and a green pixel 321G2 (2nd row, 2nd column) are arranged in the diagonal direction, and the other two pixels are a red pixel 321R (1st row, 2nd column) and a blue pixel 321B (2nd row, 1st column).

[0039] FIG. 4B shows a cross section II' of the imaging element 212 shown in FIG. 4A.

[0040] In each pixel, two photoelectric conversion units (a first photoelectric conversion unit 326 and a second photoelectric conversion unit 327) for photoelectrically converting received light are arranged in a light receiving layer 325. The microlens 322 is arranged so that the exit pupil and the light receiving layer 325 are optically conjugate. As a result, the first photoelectric conversion unit 326 mainly receives the first light beam 311, and the second photoelectric conversion unit 327 mainly receives the second light beam 212.

[0041] FIG. 4C shows the first light beam 311 received by the first photoelectric conversion section 326 and the second light beam 212 received by the second photoelectric conversion section 327.

[0042] The first photoelectric conversion unit 326 photoelectrically converts the received light beam to generate an electrical signal. Similarly, the second photoelectric conversion unit 327 photoelectrically converts the received light beam to generate an electrical signal. A first ranging image signal is generated from a collection of electrical signals generated by the first photoelectric conversion units 326 of each pixel of the image sensor 212. Similarly, a second ranging image signal is generated from a collection of electrical signals generated by the second photoelectric conversion units 327 of each pixel of the image sensor 212. The intensity distribution of the image formed on the image sensor 212 by the first light beam 311 can be obtained from the first ranging image signal, and the intensity distribution of the image formed on the image sensor 212 by the second light beam 212 can be obtained from the second ranging image signal. In addition, since the pixel group 321 has color filters corresponding to the blue, green, and red wavelength ranges, the first ranging image signal and the second ranging image signal contain three color information. That is, in the image sensor 212 of this embodiment, each pixel has both an image capturing function and a distance measuring function. In this way, the image capturing device 210 of this embodiment has both an image capturing function and a distance measuring function.

[0043] The imaging device 210 of this embodiment may be configured as a stereo camera or a combination of a camera for imaging and a LiDAR (Light Detection and Ranging) for acquiring distance information.

[0044] Next, based on the above description of the hardware, an overview of the processing of each functional unit shown in FIG. 2 will be described.

[0045] As described above, the ranging processor 111 calculates the amount of parallax between the first ranging image signal and the second ranging image signal, which are a collection of electrical signals generated by the first photoelectric converter 326 and the second photoelectric converter 327, respectively. Specifically, the ranging processor 111 sets a point of interest within the first ranging image signal and sets a matching range centered on the point of interest. The matching range is, for example, a rectangle with a side length of a predetermined number of pixels and centered on the point of interest. The ranging processor 111 then sets a reference point within the second ranging image signal and sets a reference range centered on the reference point. The reference range has the same size and shape as the matching range. The ranging processor 111 calculates the degree of correlation between the first ranging image signal included in the matching range and the second ranging image signal included in the reference range while sequentially moving the reference point, and determines the reference point with the highest correlation as the corresponding point corresponding to the point of interest. The amount of relative positional deviation between the point of interest and the corresponding point is the amount of parallax at the point of interest. The ranging processor 111 can calculate the amount of parallax at a plurality of pixel positions by calculating the amount of parallax while sequentially moving the point of interest. A known method can be used to calculate the degree of correlation. For example, a method called NCC (Normalized Cross-Correlation) that evaluates the normalized cross-correlation between image signals, SSD (Sum of Squared Difference) that evaluates the sum of the squares of the differences between image signals, or SAD (Sum of Absolute Difference) that evaluates the sum of the absolute values ​​of the differences can be used.

[0046] Next, the distance measurement processing unit 111 uses a predetermined conversion coefficient to convert the amount of parallax into a defocus amount, which is the distance from the image sensor 212 to the focal point of the imaging optical system 211. When the predetermined conversion coefficient is K, the defocus amount is ΔL, and the parallax amount is d, the parallax amount d can be converted into the defocus amount ΔL by the following (Equation 1).

[0047] ΔL=K×d…(Formula 1)

[0048] By performing such distance information generation processing for a plurality of pixel positions, it is possible to generate a distance map that includes the defocus amounts for a plurality of pixel positions as distance information.

[0049] Next, the defocus amount is converted into the subject distance. The defocus amount can be converted into the subject distance using the imaging relationship of the imaging optical system 211. For example, the focal length and principal point position of the imaging optical system 211 can be used for the conversion. Through these processes, a distance map containing the subject distance as distance information can be generated. Here, the distance measurement processing unit 111 can measure at least the distance from the shooting position of the imaging device attached to the door to the edge of the door (the specifics of how this is done will be described later).

[0050] The generated distance map is stored in non-volatile memory 204 and can be referenced by obstacle detection unit 113. Obstacle detection unit 113 references the distance map generated by distance measurement processing unit 111, and if there is a distance value less than the predetermined distance within a predetermined range, detects that there is an obstacle at a position less than the predetermined distance. Here, the predetermined range is, for example, the driving range of the door to which imaging device 210 is attached. Also, the predetermined distance value is, for example, the distance from the position of imaging device 210 attached to the door to the edge of the door.

[0051] The obstacle detection unit 113 stores the coordinates of the obstacle in the distance map and distance information as the obstacle detection result in, for example, the non-volatile memory 204.

[0052] The image processing unit 313 performs image processing on the image signal output from the imaging unit 101. Note that part or all of the image processing unit 313 may be performed by a signal processing unit stacked in the imaging element 212.

[0053] Specifically, the image processing unit 313 performs de-Bayer processing on image data input from the image sensor 212 in accordance with the Bayer array, converting it into RGB raster image data. The Bayer array is a typical pixel array used in image sensors for color cameras. Furthermore, the image processing unit 313 performs various correction processes, such as white balance adjustment, distortion correction, gain / offset adjustment, gamma processing, color matrix processing, and lossless compression processing. However, it is preferable not to perform lossy compression processing.

[0054] The processed image data is stored in the non-volatile memory 204 and is referenced by the image recognition unit 112 .

[0055] The image recognition unit 112 recognizes objects in the image data by applying a predetermined image recognition algorithm to the image data input from the image processing unit 110. The predetermined algorithm may be configured to perform detection using an inference model that has been trained in advance by deep learning by inputting a data set that includes at least walls, ceilings, and people as training data.

[0056] As a result of object recognition, the type of object and coordinate information within the image data are stored in the nonvolatile memory 204.

[0057] The obstacle detection unit 113 performs processing based on the coordinates of the obstacle in the distance map and distance information output from the obstacle detection unit 113 stored in the nonvolatile memory 204, and the type of object and coordinate information in the image data output from the image recognition unit 112. Specifically, it obtains information necessary for door control and notifies the opening / closing body control unit 115. In addition, if there is a possibility of collision between the door and the obstacle, it notifies the warning unit 114 to issue a warning.

[0058] Next, the obstacle detection process performed by the obstacle detection system of the first embodiment will be described with reference to FIGS. 5A to 7B. First, a state when an obstacle is present near a vehicle according to the prior art will be described with reference to FIGS. 5A to 5C. FIG. 5A is a side view of the rear of a vehicle according to the prior art when there is a pole as an obstacle. FIG. 5B is a side view of the rear of a vehicle according to the prior art when there is a wall as an obstacle. FIG. 5C is a side view of the rear of a vehicle according to the prior art when there is a wall as an obstacle.

[0059] When opening the back door 10, the driver or passenger of the vehicle 1 must be careful because, as shown in Fig. 5A, a pole 40 may be installed as an obstacle within the driving range 20 (the range in which the back door 10 moves when opening it). In this case, there is a possibility that the back door 10 may collide with the pole 40 when opened.

[0060] Similarly, when opening the back door 10, care must be taken because, as shown in FIG. 5B, a wall 41 of a parking lot or the like may be close to the driving range 20. In this case, there is a possibility that the back door 10 may collide with the wall 41 when opened.

[0061] Similarly, when opening the back door 10, care must be taken if, within the driving range 20, the garage ceiling 42 is lower than the height of the vehicle 1, as shown in FIG. 5C. In this case, there is a possibility that the back door 10 may collide with the ceiling 42 when opened.

[0062] Therefore, the process for solving the problems of the conventional technology in the obstacle detection system of this embodiment will be described below with reference to FIGS. 6A to 8B. FIG. 6A is a diagram (part 1) showing the rear side of the vehicle to explain the installation position of the imaging device in the first embodiment. FIG. 6B is a diagram (part 2) showing the rear of the vehicle from the side, for explaining the installation position of the imaging device in the first embodiment. FIG. 7A is a flowchart (part 1) showing the processing of the obstacle detection system according to the first embodiment of the present invention. FIG. 7B is a flowchart showing the processing of the obstacle detection system according to the first embodiment of the present invention (part 2). FIG. 8A is a diagram showing the state of a distance map (part 1). FIG. 8B is a diagram showing the state of the distance map (part 2).

[0063] 6A, in the obstacle detection system of this embodiment, the imaging device 210 is positioned so that the surrounding area of ​​the surface in the driving direction of the back door 10 (the rear surface of the vehicle) and at least a part of the back door driving range 20 fall within the imaging and ranging range 30. The imaging and ranging range 30 is the image acquisition range obtained by imaging from the imaging range of the imaging device 210, and is also the range for acquiring distance measurement data.

[0064] In this way, the imaging device 210 is installed so that the imaging and ranging position changes in conjunction with the back door, so that the imaging and ranging range 30 can always capture images and measure distances within the range that is closest to the back door 10 when it opens. The integrated processing unit 300 is housed inside the vehicle 1.

[0065] The angle of view of the imaging device 210 may be narrow as long as it can be installed so as to capture the most recent driving range 20 of the back door 10 and the surface in the driving direction of the back door 10, as described above. More preferably, the angle of view is wide enough to capture the rear of the vehicle 1, so that rear monitoring of the vehicle 1 can also be performed.

[0066] Fig. 6B shows the imaging and ranging range 30 in which imaging device 210 can capture images and measure distances when back door 10 is open, as shown in Fig. 6A. As shown in Fig. 6B, by attaching imaging device 210 to back door 10 so that the imaging and ranging position moves in the same way in conjunction with back door 10, imaging and ranging can always be performed within back door driving range 20 within the range in which back door 10 most recently moves.

[0067] Next, a method for detecting an obstacle near the back door by the obstacle detection system of this embodiment will be described with reference to FIGS. 7A and 7B.

[0068] Here, the back door 10 will be described as a power back door that is actuated by a user's switch operation or the like as a trigger. The opening / closing body in Figures 7A and 7B is the back door 10, and the opening / closing body will be described as the back door 10 below.

[0069] First, the distance measurement processing unit 111 of the obstacle detection system generates a distance map (S101).

[0070] Next, the obstacle detection unit 113 of the obstacle detection system performs obstacle detection processing (S102).

[0071] The obstacle detection unit 113 determines whether or not there is an obstacle within the opening / closing body driving range (S103), and if there is an obstacle (S103: Yes), the process proceeds to S106, and if there is no obstacle (S103: No), the process proceeds to S113 (FIG. 7B). Although not shown in the flowchart, if there is an obstacle, the warning unit 114 may issue a warning notice to the user.

[0072] The image processing unit 110 of the obstacle detection system generates an image for recognition in parallel with S101 and S102 (S104).

[0073] Next, the image recognition unit 112 of the obstacle detection system performs image recognition processing in the image recognition unit 112 based on the generated recognition image (S105).

[0074] If it is determined in S103 that an obstacle is present within the driving range, the obstacle detection unit 113 detects the coordinates (referred to as coordinates C1) on the distance map of the obstacle that will collide with the detected back door 10 (S106). Specifically, when the distance to the back door 10 on the distance map is less than a certain threshold value Th d Find the following coordinates:

[0075] Next, the obstacle detection unit 113 detects the coordinates on the distance map of the back door 10 that will collide with the detected obstacle (referred to as coordinates C2) (S107). Specifically, among the distance values ​​of the coordinates detected in S106 and multiple coordinates of the back door 10 with the same distance values, the coordinate whose coordinate in the direction perpendicular to the driving direction of the back door 10 is closest to the obstacle is detected. Note that how to determine the coordinates in S106 and S107 will be described in detail later with reference to FIGS. 8A and 8B.

[0076] Next, the obstacle detection unit 113 determines whether the difference between a certain component of the coordinate C1 acquired in S106 and a corresponding component of the coordinate C2 acquired in S107 is smaller than a threshold value Th1 (S108). Specifically, the difference between the y coordinate of C1 on the distance map acquired in S106 and the y coordinate of C2 on the distance map acquired in S107 is calculated. Note that, as shown in FIG. 6A, the coordinates are calculated such that (y coordinate of C1)>(y coordinate of C2). The threshold value Th1 is set to a value beyond which the back door 10 may collide with an obstacle if driven further. If the difference between a certain component of the coordinate C1 and a corresponding component of the coordinate C2 is smaller than the threshold value Th1 (S108: Yes), the process ends. If the difference between a certain component of the coordinate C1 and a corresponding component of the coordinate C2 is equal to or greater than the threshold value Th1 (S108: No), the process proceeds to S109.

[0077] In S108, the obstacle detection unit 113 determines whether the difference between a certain component of the coordinate C1 acquired in S106 and a corresponding certain component of the coordinate C2 acquired in S107 is smaller than a threshold value Th2 (S109). Specifically, the difference between the y coordinate on the distance map acquired in S106 and the y coordinate on the distance map acquired in S107 may be calculated. The threshold value Th2 is set to a value that prevents the back door 10 from colliding with an obstacle until S108 is performed for the next frame of an image acquired by the image capture device 210 when the drive speed of the back door 10 is set to speed V1. In S109, if the difference between the certain component of the coordinate C1 and the corresponding certain component of the coordinate C2 is smaller than the threshold value Th2 (S109: Yes), the process proceeds to S111. If the difference between the certain component of the coordinate C1 and the corresponding certain component of the coordinate C2 is equal to or greater than the threshold value Th2 (S109: No), the process proceeds to S110 (FIG. 7B).

[0078] If the difference between a certain component of coordinate C1 and a corresponding certain component of coordinate C2 is equal to or greater than threshold Th2, the obstacle detection unit 113 determines what kind of object the obstacle within the driving range of the back door 10, acquired by the obstacle detection unit 113, is recognized as by the image recognition unit 112 (S110). If, based on the determination result of S110, the recognition result by the image recognition unit 112 is a moving object such as a person or animal, or an unknown object that has not been recognized (S110: moving object or unknown object), the process proceeds to S111 (FIG. 7A). Otherwise, the object is determined to be a static object such as a wall, ceiling, or pole (S110: static object), and the process proceeds to S112.

[0079] If the recognition result is a moving object such as a person or an animal, or an unknown object that cannot be recognized, the obstacle detection unit 113 notifies the opening / closing body control unit 115, and the opening / closing body control unit 115 sets the driving speed of the back door 10 to speed V1 (S111).

[0080] If the recognition result indicates a stationary object, the obstacle detection unit 113 determines whether the difference between a certain component of the coordinate C1 acquired in S106 and a corresponding component of the coordinate C2 acquired in S107 is smaller than a threshold value Th3 (S112). The threshold value Th3 is set to a value that prevents the back door 10 from colliding with an obstacle until S108 is performed for the next frame of an image acquired by the imaging device 210 when the drive speed of the back door 10 is set to speed V2. If the difference between a certain component of the coordinate C1 and a corresponding component of the coordinate C2 is smaller than the threshold value Th3 in S112 (S112: Yes), the process proceeds to S114. If the difference is equal to or greater than the threshold value Th3 (S112: No), the process proceeds to S113.

[0081] If the difference between a certain component of coordinate C1 and a corresponding certain component of coordinate C2 is equal to or greater than threshold value Th3, obstacle detection unit 113 checks for any objects recognized by image recognition unit 112 that are outside the driving range of back door 10 (S113). If there is a dynamic object such as a person or animal outside the driving range of back door 10 (S113: Yes), the process proceeds to S114. Otherwise (S113: No), the process proceeds to S115. At this time, the distance to the dynamic object outside the driving range may be taken into consideration.

[0082] If there is a moving object such as a person or an animal outside the driving range of the back door 10, the obstacle detection unit 113 notifies the opening / closing body control unit 115, and the opening / closing body control unit 115 sets the driving speed of the back door 10 to speed V2 (S114). Here, the relationship between speed V1 and speed V2 is speed V1<speed V2.

[0083] If there is no moving object such as a person or an animal outside the driving range of the back door 10, the obstacle detection unit 113 notifies the opening / closing body control unit 115, and the opening / closing body control unit 115 sets the driving speed of the back door 10 to speed V3 (S115). Here, the relationship between speed V2 and speed V3 is speed V2<speed V3.

[0084] After execution of S111, or if the recognition result in S110 is a moving object such as a person or an animal, or if it is determined to be an unknown object that cannot be recognized, the opening / closing body control unit 115 notifies the ECU 220 of the driving speed of the tailgate, and the ECU 220 performs tailgate driving control (S116).

[0085] Next, the opening / closing body control unit 115 determines whether the back door 10 is no longer driven, i.e., whether it is fully opened (S117: FIG. 7A), and if it is fully opened (S117: Yes), controls to end the opening / closing operation. Otherwise (S117: No), the process returns to S101 and repeats the process.

[0086] Next, how to obtain coordinates in S106 and S107 will be described in detail with reference to FIGS. 8A and 8B.

[0087] The processing of S106 is a processing in which the obstacle detection unit 113 detects the coordinates on the distance map of an obstacle that will collide with the tailgate 10, and the processing of S107 is a processing in which the method of calculating the coordinates is to detect the coordinates on the distance map of the tailgate 10 that will collide with the detected obstacle.

[0088] FIG. 8A shows a distance map obtained in S101 from the imaging angle of the imaging device 210. The upper left corner of the distance map is the origin, the h direction is the rightward direction relative to the origin, and the v direction is the downward direction relative to the origin. The darker the color, the closer the distance from the imaging device 210, and the whiter the color, the farther the distance. The distance map stores distance information for the tailgate 10 and the pole 40 from the imaging device 210. The method for detecting the position 50 of the pole 40 where the tailgate 10 first collides with the tailgate 10 in S106 will now be described. The coordinates and distance values ​​of the distances within the distance map that are less than a predetermined distance, i.e., within the driving range of the tailgate 10, are stored in the non-volatile memory 204 by the obstacle detection unit 113. The coordinates of these distance values ​​that are closest to the tailgate 10 in the driving direction of the tailgate 10 are the positions of the obstacles that the tailgate 10 will first collide with. Because the tailgate 10 moves in the +y direction, the position of the pole 40 with the largest y coordinate corresponds to the position of the pole 40 within the driving range of the tailgate 10.

[0089] Next, a method for detecting the position of collision with the pole 40 within the back door 10 in S107 will be described. Among the coordinates of the back door 10 with the distance value detected in S106 and the same distance value, the coordinate whose coordinate perpendicular to the driving direction of the back door 10 is the same as that of the pole 40 is determined as the collision position 51 of the obstacle. In this embodiment, the coordinate area of ​​the back door 10 with the same y coordinate as the distance value of the pole 40 in S106 and the same x coordinate as the pole 40 is determined as the coordinate area of ​​the back door 10 where collision with the pole 40 occurs. Here, the distance value of the back door 10 in the distance map is uniquely determined by the mounting position of the distance measurement data acquisition unit 102, and is therefore stored in the non-volatile memory 204 in advance. This makes it possible to determine the position of collision with the obstacle even if the shape of the back door 10 prevents the edge of the back door 10 from being captured by the image capture device 210.

[0090] Figure 8B is a diagram of Figure 8A immediately before tailgate 10 hits pole 40. As shown in Figure 8B, by controlling the drive of tailgate 10 to the very limit where the y coordinates of positions 50 and 51 do not overlap, tailgate 10 can be opened to the maximum extent possible without hitting pole 40. Note that obstacles are not limited to poles, and the same applies to other objects such as walls, ceilings, and people.

[0091] As described above, the obstacle detection system 100 in this embodiment can detect obstacles within the driving range of the tailgate 10. Furthermore, if the tailgate 10 is a power tailgate, the tailgate 10 can be opened to the maximum extent possible without colliding with an obstacle, and the door can be opened at a safe speed.

[0092] The information processing device 200 in this embodiment may be integrated into the ECU 220, and the processing of the information processing device may be performed by the ECU 220.

[0093] Note that even if the back door 10 is not a power back door, i.e., is manually operated, if the user detects an obstacle while opening the back door 10, the notification by the warning unit 114 effectively functions to alert the user. In this case, if the user is a specific user registered in advance in the image recognition unit 112, the specific user may be manually controlling the back door, so control may be performed so that the user is not included in the obstacles.

[0094] Furthermore, a light emitting unit that emits auxiliary light for distance measurement in synchronization with the image capturing timing of the image capturing device 210 may be mounted.

[0095] In addition, in this embodiment, when the back door 10 is opened, the imaging device 210 is disposed on the outside of the back door 10. However, by disposing the imaging device 210 on the vehicle inside of the back door 10 so that the closing surface of the back door 10 is captured, the same effect can be obtained when the back door 10 is closed.

[0096] [Embodiment 2]

[0097] Hereinafter, a second embodiment of the present invention will be described with reference to FIG. FIG. 9 is a diagram seen from above the vehicle to explain the installation position of the imaging device in the second embodiment.

[0098] The obstacle detection system of the first embodiment is a system in which the opening / closing body is the back door of a vehicle and detects obstacles that obstruct the opening and closing of the back door. The obstacle detection system of this embodiment is a system in which the opening / closing body is the front door of a vehicle and detects obstacles that obstruct the opening and closing of the front door. The configuration and operation of the obstacle detection system of this embodiment are the same as those of the first embodiment, and the following description will focus on the differences in the opening / closing body.

[0099] As shown in FIG. 9 , the obstacle detection system of this embodiment performs processing to detect surrounding obstacles when the front door 11r, which is a hinged door on the side of a vehicle, opens or closes. FIG. 9 shows the installation position of an imaging device 210 for detecting nearby obstacles when the front door 11r opens. The imaging device 210 is positioned so that at least a portion of the driving direction surface of the front door 11r and the driving range 21 are within an imaging and distance measurement range 31 in which the imaging device 210 can capture and measure distances. This installation allows the imaging device 210 to always capture and measure distances to the most recent driving range when the front door 11r opens. The imaging device 210 is also positioned, for example, on a side mirror. The angle of view of the imaging device 210 may be narrow as long as it can be installed so that the most recent driving range of the driving range 20 of the front door 11r and the surface in the driving direction of the front door 11r are captured. More preferably, the angle of view is set wide enough to capture the sides of the vehicle 1, so that the sides of the vehicle 1 can also be monitored.

[0100] The process for detecting an obstacle near the front door 11r is the same as in the first embodiment.

[0101] Moreover, by changing the position of the imaging device 210 so that the surface that moves when the front door 11r closes, that is, the surface of the front door 11r on the inside of the vehicle, is captured, it is possible to similarly detect an obstacle.

[0102] The obstacle detection system of this embodiment is not limited to the front door 11r, and can similarly detect obstacles in any hinged door or double door (a door in which the left and right doors open in opposite directions).

[0103] [Embodiment 3]

[0104] Hereinafter, a third embodiment of the present invention will be described with reference to FIGS. 10A and 10B. FIG. 10A is a diagram (part 1) seen from above the vehicle to explain the installation position of the imaging device in the third embodiment. FIG. 10B is a view (part 2) seen from above the vehicle to explain the installation position of the imaging device in the third embodiment.

[0105] The obstacle detection system of the second embodiment is a system that detects obstacles that hinder the opening and closing of a front door of a vehicle, which is an opening and closing body, and the front door of the vehicle is a hinged door.

[0106] The obstacle detection system of this embodiment is a system that detects obstacles that hinder the opening and closing of a rear door of a vehicle, which is a sliding door.

[0107] In this embodiment, as in the second embodiment, the configuration and operation of the obstacle detection system are the same as in the first embodiment, and the differences in the opening / closing body will be mainly described.

[0108] In this embodiment, the rear side sliding doors 12r and 12l of the vehicle are driven in the +x direction, the -x direction, and then the +y direction when opening from a closed state. Therefore, there are two types of drive surfaces, and the arrangement for detecting obstacles on each side will be described below.

[0109] In this embodiment, the installation position of the imaging device 210 is as shown in FIG. 10A in order to detect nearby obstacles when the rear door 12r begins to open. The imaging device 210 is positioned so that the surface of the rear door 12r that moves in the +x direction (the side surface of the vehicle 1) and at least a portion of the driving range 22 are within the imaging and distance measurement range 32 in which the imaging device 210 can capture and measure images. By installing the imaging device 210 in this manner, it is possible to capture and measure images of the range of the rear door 12r that moves in the +x direction when it slides open from a closed state. Preferably, by widening the angle of view of the imaging device 210 as in the first embodiment, it can also be used to monitor the side of the vehicle 1.

[0110] 10B shows the installation position of the imaging device 210 for detecting obstacles near the driving direction of the rear door 12r when the rear door 12r moves in the +y direction after being driven in the +x direction as shown in FIG. 10A. The imaging device 210 is positioned so that at least a portion of the surface of the rear door 12r that drives in the +y direction and the driving range 20 falls within the imaging and distance measurement range 32 of the imaging device 210. By installing the imaging device 210 in this manner, it is possible to capture images and measure distances within the range of immediate driving when the rear door 12r moves in the y direction after the operation shown in FIG. 10A when sliding open.

[0111] The process for detecting an obstacle near the rear door 12r is the same as in the first embodiment.

[0112] In addition, obstacles can be detected in the same manner by changing the position of the imaging device 210 so that it captures the surface that moves when the rear door 12r closes, i.e., the surface on the inside of the vehicle of the rear door 12r or the surface on the front door 11r side.

[0113] The obstacle detection system of this embodiment is not limited to the rear door 12r, and can similarly detect obstacles on any sliding door.

[0114] [Embodiment 4]

[0115] Hereinafter, a fourth embodiment of the present invention will be described with reference to FIGS. FIG. 11 is a view seen from above the vehicle to explain the installation position of the imaging device in the fourth embodiment. FIG. 12 is a functional configuration diagram of an obstacle detection system according to the second embodiment.

[0116] In the obstacle detection system of the first embodiment, the back door of the vehicle is the opening / closing body, and in the embodiment, the front door of the vehicle is the opening / closing body.

[0117] The obstacle detection system of this embodiment is a system in which imaging devices are installed on both the back door and the front door of the vehicle.

[0118] FIG. 11 shows the image capturing device 210bd attached to the back door 10, the image capturing and ranging range 30 of the image capturing device 210bd, and the image capturing and ranging range 31 of the image capturing device 210fr attached to the front door 11r. An overlapping range 35 of the image capturing and ranging ranges 30 and 31 is also shown. In this case, the information processing device 200 connected to the image capturing devices 210bd and 210fr can process the captured images and ranging data of the image capturing device 210bd and the image capturing device 210fr, respectively. For ease of explanation, the example in FIG. 11 shows only two image capturing devices; however, they may be placed on the front door 11l, the rear doors 12r and 12l, and other positions (such as a rearview mirror, not shown) that can capture and ranging images of the front of the vehicle 1.

[0119] The obstacle detection system of this embodiment is configured by adding an object movement direction estimation unit 116 and an obstacle collision determination unit 117 to the obstacle detection system of the first embodiment shown in FIG.

[0120] The object movement direction estimation unit 116 is a functional unit that performs image recognition from image data captured by the imaging device 210bd and the imaging device 210fr, and estimates the direction of movement of the recognized object if it is a dynamic object. The obstacle collision determination unit 117 is a functional unit that determines whether the dynamic object will collide with the door based on the movement direction and movement speed of the dynamic object estimated by the object movement direction estimation unit 116 and the door drive speed.

[0121] Here, when driving the opening / closing body, if an obstacle outside the imaging / distancing range of the opening / closing body being driven is recognized by image data from an imaging device installed on another opening / closing body, the recognition result is used to reflect in the driving speed of the opening / closing body being driven.

[0122] For example, when driving the back door 10, the back door 10 is controlled as described in the first embodiment. At this time, if a moving obstacle, such as a person, is detected outside the imaging and ranging range 30 of the image capture device 210bd and within the imaging and ranging range 31 of the image capture device 210fr, the object movement direction estimation unit 116 estimates the movement direction of the obstacle. If the moving obstacle is heading toward the back door 10, the driving speed of the back door 10 is reflected in the determination of the driving speed, for example, by setting it to speed V3 described in S115 of FIG. 7B based on the determination of the obstacle collision determination unit 117. The movement direction of the moving obstacle may be estimated using optical flow as a predetermined algorithm.

[0123] In this embodiment, two image capturing devices are attached to separate doors to capture images and measure distances. Therefore, for example, when a person approaches the rear of the vehicle from the side, the direction of movement can be estimated and obstacle detection with a wider angle of view, such as controlling the opening and closing speed of the back door, can be performed, and multi-door control can be performed.

[0124] (Configuration 1) An obstacle detection system for detecting an obstacle near an opening / closing body provided in a mobile device, a distance measurement data acquisition unit that acquires distance measurement data using an imaging device that is installed so that a surface in the drive direction of the opening / closing body and at least a part of the drive range of the opening / closing body fall within a distance measurement data acquisition range; a distance measurement processing unit that detects the distance from the installation position of the imaging device to the end of the opening / closing body based on the output of the distance measurement data acquisition unit and generates a distance map; an obstacle detection unit that detects the obstacle based on whether or not a distance value less than a predetermined distance exists within the drive range of the opening / closing body in the distance map; An obstacle detection system having:

[0125] (Configuration 2) the imaging device is disposed at a position where a surface of the opening / closing body in a driving direction and a peripheral area of ​​the surface of the opening / closing body in the driving direction are within the distance measurement data acquisition range, and is moved in conjunction with the driving of the opening / closing body. 2. The obstacle detection system according to configuration 1,

[0126] (Configuration 3) the obstacle detection unit sets the predetermined distance value for detecting the obstacle as a distance value from the installation position of the imaging device to an end of the opening / closing body; 3. The obstacle detection system according to claim 1, wherein:

[0127] (Configuration 4) An opening / closing body control unit that controls the driving of the opening / closing body is further provided, The opening / closing body control unit controls the driving of the opening / closing body so as to prevent the opening / closing body from colliding with the obstacle. 4. The obstacle detection system according to any one of configurations 1 to 3.

[0128] (Configuration 5) The opening / closing body control unit sets a drive speed of the opening / closing body in accordance with the distance to the obstacle. 5. The obstacle detection system according to configuration 4.

[0129] (Configuration 6) an imaging unit that captures an image using the imaging device; further comprising an image recognition unit that performs object recognition based on the image; the imaging device is disposed so that an imaging range thereof overlaps with the distance measurement data acquisition range; The opening / closing body control unit sets a drive speed of the opening / closing body based on the result of the object recognition. 6. The obstacle detection system of claim 4 or 5.

[0130] (Configuration 7) the opening / closing body control unit determines whether the obstacle present in the driving range of the opening / closing body in the distance map is a static object based on the result of the object recognition, and sets the driving speed of the opening / closing body based on the determination result. 7. The obstacle detection system according to configuration 6,

[0131] (Configuration 8) an object movement direction estimation unit that performs image recognition of an obstacle from an image captured by the imaging device and estimates a movement direction of the obstacle; an obstacle collision determination unit that determines whether the obstacle will collide with the drive range of the opening / closing body based on the moving direction of the obstacle; The opening / closing body control unit determines the drive speed for controlling the opening / closing body based on the determination by the obstacle collision determination unit. 8. The obstacle detection system according to claim 6 or 7,

[0132] (Method 1) An obstacle detection method using an obstacle detection system that detects an obstacle near an opening / closing body provided in a mobile device, comprising: The obstacle detection system an imaging device for acquiring distance measurement data; an information processing device that executes a computer program; a distance measurement data acquisition step in which the information processing device acquires data using an imaging device that is installed so that a surface in the drive direction of the opening / closing body and at least a part of the drive range of the opening / closing body fall within a distance measurement data acquisition range; a distance measurement processing step in which the information processing device detects the distance from the installation position of the imaging device to the end of the opening / closing body based on the distance measurement data obtained in the distance measurement data acquisition step, and generates a distance map; an obstacle detection step in which the information processing device detects the obstacle based on whether or not a distance value less than a predetermined distance exists within the drive range of the opening / closing body in the distance map; An obstacle detection method comprising:

[0133] (Program 1) A computer program for causing a computer to execute each step of the obstacle detection method described in Method 1.

[0134] (Configuration 9) A mobile device including an imaging device and an opening / closing body, an obstacle detection system that detects obstacles near the opening / closing body; The obstacle detection system includes: a distance measurement data acquisition unit that acquires distance measurement data using an imaging device that is installed so that a surface in the drive direction of the opening / closing body and at least a part of the drive range of the opening / closing body fall within a distance measurement data acquisition range; a distance measurement processing unit that detects the distance from the installation position of the imaging device to the end of the opening / closing body based on the output of the distance measurement data acquisition unit and generates a distance map; an obstacle detection unit that detects the obstacle based on whether or not a distance value less than a predetermined distance exists within the drive range of the opening / closing body in the distance map; and the imaging device is disposed at a position where a surface of the opening / closing body in a driving direction and a peripheral area of ​​the surface of the opening / closing body in the driving direction are within the distance measurement data acquisition range, and is moved in conjunction with the driving of the opening / closing body. A mobile device characterized by:

[0135] (Configuration 10) The opening / closing body is one of a back door at the rear of the moving device, a front door at the side, and a rear door at the side. 10. The moving device according to configuration 9.

[0136] (Configuration 11) a first imaging device and a second imaging device; The first imaging device and the second imaging device are the surfaces of the first and second opening / closing bodies in the driving direction and the peripheral areas of the surfaces of the first and second opening / closing bodies in the driving direction are positioned within the distance measurement data acquisition range, and are arranged to move in conjunction with the driving of the surfaces of the first and second opening / closing bodies in the driving direction, The obstacle detection system an object movement direction estimation unit that performs image recognition of an obstacle from an image captured by the imaging device and estimates a movement direction of the obstacle; an obstacle collision determination unit that determines whether the obstacle will collide with the drive range of the opening / closing body based on the moving direction of the obstacle; the opening / closing body control unit determines the drive speed of the second opening / closing body based on a determination by the obstacle collision determination unit using distance measurement data acquired by imaging by the first imaging device. 11. The mobile device of either of configurations 9 or 10. [Explanation of symbols]

[0137] 1...vehicle, 10...back door, 11r...right front door, 11l...left front door, 12r...right rear door, 12l...left rear door, 20, 21, 22...driving range, 30, 31, 32...imaging and ranging range, 100... obstacle detection system, 101... imaging unit, 102... distance measurement data acquisition unit, 110... image processing unit, 111... distance measurement processing unit, 112... image recognition unit, 113... obstacle detection unit, 114... warning unit, 115... opening / closing body control unit, 120...storage unit, 121...distance measurement data, 122...image data, 123...distance map data, 124...opening / closing body control data, 200...information processing device, 201...CPU, 202...main memory, 203...SOC / FPGA, 204...non-volatile memory, 210...imaging device, 211...imaging optical system, 212...imaging element, 213...digital data processing circuit, 220...ECU, 230...opening / closing body drive system device, 240...user interface device, 250...operation device, 260...drive system device

Claims

1. An obstacle detection system for detecting an obstacle near an opening / closing body provided in a mobile device, a distance measurement data acquisition unit that acquires distance measurement data using an imaging device that is installed so that a surface in the drive direction of the opening / closing body and at least a part of the drive range of the opening / closing body fall within a distance measurement data acquisition range; a distance measurement processing unit that detects the distance from the installation position of the imaging device to the end of the opening / closing body based on the output of the distance measurement data acquisition unit and generates a distance map; an obstacle detection unit that detects the obstacle based on whether or not a distance value less than a predetermined distance exists within the drive range of the opening / closing body in the distance map; An obstacle detection system having:

2. the imaging device is disposed at a position where a surface of the opening / closing body in a driving direction and a peripheral area of ​​the surface of the opening / closing body in the driving direction are within the distance measurement data acquisition range, and is moved in conjunction with the driving of the opening / closing body.

2. The obstacle detection system of claim 1.

3. the obstacle detection unit sets the predetermined distance value for detecting the obstacle as a distance value from the installation position of the imaging device to an end of the opening / closing body; 2. The obstacle detection system of claim 1.

4. An opening / closing body control unit that controls the driving of the opening / closing body is further provided, The opening / closing body control unit controls the driving of the opening / closing body so as to prevent the opening / closing body from colliding with the obstacle.

2. The obstacle detection system of claim 1.

5. The opening / closing body control unit sets a drive speed of the opening / closing body in accordance with the distance to the obstacle.

5. The obstacle detection system according to claim 4.

6. an imaging unit that captures an image using the imaging device; further comprising an image recognition unit that performs object recognition based on the image; the imaging device is disposed so that an imaging range thereof overlaps with the distance measurement data acquisition range; The opening / closing body control unit sets a drive speed of the opening / closing body based on the result of the object recognition.

5. The obstacle detection system according to claim 4.

7. the opening / closing body control unit determines whether the obstacle present in the driving range of the opening / closing body in the distance map is a static object based on the result of the object recognition, and sets the driving speed of the opening / closing body based on the determination result.

7. The obstacle detection system according to claim 6.

8. an object movement direction estimation unit that performs image recognition of an obstacle from an image captured by the imaging device and estimates a movement direction of the obstacle; an obstacle collision determination unit that determines whether the obstacle will collide with the drive range of the opening / closing body based on a moving direction of the obstacle; the opening / closing body control unit determines the drive speed for controlling the opening / closing body based on the determination by the obstacle collision determination unit.

7. The obstacle detection system according to claim 6.

9. An obstacle detection method using an obstacle detection system that detects an obstacle near an opening / closing body provided in a mobile device, comprising: The obstacle detection system an imaging device for acquiring distance measurement data; an information processing device that executes a computer program; a distance measurement data acquisition step in which the information processing device acquires data using an imaging device that is installed so that a surface in the drive direction of the opening / closing body and at least a part of the drive range of the opening / closing body fall within a distance measurement data acquisition range; a distance measurement processing step in which the information processing device detects the distance from the installation position of the imaging device to the end of the opening / closing body based on the distance measurement data obtained in the distance measurement data acquisition step, and generates a distance map; an obstacle detection step in which the information processing device detects the obstacle based on whether or not a distance value less than a predetermined distance exists within the drive range of the opening / closing body in the distance map; An obstacle detection method comprising:

10. A computer program for causing a computer to execute each step of the obstacle detection method according to claim 9.

11. A mobile device including an imaging device and an opening / closing body, an obstacle detection system that detects obstacles in the vicinity of the opening / closing body; The obstacle detection system includes: a distance measurement data acquisition unit that acquires distance measurement data using an imaging device that is installed so that a surface in the drive direction of the opening / closing body and at least a part of the drive range of the opening / closing body fall within a distance measurement data acquisition range; a distance measurement processing unit that detects the distance from the installation position of the imaging device to the end of the opening / closing body based on the output of the distance measurement data acquisition unit and generates a distance map; an obstacle detection unit that detects the obstacle based on whether or not a distance value less than a predetermined distance exists within the drive range of the opening / closing body in the distance map; and the imaging device is disposed at a position where a surface of the opening / closing body in a driving direction and a peripheral area of ​​the surface of the opening / closing body in the driving direction are within the distance measurement data acquisition range, and is moved in conjunction with the driving of the opening / closing body. A mobile device characterized by:

12. The opening / closing body is one of a back door at the rear of the moving device, a front door at the side, and a rear door at the side.

12. The mobile device of claim 11.

13. a first imaging device and a second imaging device; The first imaging device and the second imaging device are the first and second opening / closing bodies are arranged so that the surfaces of the first and second opening / closing bodies in the driving direction and the peripheral areas of the surfaces of the first and second opening / closing bodies in the driving direction are within the distance measurement data acquisition range, and the first and second opening / closing bodies are arranged so that the surfaces of the first and second opening / closing bodies in the driving direction move in conjunction with the driving of the surfaces of the first and second opening / closing bodies in the driving direction, The obstacle detection system an object movement direction estimation unit that performs image recognition of an obstacle from an image captured by the imaging device and estimates a movement direction of the obstacle; an obstacle collision determination unit that determines whether the obstacle will collide with the drive range of the opening / closing body based on a moving direction of the obstacle; the opening / closing body control unit determines the drive speed of the second opening / closing body based on a determination by the obstacle collision determination unit using distance measurement data acquired by imaging with the first imaging device.

12. The mobile device of claim 11.

Images