Vehicle control device and program

Abstract

To provide a vehicle control device and a program capable of three-dimensional measurement of an environment surrounding a vehicle.SOLUTION: A vehicle control device 10 comprises an operation section 12 which calculates dimensions of an object based on an image of a space surrounding a vehicle captured with a first measuring section 2 and a measurement of a second measuring section 4 measuring the object existing around the vehicle using electromagnetic waves. The operation section calculates a first measurement of the dimensions of the object based on the image of the object included in the captured image, sets a second measurement as a dimension value of the object when an error between the first measurement and the second measurement of the dimensions of the object measured with the second measuring section is equal to or lower than a predetermined value; and generates three-dimensional data of the space based on the dimension value.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a vehicle control device and a program capable of three-dimensionally measuring the surroundings of a vehicle. [Background technology]

[0002] In recent years, there is a demand for obtaining three-dimensional data of the environment around a vehicle in order to assist driving of the vehicle. Patent Document 1 describes a distance calculation device that can capture images of an object in a monocular state and a stereo camera state using a stereo camera and calculate the distance to the object. This distance calculation device is configured to calculate the distance to the object based on the amount of blur between a monocular camera image captured by a single camera and a stereo camera image captured by a pair of cameras as a stereo camera. According to the technology described in Patent Document 1, it is possible to calculate the distance to the object based on either the monocular camera image or the stereo camera image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2013-186043 A Summary of the Invention [Problem to be solved by the invention]

[0004] The technology described in Patent Document 1 recognizes the area of ​​an object contained in an image, such as a vehicle that can be pattern matched in advance, and estimates the dimensions of the object. The environment around the vehicle includes terrains whose dimensions cannot be set based on pattern matching, such as terrains formed by rocks and uneven ground with unevenness formed by flat and inclined surfaces. In such terrains, it may be difficult to generate three-dimensional data using pattern matching based on the captured image captured by a camera.

[0005] An object of the present invention is to provide a vehicle control device and program capable of three-dimensionally measuring the environment around the vehicle. [Means for solving the problem]

[0006] One aspect of the present invention is a vehicle control device that includes a calculation unit that calculates a dimension of an object based on an image of the space around a vehicle captured by a first measurement unit and a measurement value of a second measurement unit that measures the object present around the vehicle based on electromagnetic waves, wherein the calculation unit calculates a first measurement value of the dimension of the object based on an image of the object included in the captured image, and if an error between the first measurement value and a second measurement value of the dimension of the object measured by the second measurement unit is equal to or less than a predetermined value, sets the second measurement value to the dimension value of the object, and generates three-dimensional data of the space based on the dimension value. Effect of the Invention

[0007] According to the present invention, the environment around the vehicle can be measured in three dimensions. [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram showing a configuration of a vehicle control device according to an embodiment; [Diagram 2] FIG. 2 is a diagram showing the relationship between the size of an object included in a captured image and pixels. [Diagram 3] 4 is a flowchart showing a process flow of a vehicle control method executed in a vehicle control device. [Figure 4] 4 is a flowchart showing a flow of a process for generating three-dimensional data executed in the vehicle control device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] 1, the vehicle 1 is communicatively connected to an external device 20 via a network W. The external device 20 is, for example, a server device provided in another vehicle or a facility capable of remotely controlling the vehicle 1. The vehicle 1 may be an autonomous vehicle controlled by an on-board vehicle control device 10, or may be a manually-driven vehicle assisted in driving by the vehicle control device 10.

[0010] The vehicle 1 includes a first measurement unit 2 and a second measurement unit 4 that measure the surrounding environment, a position detection unit 6 that detects the current position, a drive unit 8 that serves as a power source for traveling, and a vehicle control device 10 that controls the vehicle 1. The first measurement unit 2 is configured with a stereo camera that can measure the distance to an object. The first measurement unit 2 includes, for example, a pair of camera elements. The first measurement unit 2 captures an image of the object. The first measurement unit 2 may be provided not only to capture an image of the range in the forward direction of the vehicle 1, but also to capture an image of a range toward the side or rear of the vehicle 1.

[0011] The first measurement unit 2 is controlled by the vehicle control device 10. The captured image captured by the first measurement unit 2 is output to the vehicle control device 10. The vehicle control device 10 calculates the distance to the object and the dimensions of the object based on the captured image as described below.

[0012] The second measurement unit 4 is configured with a detection device capable of measuring an object existing around the vehicle based on electromagnetic waves. The second measurement unit 4 is, for example, a radar device using millimeter waves or a lidar device using laser light. The second measurement unit 4 emits electromagnetic waves and receives the reflected waves reflected by the object. The second measurement unit 4 measures the distance to the object based on the phase difference between the emitted electromagnetic waves and the received reflected waves. The second measurement unit 4 may be provided not only to measure the range in the forward direction of the vehicle 1, but also to measure the range toward the side or rear of the vehicle 1.

[0013] The second measurement unit 4, for example, scans electromagnetic waves in three-dimensional directions in the direction of the radio wave emission surface and receives reflected waves. The second measurement unit 4 measures the distance and dimensions of an object based on the electromagnetic waves emitted while scanning and the reflected waves of the electromagnetic waves. The measurement values ​​measured by the second measurement unit 4 are output to the vehicle control device 10. The position detection unit 6 is composed of sensors such as a position sensor capable of detecting the current position, such as a GPS (Global Positioning System), an accelerometer, and an angular velocity meter. The position detection unit 6 detects the position data, acceleration, and angular acceleration of the vehicle 1 currently traveling, and outputs them to the vehicle control device 10.

[0014] The vehicle control device 10 calculates the current position of the vehicle 1 based on the detection result of the position detection unit 6. The vehicle control device 10 calculates the distance to an object and the dimensions of the object based on the measurement values ​​as described below, and generates three-dimensional data of the surroundings of the vehicle 1 at the current position. The vehicle control device 10 executes driving assistance and driving control of the vehicle 1 based on the generated three-dimensional data.

[0015] The drive unit 8 includes a prime mover that serves as a drive source necessary for running the vehicle 1, and a steering device for controlling the direction of the vehicle 1. The prime mover and the steering device may be configured separately or integrally. The drive unit 8 is controlled by a vehicle control device 10.

[0016] The vehicle control device 10 is configured, for example, by a computer device mounted on the vehicle 1. The vehicle control device 10 includes a calculation unit 12 that executes calculation processing. The vehicle control device 10 calculates the current position using the position detection unit 6. The vehicle control device 10 generates three-dimensional data of the space around the vehicle 1 based on the measurement results of the first measurement unit 2 and the second measurement unit 4. The vehicle control device 10 calculates a target vehicle speed at the current position based on the three-dimensional data, controls the drive unit 8, and causes the vehicle 1 to travel.

[0017] The vehicle control device 10 includes a storage unit 14 that stores programs and data necessary for processing by the calculation unit 12. The storage unit 14 is configured from a non-transitory storage medium such as a hard disk drive (HDD) or a solid state disk (SSD). The vehicle control device 10 includes a communication unit 16 that can be connected to a network W. The communication unit 16 is configured from a communication device capable of wireless communication. The vehicle control device 10 includes a display unit 18 that can display various information. The display unit 18 is configured from a display device such as a liquid crystal display or an organic EL (Electro-Luminescence) display.

[0018] The calculation unit 12 is composed of at least one hardware processor such as a CPU (Central Processing Unit). The calculation unit 12 generates three-dimensional data of the space around the vehicle 1 based on the captured image captured by the first measurement unit 2 at the current position and the measurement values ​​measured by the second measurement unit 4. The calculation unit 12 acquires the captured image of the space around the vehicle 1 captured by the first measurement unit 2. The calculation unit 12 analyzes the captured image acquired from the first measurement unit 2. The calculation unit 12 calculates the distance to an object included in the captured image based on the captured image. The calculation unit 12 extracts an object included in the captured image captured at a predetermined distance based on the captured image.

[0019] The calculation unit 12 calculates a first measurement value of the dimension of the object based on the image of the object included in the captured image. The calculation unit 12 calculates the dimension of the object based on, for example, the relationship between the number of pixels of the object included in the captured image and the distance to the object.

[0020] As shown in Fig. 2, the calculation unit 12 extracts the number of pixels of an object T1 included in a captured image M1 captured at a predetermined distance from the current position. In the figure, a part of the captured image M1 consisting of unit pixels Mm arranged in a 6 x 4 grid is excerpted and illustrated. The captured image M1 captures an image of the object T1. The dimensions corresponding to the unit pixels Mm of the captured image M1 are stored in advance as parameters in the storage unit 14 based on the design values ​​of the camera and lens.

[0021] The calculation unit 12 extracts the number of pixels of the object T1 at a predetermined distance from the captured image M1 and calculates the dimensions of the object. The first measurement value (x1, y1, z1) of the dimensions of the object T1 is calculated, for example, from the distance to the object T1: x1, the horizontal dimension of the object T1: y1, and the vertical dimension of the object T1: z1. For example, the calculation unit 12 calculates the first measurement value (x1, y1, z1) based on the correspondence between a predetermined pixel area (e.g., 2×2 pixels) on the image of the object T1 captured at the predetermined distance x1 in the captured image M1 including a plurality of unit pixels Mm and the actual dimensions. The correspondence relationship between the pixel area and the actual dimensions is an example, and is not limited to the numerical values ​​of the pixels illustrated, and other numerical values ​​may be applied.

[0022] The illustrated object T1 can be measured at a maximum pixel area (e.g., 3×3 pixels) depending on its position in the actual captured image M1. A first dimension of the object T1 can be calculated up to a dimension corresponding to the range of the maximum pixel area of ​​3×3 pixels. Thus, the first measurement value includes a first measurement error e1 based on the change in the position of the object T1 in the captured image M1. The correspondence relationship between the maximum pixel area and the actual dimensions is an example, and is not limited to the illustrated pixel values, and other values ​​may be applied.

[0023] The calculation unit 12 acquires a second measurement value of the dimension of the object T1 measured by the second measurement unit 4. The second measurement unit 4 detects a second measurement value (x2, y2, z2) including the distance to the object T1: x2, the horizontal dimension of the object T1: y2, and the vertical dimension of the object T1: z2. The second measurement value includes a second measurement error e2 that depends on the performance of the second measurement unit 4. The second measurement error e2 is calculated according to the magnitude of the second measurement value.

[0024] The calculation unit 12 calculates the error between a first measurement value of a first dimension of the object T1 at the distance X to the object T1 and a second measurement value of a second dimension of the object measured by the second measurement unit 4. The calculation unit 12 calculates the difference between the first measurement value and the second measurement value. The calculation unit 12 compares the difference obtained by subtracting the second measurement value from the first measurement value with the sum of the first measurement error e1 and the second measurement error e2.

[0025] The calculation unit 12 determines whether or not the difference obtained by subtracting the second measurement value from the first measurement value is equal to or less than the sum of the first measurement error e1 and the second measurement error e2 (equal to or less than a predetermined value). If the difference obtained by subtracting the second measurement value from the first measurement value is equal to or less than the sum of the first measurement error e1 and the second measurement error e2, the calculation unit 12 determines whether or not the second dimension of the object based on the second measurement value matches the first dimension within a predetermined reference range. For example, the calculation unit 12 determines whether or not the horizontal dimension y2 of the second measurement value matches the number of pixels corresponding to the horizontal dimension y1 of the object in the captured image M1.

[0026] The calculation unit 12 may determine whether the vertical dimension z2 of the second measurement value matches the number of pixels corresponding to the vertical dimension z1 of the object in the captured image M1. When the second dimension of the object based on the second measurement value measured at a predetermined distance matches the first dimension within a predetermined reference range, the calculation unit 12 sets the second measurement value as the dimension value of the object.

[0027] The calculation unit 12 generates three-dimensional data of the space around the vehicle 1 based on the set dimensional values ​​of the object. The calculation unit 12 controls the vehicle 1 based on the calculated three-dimensional data. Based on the calculated three-dimensional data, the calculation unit 12 can travel in a space where a plurality of objects of different sizes exist. Based on the above processing, the calculation unit 12 can recognize objects whose length or size cannot be assumed by the vehicle 1 through pattern matching.

[0028] Based on the above process, the vehicle 1 can travel on rough ground whose shape and dimensions cannot be assumed by pattern matching. Based on the above process, the vehicle 1 can travel on rough ground including, for example, rocks, flat ground, sloping ground, terrain with depressions, etc. Based on the above process, the vehicle 1 can measure the dimensions of the target object based on different measuring means of the first measuring unit 2 and the second measuring unit 4, even if the shape and dimensions of the object cannot be assumed by pattern matching, and can determine that the measurement value of either one of the two measuring means is relatively normal.

[0029] 3 shows a process flow of a vehicle control method executed in the vehicle control device 10. The vehicle control method is executed based on a computer program installed in a computer mounted on the vehicle control device 10. The program causes the computer to execute each of the following processes.

[0030] The calculation unit 12 detects the initial position of the vehicle 1 based on the detection value of the position detection unit 6 (step S100). The calculation unit 12 acquires the detection values ​​of the sensors of the position detection unit 6, the first measurement unit 2, and the second measurement unit 4 (step S102). The calculation unit 12 calculates the current position of the vehicle 1 based on the detection value of the position detection unit 6 (step S104). The calculation unit 12 calculates the dimensions of an object present around the vehicle 1 based on the first measurement value of the first measurement unit 2 and the second measurement value of the second measurement unit 4, and generates three-dimensional data of the space around the vehicle 1 based on the calculated dimensions of the object (step S106). The calculation unit 12 calculates a target speed of the vehicle 1 based on the calculated three-dimensional data, and controls the drive unit 8 to run the vehicle 1 (step S108).

[0031] 4 shows the flow of the calculation process executed in step S106. The calculation unit 12 determines whether or not the difference between the first measurement value of the first measurement unit 2 and the second measurement value of the second measurement unit 4 is equal to or less than the sum of the first measurement error e1 and the second measurement error e2 of the first measurement value (step S200).

[0032] If the determination in step S200 is positive, the calculation unit 12 determines whether or not a second dimension of the object based on a second measurement value measured at a predetermined distance from a first dimension of the object based on the number of pixels of the object included in the captured image captured at a predetermined distance matches within a predetermined reference range (for example, within a range between a predetermined pixel area and a maximum pixel area) (step S202). If the determination in step S202 is positive, the calculation unit 12 sets the second measurement value as the dimension of the object (step S204).

[0033] The calculation unit 12 generates three-dimensional data of the space around the vehicle 1 based on the dimensions of the object (step S206). If the determination in step S202 is negative, the calculation unit 12 sets the first measurement value as the dimension of the object. The calculation unit 12 repeatedly executes the processes of steps S200 to S206 to update the three-dimensional data of the space around the vehicle 1.

[0034] If the difference between the first measurement value and the second measurement value exceeds a predetermined error level in step S200, the calculation unit 12 determines that an abnormality has occurred in either the first measurement value or the second measurement value, and outputs a predetermined notification to the external device 20 via the network W (step S210). The calculation unit 12 may cause the display unit 18 of the vehicle 1 to display a notification informing the user of the abnormality.

[0035] As described above, the vehicle control device 10 can calculate the dimensions of an object whose dimensions cannot be assumed by pattern matching, using the first measurement value of the first measurement unit 2 and the second measurement value of the second measurement unit 4. The vehicle control device 10 can use the first measurement value of the first measurement unit 2 and the second measurement value of the second measurement unit 4 to determine whether or not one of the measurement values ​​is normal.

[0036] In the above-described embodiment, the computer program executed in each component of the vehicle control device 10 may be provided in a form recorded on a computer-readable portable recording medium, such as a semiconductor memory, a magnetic recording medium, or an optical recording medium. [Explanation of symbols]

[0037] 1 vehicle 2. First Measurement Section 4. Second Measurement Section 6 Position detection section 8 Drive unit 10 Vehicle control device 12 Arithmetic section 14 Storage section 16 Communications Department 18 Display 20 External device

Claims

1. a calculation unit that calculates a size of the object based on a captured image of the space around the vehicle captured by a first measurement unit and a measurement value of a second measurement unit that measures the object around the vehicle based on electromagnetic waves, The calculation unit is calculating a first measurement of a dimension of the object based on an image of the object included in the captured image; When an error between the first measurement value and a second measurement value of the dimension of the object measured by the second measuring unit is equal to or smaller than a predetermined value, the second measurement value is set as a dimension value of the object; generating three-dimensional data of the space based on the dimension values; Vehicle control device.

2. The calculation unit is calculating a first dimension of the object based on a number of pixels of the object included in the captured image captured at a predetermined distance; When a second dimension of the object based on the second measurement value measured at the predetermined distance matches the first dimension within a predetermined reference range, the second measurement value is set as the dimension value of the object. The vehicle control device according to claim 1.

3. The calculation unit is When a second dimension of the object based on the second measurement value measured at the predetermined distance exceeds a predetermined standard for the first dimension, the first measurement value is set as the dimension value of the object. The vehicle control device according to claim 2.

4. The calculation unit is When a difference between the first measurement value and the second measurement value exceeds a predetermined error level, it is determined that an abnormality has occurred in either the first measurement value or the second measurement value, and a predetermined notification is output. The vehicle control device according to claim 1.

5. A program installed in a computer mounted on a vehicle control device, Obtaining an image of a space around the vehicle by a first measurement unit; measuring an object present around the vehicle based on electromagnetic waves by a second measurement unit; calculating a first measurement of a dimension of the object based on an image of the object included in the captured image; When an error between the first measurement value and a second measurement value of the dimension of the object measured by the second measuring unit is equal to or smaller than a predetermined value, the second measurement value is set as a dimension value of the object; A program that causes a computer to execute a process of generating three-dimensional data of the space based on the dimensional values.

Images