Object recognition device, computer program, and recording medium

Abstract

To provide an object recognition apparatus, a computer program, and a recording medium capable of improving the reliability of object recognition results while reducing the arithmetic processing load of a conventional system dependent on an imaging device.SOLUTION: An object recognition apparatus recognizes an object by using observation data of a distance measuring sensor which receives a reflected wave of an irradiated electromagnetic wave and image data generated by an imaging device, sets object existence areas where objects may exist based on the observation data of the distance measuring sensor, estimates attributes of the objects which may exist in each object existence area, sets, based on the object attributes, a processing load level to be spent for object recognition processing performed using at least the image data for the object existence area, and performs processing including performing the object recognition processing according to the set processing load level for each object existence area.SELECTED DRAWING: Figure 8

Description

Technical Field

[0001] The present disclosure relates to an object recognition device applicable to a vehicle, a computer program, and a recording medium.

Background Art

[0002] In recent years' vehicle automatic driving technologies and advanced driver assistance systems, development of object recognition technologies that utilize imaging devices such as monocular cameras or stereo cameras, and ranging sensors that use electromagnetic waves such as LiDAR (Light Detection And Ranging or Laser Imaging Detection And Ranging) or millimeter-wave radars has been underway.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, currently, systems that rely on either one of the imaging device or the ranging sensor are mainstream. By utilizing both the imaging device and the ranging sensor, it is expected to duplicate the system to enhance the reliability of the object recognition result, reduce the hardware load by reducing the computational processing load, and ultimately improve the safety of the system.

[0005] For example, in the case of a system that largely depends on an imaging device as an object recognition technology, it is required to ensure safety by performing object recognition processing, type estimation processing, relative speed estimation processing, etc. that utilize the hardware resources of the imaging device over the entire field of view of a person. However, in order to achieve such requirements only with the technology of the imaging device, it is necessary to increase the hardware resources. Specifically, since the imaging device scans the entire observation range in front of the vehicle and executes each of the above processes, in order to execute each of the above processes without increasing the hardware resources, it is necessary to reduce the resolution, such as reducing the number of pixels, which may reduce the reliability. This may become a factor limiting the realization of future diverse processes, function expansion, and improvement of system reliability.

[0006] The present disclosure has been made in view of the above problems, and an object of the present disclosure is to provide an object recognition device, a computer program, and a recording medium capable of improving the reliability of an object recognition result while reducing the computational processing load of a conventional system that depends on an imaging device.

Means for Solving the Problem

[0007] In order to solve the above problems, according to an aspect of the present disclosure, there is provided an object recognition device that recognizes an object using observation data of a distance measurement sensor that receives a reflected wave of an irradiated electromagnetic wave and image data generated by an imaging device, the object recognition device including: one or more processors; and one or more memories communicably connected to the one or more processors, wherein the processor sets an object existence region where an object may exist based on the observation data of the distance measurement sensor, estimates an attribute of an object that may exist in each object existence region, sets a level of a processing load to be spent on object recognition processing executed using at least the image data for each object existence region based on the attribute of the object, and executes a process including performing object recognition processing corresponding to the set level of the processing load for each object existence region.

[0008] Also, in order to solve the above problems, according to another aspect of the present disclosure, there is provided an object recognition device that recognizes an object using observation data of a distance measurement sensor that receives a reflected wave of an irradiated electromagnetic wave and image data generated by an imaging device. The object recognition device includes: an object existence region setting unit that sets an object existence region where an object may exist based on the observation data of the distance measurement sensor, and estimates an attribute of an object that may exist in each of the object existence regions; a processing load level setting unit that sets a level of a processing load to be spent on an object recognition process that is executed using at least the image data for each of the object existence regions based on the attribute of the object; and an object recognition processing unit that performs the object recognition process corresponding to the set processing load level for each of the object existence regions.

[0009] Also, in order to solve the above problems, according to another aspect of the present disclosure, a computer program is provided that causes a processor to execute operations including: setting an object existence region where an object may exist based on observation data of a distance measurement sensor that receives a reflected wave of an irradiated electromagnetic wave; estimating an attribute of an object that may exist in each of the object existence regions; setting a level of a processing load to be spent on an object recognition process that is executed using at least image data generated by an imaging device for each of the object existence regions based on the attribute of the object; and performing the object recognition process corresponding to the set processing load level for each of the object existence regions.

[0010] Also, in order to solve the above problems, according to another aspect of the present disclosure, a processor is caused to set an object existence region where an object may exist based on observation data of a distance measurement sensor that receives a reflected wave of an irradiated electromagnetic wave, estimate an attribute of an object that may exist in each of the object existence regions, set a level of a processing load to be spent on an object recognition process that is executed using at least image data generated by an imaging device for the object existence region based on the attribute of the object, and perform the object recognition process corresponding to the set level of the processing load for each of the object existence regions. There is provided a recording medium storing a computer program that causes an operation including these to be executed.

Advantages of the Invention

[0011] As described above, according to the present disclosure, it is possible to provide an object recognition device for a vehicle that can improve the reliability of an object recognition result while reducing the load of arithmetic processing of a conventional system that depends on an imaging device.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Embodiments for Carrying Out the Invention

[0013] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted.

[0014] <1. Overall Configuration of the Vehicle> First, an example of the overall configuration of a vehicle to which the object recognition device according to the embodiment of the present disclosure can be applied will be described. In the following embodiments, an example of an object recognition device using LiDAR as an example of a distance measurement sensor will be described.

[0015] FIG. 1 is a schematic diagram showing a configuration example of a vehicle 1 equipped with an object recognition device 50 according to the present embodiment. The vehicle 1 shown in FIG. 1 is configured as a four-wheel drive vehicle that transmits the driving torque output from a driving power source 9 that generates the driving torque of the vehicle to the left front wheel 3LF, the right front wheel 3RF, the left rear wheel 3LR, and the right rear wheel 3RR (hereinafter, collectively referred to as "wheel 3" when no particular distinction is required). The driving power source 9 may be an internal combustion engine such as a gasoline engine or a diesel engine, may be a driving motor, or may be provided with both an internal combustion engine and a driving motor.

[0016] Note that the vehicle 1 may be, for example, an electric vehicle equipped with two driving motors, a front-wheel driving motor and a rear-wheel driving motor, or an electric vehicle equipped with a driving motor corresponding to each wheel 3. Further, when the vehicle 1 is an electric vehicle or a hybrid electric vehicle, the vehicle 1 is equipped with a secondary battery that stores electric power supplied to the driving motor, and a generator such as a motor or a fuel cell that generates electric power for charging the battery.

[0017] Vehicle 1 includes, as devices used for driving control of Vehicle 1, a driving force source 9, an electric power steering device 15, and brake devices 17LF, 17RF, 17LR, 17RR (hereinafter, collectively referred to as "brake device 17" when no particular distinction is required). The driving force source 9 outputs driving torque that is transmitted to the front-wheel drive shaft 5F and the rear-wheel drive shaft 5R via a transmission (not shown), a front-wheel differential mechanism 7F, and a rear-wheel differential mechanism 7R. The driving of the driving force source 9 and the transmission is controlled by a vehicle control device 41 configured to include one or more electronic control units (ECUs: Electronic Control Unit).

[0018] An electric power steering device 15 is provided on the front-wheel drive shaft 5F. The electric power steering device 15 includes an electric motor (not shown) and a gear mechanism, and adjusts the steering angles of the left front wheel 3LF and the right front wheel 3RF by being controlled by the vehicle control device 41. During manual driving, the vehicle control device 41 controls the electric power steering device 15 based on the steering angle of the steering wheel 13 by the driver. Further, during autonomous driving, the vehicle control device 41 controls the electric power steering device 15 based on a target steering angle set according to a planned travel trajectory capable of avoiding a collision with an obstacle.

[0019] The brake devices 17LF, 17RF, 17LR, 17RR apply braking force to the front, rear, left, and right drive wheels 3LF, 3RF, 3LR, 3RR, respectively. The brake device 17 is configured as, for example, a hydraulic brake device, and generates a predetermined braking force by the hydraulic pressure supplied to each brake device 17 being controlled by the vehicle control device 41. When Vehicle 1 is an electric vehicle or a hybrid electric vehicle, the brake device 17 is used in combination with regenerative braking by the drive motor.

[0020] The vehicle control device 41 includes one or more electronic control devices that control the driving of a driving force source 9 that outputs the driving torque of the vehicle 1, an electric steering device 15 that controls the steering angle of the steering wheel 13 or the steered wheels, and a brake device 17 that controls the braking force of the vehicle 1. The vehicle control device 41 may have a function of controlling the driving of a transmission that shifts the output from the driving force source 9 and transmits it to the wheels 3. The vehicle control device 41 is configured to be able to execute automatic driving control and emergency braking control of the vehicle 1 using information on obstacles recognized by the object recognition device 50.

[0021] Further, the vehicle 1 includes a pair of left and right front cameras 31LF, 31RF, a LiDAR 31S, a vehicle state sensor 35, a GPS (Global Positioning System) sensor 37, an HMI (Human Machine Interface) 43, and an object recognition device 50. The vehicle state sensor 35, the GPS sensor 37, the HMI 43, and the object recognition device 50 are connected to the vehicle control device 41 via a dedicated line or via communication means such as CAN (Controller Area Network) or LIN (Local Inter Net).

[0022] The front cameras 31LF, 31RF capture the front of the vehicle 1 and generate image data. The front cameras 31LF, 31RF are imaging devices equipped with imaging elements such as CCD (Charged-Coupled Devices) or CMOS (Complementary Metal-Oxide-Semiconductor). The front cameras 31LF, RF are communicably connected to the object recognition device 50 by wired or wireless communication means, and transmit the generated image data to the object recognition device 50.

[0023] In the vehicle 1 shown in FIG. 1, the front cameras 31LF and 31RF are configured as a stereo camera including a pair of left and right cameras, but may also be a monocular camera consisting of a single imaging camera. In addition to the front cameras 31LF and 31RF, the vehicle 1 may be provided with a camera for photographing the rear of the vehicle 1, or a camera provided on a side mirror or the like for photographing the left rear or right rear.

[0024] LiDAR 31S is an aspect of a distance measuring sensor. It irradiates laser light (optical wave), which is a kind of electromagnetic wave, in a plurality of directions in front of the vehicle 1 and receives the reflected light (reflected wave) of the laser light, and detects the three-dimensional position of the reflection point based on the data of the received reflected light. For example, LiDAR 31S may be a ToF (Time of Flight) type LiDAR that detects the three-dimensional position of each reflection point based on the data of the direction in which the reflected light is received and the data of the time from when the laser light is irradiated until the reflected light is received. LiDAR 31S may further detect the three-dimensional position of the reflection point based on the intensity of the reflected light. Also, LiDAR 31S may be an FMCW (Frequency Modulated Continuous Wave) type LiDAR that emits light with a linearly changing frequency and detects the three-dimensional position of each reflection point based on the data of the direction in which the reflected light is received and the data of the phase difference with the frequency of the reflected light. LiDAR 31S is provided, for example, at the upper part of the front window in the vehicle interior or the front part of the vehicle body so as to be able to irradiate laser light forward.

[0025] LiDAR 31S may be a so-called scanning type LiDAR that scans a plurality of laser lights arranged in a line along the vertical direction or the horizontal direction in the horizontal direction or the vertical direction, irradiates the laser light over a wide range, images the reflected light reflected by an object with a three-dimensional distance image sensor, and generates data of a reflection point group by analyzing the three-dimensional position of the reflection point. LiDAR 31S is communicably connected to the object recognition device 50 by wired or wireless communication means, and LiDAR 31S transmits the data of the detected reflection point group to the object recognition device 50.

[0026] Note that the distance measurement sensor using electromagnetic waves is not limited to LiDAR31S, and other sensors such as radar sensors like millimeter wave radars or ultrasonic sensors may also be used.

[0027] The vehicle state sensor 35 consists of one or more sensors that detect the operating state and behavior of the vehicle 1 (hereinafter collectively referred to as the "driving state of the vehicle"). The vehicle state sensor 35 includes, for example, at least one of a steering angle sensor, an accelerator position sensor, a brake stroke sensor, a brake pressure sensor, or an engine speed sensor, and detects the operating state of the vehicle 1 such as the steering angle of the steering wheel 13 or the steering wheel, the accelerator opening, the brake operation amount, or the engine speed. Further, the vehicle state sensor 35 includes, for example, at least one of a vehicle speed sensor, an acceleration sensor, or an angular velocity sensor, and detects the behavior of the vehicle such as the vehicle speed, longitudinal acceleration, lateral acceleration, or yaw rate. The vehicle state sensor 35 transmits a sensor signal including the detected information to the vehicle control device 41.

[0028] In this embodiment, the vehicle state sensor 35 includes at least a vehicle speed sensor. The vehicle speed sensor may be an encoder that detects the wheel speed, an encoder that detects the rotation speed of the drive motor as the drive power source 9, or a laser Doppler sensor. Alternatively, the vehicle speed sensor may be a speed estimation module that utilizes the self-position estimation technology by SLAM (Simultaneous Localization And Mapping) using the distance measurement sensor 31 or a camera or the like.

[0029] The GPS sensor 37 receives satellite signals from GPS satellites. The GPS sensor 37 transmits the position information of the vehicle 1 on the map data included in the received satellite signals to the vehicle control device 41. Note that instead of the GPS sensor 37, an antenna that receives satellite signals from other satellite systems for specifying the position of the vehicle 1 may be provided.

[0030] The HMI 43 is driven by the vehicle control device 41 and presents various information to the passengers by means such as image display and voice output. The HMI 43 includes, for example, a display device provided in the instrument panel and a speaker provided in the vehicle 1. The display device may be a display device of the navigation system.

[0031] <2. Object Recognition Device> Subsequently, the object recognition device 50 according to the present embodiment will be specifically described.

[0032] (2-1. Configuration Example) FIG. 2 is a block diagram showing a configuration example of the object recognition device 50 according to the present embodiment. The object recognition device 50 includes a control unit 51 and a storage unit 53. The control unit 51 is configured to include one or more processors such as a CPU (Central Processing Unit). Part or all of the control unit 51 may be configured with updatable components such as firmware, or may be program modules executed according to instructions from a CPU or the like.

[0033] The storage unit 53 is composed of one or more storage elements (memories) such as a RAM (Random Access Memory) or a ROM (Read Only Memory) that is communicably connected to the control unit 51. However, the number and type of the storage unit 53 are not particularly limited. The storage unit 53 stores computer programs executed by the control unit 51, various parameters used for arithmetic processing, detection data, arithmetic results, and other data. In addition, the object recognition device 50 includes an interface (not shown) for transmitting and receiving data between the front cameras 31LF and 31RF, the LiDAR 31S, and the vehicle control device 41.

[0034] (2-2. Functional Configuration) The control unit 51 of the object recognition device 50 executes object recognition processing based on the data transmitted from the front cameras 31LF, RF and the LiDAR 31S. In the technology of the present disclosure, the control unit 51 sets an object presence area where an object may exist based on the detection data by the LiDAR 31S, and sets the level of the processing load spent on the object recognition processing executed for each object presence area. Then, the control unit 51 executes object recognition processing corresponding to the set processing load level for each object presence area.

[0035] Although the object recognition processing using the observation data of the LiDAR 31S has lower accuracy than the object recognition processing using the image data generated by the imaging device, the existence of some object itself is estimated with high accuracy. For this reason, in the technology of the present disclosure, an object presence area is set based on the observation data of the LiDAR 31S, and the resources of the object recognition processing using at least the image data of the front cameras 31LF, 31RF are intensively invested in the object presence area.

[0036] In the following description, the "detection range" of the LiDAR 31S is an area where object recognition processing using the detection data by the LiDAR 31 is executed, and is an area obtained by projecting the irradiation range of the laser light in the real space onto a two-dimensional plane in the vertical and horizontal directions. The vertical direction and the horizontal direction are the vertical direction and the horizontal direction for the LiDAR 31S, and are set in advance for the LiDAR 31S. The LiDAR 31S is attached so that the vertical direction is parallel to the height direction of the vehicle 1 and the horizontal direction is parallel to a plane perpendicular to the height direction of the vehicle 1. Alternatively, the vertical direction and the horizontal direction may be configured to be adjustable in an arbitrary direction by coordinate transformation of the three-dimensional space of the LiDAR 31S.

[0037] As shown in FIG. 2, the control unit 51 of the object recognition device 50 includes an object presence area setting unit 61, a processing load level setting unit 63, and an object recognition processing unit 65. The object presence area setting unit 61, the processing load level setting unit 63, and the object recognition processing unit 65 are functions realized by executing a computer program by one or more processors such as a CPU. Note that part or all of the object presence area setting unit 61, the processing load level setting unit 63, and the object recognition processing unit 65 may be configured by hardware such as an analog circuit.

[0038] (Object Presence Area Setting Unit) The object presence area setting unit 61 sets an object presence area where an object may exist based on the observation data of the LiDAR 31S, and executes a process of estimating the attributes of the objects that may exist in each object presence area. For example, the object presence area setting unit 61 acquires data of the reflection point group of the reflected light received by the LiDAR 31S from the LiDAR 31S, and determines whether an object exists in front of the vehicle 1 based on the data of the reflection point group. The object presence area setting unit 61 sets the area determined to have an object as the object presence area.

[0039] The data of the reflection point group includes information on the three-dimensional position of the detected reflection points based on the data of the direction of the reflection points viewed from the vehicle 1 and the data of the time from when the LiDAR 31S irradiates the laser light until the reflected light is received. The data of the reflection point group may further include data on the intensity of the received reflected light. For example, the object presence area setting unit 61 executes a clustering process of grouping reflection points whose distance between the reflection points is closer than a predetermined distance based on the information on the three-dimensional position of the reflection point group. Then, when the area of the area where the grouped reflection point group exists exceeds a predetermined range set in advance, the object presence area setting unit 61 sets the area as the object presence area.

[0040] In addition, the object presence area setting unit 61 estimates the attributes of the objects that may exist in each object presence area. The attributes of the objects include information on the distance from the vehicle 1 to the object, the size of the object, and the moving speed of the object. The information on the moving speed of the object may be information on the relative speed between the vehicle 1 and the object. The object presence area setting unit 61 can calculate the distance from the position of each object presence area in the detection range of the LiDAR 31S to the object. Further, the object presence area setting unit 61 can calculate the size of the object as seen from the vehicle 1 based on the size of the outer shape of the grouped reflection point group and the distance to the object in the detection range of the LiDAR 31S. Alternatively, when the irradiation density of the laser light by the LiDAR 31S is uniform, the object presence area setting unit 61 can calculate the size of the object as seen from the vehicle 1 based on the number of the grouped reflection point groups.

[0041] In addition, the object presence area setting unit 61 can calculate the relative speed between the vehicle 1 and the object in the direction crossing the traveling direction of the vehicle 1 based on the time change of the distance to the object calculated in time series. Further, the object presence area setting unit 61 can calculate the relative speed between the vehicle 1 and the object in the traveling direction of the vehicle 1 based on the time change of the distance to the object or the time change of the size of the object. Furthermore, based on the relative speed of the object in the direction crossing the traveling direction of the vehicle 1 or the relative speed of the object in the traveling direction of the vehicle 1, and the moving speed of the vehicle 1 in each direction, the moving speed of the object in each direction can be calculated. The time change of the distance to the object or the time change of the size of the object may be calculated, for example, by comparing the image frame of the inspection range generated by one scanning process of the LiDAR 31S with the image frames 2 to 30 frames before. The number of frame intervals can be appropriately set based on the reliability of the LiDAR 31S or the intensity of the reflected light, etc.

[0042] However, the content of the process of setting the object existence area by the object existence area setting unit 61 is not limited to the above-described example. For example, the object existence area setting unit 61 may set the object existence area along the edge of the acquired reflection point group, or may set an area obtained by expanding a predetermined size with respect to the edge as the object existence area. Similarly, the content of the process of estimating the attributes of the object by the object existence area setting unit 61 is not limited to the above-described example.

[0043] (Processing load level setting unit) The processing load level setting unit 63 executes a process of setting the level of the processing load to be expended on the object recognition process executed using at least the image data generated by the front cameras 31LF and 31RF with respect to the object existence area, based on the attributes of the object calculated by the object existence area setting unit 61. The level of the processing load is set according to the collision risk of the object estimated from the attributes of the objects that may exist in each object existence area, and the higher the collision risk, the higher the level is set. The collision risk is a concept representing the possibility of collision between the object and the vehicle 1 or the magnitude of the damage assumed at the time of collision.

[0044] In the present embodiment, the processing load level setting unit 63 sets the level of the processing load to either a steady processing level with a low processing load or a fine processing level with a processing load higher than the steady processing level. The fine processing level is further divided into a low fine processing level with a relatively low processing load and a high fine processing level with a relatively high processing load.

[0045] In the present embodiment, the processing load level setting unit 63 assigns points to each of the calculated distance to the object, the size of the object, and the relative speed or moving speed of the object, and sets the processing load level according to the sum of the points. Table 1 shows an example of assigning points according to the distance to the object, the size of the object, and the moving speed of the object. Table 2 shows an example of setting the processing load level according to the sum of the points.

[0046] [Table 1]

[0047]

Table 2

[0048] In the examples shown in Table 1 and Table 2, the higher the collision risk, the higher the points assigned, and the larger the sum of the points, the higher the processing content with a higher load is set. That is, the greater the collision risk, the higher the processing load is set so that the accuracy of the object recognition process can be improved by intensively consuming hardware resources. Also, since the fine processing level is divided into a low fine processing level and a high fine processing level, appropriate hardware resources can be consumed according to the collision risk, and the object recognition process can be executed with the required accuracy.

[0049] The processing load level setting unit 63 assigns points to the object existence area for each processing process repeated at a predetermined cycle and sets the processing load level. When only moving objects are detected excluding stationary objects, the sum of the points for an object with a calculated moving speed of zero may be set to 0. Also, according to the course of the vehicle 1 estimated from the steering wheel or the steering angle of the steering wheel of the vehicle 1, an object existence area where a collision with the vehicle 1 cannot be assumed may be processed as non-existent. Thereby, the load of the arithmetic processing of the control unit 51 can be reduced without reducing the safety against collisions.

[0050] Note that the attributes of the object are only examples, and the processing load level may be set based on other information related to the collision risk. For example, the processing load level setting unit 63 may estimate the type of the object based on the size, speed, etc. of the object, and set the processing load level according to the estimated type of the object. Specifically, when the type of the object is a person such as a pedestrian, the processing load level may be set higher than that of other objects to enhance the safety for the person. Also, the processing load level setting unit 63 may calculate the kinetic energy of the object based on the weight and speed of the object estimated from the type, size, etc. of the object, and assign a higher point as the kinetic energy is larger. Also, the setting examples shown in Table 1 and Table 2 are only examples, and the thresholds for classifying the distance, size, or speed or relative speed, and the number of points to be assigned may be changed as appropriate. Also, in the example shown in Table 1, the same points are assigned in three steps for each of the distance, size, and speed, but weighting may be performed for specific items.

[0051] (Object recognition processing unit) The object recognition processing unit 65 executes object recognition processing using at least the image data generated by the front shooting cameras 31LF and 31RF according to the level of the processing load set by the processing load level setting unit 63 for each of the object existence regions set by the object existence region setting unit 61.

[0052] The object recognition processing unit 65 is configured to be able to mutually perform coordinate conversion between the three-dimensional coordinates of the observation space by the LiDAR 31S and the three-dimensional coordinates of the observation space by the front shooting cameras 31LF and 31RF. When the installation positions of the LiDAR 31S and the front shooting cameras 31LF and 31RF are aligned so that the three-dimensional coordinates of the observation space by the LiDAR 31S and the three-dimensional coordinates of the observation space by the front shooting cameras 31LF and 31RF match, the coordinate conversion function may not be provided.

[0053] FIG. 3 is an explanatory diagram showing the object recognition processing executed by the object recognition processing unit 65 in this embodiment for each processing load level. The object recognition processing unit 65 reduces the number of processes to be executed on the object existence area set at the normal processing level compared to the number of processes to be executed on the object existence areas set at the low-resolution processing level and the high-resolution processing level. Therefore, the hardware resources consumed for the object recognition processing on the object existence areas with relatively low necessity are reduced.

[0054] In the present embodiment, the object recognition processing unit 65 estimates the type of an object for the object existence area set at the normal processing level by using image data to perform white line recognition processing, curb recognition processing, sign recognition processing, and moving object detection processing by feature point matching, and executes processing for estimating the distance to the object. The white line recognition processing, curb recognition processing, and sign recognition processing may be executed, for example, by determining whether an image corresponding to the white line image data, curb image data, and sign image data stored in the storage unit 53 in advance exists within the object existence area set at the normal processing level, but may also be executed by other methods.

[0055] In addition, the moving object detection processing by feature point matching is executed by determining whether an image corresponding to the image data of the moving object stored in the storage unit 53 in advance exists within the object existence area set at the normal processing level. For example, the object recognition processing unit 65 compares the respective image data generated by the left and right front cameras 31LF and 31RF with the image data of the moving object stored in the storage unit 53 to identify the corresponding moving object. When the front camera is a monocular camera, the object recognition processing unit 65 compares the image data generated by the monocular camera with the template image data of the classified moving object to identify the corresponding moving object. Examples of the moving object include a commercial vehicle, a four-wheel vehicle, a two-wheel vehicle, a bicycle, and a pedestrian. Data of other moving objects may also be included. Further, the distance estimation processing is executed based on the parallax information when the object specified by, for example, white line recognition processing, curb recognition processing, sign recognition processing, and moving object detection processing by feature point matching is photographed by the left and right front cameras 31LF and 31RF.

[0056] The processes of estimating the type of an object and the distance to the object by these white line recognition process, curb recognition process, sign recognition process, moving object detection process by feature point matching, etc. are arithmetic processes with relatively low load. That is, for an object existence area where an object is presumed to exist but the collision risk is low, the type of the object and the distance to the object are estimated by an arithmetic process method with low processing load, while the moving speed, size, and kinetic energy of the object are not estimated.

[0057] In addition, the object recognition processing unit 65 executes a process of detecting an edge or contour of an object from image data (edge detection process) for an object existence area set at a low-resolution processing level and a high-resolution processing level, and executes a process of estimating the type of the object (type estimation process), a process of estimating the distance to the object (distance estimation process), and a process of estimating the speed of the object (speed estimation process) based on the extracted edge or contour. Further, the object recognition processing unit 65 executes a process of estimating the size of the object (size estimation process) and a process of estimating the kinetic energy of the object (energy estimation process) based on the information of the type of the object, the distance to the object, and the speed of the object estimated for the object existence area set at the low-resolution processing level and the high-resolution processing level. The process of selecting a processing method with low load when executing the type estimation process, distance estimation process, speed estimation process, size estimation process, and energy estimation process is a low-load process, and the process of selecting a processing method with high load is a high-load process.

[0058] For example, the object recognition processing unit 65 executes a process of detecting edges or contours from the image data for the object presence area set at the low-resolution processing level, and also executes a process of extracting the features of the shape set for each object based on the extracted edge or contour data, and estimates the type of the object. Alternatively, the object recognition processing unit 65 may execute a process of detecting edges or contours from the image data for the object presence area set at the low-resolution processing level, and execute a shape pattern matching process set for each object based on the extracted edge or contour data, and estimate the type of the object. The process of extracting the edges or contours of the object and estimating the type of the object is a computationally intensive process with a higher load compared to the white line recognition process, curb recognition process, sign recognition process, and moving object detection process by feature point matching executed at the steady processing level.

[0059] On the other hand, the object recognition processing unit 65 executes a process of detecting edges or contours from the image data for the object presence area set at the high-resolution processing level, and in combination with the process of extracting the features of the shape set for each object based on the extracted edge or contour data, executes a process of estimating the type of the object by machine learning. The process of estimating the type of the object by machine learning is performed by inputting the extracted edge or contour data into an object edge model that has performed machine learning using the edge or contour data of the moving object to be learned in advance and the data of the type of the moving object as learning data, and obtaining the information on the type of the object output. The machine learning model may be a computational model using a neural network such as a support vector machine, a nearest neighbor method, deep learning, or a Bayesian network. By also executing the process of estimating the type of the object by the machine learning, the estimation accuracy of the type of the object can be improved.

[0060] In addition, the object recognition processing unit 65 executes a speed estimation process based on the time change of the moving distance of the extracted edge or contour for the object presence area set at the low-resolution processing level and the high-resolution processing level. The time change of the moving distance of the edge or contour can be calculated by comparing the image frames generated in time series.

[0061] Furthermore, the object recognition processing unit 65 performs a process of estimating the size of an object and a process of estimating the kinetic energy of the object on the object presence areas set at the low-resolution processing level and the high-resolution processing level. The kinetic energy of the object (W = Fs = 1 / 2 × mv 2 ) is a factor that affects the magnitude of damage when colliding with the vehicle 1. The larger the mass m of the object and the faster the moving speed v of the object, the larger the value. At this time, at the high-resolution processing level, since the estimation accuracy of the object type is higher than that at the low-resolution processing level, the estimation accuracy of the size of the estimated object and the kinetic energy of the object can also be improved.

[0062] The object recognition processing unit 65 may be set to execute the object recognition process in the order of the sum of points being large, that is, in the order of the high-resolution processing level, the low-resolution processing level, and the steady-state processing level. Thereby, the high-resolution processing level has a higher software priority than the low-resolution processing level, and information on an object with a high collision risk can be calculated faster and more accurately.

[0063] Also, for an object once detected, the object recognition processing unit 65 may perform a labeling process (a process of assigning a label) and execute a process of tracing using the front cameras 31LF and 31RF or the LiDAR 31S, thereby omitting the process of estimating the type and size of the object. Thereby, the computational processing load of the object recognition device 50 can be further reduced.

[0064] In addition, when there is no object presence area set at the high-resolution processing level or no object presence area set at the high-resolution processing level and the low-resolution processing level, the object recognition processing unit 65 may perform part or all of the processing contents executed at the fine processing level on the object presence area set at the steady-state processing level. Thereby, the system can be duplicated in combination with the object recognition process by the LiDAR 31S, information that is not detected by the LiDAR 31S can be recognized, and it can contribute to reducing the collision risk.

[0065] Note that the content of the object recognition process according to the processing load level shown in FIG. 3 is merely an example and is not limited to the illustrated content.

[0066] Furthermore, the object recognition processing unit 65 may calculate the collision risk with the vehicle 1 based on the estimated object type, the distance to the object, the speed of the object, the size of the object, and the kinetic energy of the object. The object recognition processing unit 65 transmits the calculated object information to the vehicle control device 41. The vehicle control device 41 that has received the object information executes automatic driving control while avoiding a collision with the object, or executes emergency braking control or emergency steering control to avoid a collision with the object or reduce the impact at the time of collision.

[0067] (2-3. Operation of Object Recognition Device) Hereinafter, an example of the operation of the object recognition device 50 according to the present embodiment will be described with reference to a flowchart. FIGS. 4 and 5 are flowcharts showing the processes executed by the object recognition device 50.

[0068] First, when an in-vehicle system including the object recognition device 50 is activated (step S11), the object presence area setting unit 61 of the control unit 51 acquires the measurement data of the reflection point group transmitted from the LiDAR 31S (step S13). Next, the object presence area setting unit 61 sets an object presence area within the detection range of the LiDAR 31S based on the acquired reflection point group data (step S15). Specifically, the object presence area setting unit 61 executes a clustering process of grouping reflection points whose distance between reflection points is closer than a predetermined distance. Then, when the area of the region where the grouped reflection point group exists exceeds a predetermined range set in advance, the object presence area setting unit 61 sets the region as the object presence area.

[0069] Next, the object presence area setting unit 61 estimates the attributes of the objects that may exist in each of the set object presence areas (step S17). As described above, the object presence area setting unit 61 calculates the distance from the vehicle 1 to the object, the size of the object, and the information on the moving speed or relative speed of the object based on the data of the reflection point group detected by the LiDAR 31S.

[0070] Next, the processing load level setting unit 63 of the control unit 51 sets the level of the processing load to be consumed for the object recognition process executed using the image data generated by at least the front cameras 31LF and 31RF for each of the object presence areas based on the attributes of the objects estimated by the object presence area setting unit 61 (step S19). The level of the processing load is set according to the collision risk of the object estimated from the attributes of the objects that may exist in each of the object presence areas, and the higher the collision risk, the higher the level is set. In the present embodiment, the processing load level setting unit 63 sets the level of the processing load to any one of a steady processing level with a low processing load, a low-resolution processing level with a higher processing load than the steady processing level, or a high-resolution processing level.

[0071] Here, with reference to FIGS. 6 to 8, an example of the process from the setting of the object presence area to the setting of the processing load level will be described. FIG. 6 shows an example of the view in front of the vehicle 1. In the example shown in FIG. 6, in front of the vehicle 1, the first front vehicle 101, the second front vehicle 103, and the third front vehicle 105 are traveling in the same direction in order from the near side in the traveling direction. In addition, a two-wheeled vehicle 107 is traveling in the same direction on the left side of the second front vehicle 103. In addition, a bicycle 109 exists at the left end of the road, and a pedestrian 113 exists at the right end of the road. Furthermore, street trees 111 exist on both sides of the road.

[0072] FIG. 7 shows the object presence areas set by the object presence area setting unit 61. The object presence area setting unit 61 performs clustering processing on the data of the reflection point group detected by the LiDAR 31S, and sets the object presence areas so as to include each grouped reflection point group. Object presence areas 101a, 103a, and 105a are set at the positions where the first preceding vehicle 101, the second preceding vehicle 103, and the third preceding vehicle 105 exist, respectively. Also, object presence areas 107a, 109a, and 113a are set at the positions where the two-wheeled vehicle 107, the bicycle 109, and the pedestrian 113 exist, respectively. Furthermore, an object presence area 111a is set at the position where the street tree 111 exists. The set object presence area is not limited to a rectangular area set so as to include the grouped reflection point group, and may be a circular or elliptical area, or an area of any other appropriate shape.

[0073] FIG. 8 shows the levels of processing loads set for the respective object presence areas by the processing load level setting unit 63. For example, when the distance to the first preceding vehicle 101 is 40 m, the size of the first preceding vehicle 101 as seen from vehicle 1 is 3 m 2 and the speed of the first preceding vehicle 101 is 40 km / h, the processing load level setting unit 63 assigns 11 (5 + 3 + 3) points to the object presence area 101a corresponding to the first preceding vehicle 101 according to Table 1 above. Also, when the distance to the second preceding vehicle 103 is 60 m, the size of the second preceding vehicle 103 as seen from vehicle 1 is 3 m 2 and the speed of the second preceding vehicle 103 is 40 km / h, the processing load level setting unit 63 assigns 9 (3 + 3 + 3) points to the object presence area 103a corresponding to the second preceding vehicle 103 according to Table 1 above.

[0074] Also, when the distance to the third preceding vehicle 105 is 100 m, the size of the third preceding vehicle 105 as seen from vehicle 1 is 3 m 2When the speed of the third preceding vehicle 105 is 40 km / h, the processing load level setting unit 63 assigns 7 (1 + 3 + 3) points to the object presence area 105a corresponding to the third preceding vehicle 105 according to Table 1 above. Also, the distance to the motorcycle 107 is 60 m, and the size of the motorcycle 107 as seen from the vehicle 1 is 1.5 m 2 When the speed of the motorcycle 107 is 40 km / h, the processing load level setting unit 63 assigns 7 (3 + 1 + 3) points to the object presence area 107a corresponding to the motorcycle 107 according to Table 1 above.

[0075] Also, the distance to the bicycle 109 is 50 m, and the size of the bicycle 109 as seen from the vehicle 1 is 1 m 2 When the speed of the bicycle 109 is 5 km / h, the processing load level setting unit 63 assigns 5 (3 + 1 + 1) points to the object presence area 109a corresponding to the bicycle 109 according to Table 1 above. Also, the distance to the pedestrian 113 is 50 m, and the size of the pedestrian 113 as seen from the vehicle 1 is 0.8 m 2 When the speed of the pedestrian 113 is 2 km / h, the processing load level setting unit 63 assigns 5 (3 + 1 + 1) points to the object presence area 113a corresponding to the pedestrian 113 according to Table 1 above.

[0076] Also, since the speed of the roadside tree 111 is calculated to be 0 km / h, the processing load level setting unit 63 sets the sum of the points of the object presence area 111a corresponding to the roadside tree 111 to 0. As a result, the level of the processing load for the object presence area where the estimated moving speed of the object is zero is set to the level with the lightest processing load.

[0077] The processing load level setting unit 63 sets the processing load level for each object existence area according to Table 2 based on the sum of the given points. In the example shown in FIG. 8, the processing load level setting unit 63 sets the object existence areas 101a and 103a corresponding to the first preceding vehicle 101 and the second preceding vehicle 103 to the high-definition processing level. Also, the processing load level setting unit 63 sets the object existence areas 105a, 107a, 109a, and 113a corresponding to the third preceding vehicle 105, the motorcycle 107, the bicycle 109, and the pedestrian 113 to the low-definition processing level. Further, the processing load level setting unit 63 sets the object existence area 111a corresponding to the roadside tree 111 whose moving speed is zero and the sum of points is 0 to the steady processing level with the lightest processing load.

[0078] Returning to the flowchart of FIG. 4, after the processing load level of each object existence area is set in step S19, the object recognition processing unit 65 of the control unit 51 acquires the image data generated by the front cameras 31LF and 31RF (step S21). Next, the object recognition processing unit 65 executes object recognition processing for each object existence area according to the processing load level set by the processing load level setting unit 63 (step S23).

[0079] FIG. 5 is a flowchart showing the object recognition processing. First, the object recognition processing unit 65 determines whether there is an object existence area set to the high-definition processing level (step S31). If there is no object existence area set to the high-definition processing level (S31 / No), the object recognition processing unit 65 proceeds directly to step S35. On the other hand, if there is an object existence area set to the high-definition processing level (S31 / Yes), the object recognition processing unit 65 executes the processing (high-load processing) set to the processing content of the high-definition processing level for the corresponding object existence area (step S33). In the above-described example, the object recognition processing unit 65 executes high-load processing for the object existence areas 101a and 103a corresponding to the first preceding vehicle 101 and the second preceding vehicle 103 (see FIG. 3).

[0080] Next, the object recognition processing unit 65 determines whether there is an object presence area set at the low-resolution processing level (step S35). If there is no object presence area set at the low-resolution processing level (S35 / No), the object recognition processing unit 65 proceeds directly to step S39. On the other hand, if there is an object presence area set at the low-resolution processing level (S35 / Yes), the object recognition processing unit 65 executes the processing (low-load processing) set for the processing content at the low-resolution processing level on the corresponding object presence area (step S37). In the example described above, the object recognition processing unit 65 executes low-load processing on the object presence areas 105a, 107a, 109a, and 113a corresponding to the third preceding vehicle 105, the motorcycle 107, the bicycle 109, and the pedestrian 113 (see FIG. 3).

[0081] Next, the object recognition processing unit 65 executes the processing (steady processing) set for the processing content at the steady processing level on the object presence area set at the steady processing level (step S39). In the example described above, the object recognition processing unit 65 executes steady processing on the object presence area 111a corresponding to the street tree 111 (see FIG. 3).

[0082] In the object recognition processing shown in FIG. 5, since the object recognition processing is executed in the order of high-load processing, low-load processing, and steady processing, the priority of the processing for the area with high importance is increased, and the information of the object with a high collision risk can be calculated more quickly and accurately.

[0083] Also, in the present embodiment, in executing the object recognition processing, a labeling process (a process of assigning a label) is performed on the object detected in the previous processing process. Therefore, for an object once detected, the process of estimating the type and size of the object by using the front cameras 31LF and 31RF or the LiDAR 31S for tracing may be omitted. Thereby, the load of the arithmetic processing of the object recognition device 50 can be further reduced.

[0084] Also, when there is no object existence area set at the high-definition processing level or an object existence area set at both the high-definition processing level and the low-definition processing level, for the object existence area set at the steady processing level, part or all of the processing content executed at the high-definition processing level may be executed. As a result, the system can be duplicated in combination with the object recognition processing by LiDAR31S, information that cannot be detected by LiDAR31S can be recognized, and it can contribute to reducing the collision risk.

[0085] Returning to the flowchart of FIG. 4, after the object recognition processing according to the processing load level is executed, the object recognition processing unit 65 executes a labeling process and a process of attaching information to the detected object (step S25).

[0086] FIG. 9 shows an example of executing the labeling process and the process of attaching information. In the example shown in FIG. 9, for the first preceding vehicle 101 and the second preceding vehicle 103 for which high-load processing has been executed on the corresponding object existence areas 101a, 103a, information on the type of object (= car) is labeled, and information on the kinetic energy of the object is attached. Also, for the third preceding vehicle 105, the motorcycle 107, the bicycle 109, and the pedestrian 113 for which low-load processing has been executed on the corresponding object existence areas 105a, 107a, 109a, 113a, information on the kinetic energy of the object is attached. On the other hand, for the street tree 111 for which steady processing has been executed on the corresponding object existence area 111a, only information indicating the object existence area is attached.

[0087] Returning to the flowchart of FIG. 4, after the labeling process and the process of attaching information are executed, the object recognition processing unit 65 determines whether the in-vehicle system has stopped (step S27). If the in-vehicle system has stopped (S27 / Yes), the control unit 51 ends a series of processes. On the other hand, if the in-vehicle system has not stopped (S27 / No), it returns to step S13 and repeatedly executes the processes of each step described so far.

[0088] In this way, the object recognition device 50 according to the present embodiment sets the object existence area based on the observation data from the LiDAR 31S with relatively low reliability compared to the imaging device, and estimates the attributes of the objects that may exist in each object existence area. Based on the attributes of the objects, the level of processing load for the object recognition process executed using the image data generated by the front cameras 31LF and 31RF is set. Then, the object recognition device 50 executes the object recognition process corresponding to the set level of processing load for each object existence area.

[0089] As a result, the resources for the object recognition process using the image data of the front cameras 31LF and 31RF can be intensively allocated to the object existence areas with a high risk of collision with the vehicle 1. Therefore, while reducing the computational processing load of the conventional system that depends on the imaging device, the recognition accuracy of the objects with a high risk of collision with the vehicle 1 is ensured, and the reliability of the object recognition result can be improved.

[0090] Also, since the resources for the object recognition process can be intensively allocated to the areas with high necessity, the resources for various future processes can be secured, and the expansion of the functions of the object recognition device 50 can also be supported.

[0091] As described above, the preferred embodiments of the present disclosure have been described in detail with reference to the accompanying drawings, but the present disclosure is not limited to such examples. It is obvious that those with ordinary knowledge in the technical field to which the present disclosure belongs can conceive of various modification examples or correction examples within the scope of the technical idea described in the claims, and these are naturally understood to belong to the technical scope of the present disclosure.

[0092] For example, the object recognition processing unit 65 may reduce the resolution when performing arithmetic processing as a low-resolution processing level compared to the resolution when performing arithmetic processing as a high-resolution processing level. Specifically, the object recognition processing unit 65 may irradiate the area to be irradiated by the LiDAR 31S at a resolution obtained by multiplying the maximum irradiable resolution of the LiDAR 31S by a coefficient. For example, for an object that originally has a resolution of 40K pixels at 200×200 pixels, by setting the resolution with a coefficient of 0.25, the reflected light may be received at a ratio of one-fourth of the laser light to be irradiated. In this way, it is possible to create a difference in spatial resolution. Thereby, the hardware resources consumed for relatively less necessary object recognition areas can be reduced, and the computational load of the object recognition device 50 can be reduced. Also, more hardware resources can be consumed for relatively more necessary object recognition areas, and the reliability of the object recognition results in that area can be further enhanced.

[0093] Also, the object recognition processing unit 65 may reduce the frequency of executing arithmetic processing as a low-resolution processing level compared to the frequency of executing arithmetic processing as a high-resolution processing level. For example, in a processing process repeated at a predetermined interval, while the high-resolution processing level processing is executed every time, the low-resolution processing level processing may be executed every other time. Thereby, the hardware resources consumed for relatively less necessary object recognition areas can be reduced, and the computational load of the object recognition device 50 can be reduced. Also, more hardware resources can be consumed for relatively more necessary object recognition areas, and the reliability of the object recognition results in that area can be further enhanced. In this case, needless to say, when the object existence area set at the low-resolution processing level in the previous processing process is set at the high-resolution processing level in the current processing process, the object recognition processing at the high-resolution processing level is executed.

Explanation of Signs

[0094] 1... Vehicle, 31LF·31RF... Front camera, 31S... LiDAR (Distance measuring sensor), 35... Vehicle state sensor, 41... Vehicle control device, 43... HMI, 50... Object recognition device, 51... Control unit, 53... Memory unit, 61... Object existence area setting unit, 63... Processing load level setting unit, 65... Object recognition processing unit

Claims

1. An object recognition device that recognizes an object using observation data of a distance measurement sensor that receives a reflected wave of irradiated electromagnetic waves and image data generated by an imaging device, comprises one or more processors and one or more memories communicably connected to the one or more processors, wherein the one or more processors set an object existence region where an object may exist based on the observation data of the distance measurement sensor, estimate, for each object that may exist in each object existence region, an attribute of the object including at least information on the distance from the host vehicle to the object, the size of the object, and the speed of the object, obtain an index value indicating a collision risk representing the possibility of collision between the host vehicle and the object and the magnitude of damage assumed at the time of collision between the host vehicle and the object based on the attribute of the object, and based on the index value, set a level of processing load to be expended on object recognition processing performed using at least the image data with respect to the object existence region, and perform, for each object existence region, object recognition processing according to the set level of processing load. An object recognition device that executes a process including this.

2. The one or more processors set the level of processing load to either a steady processing level with a low processing load or a fine processing level with a higher processing load than the steady processing level. The object recognition device according to claim 1.

3. The one or more processors reduce the number of processes executed as processes at the steady processing level compared to the number of processes executed as processes at the fine processing level. The object recognition device according to claim 2.

4. The one or more processors as processing at the steady processing level, execute at least matching processing with data of a preset object and distance estimation processing of the object based on the image data, as processing at the fine processing level, execute any one or more of edge detection processing of the object, type estimation processing of the object, speed estimation processing of the object, distance estimation processing of the object, size estimation processing of the object, and energy estimation processing of the object. The object recognition device according to claim 2.

5. The fine processing level includes a low fine processing level with a relatively low processing load and a high fine processing level with a relatively high processing load. The object recognition device according to claim 4.

6. Among the edge detection process, the type estimation process, the speed estimation process, the distance estimation process, the size estimation process, and the energy estimation process, at least one of the processes is set to a low-load process with a relatively low processing load and a high-load process with a relatively high processing load. The one or more processors When executing any one of the edge detection process, the type estimation process, the speed estimation process, the distance estimation process, the size estimation process, and the energy estimation process as the process at the low-definition processing level, select and execute the low-load process. The object recognition device according to claim 5, wherein when executing any one of the edge detection process, the type estimation process, the speed estimation process, the distance estimation process, the size estimation process, and the energy estimation process as the process at the high-definition processing level, select and execute the high-load process.

7. The one or more processors The object recognition device according to claim 5, wherein the resolution when executing the edge detection process, the type estimation process, the speed estimation process, the distance estimation process, the size estimation process, or the energy estimation process as the process at the low-definition processing level is reduced compared to the resolution when executing as the process at the high-definition processing level.

8. The one or more processors The object recognition device according to claim 5, wherein the frequency of executing the edge detection process, the type estimation process, the speed estimation process, the distance estimation process, the size estimation process, or the energy estimation process as the process at the low-definition processing level is made less than the frequency of executing as the process at the high-definition processing level.

9. In an object recognition device that recognizes an object using observation data of a distance measuring sensor that receives a reflected wave of an irradiated electromagnetic wave and image data generated by an imaging device, An object presence area setting unit that sets an object presence area where an object may exist based on the observation data of the distance measuring sensor, and for each object that may exist in the object presence area, estimates the attributes of the object including at least the distance from the host vehicle to the object, the size of the object, and the speed of the object. Based on the attributes of the object, an index value indicating a collision risk representing the possibility of a collision between the host vehicle and the object and the magnitude of damage assumed at the time of the collision between the host vehicle and the object is obtained, and based on the index value, a processing load level setting unit that sets the level of the processing load incurred in the object recognition process performed on the object existence region using at least the image data; An object recognition processing unit that performs the object recognition process according to the set level of the processing load for each of the object existence regions; An object recognition device comprising: **Claim 10**: One or more processors are caused to Set an object existence region where there may be an object based on the observation data of a distance measurement sensor that receives the reflected wave of the irradiated electromagnetic wave; Estimate the attributes of the object that may exist in each of the object existence regions, including at least the distance from the host vehicle to the object, the size of the object, and the speed of the object; Based on the attributes of the object, an index value indicating a collision risk representing the possibility of a collision between the host vehicle and the object and the magnitude of damage assumed at the time of the collision between the host vehicle and the object is obtained, and based on the index value, the level of the processing load incurred in the object recognition process performed on the object existence region using at least the image data generated by an imaging device is set; Perform the object recognition process according to the set level of the processing load for each of the object existence regions; A computer program that causes the execution of operations including: **Claim 11**: One or more processors are caused to Set an object existence region where there may be an object based on the observation data of a distance measurement sensor that receives the reflected wave of the irradiated electromagnetic wave; Estimate the attributes of the object that may exist in each of the object existence regions, including at least the distance from the host vehicle to the object, the size of the object, and the speed of the object; Based on the attributes of the object, an index value indicating a collision risk representing the possibility of a collision between the host vehicle and the object and the magnitude of damage assumed at the time of the collision between the host vehicle and the object is obtained, and based on the index value, the level of the processing load incurred in the object recognition process performed on the object existence region using at least the image data generated by an imaging device is set; Performing the object recognition process according to the set level of the processing load for each of the object presence regions; A recording medium recording a computer program for executing an operation including this.

Images