Mobile body control device, mobile body control method, and storage medium

Abstract

The present application provides a mobile body control device, a mobile body control method, and a storage medium that can perform more appropriate mobile body control. The mobile body control device of the embodiment includes an identification unit that identifies a surrounding situation of a mobile body based on a first image captured by an imaging unit, a movement plan generation unit that generates a future movement plan of the mobile body based on an identification result identified by the identification unit, and a driving control unit that controls at least one of a steering and a speed of the mobile body based on the movement plan generated by the movement plan generation unit, the identification unit identifies an object in the surroundings of the mobile body in a first identification mode using a second image in which a data amount of the first image is reduced, and in a case where an accuracy of the identified object is less than a threshold value, an object other than the identified object that has a risk area greater than that of the identified object is identified.

Description

TECHNICAL FIELD

[0001] The present application relates to a mobile body control device, a mobile body control method, and a storage medium. BACKGROUND

[0002] A technology is known in which, in a vehicle control device that plans a target track of a vehicle based on recognition information from an external sensor such as a camera, in a case where an object in the surroundings of the vehicle is recognized from the external sensor, a target track that widens the actual sensing range of the external sensor is planned (for example, Japanese Patent Application Publication No. 2021-100827). SUMMARY

[0003] However, in general, the higher the resolution of the camera, the better the recognition performance, but the amount of data is large, so the load on image processing is heavy, and the processing time is long. On the other hand, in a case where the resolution is reduced in order to shorten the processing time, it is possible that the object will be misrecognized, but in the conventional technology, no consideration is given to processing with respect to an object for which the possibility of misrecognition is high. Therefore, sometimes the target track corresponding to the situation in the surroundings of the mobile body cannot be generated, and proper driving control cannot be performed.

[0004] The present application is achieved in consideration of such a situation, and one of the objects thereof is to provide a mobile body control device, a mobile body control method, and a storage medium that can perform more proper mobile body control.

[0005] The mobile body control device, the mobile body control method, and the storage medium of the present application adopt the following structure.

[0006] (1) The mobile body control device of one aspect of the present application includes: an identification unit that identifies a situation in the surroundings of a mobile body based on a first image captured by an imaging unit; a behavior plan generation unit that generates a future behavior plan of the mobile body based on an identification result identified by the identification unit; and a driving control unit that controls at least one of the steering and the speed of the mobile body based on the behavior plan generated by the behavior plan generation unit, the identification unit identifies an object in the surroundings of the mobile body in a first identification mode in which a second image in which the amount of data of the first image is reduced is used for identification, and in a case where the accuracy of the identified object is less than a threshold value, it is assumed that another object whose risk area is larger than that of the identified object has been identified.

[0007] (2) In the aspect of (1) above, the recognition unit extracts a first partial image region containing the object from the first image and recognizes the object with respect to the extracted first partial image region using a second recognition mode that uses a third image having a larger data amount than the second image, in a case where the accuracy of the object recognized by the first recognition mode is less than a threshold value, and the action plan generation unit generates the action plan based on information of the object recognized by the second recognition mode in a case where the object recognized by the first recognition mode is different from the object recognized by the second recognition mode.

[0008] (3) In the aspect of (1) above, the data amount includes at least one of resolution and frame rate.

[0009] (4) In the aspect of (1) above, the recognition result recognized by the recognition unit includes at least a position, a size, and a kind of the object.

[0010] (5) In the aspect of (1) above, the recognition unit extracts a second partial image region containing a division line that divides a region in which the mobile object moves from the first image and recognizes the division line with respect to the extracted second partial image region using a second recognition mode that uses a third image having a larger data amount than the second image, in a case where the object recognized by the first recognition mode includes another mobile object and the division line that divides the region in which the mobile object moves, and recognizes a position of the another mobile object based on a positional relationship between the recognized division line and the another mobile object.

[0011] (6) In the aspect of (5) above, the recognition unit recognizes the position of the another mobile object based on a positional relationship between an imaginary line obtained by extending the division line that divides the region in which the mobile object moves and a division line existing within a prescribed distance from the another mobile object.

[0012] (7) In the aspect of (1) above, the other object having a large risk region is an object predicted to have a larger movement amount in a prescribed time than a movement amount in the prescribed time of the object recognized from the second image.

[0013] (8) A mobile body control method of one aspect of the present application causes a computer to perform the following processing: recognizing a surrounding situation of a mobile body based on a first image captured by an imaging section; generating a future action plan of the mobile body based on a result of the recognition; controlling at least one of a steering and a speed of the mobile body based on the generated action plan; recognizing an object in the surroundings of the mobile body in a first recognition mode that uses a second image in which a data amount of the first image is reduced; and in a case where an accuracy of the recognized object is less than a threshold value, recognizing an other object whose risk area is greater than that of the recognized object as a risk area.

[0014] (9) A storage medium of one aspect of the present application stores a program that causes a computer to perform the following processing: recognizing a surrounding situation of a mobile body based on a first image captured by an imaging section; generating a future action plan of the mobile body based on a result of the recognition; controlling at least one of a steering and a speed of the mobile body based on the generated action plan; recognizing an object in the surroundings of the mobile body in a first recognition mode that uses a second image in which a data amount of the first image is reduced; and in a case where an accuracy of the recognized object is less than a threshold value, recognizing an other object whose risk area is greater than that of the recognized object as a risk area.

[0015] According to the aspects (1) to (9) described above, more appropriate mobile body control can be performed. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 is a configuration diagram of a vehicle system including a mobile body control device of the first embodiment.

[0017] Figure 2 is a functional configuration diagram of the first control section and the second control section of the first embodiment.

[0018] Figure 3 is a diagram for explaining generation of an action plan based on a recognition result recognized by a recognition section.

[0019] Figure 4 is a diagram for explaining a risk area of each object.

[0020] Figure 5 is a diagram for explaining a risk area in a case where an accuracy of recognition of an object is less than a threshold value.

[0021] Figure 6 is a flowchart showing one example of a flow of processing performed by the automatic driving control device in the first embodiment.

[0022] Figure 7is a flowchart showing an example of a flow of processing performed by the automatic driving control device in the second embodiment.

[0023] Figure 8 is a diagram for explaining recognition of a local image region containing a lane in the second recognition mode.

[0024] Figure 9 is a diagram for explaining recognition of a positional relationship of a division line.

[0025] Figure 10 is a flowchart showing an example of a flow of processing performed by the automatic driving control device in the third embodiment. DETAILED DESCRIPTION

[0026] Embodiments of a mobile body control device, a mobile body control method, and a storage medium of the present application will be described below with reference to the accompanying drawings. In the following description, an example in which the mobile body control device is mounted on a mobile body will be described. The mobile body refers to a structure that is capable of moving by a driving mechanism provided therein, such as a vehicle, a micro-mobility, an autonomous mobile robot, a ship, a drone, and the like. The control of the mobile body includes, for example, driving control in which the mobile body is autonomously moved by temporarily controlling one or both of the steering and the speed of the mobile body. The control of the mobile body can include, in addition to the driving control, an intervention control in which a certain degree of intervention is performed by giving a suggestion for a driving operation by voice, display, or the like, and a control in which the operation of a protection device for protecting an occupant of the mobile body is controlled. In the following description, a vehicle that moves on the ground will be assumed as the mobile body, and a structure and a function for moving the vehicle on the ground will be described. The driving control of the vehicle can include, for example, various driving controls such as automatic driving control of LKAS (Lane Keeping Assistance System), ALC (Auto Lane Changing), ACC (Adaptive Cruise Control), and the like, contact avoidance control in manual driving, emergency stop control, lane departure avoidance control, and the like.

[0027] (First Embodiment)

[0028] [Overall Structure]

[0029] Figure 1 is a structural diagram of a vehicle system 1 including the mobile body control device of the first embodiment. A vehicle (hereinafter referred to as the host vehicle M) on which the vehicle system 1 is mounted is, for example, a two-wheeled, three-wheeled, four-wheeled, or the like vehicle, and a driving source thereof is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination thereof. The electric motor operates using generated electric power from a generator coupled to the internal combustion engine, or discharge electric power from a secondary battery or a fuel cell.

[0030] Vehicle system 1 includes, for example, a camera 10, a radar device 12, a LiDAR (Light Detection and Ranging) system 14, a communication device 20, an HMI (Human Machine Interface) 30, vehicle sensors 40, a navigation device 50, an MPU (Map Positioning Unit) 60, driving controls 80, an automatic driving control device 100, a driving force output device 200, a braking device 210, and a steering device 220. These devices and equipment are interconnected through multiple communication lines such as CAN (Controller Area Network) communication lines, serial communication lines, and wireless communication networks. Figure 1 The structure shown is just one example; a part of the structure may be omitted, or other structures may be added. Camera 10 is an example of a "camera unit." Automatic driving control device 100 is an example of a "movement control device."

[0031] Camera 10 is, for example, a digital camera utilizing a solid-state imaging element such as CCD (Charge Coupled Device) or CMOS (Complementary Metal-Oxide Semiconductor). Camera 10 is mounted anywhere on the vehicle M equipped with vehicle system 1. When shooting forward, camera 10 is mounted on the upper part of the windshield, behind the rearview mirror inside the vehicle, etc. Camera 10 periodically and repeatedly captures images of the surroundings of the vehicle M. Camera 10 can also be a stereo camera. Camera 10 can also be a camera capable of capturing images of the surroundings of the vehicle M with a wide angle (e.g., 360 degrees). Camera 10 can also be implemented by combining multiple cameras. When camera 10 has multiple cameras, each camera can capture images at a different resolution and frame rate (FPS).

[0032] Radar device 12 radiates millimeter-wave or other radio waves around the vehicle M and detects the radio waves reflected by objects (reflected waves) to detect at least the position (distance and orientation) of the objects. Radar device 12 can be installed at any location on the vehicle M. Radar device 12 can also detect the position and speed of objects using FM-CW (Frequency Modulated Continuous Wave) method.

[0033] The LIDAR14 illuminates the periphery of the vehicle M with light (or electromagnetic waves with wavelengths close to light) and measures the scattered light. The LIDAR14 determines the distance to the object based on the time from the emission of light to the reception of light. The illuminating light can be, for example, a pulsed laser. The LIDAR14 can be mounted at any location on the vehicle M.

[0034] The communication device 20 communicates with other vehicles existing in the periphery of the host vehicle M using a cellular network, a Wi-Fi network, Bluetooth (registered trademark), DSRC (Dedicated Short Range Communication), or the like, for example, or communicates with various server devices via a wireless base station.

[0035] The HMI 30 outputs various information to the occupant of the host vehicle M by the control of the HMI control section 170. The HMI 30 can also function as an acceptance section that accepts input operations by the occupant. The HMI 30 includes a display device, a speaker, a microphone, a buzzer, a key, an indicator, and the like, for example. The display device is an LCD (Liquid Crystal Display), an organic EL (Electro Luminescence) display device, or the like, for example.

[0036] The vehicle sensor 40 includes a vehicle speed sensor that detects the speed of the host vehicle M, an acceleration sensor that detects acceleration, a yaw rate sensor that detects the angular velocity about the vertical axis, a direction sensor that detects the orientation of the host vehicle M, and the like. The vehicle sensor 40 can include a position sensor that acquires the position of the vehicle M. The position sensor is a sensor that acquires position information (longitude, latitude information) from a GPS (Global Positioning System) device, for example. The position sensor can also be a sensor that acquires position information using a GNSS (Global Navigation Satellite System) receiver 51 of the navigation device 50.

[0037] The navigation device 50 includes, for example, a GNSS receiver 51, a navigation HMI 52, and a route decision section 53. The navigation device 50 holds first map information 54 in a storage device such as a hard disk drive (HDD), a flash memory, or the like. The GNSS receiver 51 determines the position of the host vehicle M based on signals received from GNSS satellites. The position of the host vehicle M can also be determined or supplemented by an inertial navigation system (INS) that utilizes the output of the vehicle sensor 40. The navigation HMI 52 includes a display device, a speaker, a touch panel, a button, or the like. The navigation HMI 52 can also be partially or wholly shared with the aforementioned HMI 30. The route decision section 53 determines a route (hereinafter referred to as an on-map route) from the position of the host vehicle M determined by the GNSS receiver 51 (or an arbitrary position input) to a destination input by an occupant using the navigation HMI 52, for example, with reference to the first map information 54. The first map information 54 is information that represents the shape of a road, for example, by representing road segments and nodes connected by the road segments. The first map information 54 can also include the curvature of a road, point of interest (POI) information, or the like. The on-map route is output to the MPU 60. The navigation device 50 can also perform route guidance using the navigation HMI 52 based on the on-map route. The navigation device 50 can also be implemented by the functions of a terminal device such as a smartphone, a tablet terminal, or the like held by an occupant. The navigation device 50 can also transmit the current position and the destination to a navigation server via the communication device 20 and acquire a route equivalent to the on-map route from the navigation server.

[0038] The MPU 60 includes, for example, a recommended lane decision section 61 and holds second map information 62 in a storage device such as an HDD, a flash memory, or the like. The recommended lane decision section 61 divides the on-map route provided from the navigation device 50 into a plurality of blocks (for example, every 100 [m] in the direction of travel of the vehicle) and determines a recommended lane for each block with reference to the second map information 62. The recommended lane decision section 61 makes a decision as to which lane to travel on, for example, the nth lane from the left. The recommended lane decision section 61 determines a recommended lane so that the host vehicle M can travel on a reasonable route for traveling to a branched destination in the case where there is a branch point on the on-map route.

[0039] The second map information 62 is map information that is more accurate than the first map information 54. The second map information 62 includes, for example, information on the center of a lane or information on the boundary of a lane, or the like. The second map information 62 can include road information, traffic restriction information, residential information (residence, postal code), facility information, telephone number information, or the like. The second map information 62 can be updated at any time by communicating with other devices via the communication device 20.

[0040] The driving operation member 80 includes, for example, a steering wheel, an accelerator pedal, a brake pedal, a shift lever, and other operation members. Sensors that detect the operation amount or the presence or absence of operation are installed in the driving operation member 80, and the detection results are output to the automatic driving control device 100 or to some or all of the travel driving force output device 200, the brake device 210, and the steering device 220.

[0041] Next, before the description of the automatic driving control device 100, the travel driving force output device 200, the brake device 210, and the steering device 220 are described. The travel driving force output device 200 outputs a travel driving force (torque) for traveling of the host vehicle M to the drive wheels. The travel driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, a transmission, and the like, and an ECU (Electronic Control Unit) that controls them. The ECU controls the above-described configuration in accordance with information input from the automatic driving control device 100 (specifically, the second control section 160 described later) or information input from the driving operation member 80.

[0042] The brake device 210 includes, for example, a brake caliper, a hydraulic cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the hydraulic cylinder, and a brake ECU. The brake ECU controls the electric motor in accordance with information input from the second control section 160 or information input from the driving operation member 80 to output a brake torque corresponding to a brake operation to each wheel. The brake device 210 can include a mechanism that transmits hydraulic pressure generated by operation of a brake pedal included in the driving operation member 80 to the hydraulic cylinder via a master hydraulic cylinder as a backup. The brake device 210 is not limited to the configuration described above, and can be an electronically controlled hydraulic brake device that controls an actuator in accordance with information input from the second control section 160 to transmit hydraulic pressure of the master hydraulic cylinder to the hydraulic cylinder.

[0043] The steering device 220 includes, for example, a steering ECU and an electric motor. The electric motor, for example, applies force to a rack-and-pinion mechanism to change the orientation of the steered wheels. The steering ECU drives the electric motor in accordance with information input from the second control section 160 or information input from the steering wheel 82 of the driving operation member 80 to change the orientation of the steered wheels.

[0044] Next, the automatic driving control device 100 will be described. The automatic driving control device 100 includes, for example, a first control section 120, a second control section 160, an HMI control section 170, and a storage section 180. The first control section 120 and the second control section 160 are each implemented by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Part or all of these components can also be implemented by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), a GPU (Graphics Processing Unit), or the like, and can also be implemented by a combination of software and hardware. The program can be stored in advance in a storage device (a storage device including a non-transitory storage medium) such as an HDD, a flash memory, or the like of the automatic driving control device 100, or can be stored in a removable storage medium such as a DVD or a CD-ROM, and installed in the HDD or the flash memory of the automatic driving control device 100 by mounting the storage medium (non-transitory storage medium) in a drive device. The second control section 160 is an example of a "driving control section".

[0045] The storage section 180 can also be implemented by various storage devices described above, or an SSD (Solid State Drive), an EEPROM (Electrically Erasable Programmable Read Only Memory), a ROM (Read Only Memory), or a RAM (Random Access Memory), or the like. The storage section 180 stores, for example, a program, and other various information, and the like. The map information (first map information 54, second map information 62) described above can also be stored in the storage section 180.

[0046] Figure 2 is a functional configuration diagram of the first control section 120 and the second control section 160 of the first embodiment. The first control section 120 includes, for example, an identification section 130 and a behavior plan generation section 140. The first control section 120 implements, for example, functions based on AI (Artificial Intelligence) and functions based on a model given in advance in parallel. For example, the function of "identifying an intersection" can be implemented by "performing identification of an intersection based on deep learning or the like and identification based on a condition given in advance (presence of a signal, a road sign, or the like that can be pattern-matched, and the like) in parallel, and comprehensively evaluating both by scoring them".

[0047] The recognition unit 130 recognizes the situation around the host vehicle M. The recognition unit 130 includes, for example, a first acquisition unit 132, a recognition mode setting unit 134, an object recognition unit 136, and a risk area derivation unit 138.

[0048] The first acquisition unit 132 acquires, for example, data of detection results detected by some or all of the camera 10, the radar device 12, and the LIDAR 14. The first acquisition unit 132 can also perform processing of transforming a coordinate system of an image (hereinafter referred to as a camera image) captured by the camera 10 (for example, a camera coordinate system of a front field angle) into a coordinate system (a vehicle coordinate system, an overhead view coordinate system) based on the position of the host vehicle M when the host vehicle M is viewed from above. The camera image is an example of a "first image" and includes not only a still image but also a moving image.

[0049] The recognition mode setting unit 134 sets a recognition mode when the object recognition unit 136 performs object recognition with respect to data acquired by the first acquisition unit 132. The recognition mode includes, for example, a first recognition mode and a second recognition mode in which a larger amount of data is used for object recognition than in the first recognition mode, but can include other recognition modes in which the amount of data is different. For example, in a case where the data acquired by the first acquisition unit 132 is image data (hereinafter referred to as a camera image) captured by the camera 10, at least one of the resolution (number of pixels) and the frame rate of the image is included in the amount of data. Whether or not detection results detected by the radar device 12 and the LIDAR 14 are included in the amount of data (for example, the detection range, the detection period).

[0050] The recognition mode setting unit 134 can set a predetermined recognition mode among a plurality of recognition modes, or can set the recognition mode based on a recognition result recognized by the object recognition unit 136. For example, the recognition mode setting unit 134 sets the first recognition mode in a case where a predetermined object is not recognized by the object recognition unit 136 or in a case where the accuracy of the recognized object is equal to or higher than a threshold value. The predetermined object refers to an object that has an influence on the driving control of the host vehicle M, and includes, for example, other vehicles (including two-wheeled vehicles), pedestrians, bicycles, and the like, traffic participants, a division line that divides an area in which the host vehicle M travels (a moving body moves), a curb, a median strip, and the like. The accuracy refers to an index value that indicates the accuracy of the object. The accuracy can also be referred to as reliability or likelihood. The recognition mode setting unit 134 sets the second recognition mode, for example, in a case where the accuracy of the recognized object is less than the threshold value.

[0051] The recognition mode setting section 134 can also generate data corresponding to the set recognition mode from the data acquired by the first acquisition section 132. For example, the recognition mode setting section 134 generates a second image having a resolution lower (a smaller value of resolution) than the resolution of the camera image (reference resolution) when the first recognition mode is set. The recognition mode setting section 134 generates a third image having a resolution higher than the second image when the second recognition mode is set. The resolution of the third image is lower than the resolution of the camera image. Therefore, the recognition mode setting section 134 can also use the camera image as the third image. The recognition mode setting section 134 can generate the second image and the third image in which the frame rate of the camera image is adjusted instead of (or in addition to) adjusting the resolution. When the frame rate is adjusted, the recognition mode setting section 134, for example, thins out the image frames at a prescribed interval from the frame rate of the camera image (reference frame rate), whereby the frame rate is reduced.

[0052] The recognition mode setting section 134 can also select the camera image corresponding to the set recognition mode when the camera 10 is capturing at different resolutions and frame rates.

[0053] The object recognition section 136 recognizes an object existing within a prescribed distance from the host vehicle M using the recognition mode set by the recognition mode setting section 134 by performing sensor fusion processing on the data of the detection results detected by a part or all of the camera 10, the radar device 12, and the LIDAR 14 acquired by the first acquisition section 132. The object includes, for example, other vehicles, pedestrians, bicycles, and the like, traffic participants, lane dividers, curbs, median strips, road signs, traffic signal machines, intersections, pedestrian crossings, and the like, road structures.

[0054] For example, the object recognition section 136 performs prescribed image analysis processing using an image of a data amount corresponding to the recognition mode set by the recognition mode setting section 134, and refers to a model or the like defined in advance for pattern matching on the basis of the image information of the analysis result, and recognizes the object included in the image by matching processing. The image analysis processing includes, for example, edge extraction processing of extracting edge points having a large luminance difference between adjacent pixels and connecting the extracted edge points to obtain the outline of the object, processing of extracting a feature amount from the color, shape, size, and the like within the outline, and the like. The model refers to a learned model such as a DNN (Deep Neural Network) learned by inputting the analysis result or the data acquired by the first acquisition section 132 and outputting the kind (category) of the object included in the image, but is not limited thereto. The model can be stored in the storage section 180, or can be acquired from an external device via the communication device 20. The model can be appropriately updated by feedback control of the recognition result, update data from an external device, and the like.

[0055] The object recognition unit 136 can also output the accuracy of the object in the image being the object defined by the model (the kind of object) by using the matching process using the analysis result and the model. The accuracy can be defined by the model, for example, and can be set based on the degree of agreement (similarity) or the like in the matching process.

[0056] The object recognition unit 136, in a case where an object is recognized, recognizes the position, speed (absolute speed, relative speed), acceleration (absolute acceleration, relative acceleration), direction of travel (direction of movement), or the like of the object. The position of the object is recognized as a position on an absolute coordinate with a representative point (center of gravity, center of driving shaft, or the like) of the host vehicle M as the origin, for example, and is used for control. The position of the object can be represented by a representative point such as the center of gravity or a corner of the object, or can be represented by an area. The "state" of the object can include the acceleration, jerk, or "behavioral state" (for example, whether or not a lane change is being made or is about to be made) of another moving body such as another vehicle, for example, in a case where the object is another moving body.

[0057] The object recognition unit 136, for example, can also recognize a lane (travel lane) in which the host vehicle M is traveling in a case where a division line is recognized. For example, the object recognition unit 136 recognizes the travel lane by comparing a pattern of a road division line (for example, an arrangement of solid lines and dashed lines) obtained from the second map information 62 with a pattern of a division line (road division line) of the periphery of the host vehicle M recognized from an image captured by the camera 10. The object recognition unit 136 is not limited to recognizing a division line, and can recognize a travel road boundary (road boundary) including a curbstone, a median strip, or the like. In this recognition, the position of the host vehicle M obtained from the navigation device 50 or a processing result based on the INS can be taken into consideration.

[0058] The object recognition unit 136 can also recognize the position and posture of the host vehicle M with respect to the travel lane when recognizing the travel lane of the host vehicle M. In this case, the object recognition unit 136 recognizes the deviation of a reference point of the host vehicle M from the center of the lane and the angle of the direction of travel of the host vehicle M with respect to a line connecting the center of the lane as the relative position and posture of the host vehicle M with respect to the travel lane. Instead, the object recognition unit 136 can recognize the position of a reference point of the host vehicle M with respect to an arbitrary side end portion (road boundary) of the travel lane or the like as the relative position of the host vehicle M with respect to the travel lane.

[0059] The risk region derivation unit 138 derives a region in which a risk is potentially distributed or potentially present around the object recognized by the object recognition unit 136 (hereinafter, referred to as a risk region). The risk refers to, for example, a risk that the object brings to the host vehicle M. More specifically, the risk is, for example, a risk that the host vehicle M is forced to suddenly brake, turn, or the like due to a sudden movement of the object. The risk can also be a risk that the host vehicle M brings to the object. Hereinafter, the degree of such a risk is handled as a quantitative index value, and the index value is referred to as "potential risk" to be described. The potential risk is derived as a region that is equal to or higher than a threshold value. The risk region derivation unit 138 can also adjust the risk region based on the accuracy with respect to the object recognized by the object recognition unit 136. The object recognition unit 136 outputs the recognition result to the action plan generation unit 140.

[0060] The action plan generation unit 140 generates a future action plan of the host vehicle M based on the recognition result recognized by the recognition unit 130. For example, the action plan generation unit 140 generates a target track in which the host vehicle M automatically (independently of the operation of the driver) travels in the future in such a manner that the host vehicle M travels on the recommended lane decided by the recommended lane decision unit 61 in principle and can cope with the situation around the host vehicle M. The target track includes, for example, a speed element. For example, the target track is expressed as a track in which places (track points) where the host vehicle M should arrive are arranged in order. The track points are places where the host vehicle M should arrive at every prescribed travel distance (for example, several [m]) in terms of the distance along the route, and, in addition thereto, a target speed and a target acceleration at every prescribed sampling time (for example, several [sec]) are generated as a part of the target track. The track points can also be positions where the host vehicle M should arrive at every prescribed sampling time at the sampling time. In this case, information of the target speed and the target acceleration is expressed by the interval of the track points.

[0061] The action plan generation unit 140 can set an automatic driving event when generating the target track. The automatic driving event includes, for example, a constant speed driving event, a low speed following driving event, a lane change event, a branch event, a merging event, a contact avoidance event, an emergency stop event, a takeover event, and the like. The action plan generation unit 140 generates a target track corresponding to the started event. The action plan generation unit 140 can also propose (recommend) the driver control, the execution of the event, and the like to the occupant in a case where the driver control of the host vehicle M, the prescribed event, or the like is executed, and generate a corresponding target track in a case where the proposal is acknowledged.

[0062] The second control unit 160 controls the travel driving force output device 200, the brake device 210, and the steering device 220 so that the host vehicle M passes through the target track generated by the action plan generation unit 140 at a predetermined time.

[0063] The second control section 160 includes, for example, a second acquisition section 162, a speed control section 164, and a steering control section 166. The second acquisition section 162 acquires information of the target trajectory (trajectory point) generated by the movement plan generation section 140 and stores it in a memory (not shown). The speed control section 164 controls the travel drive force output device 200 or the brake device 210 based on a speed element attached to the target trajectory stored in the memory. The steering control section 166 controls the steering device 220 according to the curvature of the target trajectory stored in the memory. The processing of the speed control section 164 and the steering control section 166 is implemented, for example, by a combination of feedforward control and feedback control. As an example, the steering control section 166 performs a combination of feedforward control corresponding to the curvature of the road ahead of the host vehicle M and feedback control based on deviation from the target trajectory.

[0064] The HMI control section 170 notifies a passenger (driver) of the host vehicle M of prescribed information through the HMI 30. The prescribed information includes, for example, information related to the driving control of the host vehicle M, prescribed events. For example, the HMI control section 170 causes the HMI 30 to output warning information or the like in the case where the contact avoidance control, the emergency stop control, or the like is performed. The HMI control section 170 can generate an image containing the prescribed information described above and cause the display device of the HMI 30 to display the generated image, or can generate a sound representing the prescribed information and cause the generated sound to be output from the speaker of the HMI 30. The HMI control section 170 can also output information accepted by the HMI 30 to the communication device 20, the navigation device 50, the first control section 120, or the like, for example.

[0065] [Generation of movement plan based on recognition result]

[0066] Next, a specific example of the generation of the movement plan based on the recognition result recognized by the recognition section 130 will be described. In the following description, the object recognition based on the camera image will be mainly described, but the object recognition based on the detection result of the radar device 12, the LIDAR 14 is also implemented.

[0067] Figure 3 is a diagram for explaining the generation of the movement plan based on the recognition result recognized by the recognition section 130. In Figure 3 In the example of, two lanes L1, L2 that can travel in the same direction (X-axis direction in the drawing) are shown. The lane L1 is divided by a division line LM1 and a division line LM2, and the lane L2 is divided by the division line LM2 and a division line LM3. In Figure 3 In the example of, the host vehicle M traveling at a speed VM toward the extension direction (X-axis direction in the drawing) of the lane L1, and an object OB1 moving at a speed VI ahead of the host vehicle M on the lane L1 are shown. The object OB1 is assumed to be a pedestrian.

[0068] For example, when the occupants of vehicle M receive an instruction to start autonomous driving control via HMI30, the recognition mode setting unit 134 sets a first recognition mode as the initial recognition mode. The object recognition unit 136 uses a second image that corresponds to the first recognition mode to recognize the surrounding environment of vehicle M, identifying the position, size, type, etc., of object OB1. The object recognition unit 136 obtains the accuracy of the identified object OB1, the speed V1 of the identified object OB1, and the direction of movement. The object recognition unit 136 also identifies the position, size, and type of dividing lines LM1 to LM3.

[0069] Based on the recognition results identified by the object recognition unit 136, the risk area derivation unit 138 derives the risk area of ​​the object OB1 that may come into contact with the vehicle M.

[0070] Figure 4 This is a diagram used to illustrate the risk areas for each object. Figure 4 In the examples, pedestrians, two-wheeled vehicles (motorcycles), and four-wheeled vehicles are shown as examples of objects. Hereinafter, for ease of explanation, pedestrians OBa, two-wheeled vehicles OBB, and four-wheeled vehicles OBC are sometimes referred to as such.

[0071] The risk area derivation unit 138 derives risk areas where the closer the object is to the object, the higher the potential risk, and the farther away from the object, the lower the potential risk. Alternatively, the risk area derivation unit 138 may adjust the risk area so that the closer the vehicle M is to the object, the higher the potential risk (in other words, so that the farther away, the lower the potential risk).

[0072] The risk area export unit 138 can also adjust the risk area according to the type of object. For example, in the case of a pedestrian OBa and a two-wheeled vehicle OBa, even if the position, direction of movement, and speed at the time of recognition are different... Figure 4 The speeds Va and Vb shown are the same, but the amount of movement over the subsequent specified time can vary significantly. The size of the objects differs among pedestrians (OBa), two-wheeled vehicles (OBb), and four-wheeled vehicles (OBc). Therefore, the risk area derivation unit 138 derives a risk area based on the type of object. Specifically, the risk area derivation unit 138 adjusts the risk area in a manner that the larger the predicted value of the movement over the specified time (hereinafter referred to as the predicted movement), the larger the risk area. Alternatively, the risk area derivation unit 138 may adjust the risk area in a manner that the larger the size of the object, the larger the risk area.

[0073] exist Figure 4In the example of FIG. 9, the two-wheeled vehicle OBb is able to move faster than the pedestrian OBa, and thus the predicted movement amount is large. Therefore, the risk area RAb of the two-wheeled vehicle OBb is larger than the risk area RAa of the pedestrian OBa. The four-wheeled vehicle OBc is larger in size than the two-wheeled vehicle OBb even if the predicted movement amount is the same as that of the two-wheeled vehicle OBb. Therefore, the risk area Rac of the four-wheeled vehicle OBc is larger than the risk area RAb. In this way, by deriving the risk area in accordance with the kind of the object, the risk area can be derived more quickly.

[0074] The risk area derivation unit 138 can also adjust the risk area in accordance with the speed and the moving direction of the object. For example, the greater the absolute speed and the absolute acceleration of the object, the greater the potential risk that the risk area derivation unit 138 increases and derives the risk area based on the potential risk. The potential risk can also be appropriately determined in place of or in addition to the absolute speed and the absolute acceleration of the object in accordance with the relative speed and the relative acceleration between the host vehicle M and the object, the TTC (Time to Collision), the contact prediction position, and the like. The potential risk can also be adjusted in accordance with the surrounding conditions such as the road shape, the congestion degree, the weather, the time period, and the like.

[0075] Here, the object recognition unit 136 determines whether the accuracy of the recognized object is equal to or higher than a threshold value, and in the case where it is determined that the accuracy is equal to or higher than the threshold value, causes the risk area derivation unit 138 to derive the risk area for the recognized object. In contrast, in the case where it is determined that the accuracy is not equal to or higher than the threshold value (in the case where it is determined that the accuracy is lower than the threshold value), the object recognition unit 136 assumes that another object whose risk area is larger than the recognized object has been recognized, and causes the risk area derivation unit 138 to derive the risk area. The other object whose risk area is larger, for example, refers to an object whose predicted movement amount is larger than that of the recognized object (for example, an object that is able to move at a higher speed than the recognized object). The other object, for example, refers to an object whose kind is different from that of the recognized object.

[0076] For example, the object recognition unit 136 assumes that another object that is predetermined has been recognized in the case where the kind of the object whose accuracy is lower than the threshold value is a pedestrian, in the case where the kind is a two-wheeled vehicle, in the case where the kind is a four-wheeled vehicle, and the like. The object recognition unit 136 can also select another object in accordance with the accuracy, the speed of the object, and the like. In this case, the smaller the accuracy and / or the greater the speed, the more the object recognition unit 136 selects an object whose risk area is larger. For example, the object recognition unit 136 assumes that another object that is predetermined has been recognized in the case where the accuracy when a pedestrian is recognized is lower than a first predetermined value, in the case where the accuracy is lower than a second predetermined value that is smaller than the first predetermined value, and the like.

[0077] In Figure 3In the example of FIG. 10, the accuracy of recognizing the object OB1 as a pedestrian is above the threshold value, and thus the risk region RAa in which the correspondence with the pedestrian is established is set with reference to the position of the object OB1. Figure 5 is a view for explaining the risk region in a case where the accuracy of recognizing the object OB1 is less than the threshold value. In the example of FIG. 11, the accuracy of recognizing the object OB1 as a pedestrian is less than the threshold value, and thus the object OB1 is regarded as a two-wheeled vehicle, and the risk region RAb in which the correspondence with the two-wheeled vehicle is established is set with reference to the position of the object OB1. Figure 5

[0078] The action plan generation section 140 generates a target track in such a manner that the host vehicle M travels at a position that is separated from the risk region by a prescribed distance, so as to avoid contact of the risk region with the future track of the host vehicle M. Thus, in the first embodiment, the target track K1 based on the risk region RAa is generated as shown in FIG. 10 in a case where the accuracy of recognizing the object OB1 is above the threshold value, and the target track K2 based on the risk region RAb is generated as shown in FIG. 11 in a case where the accuracy of recognizing the object OB1 is less than the threshold value. Thus, in a case where the existence of the object is recognized but the kind of the object is not determined completely, a large risk region can be set, and a target track with higher safety can be generated. By generating the above-described action plan, high-speed object recognition using an image with a small amount of data can be performed, and thus more appropriate driving control of the host vehicle M corresponding to the surrounding situation can be realized. Figure 3 Figure 5 [Processing flow of the first embodiment]

[0079] [Processing flow of the first embodiment]

[0080] Next, the flow of the processing performed by the automatic driving control device 100 of the first embodiment will be described. The processing of the following flowchart is described focusing on the point that the action plan is generated based on the risk region mainly based on the recognition result of the object in the processing performed by the automatic driving control device 100. The processing shown below can be repeatedly performed at a prescribed timing.

[0081] Figure 6 is a flowchart showing an example of the flow of the processing performed by the automatic driving control device 100 in the first embodiment. In the example of FIG. 12, the processing performed by the automatic driving control device 100 is repeatedly performed at a prescribed timing. Figure 6 ​​In the example of FIG. 1, the first acquisition unit 132 acquires a camera image (first image) and the like (step S100). Next, the object recognition unit 136 performs object recognition of the periphery of the host vehicle M in a first recognition mode using a second image generated from the camera image (step S102). Next, the object recognition unit 136 determines whether the accuracy of the recognized object is equal to or higher than a threshold value in a case where an object is recognized (step S104). In a case where it is determined that the accuracy of the recognized object is equal to or higher than the threshold value, the object recognition unit 136 determines that the object as recognized is recognized. In a case where it is determined that the accuracy of the recognized object is lower than the threshold value, the object recognition unit 136 determines that another object larger than the recognized object in a risk region is recognized (step S108).

[0082] Next, the risk region derivation unit 138 derives a risk region for the object recognized by the object recognition unit 136 (step S110). Next, the action plan generation unit 140 generates a target trajectory based on the derived risk region and the like (step S112). Next, the second control unit 160 performs driving control of at least one of the steering and the speed of the host vehicle M to cause the host vehicle M to travel, so that the host vehicle M travels along the generated target trajectory (step S114). Thereby, the processing of the present flowchart ends.

[0083] According to the first embodiment as explained above, in the automatic driving control device (an example of a mobile body control device) 100, there are provided: a recognition unit 130 that recognizes a periphery situation of a host vehicle (an example of a mobile body) M based on a first image captured by a camera (an example of an imaging unit) 10; an action plan generation unit 140 that generates a future action plan of the host vehicle M based on a recognition result recognized by the recognition unit 130; and a driving control unit (a second control unit 160) that controls at least one of the steering and the speed of the host vehicle M based on the action plan generated by the action plan generation unit 140, the recognition unit 130 recognizes an object of the periphery of the host vehicle M in a first recognition mode using a second image that reduces a data amount of the first image, in a case where the accuracy of the recognized object is lower than a threshold value, determines that another object larger than the recognized object in a risk region is recognized, and thereby more appropriate mobile body control can be performed.

[0084] According to the first embodiment, for example, the recognition is performed in the first recognition mode using the image with low resolution, whereby the recognition result can be obtained earlier, and the target trajectory is generated based on the risk area corresponding to the accuracy of the recognition result, so that even in a case where there is an object with high possibility of misrecognition, more appropriate driving control corresponding to the situation around the host vehicle M can be performed. According to the first embodiment, a more real-time recognition result can be obtained, so that in a case where, for example, the host vehicle M is approaching the object, initial action of driving control such as contact avoidance, emergency stop, and the like can be performed early, and vehicle control with higher safety can be achieved.

[0085] (Second Embodiment)

[0086] Next, the second embodiment will be described. The second embodiment differs from the first embodiment in that, in a case where the accuracy of the recognition result is less than the threshold value, a partial image region (an example of the first partial image region) including the object in the camera image is extracted, and the object recognition is performed in the second recognition mode using the image of the extracted region. Therefore, hereinafter, the description will be mainly focused on the above-described difference. In the second embodiment, the same structure as the first embodiment can be applied, and therefore, hereinafter, the structure of the automatic driving control device 100 shown in the first embodiment will be described. The same applies to the third embodiment described later.

[0087] [Processing Flow of Second Embodiment]

[0088] Figure 7 is a flowchart showing an example of a flow of processing performed by the automatic driving control device 100 in the second embodiment. Figure 7 The processing shown in Figure 6 differs from the processing shown in Figure 6 the steps S100 to S114 shown in

[0089] In the example of Figure 7 after the processing of the step S114, the object recognition unit 136 determines whether the accuracy of the recognized object is equal to or greater than the threshold value (step S200). In a case where it is determined that the accuracy is not equal to or greater than the threshold value (less than the threshold value), the object recognition unit 136 performs a cropping process of extracting a partial image region including the object from the camera image based on the position information of the recognized object (step S202). The partial image region is, for example, a bounding box that encloses the outline (periphery) of the recognized object. The partial image region can be adjusted in the region of the bounding box according to the speed, moving direction, and relative position from the host vehicle M of the object, or can be adjusted so that the smaller the accuracy, the larger the partial image region.

[0090] Next, the object recognition unit 136 generates a third image having a larger amount of data than the second image with respect to the extracted partial image region, and performs object recognition in the second recognition mode using the generated third image (step S204). In the processing of step S204, the object recognition unit 136 can also increase the amount of data from the radar device 12, the LIDAR 14 to perform object recognition.

[0091] Next, the object recognition unit 136 determines whether the recognition results in the first recognition mode and the second recognition mode are different (step S206). The recognition results are different, for example, in a case where at least one of the position, the size, and the kind of the recognized object exceeds a permissible range. The case where the recognition results are different can include, in addition to the above, a case where the moving direction differs by a prescribed angle or more, and a case where the moving amount differs by a prescribed amount or more. In a case where it is determined that the recognition results in the first recognition mode and the second recognition mode are different, the risk region derivation unit 138 derives a risk region with respect to the object recognized in the second recognition mode (step S208).

[0092] Next, the action plan generation unit 140 adjusts the existing target trajectory based on the risk region or the like derived through the processing of step S208 (step S210). In the processing of step S210, the action plan generation unit 140, for example, generates a target trajectory based on a second risk region in a case where the second risk region is larger than a first risk region with respect to the object recognized in the first recognition mode. The action plan generation unit 140 can also control the amount of adjustment of the already generated target trajectory based on the expansion ratio of the second risk region with respect to the first risk region or the like. The action plan generation unit 140 can regenerate a target trajectory based on the second risk region again in a case where the second risk region is smaller than the first risk region, or can maintain the current target trajectory. In a case where the second risk region is small, even if the current target trajectory is maintained, the possibility that the host vehicle M collides with the object is low, and thus by maintaining the current target trajectory, it is possible to suppress a case where the behavior of the host vehicle M temporarily becomes large due to a change in the target trajectory during travel.

[0093] After the processing of step S210, the second control unit 160 performs driving control to cause the host vehicle M to travel along the generated target trajectory (step S212). Thereby, the processing of the present flowchart ends. In a case where it is determined in the processing of step S200 that the accuracy of the recognized object is the threshold or more, the processing of the present flowchart ends.

[0094] According to the second embodiment as explained above, in addition to the same effects as the first embodiment, the data amount of the local image region containing the object is increased to perform object recognition in the case where the accuracy is less than the threshold value, so the accuracy of the position, size, and kind of the object is improved, and more accurate object recognition can be achieved. Therefore, the target trajectory can be appropriately corrected based on the recognition result, and more appropriate driving control can be performed.

[0095] (Third Embodiment)

[0096] Next, the third embodiment will be described. The third embodiment differs from the first embodiment in that, in the case where a division line that divides a lane (region) in which the host vehicle M travels is recognized together with the object, a local image region (an example of the second local image region) containing the division line in the camera image is extracted, and the image of the extracted region is used to recognize the position of the object in the second recognition mode. Therefore, the following description will mainly focus on the above difference.

[0097] Figure 8 is a diagram for explaining the recognition of the local image region containing the lane in the second recognition mode. In Figure 8 In the example of

[0098] In the example of Figure 8 In the example of

[0099] For example, the object recognition unit 136 can extract a partial image area including all of the division lines recognized in the first recognition mode, or can extract a partial image area including a part of the division lines. The part of the division lines refers to, for example, a division line whose length is less than a predetermined length. In the case where the line is less than the predetermined length, the division line is likely to be interrupted, and thus by performing recognition in the third image (high-precision image) with the area including the line as a target, the division line can be more accurately recognized.

[0100] As the part of the division line, there can be a division line referred to for determining the position of the object OB2 observed from the host vehicle M, and more specifically, there can be a division line present between the host vehicle M and the object OB2. For example, in the case where the object OB2 is present in front of the right side of the host vehicle M, a partial image area including the division line on the right side of the host vehicle M and the division line on the left side of the object OB2 is extracted. A plurality of division lines can also be included in the part of the division line. In the example of FIG. 12, as the partial image area, a partial image area PA including a part of the division line LM2a and the division line LM2b is extracted. Figure 8

[0101] The object recognition unit 136 performs recognition of the division lines in the second recognition mode using the third image corresponding to the extracted partial image area PA, and recognizes the positional relationship between the division line in the vicinity of the host vehicle M and the division line in the vicinity of the object OB2 on the basis of the recognition result. The division line in the vicinity of the host vehicle M can also be acquired by referring to the map information on the basis of the positional information of the host vehicle M acquired from the vehicle sensor 40 instead of (or in addition to) being recognized from the third image.

[0102] The object recognition unit 136 recognizes the positional relationship between the division line LM2a (division line dividing the host lane) present on the right side closest to the host vehicle M and the division line LM2b (division line dividing the travel lane of the object) present on the left side closest to the object OB2. Figure 9 is a diagram for explaining recognition of the positional relationship of the division lines. In Figure 9 ​In the example, the object recognition unit 136 sets an imaginary line VL by extending the dividing line LM2a on the right side of the vehicle M along the extension direction of lanes L1 and L2 (X-axis direction in the figure). Based on the positional relationship between the set imaginary line VL and the dividing line LM2b on the left side of the object OB2, it determines whether dividing lines LM2a and LM2b are the same dividing line. If the lateral distance D between the imaginary line VL and the dividing line LM2b (road width direction, Y-axis direction) is less than a threshold, the object recognition unit 136 identifies dividing lines LM2a and LM2b as the same dividing line. If the distance D is greater than or equal to the threshold, the object recognition unit 136 identifies dividing lines LM2a and LM2b as different dividing lines. When there are multiple dividing lines on the right side of the vehicle M and the left side of the object OB2, the above determination can be performed for each dividing line. After the dividing line is identified, the object recognition unit 136 identifies the position of object OB2 based on the identification result of the dividing line performed by the second recognition mode. This allows for more accurate identification of the position of object OB2, enabling the allocation of the risk zone for object OB2 at the appropriate location and the generation of a more suitable target trajectory.

[0103] [Processing flow of the third embodiment]

[0104] Figure 10 This is a flowchart illustrating an example of the processing flow performed by the automatic driving control device 100 in the third embodiment. Figure 10 The processing shown is the same as Figure 6 The difference between steps S100 to S114 is that steps S300 to S304 are added between steps S108 and S110. Therefore, the following explanation will focus on the processes of steps S300 to S304.

[0105] exist Figure 10 In the example, after the processing in step S108, the object recognition unit 136 determines whether the object in the recognition result contains a dividing line (step S300). If it is determined that the object contains a dividing line, the object recognition unit 136 extracts the local image region containing the dividing line, and uses a third image to recognize the dividing line in a second recognition mode for the extracted local image region (step S302), and recognizes the position of the object based on the position of the recognized lane (step S304). After the processing in step S106 or S304 is completed, the processing after step S110 is executed.

[0106] According to the third embodiment as explained above, in addition to the same effects as the first embodiment, it is possible to recognize a division line with high accuracy in the second recognition mode in the case where the recognition result based on the first recognition mode includes a division line, and to recognize the position of an object viewed from the host vehicle M with high accuracy based on the position of the recognized division line. According to the third embodiment, only a division line having a certain degree of shape, size, or the like, which is limited compared to a traffic participant such as a pedestrian or a vehicle, is recognized as a recognition target, so even in the case where recognition processing is performed in the second recognition mode having a higher resolution than the first recognition mode, it is possible to quickly recognize a division line. Therefore, it is possible to generate a more appropriate target trajectory in accordance with the surrounding situation.

[0107] In the third embodiment, a road structure (for example, a curbstone, a median strip) other than a division line can also be recognized instead of (or in addition to) a division line.

[0108] The first to third embodiments described above can each be combined with a part or all of the other embodiments. For example, by combining the division line recognition of the third embodiment with the second embodiment, it is possible to set a more appropriate risk area at a more appropriate position.

[0109] The embodiments described above can be expressed as follows.

[0110] A mobile body control device includes:

[0111] a storage medium that stores commands that can be read by a computer; and

[0112] a processor connected to the storage medium,

[0113] the processor performs the following processing by executing the commands that can be read by the computer:

[0114] recognizing a surrounding situation of a mobile body based on a first image captured by an imaging section;

[0115] generating a future action plan of the mobile body based on the recognition result;

[0116] controlling at least one of a steering and a speed of the mobile body based on the generated action plan;

[0117] recognizing an object in the surroundings of the mobile body in a first recognition mode that uses a second image having a reduced data amount of the first image;

[0118] in the case where the accuracy of the recognized object is less than a threshold value, recognizing another object that is larger than the recognized object as a risk area.

[0119] The above-described embodiments illustrate the present application by using the embodiments, but the present application is not limited to such embodiments at all, and various modifications and substitutions can be applied within the scope of the gist of the present application.

Claims

1. A mobile body control device, wherein, The moving body control device includes: The recognition unit identifies the surrounding environment of the moving object based on the first image captured by the camera unit; An action plan generation unit generates a future action plan for the mobile body based on the identification results identified by the identification unit. as well as The driving control unit controls at least one of the steering and speed of the moving body based on the action plan generated by the action plan generation unit. The recognition unit identifies objects around the moving body using a first recognition mode that uses a second image with reduced data volume of the first image. If the accuracy of the identified object is less than a threshold, it is assumed that other objects with a larger risk area than the identified object have been identified.

2. The moving body control device according to claim 1, wherein, If the accuracy of the object identified by the first identification mode is less than a threshold, the identification unit extracts a first local image region containing the object from the first image, and then identifies the object using a second identification mode that uses a third image with more data than the second image for the extracted first local image region. When the object identified by the first recognition mode is different from the object identified by the second recognition mode, the action plan generation unit generates the action plan based on the information of the object identified by the second recognition mode.

3. The moving body control device according to claim 1, wherein, The amount of data includes at least one of resolution and frame rate.

4. The moving body control device according to claim 1, wherein, The identification result identified by the identification unit includes at least the position, size, and type of the object.

5. The moving body control device according to claim 1, wherein, When the object identified by the first identification mode contains other moving objects and dividing lines that divide the area where the moving objects move, the identification unit extracts a second local image region containing the dividing lines from the first image, and identifies the dividing lines using a second identification mode that uses a third image with more data than the second image for the extracted second local image region, and identifies the position of the other moving objects based on the positional relationship between the identified dividing lines and the other moving objects.

6. The moving body control device according to claim 5, wherein, The identification unit identifies the position of the other moving bodies based on the positional relationship between an imaginary line obtained by extending the dividing line that divides the area where the moving body moves and a dividing line that exists within a specified distance from the other moving bodies.

7. The moving body control device according to claim 1, wherein, Other objects in the risk area are those whose predicted movement over a specified time is greater than that of the objects identified from the second image over the same specified time.

8. A method for controlling a moving body, wherein, The moving body control method causes the computer to perform the following processing: Identify the surrounding environment of the moving object based on the first image captured by the camera unit; Based on the identification results, a future action plan for the moving entity is generated; Based on the generated action plan, control at least one of the movement's direction and speed; Objects surrounding the moving object are identified using a first recognition pattern that uses a second image with reduced data volume from the first image. If the accuracy of the identified object is less than a threshold, it is assumed that other objects with a risk area larger than the risk area of ​​the identified object have been identified.

9. A storage medium storing a program, wherein, The program causes the computer to perform the following processing: Identify the surrounding environment of the moving object based on the first image captured by the camera unit; Based on the identification results, a future action plan for the moving entity is generated; Based on the generated action plan, control at least one of the movement's direction and speed; Objects surrounding the moving object are identified using a first recognition pattern that uses a second image with reduced data volume from the first image. If the accuracy of the identified object is less than a threshold, it is assumed that other objects with a risk area larger than the risk area of ​​the identified object have been identified.

Images