Recognition device, moving body control device, recognition method, and program

Abstract

A recognition device that recognizes the position and the speed of an object present around a moving body on the basis of the output of a detection device for detecting the conditions around the moving body, the recognition device comprising: a first recognition unit that repeatedly outputs the position of the object, which is a result of performing a process for recognizing the position of the object, at a first cycle; and a second recognition unit that repeatedly outputs the speed of the object, which is a result of performing a process for recognizing the speed of the object, at a second cycle shorter than the first cycle.

Description

Recognition device, mobile object control device, recognition method, and program

[0001] The present invention relates to a recognition device, a control device for a moving object, a recognition method, and a program.

[0002] In recent years, research and practical application of autonomously moving mobile objects such as vehicles has been progressing. In most cases, such technologies require a technology for recognizing objects present around the mobile object (hereinafter referred to as "periphery recognition technology"). Regarding the peripheral recognition technology, an invention has been disclosed in which, based on driving scene data and driving behavior data in a driving scene where the vehicle is currently located, it is determined whether a predetermined stable driving condition is satisfied, and if the predetermined stable driving condition is satisfied, the driving scene data and driving behavior data are collected at a frequency lower than a predetermined sampling frequency, thereby reducing data redundancy in similar scenes and similar driving modes (Patent Document 1).

[0003] Special Publication No. 2021-526699

[0004] To realize a surroundings recognition technology, it is necessary to reduce the processor load while quickly responding to sudden events. While the above-mentioned conventional technology comprehensively assesses the environment of a moving object, there is a concern that it cannot quickly respond to local changes. Specifically, the conventional technology may not be able to quickly respond to changes in the speed of an object.

[0005] The present invention has been made in consideration of the above circumstances, and one of its objects is to provide a recognition device, a control device for a moving object, a recognition method, and a program that can quickly follow changes in the speed of an object.

[0006] The recognition device, the control device for a moving body, the recognition method, and the program according to the present invention employ the following configuration: (1): A recognition device according to one aspect of the present invention is a recognition device that recognizes the position and speed of an object present around a moving body based on the output of a detection device that detects the surrounding conditions of the moving body, and includes a first recognition unit that repeatedly outputs the position of the object, which is a result of processing to recognize the position of the object, in a first period, and a second recognition unit that repeatedly outputs the speed of the object, which is a result of processing to recognize the speed of the object, in a second period that is shorter than the first period.

[0007] (2): In the aspect of (1) above, the second recognition unit has a function of provisionally recognizing the position of an object whose position has not been output by the first recognition unit, and when the position is output from the first recognition unit and the velocity is output from the second recognition unit, the second recognition unit outputs the position output by the first recognition unit and the velocity output by the second recognition unit as the state of the object, and further includes a fusion unit that outputs the position and velocity output by the second recognition unit as the state of the object for an object whose position has not been output by the first recognition unit but whose position and velocity have been output by the second recognition unit.

[0008] (3) In the above aspect (2), the second recognition unit has a function of recognizing the presence of an object whose position has not been output by the first recognition unit.

[0009] (4) In the above aspect (1), the first recognition unit and the second recognition unit each perform a process by executing a processing procedure that is at least partially common to both units.

[0010] (5): In the above aspect (1), each of the first recognition unit and the second recognition unit is realized by one or more processors performing processing as the first recognition unit and processing as the second recognition unit in a time-sharing manner.

[0011] (6) In the above aspect (1), the first recognition unit and the second recognition unit are realized by separate processors performing processing.

[0012] (7): In the above aspect (1), the first recognition unit has a function of repeatedly outputting the position and velocity of the object in the first period, the second recognition unit has a function of repeatedly outputting the position and velocity of the object in the second period, and at the timing when the first recognition unit outputs for the first time, a fusion unit is further provided that outputs the position output by the first recognition unit and the velocity output by the second recognition unit as the state of the object.

[0013] (8): In the above aspect (7), the fusion unit outputs the position and velocity output by the first recognition unit as the state of the object when the first recognition unit outputs for the second time or later.

[0014] (9): In the aspect (1) above, the first recognition unit has a function of repeatedly outputting the position and velocity of the object in the first period, and further includes a fusion unit that outputs the position and velocity output by the first recognition unit as the state of the object at the timing when the first recognition unit outputs.

[0015] (10): In the aspect (1) above, the first recognition unit has a function of repeatedly outputting the position and velocity of the object in the first period, the second recognition unit has a function of repeatedly outputting the position and velocity of the object in the second period, and at the timing when the first recognition unit outputs for the first time, a fusion unit is further provided that outputs, as the state of the object, the position output by the first recognition unit and the velocity output by the first recognition unit based on the position output by the first recognition unit and the position output by the second recognition unit at the previous timing before the timing.

[0016] (11): In any of the above aspects (1) to (10), after the first recognition unit outputs the first position at a first timing, at a second timing when the first recognition unit does not output and the second recognition unit outputs, the second recognition unit updates the first position by performing linear interpolation based on historical information of the positions and velocities of the object that have been recognized in the past.

[0017] (12): A control device for a moving body according to another aspect of the present invention includes a recognition device according to aspect (1) above, and a driving control unit that moves the moving body so as to avoid approaching an object whose state has been output by the recognition device.

[0018] (13): Another aspect of the present invention provides a recognition method that is executed using a recognition device that recognizes the position and speed of an object present around a moving body based on the output of a detection device that detects the surrounding conditions of the moving body, and that includes repeatedly outputting the position of the object, which is the result of processing to recognize the position of the object, in a first period, and repeatedly outputting the speed of the object, which is the result of processing to recognize the speed of the object, in a second period that is shorter than the first period.

[0019] (14): Another aspect of the present invention is a program for causing a processor of a recognition device that recognizes the position and speed of an object present around a moving body based on the output of a detection device for detecting the surrounding conditions of the moving body to repeatedly output the position of the object, which is the result of processing to recognize the position of the object, in a first period, and repeatedly output the speed of the object, which is the result of processing to recognize the speed of the object, in a second period shorter than the first period.

[0020] According to the aspects (1) to (14), it is possible to quickly follow the change in the velocity of the object.

[0021] FIG. 1 is a configuration diagram of a vehicle system 1 that uses a recognition device and a mobile body control device according to a first embodiment. FIG. 2 is a functional configuration diagram of a first control unit and a second control unit. FIG. 3 is a diagram showing an example of the configuration of the recognition unit. FIG. 4 is a diagram showing an example of the relationship between the first recognition unit and the second recognition unit. FIG. 5 is a diagram showing an example of the operation of the first recognition unit and the second recognition unit in a certain scene. FIG. 6 is a diagram for explaining the processing of a future position prediction / risk setting unit. FIG. 7 is a diagram showing an example of the operation of the first recognition unit and the second recognition unit in a certain scene according to a second embodiment. FIG. 8 is a diagram showing an example of the operation of the first recognition unit and the second recognition unit in a certain scene according to a third embodiment. FIG. 9 is a diagram showing an example of the operation of the first recognition unit and the second recognition unit in a certain scene according to a fourth embodiment.

[0022] [Overview] Hereinafter, with reference to the drawings, embodiments of a recognition device, a control device for a mobile body, a recognition method, and a program of the present invention will be described. The recognition device recognizes at least the position and speed of objects present around the mobile body. The recognition device is, for example, mounted on the mobile body, but may also be installed external to the mobile body. The control device for the mobile body controls the drive device of the mobile body to move the mobile body. A mobile body refers to any object that can move through space using mechanical power, such as a four-wheeled or two-wheeled vehicle, micromobility, or a drone, aircraft, or ship. The mobile body may move with a person or animal on board, or may be unmanned. In the following description, the mobile body is assumed to be a vehicle that moves on roads with a person on board, and an "autonomous driving control device" will be used as an example of a control device.

[0023] <First embodiment> [Overall configuration] Fig. 1 is a configuration diagram of a vehicle system 1 that uses a recognition device and a mobile object control device according to the first embodiment. The vehicle on which the vehicle system 1 is mounted may be, for example, a two-wheeled, three-wheeled, or four-wheeled vehicle, and its drive source may be an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination of these. The electric motor operates using power generated by a generator connected to the internal combustion engine, or discharged power from a secondary battery or a fuel cell.

[0024] The vehicle system 1 includes, for example, a camera 10, a radar device 12, a light detection and ranging (LIDAR) 14, an object recognition device 16, a communication device 20, a human machine interface (HMI) 30, vehicle sensors 40, a navigation device 50, a map positioning unit (MPU) 60, a driving operator 80, an automatic driving control device 100, a driving force output device 200, a brake device 210, and a steering device 220. These devices and equipment are connected to each other via multiplexed communication lines such as a controller area network (CAN) communication line, serial communication lines, a wireless communication network, etc. Note that the configuration shown in FIG. 1 is merely an example, and some of the configuration may be omitted, or other configurations may be added.

[0025] The camera 10 is a digital camera that uses a solid-state imaging element such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). The camera 10 is attached to any location of a vehicle (hereinafter referred to as vehicle M) in which the vehicle system 1 is installed. When capturing an image of the front, the camera 10 is attached to the top of the front windshield, the back of the rearview mirror, or the like. The camera 10 periodically and repeatedly captures images of the periphery of the vehicle M, for example. The camera 10 may be a stereo camera.

[0026] The radar device 12 emits radio waves such as millimeter waves around the vehicle M and detects radio waves reflected by objects (reflected waves) to detect at least the position (distance and direction) of the objects. The radar device 12 is attached to any location on the vehicle M. The radar device 12 may detect the position and speed of the objects using an FM-CW (Frequency Modulated Continuous Wave) method.

[0027] The LIDAR 14 irradiates the periphery of the vehicle M with light (or electromagnetic waves with wavelengths similar to light) and measures the scattered light. The LIDAR 14 detects the distance to an object based on the time between light emission and light reception. The irradiated light is, for example, pulsed laser light. The LIDAR 14 is attached to any location on the vehicle M.

[0028] The object recognition device 16 performs sensor fusion processing on the detection results from some or all of the camera 10, the radar device 12, and the LIDAR 14 to recognize the position, type, speed, etc. of the object. The object recognition device 16 outputs the recognition results to the autonomous driving control device 100. The object recognition device 16 may output the detection results from the camera 10, the radar device 12, and the LIDAR 14 directly to the autonomous driving control device 100. The object recognition device 16 may be omitted from the vehicle system 1.

[0029] The communication device 20 communicates with other vehicles in the vicinity of the vehicle M, or with various server devices via a wireless base station, for example, using a cellular network, a Wi-Fi network, Bluetooth (registered trademark), or DSRC (Dedicated Short Range Communication).

[0030] The HMI 30 presents various information to the occupants of the vehicle M and accepts input operations by the occupants. The HMI 30 includes various display devices, speakers, buzzers, touch panels, switches, keys, and the like.

[0031] The vehicle sensor 40 includes a vehicle speed sensor that detects the speed of the vehicle M, an acceleration sensor that detects acceleration, a yaw rate sensor that detects angular velocity around a vertical axis, a direction sensor that detects the direction of the vehicle M, and the like.

[0032] The navigation device 50 includes, for example, a Global Navigation Satellite System (GNSS) receiver 51, a navigation HMI 52, and a route determination unit 53. The navigation device 50 stores first map information 54 in a storage device such as a hard disk drive (HDD) or flash memory. The GNSS receiver 51 determines the position of the vehicle M based on signals received from GNSS satellites. The position of the vehicle M may be determined or supplemented by an inertial navigation system (INS) that uses the output of the vehicle sensors 40. The navigation HMI 52 includes a display device, a speaker, a touch panel, keys, etc. The navigation HMI 52 may share some or all of the components with the HMI 30 described above. The route determination unit 53 determines a route (hereinafter, a map route) from the position of the vehicle M determined by the GNSS receiver 51 (or any input position) to a destination input by the occupant using the navigation HMI 52, for example, by referring to the first map information 54. The first map information 54 is information that represents road shapes using, for example, links indicating roads and nodes connected by the links. The first map information 54 may also include information such as road curvature and POI (Point of Interest) information. The route on the map is output to the MPU 60. The navigation device 50 may provide route guidance using the navigation HMI 52 based on the route on the map. The navigation device 50 may be realized by, for example, the functions of a terminal device such as a smartphone or tablet device owned by the occupant. The navigation device 50 may transmit the current position and destination to a navigation server via the communication device 20 and obtain a route equivalent to the route on the map from the navigation server.

[0033] The MPU 60 includes, for example, a recommended lane determination unit 61, and stores second map information 62 in a storage device such as an HDD or flash memory. The recommended lane determination unit 61 divides the route on the map provided by the navigation device 50 into a plurality of blocks (for example, every 100 meters in the vehicle's traveling direction), and determines a recommended lane for each block by referring to the second map information 62. The recommended lane determination unit 61 determines, for example, which lane from the left the vehicle should travel in. When a branch point is present on the route on the map, the recommended lane determination unit 61 determines a recommended lane so that the vehicle M can travel on a reasonable route to the branch point.

[0034] The second map information 62 is map information with higher accuracy than the first map information 54. The second map information 62 includes, for example, information on the centers of lanes or information on lane boundaries. The second map information 62 may also include road information, traffic regulation information, address information (address and postal code), facility information, telephone number information, information on prohibited sections where mode A or mode B (described later) is prohibited, and the like. The second map information 62 may be updated as needed by the communication device 20 communicating with another device.

[0035] The driving operators 80 include, for example, a steering wheel, an accelerator pedal, a brake pedal, a shift lever, and other operators. The driving operators 80 are fitted with sensors that detect the amount of operation or the presence or absence of operation, and the detection results are output to the automatic driving control device 100 or some or all of the driving force output device 200, the brake device 210, and the steering device 220.

[0036] The autonomous driving control device 100 includes, for example, a first control unit 120 and a second control unit 160. The first control unit 120 and the second control unit 160 are each realized by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by a combination of software and hardware. The program may be stored in advance in a storage device (a storage device having a non-transitory storage medium) such as an HDD or flash memory of the autonomous driving control device 100, or may be stored in a removable storage medium such as a DVD or CD-ROM, and installed in the HDD or flash memory of the autonomous driving control device 100 by inserting the storage medium (non-transitory storage medium) into a drive device.

[0037] The automatic driving control device 100 is an example of a "control device for a moving body", the recognition unit 130 is an example of a "recognition device", and the combination of the action plan generation unit 140 and the second control unit 160 is an example of a "driving control unit".

[0038] FIG. 2 is a functional configuration diagram of the first control unit 120 and the second control unit 160. The first control unit 120 includes, for example, a recognition unit 130 and an action plan generation unit 140. The first control unit 120, for example, implements functions based on AI (artificial intelligence) and functions based on a predefined model in parallel. For example, the "intersection recognition" function may be implemented by executing intersection recognition using deep learning or the like and recognition based on predefined conditions (such as traffic lights and road signs that can be pattern-matched) in parallel, and then scoring and comprehensively evaluating both. This ensures the reliability of autonomous driving.

[0039] The recognition unit 130 recognizes the presence, position, speed, acceleration, and other states of objects around the vehicle M based on information input from the camera 10, the radar device 12, and the LIDAR 14 via the object recognition device 16. The position of an object is recognized as a position on an absolute coordinate system with a representative point of the vehicle M (such as the center of gravity or the center of the drive shaft) as the origin, and is used for control. The position of an object may be represented by a representative point such as the center of gravity or a corner of the object, or may be represented by an area. The "state" of an object may include the acceleration or jerk of the object, or the "behavioral state" (for example, whether or not the object is changing lanes or is about to change lanes).

[0040] The recognition unit 130 also recognizes, for example, the lane in which the vehicle M is traveling (driving lane). For example, the recognition unit 130 recognizes the driving lane by comparing the pattern of road dividing lines (e.g., an arrangement of solid and dashed lines) obtained from the second map information 62 with the pattern of road dividing lines around the vehicle M recognized from the image captured by the camera 10. Note that the recognition unit 130 may recognize the driving lane by recognizing road boundaries (road boundaries) including not only road dividing lines but also road dividing lines, shoulders, curbs, medians, guardrails, etc. In this recognition, the position of the vehicle M obtained from the navigation device 50 and the processing results of the INS may be taken into consideration. The recognition unit 130 also recognizes stop lines, obstacles, red lights, toll booths, and other road phenomena.

[0041] When recognizing the driving lane, the recognition unit 130 recognizes the position and orientation of the vehicle M with respect to the driving lane. For example, the recognition unit 130 may recognize the deviation of the reference point of the vehicle M from the center of the lane and the angle it forms with a line connecting the centers of the lanes in the traveling direction of the vehicle M as the relative position and orientation of the vehicle M with respect to the driving lane. Alternatively, the recognition unit 130 may recognize the position of the reference point of the vehicle M with respect to either side edge of the driving lane (a road dividing line or a road boundary) as the relative position of the vehicle M with respect to the driving lane.

[0042] The behavior plan generation unit 140 automatically (without driver input) generates a target trajectory for the vehicle M to travel in the future so that, in principle, the vehicle M will travel in the recommended lane determined by the recommended lane determination unit 61 and avoid approaching any objects recognized by the recognition unit 130 (excluding objects that can be overcome, such as road dividing lines, road markings, and manholes). For example, the recognition unit 130 sets a risk area centered on the object whose status has been output, and within the risk area, the recognition unit 130 sets a risk as an index value indicating the degree to which the vehicle M should not approach. The behavior plan generation unit 140 generates a target trajectory so that the vehicle M does not pass through points where the risk is equal to or greater than a predetermined value. Because some objects are moving, the risk distribution is not one per control cycle, but is set for multiple future points in time, taking into account the future position of the object predicted based on the object's speed. The target trajectory includes, for example, a speed element. For example, the target trajectory is expressed as a sequential list of points (trajectory points) that the vehicle M should reach. The trajectory points are points that the vehicle M should reach at every predetermined travel distance (e.g., about several meters) along the road, and separately, a target speed and a target acceleration are generated for every predetermined sampling time (e.g., about a few tenths of a second) as part of the target trajectory. Alternatively, the trajectory points may be positions that the vehicle M should reach at every predetermined sampling time. In this case, the information on the target speed and target acceleration is expressed as the interval between the trajectory points.

[0043] The second control unit 160 controls the driving force output device 200, the braking device 210, and the steering device 220 so that the vehicle M passes through the target trajectory generated by the action plan generation unit 140 at the scheduled time.

[0044] The second control unit 160 includes, for example, an acquisition unit 162, a speed control unit 164, and a steering control unit 166. The acquisition unit 162 acquires information on the target trajectory (trajectory points) generated by the action plan generation unit 140 and stores it in a memory (not shown). The speed control unit 164 controls the driving force output device 200 or the brake device 210 based on a speed element associated with the target trajectory stored in the memory. The steering control unit 166 controls the steering device 220 according to the curvature of the target trajectory stored in the memory. The processing of the speed control unit 164 and the steering control unit 166 is realized, for example, by a combination of feedforward control and feedback control. As an example, the steering control unit 166 executes a combination of feedforward control according to the curvature of the road ahead of the vehicle M and feedback control based on the deviation from the target trajectory.

[0045] Driving force output device 200 outputs driving force (torque) to the drive wheels to drive the vehicle. Driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, a transmission, etc., and an ECU (Electronic Control Unit) that controls these. The ECU controls the above components in accordance with information input from second control unit 160 or information input from driving operator 80.

[0046] Brake device 210 includes, for example, a brake caliper, a cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor according to information input from second control unit 160 or information input from driving operator 80, so that brake torque corresponding to the braking operation is output to each wheel. Brake device 210 may include a backup mechanism that transmits hydraulic pressure generated by operation of a brake pedal included in driving operator 80 to the cylinder via a master cylinder. Note that brake device 210 is not limited to the configuration described above, and may also be an electronically controlled hydraulic brake device that controls an actuator according to information input from second control unit 160 to transmit hydraulic pressure from a master cylinder to the cylinder.

[0047] The steering device 220 includes, for example, a steering ECU and an electric motor. The electric motor applies a force to, for example, a rack and pinion mechanism to change the direction of the steered wheels. The steering ECU drives the electric motor to change the direction of the steered wheels in accordance with information input from the second control unit 160 or information input from the driving operator 80.

[0048] [Regarding the More Detailed Configuration of the Recognition Unit] Here, the configuration of the recognition unit 130 will be described in more detail. To realize the above functions, the recognition unit 130 has the configuration described below. FIG. 3 is a diagram showing an example of the configuration of the recognition unit 130. The recognition unit 130 includes, for example, a first recognition unit 132, a second recognition unit 134, a fusion unit 136, and a future position prediction / risk setting unit 138.

[0049] The first recognition unit 132 repeatedly outputs the position of the object, which is the result of processing for recognizing the position of the object, in a first cycle.

[0050] The second recognition unit 134 repeatedly outputs the object's velocity, which is the result of processing to recognize the object's velocity, at a second period shorter than the first period. In other words, the second recognition unit 134 performs processing at a higher speed than the first recognition unit 132. For example, the second recognition unit 134 performs processing at a frequency several to several dozen times higher than the first recognition unit 132. The second recognition unit 134 has a function of provisionally recognizing the presence and position of an object whose position has not been output by the first recognition unit 132. There are no particular restrictions on this function, and it is sufficient if it is simpler and less burdensome than the processing performed by the first recognition unit 132. As an example, this function is realized by a process that combines a contour extraction process and a size recognition process. This function may be stopped when the first recognition unit 132 begins to output the position, or it may be continued even after the first recognition unit 132 begins to output the position.

[0051] When the first recognition unit 132 outputs a position and the second recognition unit 134 outputs a velocity (and position), the fusion unit 136 outputs the position output by the first recognition unit 132 and the velocity output by the second recognition unit 134 as the state of the object; and when the first recognition unit 132 does not output a position but the second recognition unit 134 outputs a position (provisionally recognized as described above) and velocity, the fusion unit 136 outputs the position and velocity output by the second recognition unit 134 as the state of the object.

[0052] The first recognition unit 132 and the second recognition unit 134 each perform processing by executing at least a portion of a common processing procedure. For example, the first recognition unit 132 and the second recognition unit 134 each input an image captured by the camera 10 (or an image that has undergone preprocessing) into a trained model such as a deep neural network (DNN) to recognize and output the position, velocity, etc. of an object. The second recognition unit 134 compresses the input image to a lower resolution than the image input to the first recognition unit 132, thereby performing processing faster than the first recognition unit 132. FIG. 4 is a diagram illustrating an example of the relationship between the first recognition unit 132 and the second recognition unit 134. In this case, the input image input to the first recognition unit 132 is temporally thinned out, resulting in a smaller number of frames than the input image input to the second recognition unit 134.

[0053] 5 is a diagram showing an example of the operation of the first recognition unit 132 and the second recognition unit 134 in a certain scene. In this diagram, the second recognition unit 134 performs processing at a frequency three times that of the first recognition unit 132. The control timing is a virtual time that arrives at a predetermined cycle (which is the same as the processing cycle of the second recognition unit 134 in this embodiment). At each control timing, an image captured by the camera 10 is repeatedly input. For convenience, in this diagram, the control timing is assumed to start from 1.

[0054] At control timing 1, the second recognition unit 134 provisionally recognizes the presence and position of an object for the first time. The second recognition unit 134 also recognizes the object's velocity. At this time, the first recognition unit 132 or the second recognition unit 134 (or another functional unit) predicts the object's position and velocity at the next control timing 2, for example, by calculating a covariance matrix generated by arranging the states calculated before that control timing, its eigenvalues, and interpolated values ​​for future expansion (next prediction). This next prediction process is repeatedly executed at each control timing.

[0055] At control timing 1, the state output by the recognition unit 130 is defined as a provisional recognition of the object's position and a definitive recognition of its speed. The degree of control over the object is changed depending on whether the recognition is provisional or definitive. For example, for an object whose position is provisionally recognized, the risk is calculated to be smaller than for an object whose position is definitively recognized, and processing is performed to relax the degree of control.

[0056] At control timing 2, the second recognition unit 134 updates the position and velocity of the object based on the result of the next prediction at control timing 1 and the result of processing the input image.

[0057] At control timing 3, the first recognition unit 132 updates the object position based on the result of the next prediction at control timing 2 and the result of processing the input image. Also, the second recognition unit 134 updates the object speed based on the result of the next prediction at control timing 2 and the result of processing the input image. At this time, it is defined that the presence of the object is confirmed, and the position of the object is also confirmed to be recognized definitively.

[0058] At control timing 4, the second recognition unit 134 updates the object velocity based on the result of the next prediction at control timing 3 and the result of processing the input image.

[0059] At control timing 5, the second recognition unit 134 updates the object velocity based on the result of the next prediction at control timing 4 and the result of processing the input image.

[0060] At control timing 6, the first recognition unit 132 updates the object's position based on the result of the next prediction at control timing 5 and the result of processing the input image. The second recognition unit 134 also updates the object's speed based on the result of the next prediction at control timing 5 and the result of processing the input image. Thereafter, the same processing as at control timings 4 to 6 is repeatedly executed.

[0061] The future position prediction / risk setting unit 138 predicts the future position of the object based on the object's state (position, velocity) output by the fusion unit 136 and sets a risk for the object. FIG. 6 is a diagram for explaining the processing of the future position prediction / risk setting unit 138. The future position prediction / risk setting unit 138 sets risk, which is an index value indicating the degree to which the vehicle M should not enter or approach, on an assumed plane S, which is a virtual plane that represents the space around the vehicle M as a two-dimensional plane viewed from above. A larger risk value indicates that the vehicle M should not enter or approach, and a value closer to zero indicates that it is more favorable for the vehicle M to travel. However, this relationship may be reversed. If the moving object is a flying object such as a drone rather than a vehicle, the future position prediction / risk setting unit 138 may perform similar processing in three-dimensional space rather than on the assumed plane S.

[0062] The future position prediction / risk setting unit 138 sets the risk on the imaginary plane S not only for the current time, but also for the future position of the object predicted in advance, which is specified at regular time intervals, such as the current time t, after Δt (time t + Δt), after 2Δt (time t + 2Δt), etc.

[0063] The future position prediction / risk setting unit 138 sets risks for traffic participants (moving targets) such as vehicles, pedestrians, and bicycles on an assumed plane S, with contour lines of ellipses or circles based on the direction of travel and speed, and sets a fixed value of risk for impassable areas. In the figure, DM is the direction of travel of vehicle M. R(M1) is the risk of stopped vehicle M1, and R(P) is the risk of pedestrian P. Since pedestrian P is moving in a direction crossing the road, a risk is set at each future point in time at a position different from the current time. The same applies to moving vehicles, bicycles, etc. R(BD) is the risk of impassable area BD. In the figure, the darkness of the hatching indicates the risk value, with darker hatching indicating a greater risk. The future position prediction / risk setting unit 138 may set the risk so that the value increases the further away from the center of the lane.

[0064] By performing processing with this configuration, the recognition unit 130 and the autonomous driving control device 100 can quickly track changes in the speed of an object. As described with reference to FIGS. 3 to 5 , the second recognition unit 134 performs processing using compressed input images, thereby shortening the overall processing time and enabling faster processing than the first recognition unit 132, which performs processing using high-resolution images. The advantage of processing using high-resolution images is that it can accurately detect objects that are primarily far away from the vehicle M, but it has been found that there is not much difference in performance for objects located near the vehicle M. On the other hand, when performing processing to set risks based on the future position of an object, it is desirable to quickly track events such as sudden changes in the object's speed (e.g., a pedestrian suddenly starting to run, or another vehicle suddenly stopping). In this regard, in the recognition unit 130 of the embodiment, the second recognition unit 134, which mainly recognizes speed, performs high-speed processing, so sudden changes in the object's speed can be detected quickly. Therefore, changes in the object's speed can be quickly tracked, and the behavior of the vehicle M can be quickly controlled.

[0065] Each of the first recognition unit 132 and the second recognition unit 134 is realized, for example, by one or more processors performing processing as the first recognition unit 132 and processing as the second recognition unit 134 in a time-shared manner. The same may be true for the fusion unit 136 and the future position prediction / risk setting unit 138. Alternatively, each of the first recognition unit 132 and the second recognition unit 134 may be realized by separate processors performing processing. The same may be true for the fusion unit 136 and the future position prediction / risk setting unit 138.

[0066] Second Embodiment Next, a second embodiment will be described. The second embodiment differs from the first embodiment in that the first recognition unit 132 starts outputting the position of an object when it recognizes the position of the object, and outputs both the position and the velocity of the object after it recognizes the velocity of the object. Furthermore, the second embodiment differs from the first embodiment in that after the first recognition unit 132 recognizes the position of the object, the second recognition unit 134 updates the position of the object based on the velocity recognized by the first recognition unit 132 at a control timing when the first recognition unit 132 is not operating. In the following description, components having the same functions as those in the first embodiment will be assigned the same reference numerals and names, and detailed description thereof will be omitted.

[0067] 7 is a diagram showing an example of the operations of the first recognition unit 132 and the second recognition unit 134 in a certain scene according to the second embodiment. In this diagram, the second recognition unit 134 performs processing at a frequency three times that of the first recognition unit 132.

[0068] At control timing 1, the second recognition unit 134 provisionally recognizes the presence and position of an object for the first time. The second recognition unit 134 also recognizes the object's speed. That is, the position and speed are recognized based on the object recognition result ("high-speed object recognition result") by the second recognition unit 134, which performs object recognition at high speed. At this time, the second recognition unit 134 (or another functional unit) predicts the object's position and speed at the next control timing 2, for example, by calculating a covariance matrix generated by arranging the states calculated before that control timing, its eigenvalues, and interpolated values ​​for future expansion (next prediction). This next prediction process is repeatedly executed at each control timing. At control timing 1, the state output by the recognition unit 130 is defined as tentatively recognized for the object's position and definitively recognized for its speed.

[0069] At control timing 2, the second recognition unit 134 updates the position and velocity of the object based on the result of the next prediction at control timing 1 and the result of processing the input image.

[0070] Control timing 3 is the timing at which the first recognition unit 132 performs output for the first time. At this control timing 3, the first recognition unit 132 updates the object position based on the result of the next prediction at control timing 2 and the result of processing the input image. That is, the object position is updated based on the object recognition result ("normal object recognition result") by the first recognition unit 132 performing object recognition at normal speed. Also, the second recognition unit 134 updates the object velocity based on the result of the next prediction at control timing 2 and the result of processing the input image. That is, the object velocity is updated based on the high-speed object recognition result by the second recognition unit 134. At this time, it is defined that the presence of the object is confirmed, and the object position is also definitively recognized.

[0071] At control timing 4, the second recognition unit 134 updates the object's velocity based on the result of the next prediction at control timing 3 and the result of processing the input image. Furthermore, the second recognition unit 134 updates the object's position based on the velocity. For example, the second recognition unit 134 updates the object's position based on linear interpolation. The second recognition unit 134 estimates the object's current position by using the object's position recognized by the first recognition unit 132 at control timing 3 as a reference position and adding the object's movement amount calculated using linear interpolation (the movement amount from control timing 3 to control timing 4) to this reference position, and updates the object's position to the estimated position. For example, the linear interpolation uses a covariance matrix (position and velocity history information) generated by arranging states calculated before the control timing in question, its eigenvalues, etc.

[0072] At control timing 5, the second recognition unit 134 updates the object's velocity based on the result of the next prediction at control timing 4 and the result of processing the input image. Furthermore, the second recognition unit 134 updates the object's position based on the velocity. For example, the second recognition unit 134 updates the object's position by linear interpolation. The second recognition unit 134 estimates the current object's position by setting the object's position updated at control timing 3 as a reference position and adding the object's movement amount calculated using linear interpolation (the movement amount from control timing 3 to control timing 5) to this reference position, and updates the object's position to the estimated position. Alternatively, the second recognition unit 134 estimates the current object's position by setting the object's position updated at control timing 4 as a reference position and adding the object's movement amount calculated using linear interpolation (the movement amount from control timing 4 to control timing 5) to this reference position, and updates the object's position to the estimated position.

[0073] That is, after the first recognition unit 132 outputs the first position at a first timing, at a second timing when the first recognition unit 132 does not output but the second recognition unit 134 does output, the second recognition unit 134 updates the first position by performing linear interpolation based on historical information of the positions and velocities of objects that have been recognized in the past.

[0074] Control timing 6 is the timing at which the first recognition unit 132 performs the second or subsequent output. At control timing 6, the first recognition unit 132 updates the object's position based on the result of the next prediction at control timing 5 and the result of processing the input image. Also, at control timing 6, information on the object's position recognized by the first recognition unit 132 has been accumulated, making it possible to estimate the object's speed based on the position information. Therefore, the first recognition unit 132 updates the object's speed based on the recognition result of the object's position previously recognized by the first recognition unit 132 (e.g., the recognition result at control timing 3) and the result of processing the input image. Thereafter, the same processing as at control timings 4 to 6 is repeatedly executed.

[0075] That is, the first recognition unit 132 has a function of repeatedly outputting the position and velocity of an object in a first period. The second recognition unit 134 has a function of repeatedly outputting the position and velocity of an object in a second period. When the first recognition unit 132 outputs for the first time, the fusion unit 136 outputs the position output by the first recognition unit 132 and the velocity output by the second recognition unit 134 as the state of the object. Furthermore, when the first recognition unit 132 outputs for the second time or later, the fusion unit 136 outputs the position and velocity output by the first recognition unit 132 as the state of the object.

[0076] According to the second embodiment described above, it is possible to quickly track changes in the velocity of an object, thereby quickly controlling the behavior of the moving object. Furthermore, after the velocity of the object is recognized by the first recognition unit 132, the position and velocity recognized by the first recognition unit 132 are output as the state of the object, thereby improving the accuracy of estimating the position and velocity of the object. Furthermore, after the position of the object is recognized by the first recognition unit 132, at a control timing when the first recognition unit 132 is not operating, the second recognition unit 134 updates the object's position based on the velocity, thereby improving the accuracy of estimating the object's position and effectively reducing the impact of position errors on the fusion unit and improving fusion performance.

[0077] <Third Embodiment> Next, a third embodiment will be described. The third embodiment differs from the first and second embodiments in that the first recognition unit 132 does not output anything until it recognizes the speed, and outputs both the speed and the position after it recognizes the speed. In the following description, components having the same functions as those in the first and second embodiments will be given the same reference numerals and names, and detailed descriptions thereof will be omitted.

[0078] 8 is a diagram showing an example of the operations of the first recognition unit 132 and the second recognition unit 134 in a certain scene according to the third embodiment. In this diagram, the second recognition unit 134 performs processing at a frequency three times that of the first recognition unit 132.

[0079] At control timing 1, the second recognition unit 134 provisionally recognizes the presence and position of an object for the first time. The second recognition unit 134 also recognizes the object's speed. At this time, the second recognition unit 134 (or another functional unit) predicts the object's position and speed at the next control timing 2, for example, by calculating a covariance matrix generated by arranging the states calculated before that control timing, its eigenvalues, and interpolated values ​​for future expansion (next prediction). This next prediction process is repeatedly executed at each control timing. At control timing 1, the state output by the recognition unit 130 is defined as tentatively recognized for the object's position and definitively recognized for its speed.

[0080] At control timing 2, the second recognition unit 134 updates the position and velocity of the object based on the result of the next prediction at control timing 1 and the result of processing the input image.

[0081] At control timing 3, the second recognition unit 134 updates the position and speed of the object based on the result of the next prediction at control timing 2 and the result of processing the input image. Control timing 3 is also the timing at which the first recognition unit 132 performs its first operation. At this control timing 3, the first recognition unit 132 recognizes the position of the object based on the result of the next prediction at control timing 2 and the result of processing the input image, but does not output the recognized position but stores it in memory (not shown).

[0082] At control timing 4, the second recognition unit 134 updates the position and velocity of the object based on the result of the next prediction at control timing 3 and the result of processing the input image.

[0083] At control timing 5, the second recognition unit 134 updates the position and velocity of the object based on the result of the next prediction at control timing 4 and the result of processing the input image.

[0084] Control timing 6 is the timing at which the first recognition unit 132 first outputs. At this control timing 6, the first recognition unit 132 updates the object's position based on the result of the next prediction at control timing 5 and the result of processing the input image. At this time, the presence of the object is confirmed, and the object's position is also defined as being definitively recognized. Furthermore, at this control timing 6, information on the object's position recognized by the first recognition unit 132 has been accumulated, making it possible to estimate the object's speed based on this position information. Therefore, the object's speed is updated based on the object's position recognition result previously recognized by the first recognition unit 132 (e.g., the recognition result at control timing 3) and the result of processing the input image.

[0085] At control timing 7, the second recognition unit 134 updates the object's velocity based on the result of the next prediction at control timing 6 and the result of processing the input image. Furthermore, the second recognition unit 134 updates the object's position based on the velocity. For example, the second recognition unit 134 updates the object's position by linear interpolation. The second recognition unit 134 uses the object's position updated at control timing 6 as a reference position and adds the object's movement amount calculated using linear interpolation (the movement amount from control timing 6 to control timing 7) to this reference position to estimate the object's current position, and updates the object's position to the estimated position.

[0086] At control timing 8, the second recognition unit 134 updates the object's velocity based on the result of the next prediction at control timing 7 and the result of processing the input image. Furthermore, the second recognition unit 134 updates the object's position based on the velocity. For example, the second recognition unit 134 updates the object's position by linear interpolation. The second recognition unit 134 estimates the current object's position by setting the object's position updated at control timing 6 as a reference position and adding the object's movement amount calculated using linear interpolation (the movement amount from control timing 6 to control timing 8) to this reference position, and updates the object's position at the estimated position. Alternatively, the second recognition unit 134 estimates the current object's position by setting the object's position updated at control timing 7 as a reference position and adding the object's movement amount calculated using linear interpolation (the movement amount from control timing 7 to control timing 8) to this reference position, and updates the object's position at the estimated position.

[0087] Control timing 9 is the timing at which the first recognition unit 132 performs a second output. At this control timing 9, the first recognition unit 132 updates the object's position based on the result of the next prediction at control timing 8 and the result of processing the input image. The first recognition unit 132 also updates the object's velocity based on the recognition results of the object's position previously recognized by the first recognition unit 132 (e.g., the recognition results at control timings 3 and 6) and the result of processing the input image. As a result, at the timing at which the first recognition unit 132 performs an output, the fusion unit 136 outputs the position and velocity output by the first recognition unit 132 as the object's state. Thereafter, processing similar to that at control timings 7 to 9 is repeatedly executed.

[0088] According to the third embodiment described above, it is possible to quickly track changes in the velocity of an object, and therefore to quickly control the behavior of a moving object. Furthermore, after information on the position of an object recognized by the first recognition unit 132 is accumulated and velocity estimation becomes possible, the velocity of the object is updated based on the velocity recognized by the first recognition unit 132 at the control timing when the first recognition unit 132 operates. This improves the accuracy of estimating the position and velocity of the object. Furthermore, it is possible to maintain design diversity, for example, when incorporating the second recognition unit 134 into an existing system that only includes the first recognition unit 132.

[0089] <Fourth Embodiment> Next, a fourth embodiment will be described. The fourth embodiment differs from the first to third embodiments in that, at the control timing when the first recognition unit 132 operates for the first time, the velocity of the object is updated based on the recognition result by the second recognition unit 134 at the control timing immediately preceding the control timing in question and the recognition result by the first recognition unit 132 at the control timing in question. In the following description, components having the same functions as those in the first to third embodiments will be assigned the same reference numerals and names, and detailed description thereof will be omitted.

[0090] 9 is a diagram showing an example of the operations of the first recognition unit 132 and the second recognition unit 134 in a certain scene according to the fourth embodiment. In this diagram, the second recognition unit 134 performs processing at a frequency three times that of the first recognition unit 132.

[0091] At control timing 1, the second recognition unit 134 provisionally recognizes the presence and position of an object for the first time. The second recognition unit 134 also recognizes the object's speed. At this time, the second recognition unit 134 (or another functional unit) predicts the object's position and speed at the next control timing 2, for example, by calculating a covariance matrix generated by arranging the states calculated before that control timing, its eigenvalues, and interpolated values ​​for future expansion (next prediction). This next prediction process is repeatedly executed at each control timing. At control timing 1, the state output by the recognition unit 130 is defined as tentatively recognized for the object's position and definitively recognized for its speed.

[0092] At control timing 2, the second recognition unit 134 updates the position and velocity of the object based on the result of the next prediction at control timing 1 and the result of processing the input image.

[0093] Control timing 3 is the timing at which the first recognition unit 132 first outputs. At control timing 3, the first recognition unit 132 updates the object's position based on the result of the next prediction at control timing 2 and the result of processing the input image. At this time, the presence of the object is confirmed, and the object's position is defined as having been definitively recognized. The first recognition unit 132 also updates the object's velocity based on the recognition result by the second recognition unit 134 at control timing 2 (the position resulting from the previous high-speed object recognition) and the recognition result by the first recognition unit 132 at control timing 3 (the position resulting from the current normal object recognition). As a result, at the timing at which the first recognition unit 132 first outputs, the fusion unit 136 outputs, as the object's state, the position output by the first recognition unit 132 and the velocity output by the first recognition unit 132 based on the position output by the first recognition unit 132 and the position output by the second recognition unit 134 at the previous timing before that timing.

[0094] At control timing 4, the second recognition unit 134 updates the object's velocity based on the result of the next prediction at control timing 3 and the result of processing the input image. Furthermore, the second recognition unit 134 updates the object's position based on the velocity. For example, the second recognition unit 134 updates the object's position by linear interpolation. The second recognition unit 134 estimates the current object position by using the object position recognized by the first recognition unit 132 at control timing 3 as a reference position and adding the object's movement amount calculated using linear interpolation (the movement amount from control timing 3 to control timing 4) to this reference position, and updates the object's position to the estimated position.

[0095] At control timing 5, the second recognition unit 134 updates the object's velocity based on the result of the next prediction at control timing 4 and the result of processing the input image. Furthermore, the second recognition unit 134 updates the object's position based on the velocity. For example, the second recognition unit 134 updates the object's position by linear interpolation. The second recognition unit 134 estimates the current object's position by setting the object's position updated at control timing 3 as a reference position and adding the object's movement amount calculated using linear interpolation (the movement amount from control timing 3 to control timing 5) to this reference position, and updates the object's position to the estimated position. Alternatively, the second recognition unit 134 estimates the current object's position by setting the object's position updated at control timing 4 as a reference position and adding the object's movement amount calculated using linear interpolation (the movement amount from control timing 4 to control timing 5) to this reference position, and updates the object's position to the estimated position.

[0096] Control timing 6 is the timing at which the first recognition unit 132 performs the second or subsequent output. At control timing 6, the first recognition unit 132 updates the object's position based on the result of the next prediction at control timing 5 and the result of processing the input image. Also, at control timing 6, information on the object's position recognized by the first recognition unit 132 has been accumulated, making it possible to estimate the object's speed based on the position information. Therefore, the first recognition unit 132 updates the object's speed based on the recognition result of the object's position previously recognized by the first recognition unit 132 (e.g., the recognition result at control timing 3) and the result of processing the input image. Thereafter, the same processing as at control timings 4 to 6 is repeatedly executed.

[0097] According to the fourth embodiment described above, it is possible to quickly track changes in the velocity of an object, and therefore to quickly control the behavior of a moving object. Furthermore, at the control timing when the first recognition unit 132 operates for the first time, the velocity of the object is updated based on the recognition result by the second recognition unit 134 at the previous timing and the recognition result by the first recognition unit 132 at the current control timing. This allows for design diversity to be maintained, for example, when the second recognition unit 134 is incorporated into an existing system that only includes the first recognition unit 132.

[0098] In the above description, the control device is assumed to be mounted on the vehicle M (i.e., a moving body), but this is not limited to this. The control device may be installed at a location away from the moving body, acquire output data from the camera 10, radar device 12, etc. via communication, and transmit drive instruction signals to the moving body, i.e., remotely control the moving body.

[0099] Furthermore, the embodiments described above are merely examples, and the present invention is not limited to the configurations of these embodiments. It is also possible to combine the functions or configurations included in each embodiment as appropriate. For example, as described in the second to fourth embodiments, it is also possible to incorporate into the first embodiment a configuration in which, after the first recognition unit 132 recognizes the position of an object, the position of the object is updated based on the velocity recognized by the second recognition unit 134 at a control timing when the first recognition unit 132 is not operating.

[0100] The above-described embodiment can be expressed as follows: A recognition device that recognizes the position and speed of an object present in the vicinity of a moving body based on the output of a detection device that detects the surrounding conditions of the moving body, comprising: one or more storage media that store computer-readable instructions; and a processor connected to the one or more storage media, wherein the processor executes the computer-readable instructions to: repeatedly output the position of the object, which is a result of processing to recognize the position of the object, in a first period; and repeatedly output the speed of the object, which is a result of processing to recognize the speed of the object, in a second period that is shorter than the first period.

[0101] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention.

[0102] REFERENCE SIGNS LIST 10 Camera 12 Radar device 14 LIDAR 16 Object recognition device 100 Automatic driving control device 120 First control unit 130 Recognition unit 132 First recognition unit 134 Second recognition unit 136 Fusion unit 138 Future position prediction / risk setting unit 140 Action plan generation unit 160 Second control unit

Claims

1. A recognition device that recognizes the position and speed of an object present around a moving body based on an output of a detection device for detecting a surrounding situation of the moving body, a first recognition unit that repeatedly outputs the position of the object, which is a result of performing processing to recognize the position of the object, in a first period; a second recognition unit that repeatedly outputs the velocity of the object, which is a result of processing to recognize the velocity of the object, at a second period that is shorter than the first period; Equipped with the second recognition unit performs processing to recognize the speed of the object using a low-resolution image having a lower resolution than that used in processing to recognize the position of the object by the first recognition unit; recognition device.

2. the second recognition unit has a function of provisionally recognizing a position of an object whose position has not been output by the first recognition unit, When the position is output from the first recognition unit and the velocity is output from the second recognition unit, the position output from the first recognition unit and the velocity output from the second recognition unit are output as the state of the object; a fusion unit that outputs the position and velocity output by the second recognition unit as the state of an object for which the position is not output from the first recognition unit but the position and velocity are output from the second recognition unit, The recognition device according to claim 1.

3. the second recognition unit has a function of recognizing the presence of an object whose position has not been output by the first recognition unit; The recognition device according to claim 2.

4. The first recognition unit and the second recognition unit each perform a process by executing a processing procedure that is at least partially common to both the first recognition unit and the second recognition unit. The recognition device according to claim 1.

5. The first recognition unit and the second recognition unit are each realized by one or more processors performing processing as the first recognition unit and processing as the second recognition unit in a time-division manner. The recognition device according to claim 1.

6. The first recognition unit and the second recognition unit are realized by separate processors performing processing, The recognition device according to claim 1.

7. the first recognition unit has a function of repeatedly outputting the position and velocity of the object in the first period; the second recognition unit has a function of repeatedly outputting the position and velocity of the object at the second period; a fusion unit that outputs the position output by the first recognition unit and the velocity output by the second recognition unit as the state of the object at the timing when the first recognition unit performs an output for the first time, The recognition device according to claim 1.

8. the fusion unit outputs the position and velocity output by the first recognition unit as the state of the object at the timing when the first recognition unit performs a second or subsequent output; The recognition device according to claim 7.

9. the first recognition unit has a function of repeatedly outputting the position and velocity of the object in the first period; a fusion unit that outputs the position and velocity output by the first recognition unit as the state of the object at the timing when the first recognition unit outputs the position and velocity, The recognition device according to claim 1.

10. the first recognition unit has a function of repeatedly outputting the position and velocity of the object in the first period; the second recognition unit has a function of repeatedly outputting the position and velocity of the object at the second period; a fusion unit that, at a timing when the first recognition unit performs an output for the first time, outputs, as a state of the object, a position output by the first recognition unit and a velocity output by the first recognition unit based on the position output by the first recognition unit and a position output by the second recognition unit at a previous timing before the timing. The recognition device according to claim 1.

11. After the first recognition unit outputs the first position at a first timing, at a second timing when the first recognition unit does not output but the second recognition unit outputs, the second recognition unit updates the first position by performing linear interpolation based on history information of positions and velocities of the object that have been recognized in the past. A recognition device according to any one of claims 1 to 10.

12. The recognition device according to claim 1; a driving control unit that moves the moving body so as to avoid approaching the object whose state is output by the recognition device; A control device for a moving body comprising:

13. A recognition method executed by a recognition device that recognizes positions and velocities of objects present around a moving body based on an output of a detection device for detecting a surrounding situation of the moving body, the method comprising: repeatedly outputting the position of the object, which is a result of performing processing for recognizing the position of the object, in a first period; repeatedly outputting the velocity of the object, which is a result of performing processing to recognize the velocity of the object, at a second period shorter than the first period; Equipped with The process for recognizing the velocity of the object is performed using a low-resolution image having a lower resolution than the process for recognizing the position of the object. Recognition method.

14. a processor of a recognition device that recognizes the position and speed of an object present around a moving object based on an output of a detection device for detecting a surrounding situation of the moving object; repeatedly outputting the position of the object, which is a result of performing processing for recognizing the position of the object, in a first period; repeatedly outputting the velocity of the object, which is a result of performing processing to recognize the velocity of the object, at a second period shorter than the first period; A program for executing The process for recognizing the velocity of the object is performed using a low-resolution image having a lower resolution than the process for recognizing the position of the object. program.

15. the second recognition unit performs processing with a lower load than processing performed by the first recognition unit; The recognition device according to claim 1.

16. The vehicle further includes a setting unit that sets a non-travelable area for the object based on a traveling direction or a speed of the object. The recognition device according to claim 1.