Object recognition device

Abstract

To provide a technique for suppressing a processing load of object recognition in an object recognition device mounted on a vehicle in a manner that does not significantly impair performance of object recognition.SOLUTION: An object recognition unit 111 includes a plurality of learned models for recognizing an object in an image imaged by a camera 21 mounted on a vehicle, which is configured to be able to recognize the object using the plurality of learned models. A peripheral situation determination unit 112 is configured to determine a peripheral situation of the vehicle. The object recognition unit 111 identifies, on the basis of a determination result of the peripheral situation input from the peripheral situation determination unit 112, a necessary learned model, which is a learned model highly necessary for recognizing the object, among the plurality of learned models. Then, when recognizing the object, the object recognition unit 111 suppresses use of a learned model other than the necessary learned models.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to an object recognition device. [Background technology]

[0002] 2. Description of the Related Art There is known an object recognition device that recognizes an object in an image captured by a camera mounted on a vehicle. For example, Patent Document 1 discloses a device that changes a target area for object recognition processing in a captured image in accordance with the situation of the vehicle, and executes the object recognition processing in the changed target area. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2015-201584 A Summary of the Invention [Problem to be solved by the invention]

[0004] In object recognition processing, the processing load tends to increase as the performance of object recognition is improved, for example by increasing the resolution of the image to be processed. On the other hand, there are strict resource constraints required for devices mounted on vehicles. As a result of detailed studies by the inventors, it has been found that it is necessary to suppress the processing load of object recognition in a manner that does not significantly impair the performance of object recognition in an object recognition device mounted on a vehicle.

[0005] One aspect of the present disclosure provides a technique for reducing the processing load of object recognition in an object recognition device mounted on a vehicle without significantly impairing the object recognition performance. [Means for solving the problem]

[0006] One aspect of the present disclosure is an object recognition device mounted on a vehicle, comprising an object recognition unit (111) and a surrounding situation determination unit (112). The object recognition unit has a plurality of trained models for recognizing objects in an image captured by a camera mounted on the vehicle, and is configured to be able to recognize objects using the plurality of trained models. The surrounding situation determination unit is configured to determine the surrounding situation of the vehicle. The object recognition unit identifies a necessary trained model, which is a trained model that is highly necessary for recognizing an object, from among the plurality of trained models based on the surrounding situation determined by the surrounding situation determination unit, and suppresses the use of trained models other than the necessary trained model.

[0007] According to this configuration, in an object recognition device mounted on a vehicle, the processing load for object recognition can be reduced without significantly impairing the object recognition performance. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing a configuration of an object recognition device. [Diagram 2] FIG. 2 is a block diagram showing a functional configuration of the object recognition device. [Diagram 3] 13 is a table showing conditions under which the object recognition unit identifies a necessary trained model. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, exemplary embodiments of the present disclosure will be described with reference to the drawings. [1. Configuration] [1-1. Overall composition] 1 is a device that is mounted on a vehicle and recognizes various objects present around the vehicle. Hereinafter, the vehicle on which the object recognition device 1 is mounted is referred to as the host vehicle. In this embodiment, the object recognition device 1 is a device that recognizes objects in a captured image that is an image captured by a camera 21, which will be described later.

[0010] The object recognition device 1 is mainly configured with a microcomputer (hereinafter, microcomputer) having a CPU 101, a ROM 103, a RAM 104, a semiconductor memory such as a flash memory (hereinafter, memory 102). Various functions of the object recognition device 1 are realized by the CPU 101 executing a program stored in a non-transitive substantial recording medium. In this example, the memory 102 corresponds to the non-transitive substantial recording medium storing the program. Furthermore, the program is executed to execute a method corresponding to the program. The number of microcomputers constituting the object recognition device 1 may be one or more. Furthermore, the method of realizing various functions possessed by the object recognition device 1 is not limited to software, and some or all of the elements may be realized using one or more hardware. For example, when the above functions are realized by an electronic circuit that is hardware, the electronic circuit may be realized by a digital circuit including a large number of logic circuits, an analog circuit, or a combination of these.

[0011] The vehicle is equipped with a camera 21, a distance measuring device 22, a GPS receiver 23, a road traffic information acquisition device 24, a map information storage device 25, a vehicle speed sensor 26, a steering angle sensor 27, a turn signal sensor 28, a speaker 31, a display 32, and a vehicle control device 33.

[0012] The camera 21 is an in-vehicle camera that is mounted on the front side of the vehicle and is configured to be able to capture an image of the area ahead of the vehicle. The distance measuring device 22 is mounted on the front side of the vehicle and measures the distance to an object present in a predetermined area in front of the vehicle. The distance measuring device 22 in this embodiment is a LiDAR device that measures the distance to an object by irradiating light and detecting the reflected light of the irradiated light from the object. Note that LiDAR is an abbreviation for Light Detection and Ranging.

[0013] The GPS receiver 23 is a device that receives GPS signals transmitted from GPS satellites and detects the current position of the vehicle. GPS is an abbreviation for Global Positioning System.

[0014] The road traffic information acquisition device 24 is a device that acquires road traffic information from an external information and communication system. Examples of the external information and communication system include VICS (registered trademark) and TSPS. Examples of the road traffic information include information on traffic congestion, traffic regulations, road construction, accidents, etc.

[0015] The map information storage device 25 is a device that stores map information. Here, the map information stored in the map information storage device 25 may be pre-stored in the vehicle itself, or may be acquired from an external server or the like.

[0016] The vehicle speed sensor 26 is a sensor that detects the speed of the host vehicle. The steering angle sensor 27 is a sensor that detects the steering angle of the steering wheel of the host vehicle. The turn indicator sensor 28 is a sensor that detects the operating state of the turn indicator of the vehicle. Examples of the operating state of the turn indicator include a state in which the left turn indicator is flashing, a state in which the right turn indicator is flashing, or a state in which none of the turn indicators are operating.

[0017] The speaker 31 is a speaker mounted in the passenger compartment of the vehicle. The display 32 is a display mounted in the vehicle cabin. In this embodiment, the display 32 is a display device that displays an image that can be viewed by the occupants of the vehicle. However, the display 32 is not limited to a display device, and may be, for example, an LED light that emits light that can be viewed by the occupants of the vehicle.

[0018] The vehicle control device 33 is a device that executes vehicle control for the host vehicle. Examples of vehicle control include deceleration, stopping, steering in a direction that can avoid a collision between the host vehicle and an object, and the like.

[0019] [1-2. Functional configuration of the object recognition device] Next, the functional configuration of the object recognition device 1 will be described with reference to FIG. The object recognition device 1 includes an object recognition unit 111, a surrounding conditions determination unit 112, and a driving condition determination unit 113 as functional blocks realized by the CPU 101 executing a program, that is, as virtual components.

[0020] [1-2-1. Object recognition section] The object recognition unit 111 is configured to receive a captured image from the camera 21 and recognize objects in the captured image. Examples of objects recognized by the object recognition unit 111 include traffic signals, preceding vehicles, pedestrians, a road on which the host vehicle is traveling, bicycle paths, pedestrian paths, and crosswalks. Note that the traffic signals are also recognized, for example, by their illumination colors and whether or not their arrow lights are illuminated. The object recognition device 1 is configured to recognize various objects present around the host vehicle by the object recognition unit 111 recognizing objects in the captured image.

[0021] The object recognition unit 111 has a trained model for recognizing an object in a captured image. The trained model is, for example, generated by machine learning and configured by a multi-layer neural network. The object recognition unit 111 is configured to be able to recognize an object using the trained model. Specifically, the object recognition unit 111 inputs an image of a recognition target range, which is at least a part of a captured image, to the trained model. Then, the trained model recognizes an object in the image to be recognized and outputs an object recognition result. In this way, the object recognition unit 111 is configured to be able to recognize an object.

[0022] The object recognition unit 111 has a plurality of trained models with different recognition target ranges. In this embodiment, the object recognition unit 111 has four types of trained models: a close-distance model, a long-distance model, a left model, and a right model.

[0023] The short-distance model is a trained model for recognizing objects that may exist in a nearby range, which is a range close to the vehicle itself. The recognition range of an image that the short-distance model recognizes is the entire range of the captured image, as shown in FIG. 2. However, an image with a lower resolution than the captured image is used for the image that the short-distance model recognizes. This is because objects that may exist in the nearby range are displayed large in the captured image, and therefore can be sufficiently recognized even in an image with a lower resolution.

[0024] The long distance model is a trained model for recognizing objects that may exist in a long distance range, which is a range farther than the near distance range. The recognition range of the image recognized by the long distance model is a part of the range centered on the infinity point in the captured image, which indicates the far distance, as shown in Figure 2. The resolution of the image recognized by the long distance model is the same as the resolution of the captured image. That is, the image that is the recognition target of the close-range model is an image with a wider recognition target range and lower resolution than the image that is the recognition target of the long-range model.

[0025] The left model is a trained model for recognizing objects that may exist on the left side in front of the vehicle. The recognition target range of the image to be recognized by the left model is a range located on the left side of the captured image, and in this embodiment, it is the left half range of the captured image as shown in Fig. 2. The resolution of the image to be recognized by the left model is the same as the resolution of the captured image.

[0026] The right model is a trained model for recognizing objects that may exist in front of the vehicle on the right side. The recognition target range of the image recognized by the right model is a range located on the right side of the captured image, and in this embodiment, it is the right half of the captured image as shown in Fig. 2. The resolution of the image recognized by the right model is the same as the resolution of the captured image.

[0027] The object recognition unit 111 recognizes objects that may exist in the near range, far range, and on the left and right sides in front of the vehicle by using a near-distance model, a far-distance model, a left model, and a right model, respectively. That is, the images to be recognized by the four types of trained models have different recognition target ranges. For this reason, the four types of trained models are used in parallel.

[0028] However, depending on the surrounding conditions of the vehicle, a trained model that is less necessary for object recognition may occur. For example, when a traffic light in front of the vehicle displays a stop instruction, the need for a long-distance model is reduced. The object recognition unit 111 suppresses the use of the trained model with low necessity in order to suppress the processing load of object recognition. This is because if the trained model with low necessity is used, the effect on the performance of object recognition is small even if its use is suppressed.

[0029] Specifically, the object recognition unit 111 identifies a necessary trained model, which is a trained model that is highly necessary for recognizing an object, from among the four types of trained models, by a method described below. Then, the object recognition unit 111 suppresses the use of trained models other than the identified necessary trained model when recognizing an object. In this embodiment, the object recognition unit 111 suppresses the use of trained models other than the necessary trained model by not using trained models other than the necessary trained model when recognizing an object.

[0030] [1-2-2. Surrounding Situation Judgment Section] The surrounding situation determination unit 112 is configured to determine the surrounding situation, which is the situation around the host vehicle, and output the determination result to the object recognition unit 111 .

[0031] Specifically, the surrounding situation determination unit 112 determines the situation regarding the traffic lights around the vehicle based on the object recognition result input from the object recognition unit 111. The object recognition result is information indicating an object recognized by the object recognition unit 111 and an area in the captured image in which the object exists. In this embodiment, the surrounding situation determination unit 112 determines whether or not a traffic light exists ahead of the vehicle as the situation regarding the traffic lights around the vehicle, and if a traffic light exists, the display content of the traffic light. The display content of the traffic light is classified into a stop instruction and a proceed permission. If the traffic light is green or yellow, the surrounding situation determination unit 112 determines that the traffic light indicates the proceed permission. Also, if the traffic light is red, the surrounding situation determination unit 112 determines that the traffic light indicates the stop instruction. However, even if the traffic light is red, if the arrow light is on, the surrounding situation determination unit 112 determines that the traffic light indicates the proceed permission.

[0032] Furthermore, the surrounding situation determination unit 112 determines whether or not a pedestrian is present around the host vehicle based on the object recognition result input from the object recognition unit 111 . Moreover, the surrounding situation judgment unit 112 judges whether or not the width of the road on which the vehicle is traveling is narrower than a predetermined width based on the object recognition result input from the object recognition unit 111. Specifically, the surrounding situation judgment unit 112 measures the width of the road on which the vehicle is traveling from an area in the captured image where the road on which the vehicle is traveling exists, and judges whether or not the measured width is narrower than a predetermined width. Here, the predetermined width is, for example, a width that makes it difficult for an oncoming vehicle to pass each other. However, the predetermined width is not limited to this.

[0033] Furthermore, the surrounding situation determination unit 112 determines whether or not a bicycle path or a pedestrian path exists around the vehicle, based on the object recognition result input from the object recognition unit 111. The surrounding situation determination unit 112 also determines the situation of the preceding vehicle based on the measurement result of the distance between the host vehicle and the preceding vehicle input from the distance measuring device 22. Here, the situation of the preceding vehicle refers to whether or not there is a preceding vehicle ahead of the host vehicle, and if there is a preceding vehicle, whether the distance between the vehicles is narrow or wide. The surrounding situation determination unit 112 determines that the distance between the vehicles is narrow when the distance between the vehicles is less than a predetermined distance, and determines that the distance between the vehicles is wide when the distance between the vehicles is equal to or greater than the predetermined distance. The predetermined distance is, for example, 50 m. However, the predetermined distance is not limited to this.

[0034] In addition, the surrounding condition judgment unit 112 judges whether or not the vehicle is located within an intersection based on the detection result of the current position of the vehicle input from the GPS receiver 23 and the map information input from the map information storage device 25.

[0035] The surrounding situation determination unit 112 also determines the situation regarding traffic congestion around the vehicle based on the road traffic information input from the road traffic information acquisition device 24. In this embodiment, the surrounding situation determination unit 112 determines whether or not there is traffic congestion ahead of the vehicle as the situation regarding traffic congestion around the vehicle. The destination of the vehicle is, for example, the planned travel route if a navigation device is providing driving guidance along the planned travel route, and is the planned travel route if the vehicle travels along the road in other circumstances. When the surrounding situation determination unit 112 inputs road traffic information including information that traffic congestion occurs within a predetermined range at the destination of the vehicle, the surrounding situation determination unit 112 determines that there is traffic congestion ahead of the vehicle. In addition, since traffic regulations, road construction, accidents, etc. may cause traffic congestion, the surrounding situation determination unit 112 also determines that there is traffic congestion ahead of the vehicle when it inputs road traffic information including information that traffic regulations, road construction, accidents, etc. occur within a predetermined range at the destination of the vehicle. The predetermined range is, for example, a range of 500 m from the vehicle. However, the predetermined range is not limited to this.

[0036] [1-2-3. Driving condition determination unit] The driving state determination unit 113 is configured to determine a driving state, which is the state of the host vehicle while it is driving, and output the determination result to the object recognition unit 111. The driving state includes the vehicle speed and the steering state.

[0037] Specifically, the traveling state determination unit 113 determines whether the host vehicle is in one of four states, namely, high-speed traveling, low-speed traveling, stopped, and completely stopped, based on the vehicle speed input from the vehicle speed sensor 26. The traveling state determination unit 113 determines that the host vehicle is in high-speed traveling when the vehicle speed is equal to or higher than a first speed, and determines that the host vehicle is in low-speed traveling when the vehicle speed is equal to or higher than a second speed and lower than the first speed. The first speed is, for example, 30 km / h per hour, and the second speed is, for example, 3 km / h per hour. However, the first speed and the second speed are not limited thereto. The traveling state determination unit 113 also determines that the host vehicle is stopped when the vehicle speed is lower than the second speed, and in particular, determines that the host vehicle is in a completely stopped state when the vehicle speed continues to be 0 km / h per hour for a predetermined time or more. The predetermined time is, for example, 5 seconds. However, the predetermined time is not limited thereto. Furthermore, when the brakes of the host vehicle are released while the host vehicle is completely stopped, the traveling state determination unit 113 determines that the host vehicle is stopped.

[0038] Further, the driving state determination unit 113 determines whether the host vehicle is in one of three states, that is, turning left, turning right, or going straight, based on the steering angle of the steering wheel input from the steering angle sensor 27. When the steering angle is equal to or greater than a predetermined angle to the left, the driving state determination unit 113 determines that the host vehicle is turning left. On the other hand, when the steering angle is equal to or greater than a predetermined angle to the right, the driving state determination unit 113 determines that the host vehicle is turning right. Then, when the host vehicle is not determined to be turning left or right, the driving state determination unit 113 determines that the host vehicle is going straight. The steering state may be determined by taking into account the operating state of the turn indicator input from the turn indicator sensor 28.

[0039] [1-3. How to identify the required trained model] Next, a method for the object recognition unit 111 to identify a necessary trained model will be described. The object recognition unit 111 identifies a necessary trained model from four types of trained models based on the judgment result of the driving state input from the driving state judgment unit 113 and the judgment result of the surrounding situation input from the surrounding situation judgment unit 112. The object recognition unit 111 may identify multiple trained models as the necessary trained models.

[0040] In this embodiment, the object recognition unit 111 identifies the necessary learned model according to the conditions shown in Fig. 3. Specifically, the left model is identified as the necessary learned model when the host vehicle is turning left. Similarly, the right model is identified as the necessary learned model when the host vehicle is turning right. Below, the conditions under which the short-distance model and the long-distance model are identified as the necessary learned models will be described. Note that when the host vehicle is completely stopped, the object recognition process is not performed, and therefore the necessary learned model is not identified.

[0041] The close-range model is identified as the necessary trained model unless all of the following conditions (1) to (6) are met. (1) The vehicle is traveling at high speed. (2) The vehicle is traveling straight. (3) There is no traffic light ahead of the vehicle. (4) The vehicle is not located within the intersection. (5) There is no vehicle ahead or the distance between the vehicle ahead is large. (6) There are no pedestrians around the vehicle.

[0042] The case where all of the above conditions (1) to (6) are satisfied is when the vehicle is traveling at high speed and there is no danger around the vehicle, for example, when the vehicle is traveling on a highway or a rural road with good visibility, etc. In this case, the long-distance model is identified as the necessary trained model.

[0043] On the other hand, the long-distance model is identified as a necessary trained model unless the following condition (7) or (8) is met. (7) Your vehicle is stopped. (8) The vehicle is traveling at a low speed and a traffic signal in front of the vehicle is displaying a stop signal.

[0044] The object recognition unit 111 specifies the necessary learned model under the above conditions. However, the above conditions are merely examples, and at least some of the conditions may be added, deleted, or changed. For example, if the width of the road on which the vehicle is traveling is narrower than a predetermined width, or if a bicycle road or pedestrian road exists around the vehicle, both the left model and the right model may be specified as the necessary learned model. Also, for example, if there is traffic congestion ahead of the vehicle, both the short-distance model and the long-distance model may be specified as the necessary learned model.

[0045] After identifying the necessary trained model, the object recognition unit 111 recognizes the object. In this embodiment, as described above, the object recognition unit 111 suppresses the use of trained models other than the necessary trained model by not using trained models other than the necessary trained model when recognizing an object.

[0046] The object recognition result by the object recognition unit 111 is used for, for example, driving assistance and driving control performed by the speaker 31, the display 32, and the vehicle control device 33. The object recognition unit 111 outputs information necessary for driving assistance and driving control to each of the speaker 31, the display 32, and the vehicle control device 33.

[0047] [2. Effects] According to the embodiment described above in detail, the following effects can be obtained. (2a) The object recognition unit 111 has a plurality of trained models for recognizing objects in a captured image, and is configured to be able to recognize objects using the plurality of trained models. Based on the surrounding situation determined by the surrounding situation determination unit 112, the object recognition unit 111 identifies a necessary trained model, which is a trained model that is highly necessary for recognizing an object, from among the plurality of trained models, and suppresses the use of trained models other than the necessary trained model.

[0048] Even if the use of trained models other than the necessary trained models, i.e., trained models that are less necessary for object recognition, is suppressed, the impact on the performance of object recognition is small. On the other hand, suppressing the use of some trained models among a plurality of trained models leads to suppression of the processing load of object recognition. Therefore, according to the configuration of this embodiment, in the object recognition device 1, the processing load of object recognition can be suppressed without significantly impairing the performance of object recognition.

[0049] (2b) The multiple trained models include a short-distance model that is a trained model for recognizing objects that may exist in a nearby range, and a long-distance model that is a trained model for recognizing objects that may exist in a distant range. The image that the short-distance model recognizes has a wider recognition range and lower resolution than the image that the long-distance model recognizes.

[0050] The size of an object in a captured image is small when the object is in a far range and large when the object is in a near range. Furthermore, among objects in a far range, objects in front of the vehicle are located in a part of the range centered on the infinity point in the captured image. Therefore, for objects in a far range, even if the recognition target range is narrowed, the object to be recognized can be included in the range. On the other hand, for objects in a near range, since they are displayed large in the captured image, it is necessary to secure a wide recognition target range for object recognition, but they can be recognized sufficiently even if the resolution of the image to be recognized is reduced.

[0051] That is, in both the close-range model and the long-range model, object recognition is performed on an image having a smaller number of pixels than the captured image, so that the processing load of object recognition is suppressed, while the performance of object recognition is also prevented from being significantly impaired due to the above-mentioned reasons. Therefore, according to the configuration of this embodiment, in the object recognition device 1, the processing load of object recognition can be suppressed without significantly impairing the performance of object recognition.

[0052] (2c) The surrounding situation determination unit 112 determines the situation regarding traffic signals around the vehicle based on the object recognition result input from the object recognition unit 111. Therefore, according to the configuration of this embodiment, it is possible to identify the necessary trained model based on the situation regarding traffic signals around the vehicle.

[0053] (2d) In this embodiment, the short-distance model is identified as the necessary learned model when the traffic light ahead of the vehicle displays a stop instruction. When the traffic light ahead of the vehicle displays a stop instruction, it is considered that the vehicle is currently stopped or will decelerate or stop immediately after. In such a situation, it is considered that there is a high need to recognize objects that may exist in the vicinity. Therefore, according to the configuration of this embodiment, by using the short-distance model in a situation where the traffic light ahead of the vehicle displays a stop instruction, it is possible to prevent a significant loss of object recognition performance.

[0054] (2e) In this embodiment, the short-distance model is specified as the necessary learned model except when all of the following conditions are met: the vehicle is traveling at high speed and going straight, there is no traffic light in front of the vehicle, the vehicle is not located in an intersection, there is no preceding vehicle or the distance between the vehicle and the preceding vehicle is wide, and there are no pedestrians around the vehicle. When all of the above conditions are met, for example, the vehicle is traveling on a highway or a rural road with good visibility. In such a situation, it is considered that there is little need to recognize objects that may exist in the vicinity. Therefore, according to the configuration of this embodiment, the use of the short-distance model can be suppressed in a situation where there is little need to recognize objects that may exist in the vicinity, thereby suppressing the processing load of object recognition without significantly impairing the performance of object recognition.

[0055] (2f) In this embodiment, the long-distance model is identified as the necessary learned model when the host vehicle is traveling at high speed and there is no danger around the host vehicle. When the host vehicle is traveling at high speed and there is no danger around the host vehicle, for example, the host vehicle may be traveling on a highway or a country road with good visibility. In such a situation, it is considered that there is a high need to recognize objects that may exist in a distant range. Therefore, according to the configuration of this embodiment, the long-distance model is used when the host vehicle is traveling at high speed and there is no danger around the host vehicle, so that the performance of object recognition is not significantly impaired.

[0056] (2g) In this embodiment, the long-distance model is identified as the necessary learned model except when the host vehicle is stopped, or when the host vehicle is traveling at a low speed and a traffic light in front of the host vehicle is displaying a stop instruction. In any of the above cases, it is considered that the host vehicle is unlikely to reach a distant location immediately after. In such a situation, it is considered that there is little need to recognize objects that may exist in a distant range. Therefore, according to the configuration of this embodiment, by making it possible to suppress the use of the long-distance model in a situation where there is little need to recognize objects that may exist in a distant range, it is possible to suppress the processing load of object recognition without significantly impairing the performance of object recognition.

[0057] (2h) When there is traffic congestion ahead of the vehicle, the need to monitor both the near and far ranges increases compared to normal conditions due to the possibility that the number of vehicles, etc. that may be present around the vehicle may increase. Therefore, according to a configuration in which both the near-distance model and the long-distance model are identified as necessary trained models when there is traffic congestion ahead of the vehicle, object recognition can be performed appropriately in a situation where there is traffic congestion ahead of the vehicle.

[0058] (2i) When the road width is narrow, the need to monitor both the left and right sides in front of the vehicle increases. Therefore, according to a configuration in which both the left model and the right model are identified as necessary trained models when the road width is narrower than a predetermined width, object recognition can be performed appropriately when the road width is narrow.

[0059] (2j) When a bicycle path or pedestrian road is present around the host vehicle, the need to monitor both the left and right sides in front of the host vehicle increases compared to when no bicycle path or pedestrian road is present around the host vehicle. Therefore, according to a configuration in which both the left model and the right model are identified as necessary trained models when a bicycle path or pedestrian road is present around the host vehicle, object recognition can be performed appropriately in a situation where a bicycle path or pedestrian road is present around the host vehicle.

[0060] (2k) In this embodiment, the object recognition unit 111 does not use trained models other than the necessary trained models. Therefore, according to the configuration of this embodiment, the processing load in the object recognition device 1 can be reduced compared to a configuration in which the object recognition unit 111 uses trained models other than the necessary trained models.

[0061] 3. Other embodiments Although the embodiments of the present disclosure have been described above, it goes without saying that the present disclosure is not limited to the above-described embodiments and can take various forms.

[0062] (3a) In the above embodiment, the surrounding situation determination unit 112 determines that a traffic signal in front of the vehicle indicates permission to proceed when the traffic signal is red but the arrow light is on. However, the configuration in which the traffic signal is determined to indicate permission to proceed is not limited to this. For example, the surrounding situation determination unit 112 may determine that a traffic signal indicates permission to proceed only when the traffic signal is red and the arrow light is on in the straight-ahead direction. Also, for example, the surrounding situation determination unit 112 may determine that a traffic signal indicates permission to proceed only when the traffic signal is red and the arrow light is on in the direction that matches the destination of the vehicle. Note that the destination of the vehicle is, as described above, a planned driving route guided by a navigation device or a planned driving route assuming that the vehicle travels along the road.

[0063] (3b) In the above embodiment, the surrounding conditions judgment unit 112 judges whether or not a bicycle road or pedestrian road exists around the vehicle based on the object recognition result input from the object recognition unit 111, but the surrounding conditions judgment unit 112 may judge whether or not a bicycle road or pedestrian road exists around the vehicle based on the object recognition result input from the object recognition unit 111. For example, the surrounding conditions judgment unit 112 may judge whether or not a bicycle road or pedestrian road exists around the vehicle based on the detection result of the current position of the vehicle input from the GPS receiver 23 and the map information input from the map information storage device 25.

[0064] (3c) In the above embodiment, the surrounding situation judgment unit 112 judges whether the width of the road on which the vehicle is traveling is narrower than a predetermined width based on the object recognition result input from the object recognition unit 111. However, the surrounding situation judgment unit 112 may judge whether the width of the road on which the vehicle is traveling is narrower than a predetermined width based on the object recognition result input from the object recognition unit 111. For example, the surrounding situation judgment unit 112 may judge the width of the road on which the vehicle is traveling based on the detection result of the current position of the vehicle output from the GPS receiver 23 and the map information output from the map information storage device 25.

[0065] (3d) In the above embodiment, the surrounding situation determination unit 112 determines whether or not the vehicle is located in an intersection based on the detection result of the current position of the vehicle input from the GPS receiver 23 and the map information input from the map information storage device 25. However, the surrounding situation determination unit 112 may determine whether or not the vehicle is located in an intersection based on the object recognition result input from the object recognition unit 111. In this case, for example, the surrounding situation determination unit 112 may determine that the vehicle is located in an intersection when an object that may be present around the intersection, such as a traffic signal or a pedestrian crossing, is recognized in the captured image.

[0066] (3e) In the above embodiment, the recognition target range of the image to be recognized by the close-up model is described as the entire range of the captured image, but the recognition target range of the image to be recognized by the close-up model is not limited to this. For example, the recognition target range of the image to be recognized by the close-up model may be a part of the captured image.

[0067] (3f) In the above embodiment, the resolution of the image to be recognized by the long-distance model is the same as the resolution of the captured image, but the image to be recognized by the long-distance model is not limited to this. For example, the image to be recognized by the long-distance model may have a lower resolution than the captured image.

[0068] (3g) In the above embodiment, the recognition target range of the image to be recognized by the left model and the right model is exemplified as being half the range on the left and right sides of the captured image, but the recognition target range of the image to be recognized by the left model and the right model is not limited to this. For example, the recognition target range of the image to be recognized by the left model and the right model may be about one-third the range on the left and right sides of the captured image. In addition, in the above embodiment, a configuration in which an image with the same resolution as the captured image is used as the image to be recognized by the left model and the right model is exemplified, but an image with a lower resolution than the captured image may be used.

[0069] (3h) In the above embodiment, the object recognition unit 111 does not use trained models other than the necessary trained models when recognizing an object, but the configuration for suppressing the use of trained models other than the necessary trained models is not limited to this. For example, the object recognition unit 111 may suppress the use of trained models other than the necessary trained models by reducing the frequency of use of trained models other than the necessary trained models.

[0070] The object recognition unit 111 is configured to periodically use each of the multiple trained models when recognizing an object. The object recognition unit 111 may, for example, lengthen the period during which trained models other than the necessary trained model are used, thereby reducing the frequency of use of trained models other than the necessary trained model. Therefore, according to this configuration, the processing load in the object recognition device 1 can be suppressed compared to a configuration in which the frequency of use of trained models other than the necessary trained model is not reduced.

[0071] (3i) In the above embodiment, the object recognition unit 111 has four types of trained models, namely, a close-distance model, a long-distance model, a left model, and a right model, but the trained models of the object recognition unit 111 are not limited to these. For example, the object recognition unit 11 may have two types of trained models, namely, a close-distance model and a long-distance model.

[0072] (3j) In the above embodiment, the surrounding situation determination unit 112 determines the surrounding situation including the situation regarding traffic signals around the vehicle, whether or not there are pedestrians around the vehicle, whether or not the width of the road on which the vehicle is traveling is narrower than a predetermined width, whether or not there is a bicycle path or a pedestrian path around the vehicle, the situation of a preceding vehicle, whether or not the vehicle is located in an intersection, and the situation regarding traffic congestion around the vehicle. However, the surrounding situation determined by the surrounding situation determination unit 112 is not limited to these. For example, the surrounding situation determination unit 112 may determine the surrounding situation including whether or not the road on which the vehicle is traveling is an expressway, that is, whether or not the vehicle is traveling on an expressway. In this case, the surrounding situation determination unit 112 may determine whether or not the vehicle is traveling on an expressway based on, for example, the detection result of the current position of the vehicle input from the GPS receiver 23 and the map information input from the map information storage device 25.

[0073] In this case, the object recognition unit 111 may specify the long-distance model as the necessary learned model when the host vehicle is traveling on a highway. When the host vehicle is traveling on a highway, it is considered that there is a high need to recognize objects that may exist in a distant range. Therefore, according to this configuration, by using the long-distance model when the host vehicle is traveling on a highway, it is possible to prevent a significant loss of object recognition performance.

[0074] In the above case, the object recognition unit 111 may specify the short-distance model as the necessary learned model, except when the vehicle is traveling on a highway. When the vehicle is traveling on a highway, it is considered that there is little need to recognize objects that may exist in the vicinity. Therefore, according to this configuration, by making it possible to suppress the use of the short-distance model in a situation where there is little need to recognize objects that may exist in the vicinity, it is possible to suppress the processing load of object recognition without significantly impairing the performance of object recognition.

[0075] (3k) In the above embodiment, a configuration was exemplified in which the surrounding situation judgment unit 112 judges whether or not there is a traffic signal ahead of the vehicle as the situation regarding traffic signals around the vehicle. However, the traffic signals judged by the surrounding situation judgment unit 112 are not limited to traffic signals that are present ahead of the vehicle.

[0076] (3l) The function of one component in the above embodiments may be distributed among multiple components, or the functions of multiple components may be integrated into one component. Also, a part of the configuration of the above embodiments may be omitted. Also, at least a part of the configuration of the above embodiments may be added to or substituted for the configuration of another of the above embodiments.

[0077] (3m) The present disclosure may be realized in various forms, including the object recognition device described above, an object recognition system including the object recognition device, a program for causing a computer to function as the object recognition device, a medium on which the program is recorded, and an object recognition method. [Explanation of symbols]

[0078] 1...object recognition device, 111...object recognition unit, 112...surrounding situation judgment unit, 113...driving condition judgment unit.

Claims

1. An object recognition device mounted on a vehicle, A surrounding situation determination unit (112) configured to determine a surrounding situation of the vehicle; An object recognition unit (111) having a plurality of trained models for recognizing objects in an image captured by a camera mounted on the vehicle and configured to be able to recognize objects using the plurality of trained models, the object recognition unit identifying a necessary trained model that is a trained model that is highly necessary for recognizing an object among the plurality of trained models based on the surrounding situation determined by the surrounding situation determination unit, and suppressing use of the trained models other than the necessary trained model; Equipped with The object recognition unit is configured to be able to recognize an object in a recognition target range, which is at least a part of the captured image and is a range set for each of the plurality of trained models; The plurality of trained models include a short-distance model, which is a trained model for recognizing an object that may exist in a nearby range that is a range near the vehicle, and a long-distance model, which is a trained model for recognizing an object that may exist in a distant range that is a range farther than the nearby range, The image to be recognized by the close-range model has a wider recognition range and a lower resolution than the image to be recognized by the long-range model; the surrounding situation determination unit determines a situation regarding traffic signals around the vehicle based on a recognition result of the object recognition unit; An object recognition device, wherein the object recognition unit identifies the short-distance model as the necessary trained model when the surrounding situation judgment unit determines that the traffic signal in front of the vehicle is displaying a stop instruction.

2. The object recognition device according to claim 1 , The surrounding situation determination unit is configured to determine a situation regarding traffic congestion around the vehicle based on road traffic information acquired from an external information and communication system, An object recognition device, wherein the object recognition unit identifies the short-distance model and the long-distance model as the necessary learned models when the surrounding situation judgment unit determines that there is traffic congestion ahead of the vehicle.

3. An object recognition device mounted on a vehicle, A surrounding situation determination unit (112) configured to determine a surrounding situation of the vehicle; An object recognition unit (111) having a plurality of trained models for recognizing objects in an image captured by a camera mounted on the vehicle and configured to be able to recognize objects using the plurality of trained models, the object recognition unit identifying a necessary trained model that is a trained model that is highly necessary for recognizing an object among the plurality of trained models based on the surrounding situation determined by the surrounding situation determination unit, and suppressing use of the trained models other than the necessary trained model; Equipped with The object recognition unit is configured to be able to recognize an object in a recognition target range, which is at least a part of the captured image and is a range set for each of the plurality of trained models; The plurality of trained models include a short-distance model, which is a trained model for recognizing an object that may exist in a nearby range that is a range near the vehicle, and a long-distance model, which is a trained model for recognizing an object that may exist in a distant range that is a range farther than the nearby range, The image to be recognized by the close-range model has a wider recognition range and a lower resolution than the image to be recognized by the long-range model; The surrounding situation determination unit is configured to determine a situation regarding traffic congestion around the vehicle based on road traffic information acquired from an external information and communication system, An object recognition device, wherein the object recognition unit identifies the short-distance model and the long-distance model as the necessary learned models when the surrounding situation judgment unit determines that there is traffic congestion ahead of the vehicle.

4. The object recognition device according to claim 3, The surrounding situation judgment unit judges the situation regarding traffic signals around the vehicle based on the recognition result of the object recognition unit.

5. The object recognition device according to any one of claims 1 to 4, The plurality of trained models further includes a left model, which is a trained model for recognizing an object that may be present on the left side in front of the vehicle, and a right model, which is a trained model for recognizing an object that may be present on the right side in front of the vehicle; An object recognition device wherein the image to be recognized by the left model has a recognition target range located to the left of the captured image, and the image to be recognized by the right model has a recognition target range located to the right of the captured image.

6. An object recognition device mounted on a vehicle, A surrounding situation determination unit (112) configured to determine a surrounding situation of the vehicle; An object recognition unit (111) having a plurality of trained models for recognizing objects in an image captured by a camera mounted on the vehicle and configured to be able to recognize objects using the plurality of trained models, the object recognition unit identifying a necessary trained model that is a trained model that is highly necessary for recognizing an object among the plurality of trained models based on the surrounding situation determined by the surrounding situation determination unit, and suppressing use of the trained models other than the necessary trained model; Equipped with The object recognition unit is configured to be able to recognize an object in a recognition target range, which is at least a part of the captured image and is a range set for each of the plurality of trained models; The plurality of trained models include a short-distance model, which is the trained model for recognizing an object that may exist in a nearby range that is a range near the vehicle, a long-distance model, which is the trained model for recognizing an object that may exist in a distant range that is a range farther than the nearby range, a left model, which is the trained model for recognizing an object that may exist on the left side in front of the vehicle, and a right model, which is the trained model for recognizing an object that may exist on the right side in front of the vehicle; The image to be recognized by the close-range model has a wider recognition range and a lower resolution than the image to be recognized by the long-range model; An object recognition device wherein the image to be recognized by the left model has a recognition target range located to the left of the captured image, and the image to be recognized by the right model has a recognition target range located to the right of the captured image.

7. The object recognition device according to claim 6, The surrounding situation judgment unit judges the situation regarding traffic signals around the vehicle based on the recognition result of the object recognition unit.

8. The object recognition device according to any one of claims 5 to 7, The surrounding situation determination unit is configured to determine a width of a road on which the vehicle is traveling, An object recognition device, wherein the object recognition unit identifies the left model and the right model as the necessary trained models when the surrounding conditions judgment unit determines that the width of the road is narrower than a predetermined width.

9. The object recognition device according to any one of claims 5 to 8, The surrounding situation determination unit is configured to determine a situation regarding a bicycle path or a pedestrian path around the vehicle, An object recognition device, wherein the object recognition unit identifies the left model and the right model as the necessary trained models when the surrounding conditions judgment unit determines that the bicycle path or the pedestrian path exists.

10. The object recognition device according to any one of claims 1 to 9, Further comprising a running state determination unit (113) configured to determine a running state of the vehicle, The object recognition device, wherein the object recognition unit identifies the long-distance model as the necessary learned model when the driving condition judgment unit judges that the vehicle is driving at high speed and the surrounding situation judgment unit judges that there is no danger around the vehicle.

11. The object recognition device according to any one of claims 1 to 10, The surrounding situation determination unit is configured to determine whether or not the host vehicle is traveling on an expressway, The object recognition device, wherein the object recognition unit identifies the long-distance model as the necessary learned model when the surrounding situation judgment unit determines that the vehicle is traveling on an expressway.

12. The object recognition device according to any one of claims 1 to 11, The object recognition device, wherein the object recognition unit suppresses use of the trained models other than the necessary trained models by not using the trained models other than the necessary trained models.

13. The object recognition device according to any one of claims 1 to 12, The object recognition device is configured such that, when recognizing an object, the object recognition unit periodically uses each of the plurality of trained models, and suppresses the use of the trained models other than the necessary trained model by reducing the frequency of use of the trained models other than the necessary trained model.

Images