Device for recognizing object

Abstract

To provide a device capable of lowering the computational load for recognizing an object.SOLUTION: An object recognizing device includes: an image sensor which acquires an image in the travelling direction of a vehicle; a controller which cuts out, from an image captured by the image sensor, an object region containing a first region being lower than the image center and a second region being upper than image center, adjacent to the first region, and having an area smaller than the first region, and sets them as a target image; and a model which is configured to take as input the positions of the target image and the object region and output a recognition result regarding road lane lines and at least one of a traffic signal and a sign-board.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an apparatus for recognizing an object. [Background technology]

[0002] Patent Document 1 discloses a device that recognizes objects based on an image captured by a camera. This device recognizes objects in the image using a CNN (Convolutional Neural Network). This device generates an inverted image by flipping the camera image upside down, and inputs the inverted image into a CNN model. In an environment where an upper limit on the number of recognizable objects is determined, this device uses the inverted image to preferentially recognize objects that appear in the lower half of the screen. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2022-084282 Summary of the Invention [Problem to be solved by the invention]

[0004] Objects that a driver should pay attention to while driving a vehicle include signals or signs that appear in the upper half of the screen. For this reason, it is required to recognize not only objects that appear in the lower half of the screen, but also signals or signs that appear in the upper half of the screen. However, if the entire screen is processed, there is a risk that the calculation load will increase. The present disclosure provides a device that can reduce the calculation load for recognizing objects. [Means for solving the problem]

[0005] According to one embodiment of the present disclosure, an object recognition device includes an image sensor, a controller, and a model. The image sensor acquires an image in the traveling direction of a vehicle. The controller cuts out a target area from the image acquired by the image sensor, the target area including a first area below the center of the image and a second area above the center of the image, adjacent to the first area and smaller in area than the first area, to obtain a target image. The model is configured to receive the target image and the position of the target area and output a recognition result regarding lane lines on a road and at least one of a traffic light and a signboard. Effect of the Invention

[0006] According to the present disclosure, a technique is provided that can reduce the computational load required for object recognition. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of a vehicle equipped with a recognition device according to an embodiment. [Diagram 2] FIG. 2 is a diagram showing an example of a target region in a camera image. [Diagram 3] FIG. 3 is a diagram showing another example of a target region in a camera image. [Figure 4] FIG. 4 is a diagram showing another example of a target region in a camera image. [Diagram 5] FIG. 5 is a flowchart showing an example of the operation of the recognition device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the description of the drawings, the same elements are given the same reference numerals, and duplicated description will be omitted.

[0009] [Vehicle configuration] Fig. 1 is a block diagram showing an example of the configuration of a vehicle equipped with an apparatus according to an embodiment. As shown in Fig. 1, a recognition apparatus 1 (an example of an apparatus) is mounted on a vehicle 2 as an example. The vehicle 2 is an autonomous vehicle as an example. The vehicle 2 is not limited to an autonomous vehicle, and may be a vehicle that assists driving operations. The assistance of driving operations also includes cases where only information is notified.

[0010] The recognition device 1 includes an image sensor 10, a yaw rate sensor 11, a turn indicator sensor 12, a processing ECU 13 (an example of a controller), and a recognition model 14 (an example of a model). The ECU (Electronic Control Unit) is an electronic control unit having a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a CAN (Controller Area Network) communication circuit, etc. The recognition device 1 does not necessarily need to include the yaw rate sensor 11 and the turn indicator sensor 12.

[0011] The image sensor 10 is, for example, a camera. The image sensor 10 acquires an image in the traveling direction of the vehicle 2. Hereinafter, the image acquired by the image sensor 10 will be referred to as a camera image. The yaw rate sensor 11 detects the yaw rate of the vehicle 2. The turn indicator sensor 12 detects the direction of a turn indicator that indicates the traveling direction of the vehicle 2.

[0012] The processing ECU 13 cuts out a target area from the camera image to generate a target image. The target image is an image that is the subject of recognition processing by the recognition model 14. The target area includes a first area and a second area. The first area is an area below the center of the camera image. The second area is an area above the center of the camera image. The second area is adjacent to the first area and has a smaller area than the first area. In other words, the target area is a convex shape, an L-shape, or an inverted L-shape.

[0013] As an example, the processing ECU 13 divides the camera image into blocks and specifies an area on the image based on an identifier assigned to the block. FIG. 2 is a diagram showing an example of a target area in a camera image. As shown in FIG. 2, the camera image G is divided into blocks. As an example, the camera image is divided into a matrix of 8 rows and 8 columns. An identifier is assigned to each block. For example, the identifier of the leftmost block in the top row is B1. The identifier of the block adjacent to the right of block B1 is B2. In this manner, an identifier Bn is assigned to each block. The processing ECU 13 can specify an area at any position using the block identifier Bn.

[0014] The processing ECU 13 determines the position of a first region R1, which is an area below the center of the camera image G, and the position of a second region R2, which is an area above the center of the camera image G. The first region R1 is provided at a position below the center of the camera image G. Generally, the lane lines L1 and L2 of the road are captured below the center of the camera image G. In other words, the first region R1 is set in the expectation that the lane lines L1 and L2 of the road will be captured. The first region R1 may be set to include the preceding vehicle V1 and the adjacent vehicle V2.

[0015] The second region R2 is provided at a position above the center of the camera image G. Generally, the signboard S and the traffic light are captured above the center of the camera image G. In other words, the second region R2 is set in the expectation that the signboard S and the traffic light will be captured. Generally, the signboard S and the signboard are smaller than the lane lines L1 and L2 of the road. Therefore, the area of ​​the second region R2 can be made smaller than that of the first region R1. The second region R2 is adjacent to the first region R1. The target region R, which is formed by connecting the first region R1 and the second region R2, can be treated as a single image.

[0016] The processing ECU 13 determines the position of the first region R1 and the position of the second region R2 based on, for example, an identifier of a block that is determined in advance as a default.

[0017] The processing ECU 13 may change the position of the first region R1 and the position of the second region R2 for each type or performance of the image sensor 10. The type or performance of the image sensor 10 is a camera performance such as wide angle or telephoto. The type or performance of the image sensor 10 can be distinguished by an identifier. For example, the processing ECU 13 has a table that associates and stores the identifier of the image sensor 10 with the position of the first region R1 and the position of the second region R2. The table is stored in the storage unit of the ECU. The processing ECU 13 acquires the identifier of the image sensor 10, and refers to the table based on the acquired identifier to determine the position of the first region R1 and the position of the second region R2. This allows the optimal target region R to be set for each image sensor 10.

[0018] The processing ECU 13 may determine the position of the first region R1 and the position of the second region R2 based on the detection result of the yaw rate sensor 11. For example, the processing ECU 13 determines the turning direction of the vehicle 2 based on the detection result of the yaw rate sensor 11. The processing ECU 13 moves the second region R2 in the turning direction of the vehicle 2. FIG. 3 is a diagram showing another example of the target region in the camera image. The scene in FIG. 3 is a scene in which the vehicle 2 runs while turning left. In this case, as shown in FIG. 3, the position of the second region R2 is moved one block to the left compared to the target region R shown in FIG. 2. As a result, the left part of the signboard S fits into the second region R2. The processing ECU 13 may also change the position of the first region R1. The processing ECU 13 may determine the movement amount of the first region R1 and the movement amount of the second region R2 according to the magnitude of the yaw rate of the vehicle 2, and set the movement amount of the second region R2 to be larger than the movement amount of the first region R1. Thereby, the processing ECU 13 can include the lane lines L1, L2 in the first region R1, while more accurately capturing the signboard S and the traffic light, and include the signboard S and the traffic light in the second region R2. In this way, it is possible to set an optimal target region R in accordance with the movement of the vehicle 2.

[0019] The processing ECU 13 may determine the position of the first region R1 and the position of the second region R2 based on the direction indicated by the turn indicator detected by the turn indicator sensor 12. For example, the processing ECU 13 moves the second region R2 in the same direction as the direction of the turn indicator. FIG. 4 is a diagram showing another example of the target region in the camera image. The scene in FIG. 4 is a scene in which the turn indicator is flashing so that the vehicle 2 moves to the right lane. In this case, as shown in FIG. 4, the position of the first region R1 and the position of the second region R2 are moved to the right by one block compared to the target region R shown in FIG. 2. As a result, the adjacent lane and the adjacent vehicle V2 are contained in the first region R1, and the right part of the signboard S is contained in the second region R2. In this way, the optimal target region R according to the movement of the vehicle 2 can be set.

[0020] The recognition model 14 is a trained model. The recognition model 14 is trained by machine learning based on teacher data or the like. The recognition model 14 may be a CNN model. The recognition model 14 receives as input a target image and a position of a target area generated by the processing ECU 13. The position of the target area is the above-mentioned block identifier Bn. When the recognition model 14 receives as input a target image and a position of the target area, the recognition model 14 is trained to output a recognition result regarding lane lines L1 and L2 of a road and at least one of a signboard S and a traffic light. Such training is executed by a server or the like, and the trained recognition model 14 is provided to the vehicle 2 via communication or the like.

[0021] The recognition model 14 outputs the positions of lane lines L1, L2 on the road, information on signboards S, and traffic light signal information as recognition results. The recognition results are output to a user interface 3. The user interface 3 is, for example, a display, a speaker, etc. The user interface 3 notifies the driver of the recognition results. Alternatively, the recognition results are output to another ECU 4. The other ECU 4 is, for example, an autonomous driving ECU, etc.

[0022] The processing ECU 13 may determine the position of the first region R1 and the position of the second region R2 based on the positions of the lane lines L1 and L2 recognized by the recognition model 14. For example, the scene in FIG. 4 is a scene in which the vehicle 2 moves to the right lane. In this case, the lane lines L1 and L2 move relatively to the left. For this reason, the processing ECU 13 moves the positions of the first region R1 and the second region R2 in a direction opposite to the direction in which the positions of the recognized lane lines L1 and L2 move. For example, as shown in FIG. 4, the positions of the first region R1 and the second region R2 have moved one block to the right compared to the target region R shown in FIG. 2. As a result, the adjacent lane and the adjacent vehicle V2 are contained in the first region R1, and the right part of the signboard S is contained in the second region R2. In this way, the optimal target region R according to the movement of the vehicle 2 can be set.

[0023] [Recognition device operation] Fig. 5 is a flowchart showing an example of the operation of the recognition device 1. The flowchart shown in Fig. 5 starts when the recognition device 1 receives a start instruction operation.

[0024] 5, first, in step S10, the image sensor 10 of the recognition device 1 acquires a camera image G. Then, in step S12, the processing ECU 13 determines a target region R.

[0025] In Step S14, the processing ECU 13 cuts out a target region R from the camera image G to set it as a target image. In Step S16, the recognition model 14 inputs the target image cut out in Step S14. In Step S18, the recognition model 14 outputs recognition results regarding lane lines L1, L2 of the road, and at least one of a signboard S and a traffic light. When Step S18 ends, the flowchart shown in FIG. 5 ends.

[0026] [Summary of the embodiment] According to the recognition device 1, a target region R including a first region R1 below the center of the camera image G and a second region R2 above the center of the camera image G, adjacent to the first region R1 and smaller in area than the first region R1, is cut out from the camera image G acquired by the image sensor 10, and is set as a target image. By making the target image an image having a convex shape, it is possible to reduce the region to be calculated while capturing the lane lines L1, L2 and at least one of the traffic light and the signboard S within the image. Therefore, the recognition model 14 can reduce the calculation load for recognizing the lane lines L1, L2 and at least one of the traffic light and the signboard S, compared to the case where calculation is performed on an entire image.

[0027] Although exemplary embodiments have been described above, various omissions, substitutions, and modifications may be made without being limited to the above-described exemplary embodiments.

[0028] The present disclosure includes the following aspects. [Clause 1] An image sensor for acquiring an image in the traveling direction of the vehicle; a controller that cuts out a target area from the image acquired by the image sensor, the target area including a first area below a center of the image and a second area above the center of the image, adjacent to the first area and having an area smaller than the first area, to obtain a target image; a model configured to receive the target image and the location of the target region and output a recognition result relating to lane lines on a road and at least one of a traffic light and a sign; An object recognition device comprising: [Clause 2] The apparatus described in clause 1, wherein the controller has a table that stores an identifier of the image sensor in association with the position of the first area and the position of the second area, and determines the position of the first area and the position of the second area based on the identifier of the image sensor and the table. [Clause 3] A yaw rate sensor is further provided to detect a yaw rate of the vehicle. 3. The apparatus of claim 1 or 2, wherein the controller determines the position of the first region and the position of the second region based on a yaw rate of the vehicle. [Article 4] The vehicle further includes a sensor for detecting a direction of a turn signal indicating a traveling direction of the vehicle, The device of any one of clauses 1 to 3, wherein the controller determines a position of the first area and a position of the second area based on a direction indicated by the direction indicator. [Article 5] 5. The device according to claim 1, wherein the controller determines a position of the first area and a position of the second area based on positions of the lane lines recognized by the model. [Explanation of symbols]

[0029] 1...recognition device (an example of a device), 2...vehicle, 10...image sensor, 11...yaw rate sensor, 12...turn signal sensor, 13...processed ECU (an example of a controller), 14...recognition model (an example of a model).

Claims

1. An image sensor for acquiring an image in the traveling direction of the vehicle; a controller that cuts out a target area from the image acquired by the image sensor, the target area including a first area below a center of the image and a second area above the center of the image, adjacent to the first area and having an area smaller than the first area, to obtain a target image; a model configured to receive the target image and the location of the target region and output a recognition result relating to lane lines on a road and at least one of a traffic light and a sign; An object recognition device comprising:

2. The device of claim 1, wherein the controller has a table that stores an identifier of the image sensor in association with the position of the first area and the position of the second area, and determines the position of the first area and the position of the second area based on the identifier of the image sensor and the table.

3. A yaw rate sensor is further provided to detect a yaw rate of the vehicle. The apparatus of claim 1 or 2, wherein the controller determines the location of the first region and the location of the second region based on a yaw rate of the vehicle.

4. The vehicle further includes a sensor for detecting a direction of a turn signal indicating a traveling direction of the vehicle, The apparatus of claim 1 or 2, wherein the controller determines the location of the first region and the location of the second region based on a direction indicated by the direction indicator.

5. The apparatus of claim 1 or 2, wherein the controller determines the position of the first region and the position of the second region based on the positions of the lane lines recognized by the model.

Images