Device for recognizing object

Abstract

The present disclosure relates to an apparatus for recognizing an object, and provides an apparatus capable of reducing the load of an operation for recognizing an object. This object recognition device is provided with: an image sensor that acquires an image in the direction of travel of a vehicle; a controller that cuts, from the image captured by the image sensor, a target region including a first region below the center of the image and a second region above the center of the image, the second region being a region adjacent to the first region and having a smaller area than the first region, as a target image; and a model configured so as to input the target image and the position of the target region, and output a recognition result regarding the lane line of the road and at least one of the traffic light and the signboard.

Description

Technical Field

[0001] The present disclosure relates to a device for identifying an object. Background Art

[0002] Patent document 1 discloses a device for recognizing objects based on an image captured by a camera. The device uses CNN (Convolutional Neural Network) to recognize objects in an image. The device generates a reversed image obtained by reversing the camera image upside down, and inputs the reversed image into a CNN model. In an environment where the upper limit of the number of recognizable objects is determined, the device preferentially recognizes objects reflected in the lower half of the screen by using the reversed image.

[0003] Prior art literature

[0004] Patent Document 1: Japanese Patent Application Publication No. 2022-084282 Summary of the invention

[0005] Problems to be solved by the invention

[0006] Objects that the driver should pay attention to during vehicle driving also include signals or signs reflected in the upper half of the screen. Therefore, it is required to identify not only objects reflected in the lower half of the screen, but also signals or signs reflected in the upper half of the screen. However, when the entire screen is used as a processing object, the computational load may increase. The present disclosure provides a device that can reduce the computational load for identifying objects.

[0007] Technical solutions to solve problems

[0008] One embodiment of the present disclosure relates to an object recognition device comprising an image sensor, a controller, and a model. The image sensor obtains an image of the direction of travel of a vehicle. The controller crops (crops) an object area from the image obtained by the image sensor as an object image, wherein the object area includes a first area below the center of the image and a second area above the center of the image, wherein the second area is adjacent to the first area and has a smaller area than the first area. The model is configured to input the position of the object image and the object area, and output a recognition result of at least one of the lane lines, signals (traffic lights), and signboards of the road.

[0009] Effects of the Invention

[0010] According to the present disclosure, it is possible to provide a technology capable of reducing the load of calculations for recognizing an object. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1This is a block diagram showing an example of the configuration of a vehicle including a recognition device according to an embodiment.

[0012] Figure 2 A diagram showing an example of a target area in a camera image.

[0013] Figure 3 This is a diagram showing another example of the object area in the camera image.

[0014] Figure 4 This is a diagram showing another example of the object area in the camera image.

[0015] Figure 5 This is a flowchart showing an example of the operation of the recognition device. DETAILED DESCRIPTION

[0016] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the description of the drawings, the same elements are denoted by the same reference numerals, and repeated descriptions are omitted.

[0017] [Vehicle configuration]

[0018] Figure 1 1 is a block diagram showing an example of a configuration of a vehicle including a device according to an embodiment. Figure 1 As shown, as an example, the recognition device 1 (an example of a device) is mounted on a vehicle 2. As an example, the vehicle 2 is an autonomous driving vehicle. The vehicle 2 is not limited to an autonomous driving vehicle, and may also be a vehicle for assisting driving operations. Driving operation assistance also includes a case where only information is notified (reported).

[0019] The recognition device 1 includes an image sensor 10, a yaw rate sensor 11, a direction indicator sensor 12, a processing ECU 13 (an example of a controller), and a recognition model 14 (an example of a model). The ECU (Electronic Control Unit) is an electronic control unit having a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and a CAN (Controller Area Network) communication circuit. The recognition device 1 may not include the yaw rate sensor 11 and the direction indicator sensor 12.

[0020] As an example, the image sensor 10 is a camera. The image sensor 10 obtains an image of the traveling direction of the vehicle 2. Hereinafter, the image obtained by the image sensor 10 is referred to as a camera image. The yaw rate sensor 11 detects the yaw rate of the vehicle 2. The direction indicator sensor 12 detects the direction of the direction indicator indicating the traveling direction of the vehicle 2.

[0021] The processing ECU 13 generates an object image by cutting out the object area from the camera image. The object image refers to an image that becomes the object of recognition processing by the recognition model 14. The object area includes a first area and a second area. The first area is an area on the lower side of the center of the camera image. The second area is an area on the upper side of the center of the camera image. The second area is adjacent to the first area and has a smaller area than the first area. In other words, the object area is a convex shape, an L shape, or a shape after the L shape is reversed.

[0022] As an example, the processing ECU 13 divides the camera image into blocks and specifies the area on the image based on the identifiers assigned to the blocks. Figure 2 is a diagram showing an example of an object area in a camera image. Figure 2 As shown, the camera image G is divided into blocks. As an example, the camera image is divided into a matrix of 8 rows and 8 columns. Blocks are assigned identifiers. For example, the identifier of the leftmost block in the top row is B1. The identifier of the block adjacent to the right of block B1 is B2. In this way, the block is assigned an identifier Bn. The processing ECU 13 can use the block identifier Bn to specify an area at an arbitrary position.

[0023] The processing ECU 13 determines the position of the first region R1, which is the area on the lower side of the center of the camera image G, and the position of the second region R2, which is the area on the upper side of the center of the camera image G. The first region R1 is set at the lower side of the center of the camera image G. Generally speaking, the lane lines L1 and L2 of the road are reflected in the lower side of the center of the camera image G. That is, the first region R1 is set so that the lane lines L1 and L2 of the road are reflected therein. The first region R1 can also be set to include the front vehicle V1 and the adjacent vehicle V2.

[0024] The second area R2 is set at a position on the upper side of the center of the camera image G. Generally speaking, the signboard S and the signal of the traffic light are reflected on the upper side of the center of the camera image G. In other words, the second area R2 is set in the expectation that the signboard S and the signal are reflected therein. In addition, generally speaking, the signboard S and the signal are smaller than the lane lines L1 and L2 of the road. Therefore, the second area R2 can be set to be smaller in area than the first area R1. The second area R2 is adjacent to the first area R1. The object area R formed by connecting the first area R1 and the second area R2 can be processed as a single image.

[0025] The machining ECU 13 determines the position of the first region R1 and the position of the second region R2 based on the identifier of the block that is predetermined as a default value, for example.

[0026] The processing ECU 13 may also change the position of the first region R1 and the position of the second region R2 according to the type or performance of the image sensor 10. The type or performance of the image sensor 10 refers to the camera performance such as wide angle and telephoto. The type or performance of the image sensor 10 can be distinguished by an identifier. For example, the processing ECU 13 has a table that stores the identifier of the image sensor 10 in association with the position of the first region R1 and the position of the second region R2. The table is stored in the storage unit of the ECU. The processing ECU 13 obtains the identifier of the image sensor 10, and based on the obtained identifier, refers to the table to determine the position of the first region R1 and the position of the second region R2. In this way, the optimal object region R can be set for each image sensor 10.

[0027] The processing ECU 13 may determine the position of the first region R1 and the position of the second region R2 based on the detection result of the yaw rate sensor 11. For example, the processing ECU 13 determines the turning direction of the vehicle 2 based on the detection result of the yaw rate sensor 11. The processing ECU 13 moves the second region R2 in the turning direction of the vehicle 2. Figure 3 is a diagram showing another example of an object area in a camera image. Figure 3 The scenario is that vehicle 2 is driving while turning left. In this case, Figure 3 As shown, Figure 2 Compared with the object area R shown, the position of the second area R2 is moved one block to the left. As a result, the left part of the sign S falls into the second area R2. The processing ECU13 can also change the position of the first area R1. The processing ECU13 can also determine the movement amount of the first area R1 and the movement amount of the second area R2 according to the yaw rate of the vehicle 2, and set the movement amount of the second area R2 to be greater than the movement amount of the first area R1. As a result, the processing ECU13 can include the lane lines L1 and L2 in the first area R1, and more accurately capture the sign S and the signal, so that the sign S and the signal are included in the second area R2. In this way, the optimal object area R that matches the movement of the vehicle 2 can be set.

[0028] The processing ECU 13 may determine the position of the first region R1 and the second region R2 based on the direction indicated by the direction indicator detected by the direction indicator sensor 12. For example, the processing ECU 13 moves the second region R2 in the same direction as the direction of the direction indicator. Figure 4 is a diagram showing another example of an object area in a camera image. Figure 4 The scenario is that the vehicle 2 is preparing to move to the right lane and flashing the direction indicator. Figure 4 As shown, Figure 2Compared with the object area R shown in FIG. 1 , the position of the first area R1 and the position of the second area R2 are shifted to the right by one block. As a result, the adjacent lane and the adjacent vehicle V2 fall into the first area R1, and the right part of the sign S falls into the second area R2. In this way, the best object area R that matches the movement of the vehicle 2 can be set.

[0029] The recognition model 14 is a learned (trained) model. The recognition model 14 is learned by machine learning based on teacher data, etc. The recognition model 14 can also be a CNN model. The recognition model 14 is input with the object image and the position of the object area generated by the processing ECU 13. The position of the object area refers to the identifier Bn of the above-mentioned block. The recognition model 14 is learned so that if the object image and the position of the object area are input, the recognition result of at least one of the lane lines L1, L2 of the road and the sign S and the signal is output. Such learning is performed in a server, etc., and the learned recognition model 14 is provided to the vehicle 2 via communication, etc.

[0030] The recognition model 14 outputs the positions of the lane lines L1 and L2 of the road, the information of the signboard S, and the signal information of the traffic light as the recognition result. The recognition result is output to the user interface 3. The user interface 3 is, for example, a display, a speaker, etc. The user interface 3 notifies the driver of the recognition result. Alternatively, the recognition result is output to other ECUs 4. Other ECUs 4 are, for example, an automatic driving ECU, etc.

[0031] In addition, the processing ECU 13 may also determine the position of the first region R1 and the position of the second region R2 based on the positions of the lane lines L1 and L2 recognized by the recognition model 14. For example, assuming Figure 4 The scenario is a scenario where the vehicle 2 moves to the right lane. In this case, the lane lines L1 and L2 move relatively to the left. Therefore, the processing ECU 13 moves the position of the first region R1 and the position of the second region R2 in the direction opposite to the direction in which the positions of the recognized lane lines L1 and L2 move. For example, Figure 4 As shown, Figure 2 Compared with the object area R shown in FIG. 1 , the position of the first area R1 and the position of the second area R2 are shifted to the right by one block. As a result, the adjacent lane and the adjacent vehicle V2 fall into the first area R1, and the right part of the sign S falls into the second area R2. In this way, the best object area R that matches the movement of the vehicle 2 can be set.

[0032] [Operation of the identification device]

[0033] Figure 5 This is a flowchart showing an example of the operation of the recognition device. Figure 5 The flowchart shown starts when the recognition device 1 receives a start instruction operation.

[0034] like Figure 5 As shown, first, as step S10, the image sensor 10 of the recognition device 1 acquires the camera image G. Next, as step S12, the processing ECU 13 determines the target region R.

[0035] As step S14, the processing ECU 13 cuts out the target area R from the camera image G and uses it as the target image. As step S16, the target image cut out in step S14 is input to the recognition model 14. As step S18, the recognition model 14 outputs the recognition result of at least one of the lane lines L1 and L2 of the road and the sign S and the signal. When step S18 is completed, Figure 5 The shown flowchart ends.

[0036] [Summary of Implementation Methods]

[0037] According to the recognition device 1, the target area R is cut out from the camera image G obtained by the image sensor 10 and used as the target image. The target area R includes a first area R1 at the lower center of the camera image G and a second area R2 at the upper center of the camera image G. The second area R2 is a region adjacent to the first area R1 and smaller in area than the first area R1. By making the target image a convex shape or the like, the lane lines L1, L2 and at least one of the signal and the sign S can be captured in the image, and the area to be calculated can be reduced. Therefore, compared with the case where the entire image is used as the object for calculation, the recognition model 14 can reduce the load of the calculation for recognizing the lane lines L1, L2 and at least one of the signal and the sign S.

[0038] As mentioned above, although the exemplary embodiment was described, it is not limited to the above-mentioned exemplary embodiment, and various omissions, substitutions, and changes are possible.

[0039] The present disclosure includes the following technical solutions.

[0040] [Project 1]

[0041] A device for identifying an object, comprising:

[0042] an image sensor that acquires an image of the vehicle's traveling direction;

[0043] a controller that crops a target area from an image acquired by the image sensor as a target image, the target area including a first area below the center of the image and a second area above the center of the image, the second area being adjacent to the first area and having a smaller area than the first area; and

[0044] The model is configured to receive the object image and the position of the object area as input, and output a recognition result of at least one of a lane line and a traffic light and a signboard on the road.

[0045] [Project 2]

[0046] The device according to item 1,

[0047] The controller includes a table storing the identifier of the image sensor in association with the position of the first area and the position of the second area, and determines the position of the first area and the position of the second area based on the identifier of the image sensor and the table.

[0048] [Item 3]

[0049] The device according to item 1 or 2,

[0050] further comprising a yaw rate sensor for detecting a yaw rate of the vehicle,

[0051] The controller determines a position of the first region and a position of the second region based on a yaw rate of the vehicle.

[0052] [Item 4]

[0053] The device according to any one of items 1 to 3,

[0054] A sensor is further provided for detecting the direction of a direction indicator indicating the traveling direction of the vehicle.

[0055] The controller determines a position of the first area and a position of the second area based on the direction indicated by the direction indicator.

[0056] [Item 5]

[0057] The device according to any one of items 1 to 4,

[0058] The controller determines a position of the first area and a position of the second area based on the position of the lane line recognized by the model.

Claims

1. A device for identifying an object, comprising: an image sensor that acquires an image of the vehicle's traveling direction; a controller that crops a target area from an image acquired by the image sensor as a target image, the target area including a first area below the center of the image and a second area above the center of the image, the second area being adjacent to the first area and having a smaller area than the first area; and The model is configured to receive the object image and the position of the object area as input, and output a recognition result of at least one of a lane line and a traffic light and a signboard on the road.

2. The device according to claim 1, The controller includes a table storing the identifier of the image sensor in association with the position of the first area and the position of the second area, and determines the position of the first area and the position of the second area based on the identifier of the image sensor and the table.

3. The device according to claim 1 or 2, further comprising a yaw rate sensor for detecting a yaw rate of the vehicle, The controller determines a position of the first region and a position of the second region based on a yaw rate of the vehicle.

4. The device according to any one of claims 1 to 3, A sensor is further provided for detecting the direction of a direction indicator indicating the traveling direction of the vehicle. The controller determines a position of the first area and a position of the second area based on the direction indicated by the direction indicator.

5. The device according to any one of claims 1 to 4, The controller determines a position of the first area and a position of the second area based on the position of the lane line recognized by the model.

Images