Object recognition device and program

Abstract

To provide an object recognition device and a program capable of appropriately recognizing an object included in an image.SOLUTION: An object recognition device 10 calculates a score indicating certainty of object recognition for each of rectangular grids divided into plural pieces vertically and horizontally in an image captured by an imaging device 22 to recognize an object based on the score. An object recognition device 22 includes: a detection unit that detects an existence area of the object in the image as a rectangular area having a rectangular shape in which each of vertical and horizontal sides is in parallel to the grid; and a correction unit that corrects the score based on a position of the rectangular area with respect to a grid line that partitions the grid.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an object recognition device and a program for recognizing an object based on an image. [Background technology]

[0002] Conventionally, as a device for recognizing an object based on an image captured by a camera, for example, one using an SSD (Single Shot MultiBox Detector) is known, as seen in Non-Patent Document 1. In SSD, a score is calculated for each rectangular grid divided vertically and horizontally, and an object is recognized based on the score. In this case, SSD does not calculate the score using grids of all possible positions and sizes on the image, but calculates the score using a grid whose position and size relative to the image are fixed. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Liu,et.al.,''SSD:Single Shot MultiBox Detector'',ECCV,pp.21-37,Springer,2016 Summary of the Invention [Problem to be solved by the invention]

[0004] However, since an object may appear at any position in an image, the position of each grid relative to the object to be recognized may not be aligned. For example, if an object exists in an image in an area that straddles a grid line, and the area is located in a state where it is biased toward one of the grid lines, only a part of the object will be included in one grid, and the score calculated for each grid will decrease. In this case, although the object itself can be recognized in the image, there is a concern that the score will be unreasonably low and object recognition will be impossible.

[0005] The present invention has been made in consideration of the above-mentioned problems, and has an object to provide an object recognition device and a program capable of properly recognizing an object included in an image. [Means for solving the problem]

[0006] The means for solving the above problem is an object recognition device that calculates a score indicating the likelihood of object recognition for each rectangular grid divided vertically and horizontally in an image captured by an imaging device, and recognizes objects based on the score, and includes a detection unit that detects an area in which an object exists in the image as a rectangular area whose vertical and horizontal sides are parallel to the grid, and a correction unit that corrects the score based on the position of the rectangular area relative to the grid lines that divide the grid.

[0007] The size of an object to be recognized on an image is unknown, and its positional relationship with respect to the grid of the image is also arbitrary. Therefore, there is a concern that only a part of the object may be captured in each grid, causing the score calculated for each grid to be unduly reduced, making it impossible to recognize the object. In this regard, the area in which the object exists is detected as a rectangular area, and the score is corrected based on the position of the rectangular area relative to the grid lines. As a result, even if the score for each grid is calculated to be unduly low when only a part of the object is captured in each grid, the score can be corrected to an appropriate value, and the object can be properly recognized. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing the configuration of an object recognition system. [Diagram 2] FIG. 1 is a diagram for explaining a technique for image-based object recognition. [Diagram 3] FIG. 13 is a diagram showing an example of a rectangular area. [Figure 4] FIG. 4 is a diagram for explaining a method for determining a deviation position. [Diagram 5] FIG. 4 is a diagram for explaining the position of a rectangular area. [Figure 6]FIG. 4 is a diagram for explaining the position of a rectangular area. [Figure 7] FIG. 13 is a diagram showing the correction values ​​of the scores in each rectangular area. [Figure 8] 11 is a flowchart showing the procedure of an object recognition process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, specific embodiments of an object recognition device according to the present invention will be described with reference to the drawings.

[0010] As shown in FIG. 1, the object recognition system 100 according to the embodiment includes an ECU 10 as an object recognition device, a radar device 21, an imaging device 22, a vehicle speed sensor 23, a steering angle sensor 24, a yaw rate sensor 25, a receiving device 26, and a controlled device 30.

[0011] The radar device 21 is, for example, a known millimeter wave radar that transmits high-frequency signals in the millimeter wave band. The radar device 21 is installed at the front end or rear end of the vehicle, and detects the position of an object within a detection range that is within a predetermined detection angle. Specifically, it transmits a search wave at a predetermined cycle and receives the reflected wave by multiple antennas. The distance to the object is calculated based on the transmission time of the search wave and the reception time of the reflected wave. In addition, the relative speed is calculated based on the frequency of the reflected wave reflected by the object, which is changed by the Doppler effect. In addition, the direction of the object is calculated based on the phase difference of the reflected waves received by the multiple antennas.

[0012] The imaging device 22 may be, for example, a monocular camera such as a CCD camera, a CMOS image sensor, or a near-infrared camera, or may be a stereo camera. Only one imaging device 22 may be installed in the vehicle, or multiple imaging devices 22 may be installed. The imaging device 22 is attached, for example, at a predetermined height in the center of the vehicle width direction, and captures an image of an area spreading in a predetermined angular range toward the front or rear of the vehicle from a bird's-eye view. The imaging device 22 outputs images captured sequentially to the ECU 10.

[0013] The vehicle speed sensor 23 is a sensor that detects the traveling speed of the host vehicle, and outputs a traveling speed signal corresponding to the traveling speed of the host vehicle to the ECU 10. The steering angle sensor 24 is a sensor that detects the steering angle of the steering wheel, and outputs a steering angle signal corresponding to a change in the steering angle to the ECU 10. The yaw rate sensor 25 is a sensor that detects the turning angular velocity of the host vehicle, and outputs a yaw rate signal corresponding to the turning angular velocity of the host vehicle to the ECU 10.

[0014] The receiver 26 is a receiver of a positioning signal from a satellite positioning system, such as a GPS receiver. The receiver 26 receives a positioning signal corresponding to the current position of the host vehicle, and outputs the received positioning signal to the ECU 10.

[0015] The functions provided by the ECU 10 can be provided by software recorded in a physical memory device and a computer that executes the software, by software alone, by hardware alone, or by a combination of these. For example, when the ECU 10 is provided by an electronic circuit that is hardware, the functions can be provided by a digital circuit including a large number of logic circuits, or by an analog circuit. For example, the ECU 10 executes a program stored in a non-transitory tangible storage medium as a storage unit included in the ECU 10. The program is executed to execute a method corresponding to the program. The storage unit is, for example, a non-volatile memory. The program stored in the storage unit can be updated, for example, via a network such as the Internet.

[0016] The ECU 10 recognizes objects present around the vehicle based on the image captured by the imaging device 22, and adjusts the driving force and braking force of the vehicle using the controlled device 30, thereby performing driving assistance control of the vehicle. The objects include vehicles including four-wheeled automobiles, motorcycles, bicycles, etc., and pedestrians. The controlled device 30 has an accelerator device 31 which is an acceleration device, a brake device 32 which is a deceleration device, and a steering device 33. The accelerator device 31 applies a driving force to the vehicle in response to the accelerator operation of the driver or a control command from the ECU 10. The brake device 32 applies a braking force to the vehicle in response to the brake operation of the driver or a control command from the ECU 10. The steering device 33 controls the steering of the vehicle in response to the steering operation of the driver or a control command from the ECU 10.

[0017] A method of object recognition based on an image will be described. As shown in Fig. 2(A), an object H, such as a person, is captured in an image captured by the imaging device 22, and the ECU 10 detects the area in which the object H exists as a rectangular area E having a rectangular shape. Here, for example, if the object H is a person, the rectangular area E is detected as a vertically long rectangular area (Fig. 3(A)). Also, if the object H is a bicycle with a person on it, the rectangular area E is detected as a horizontally long rectangular area (Fig. 3(B)).

[0018] 2B, a rectangular grid G ​​divided vertically and horizontally is set on the image. In this case, a rectangular area E is positioned so as to overlap each grid G, and each vertical and horizontal side of the rectangular area E is parallel to the grid lines GL that separate each grid G. In the following description, the grid lines GL extending vertically are also referred to as vertical grid lines GL1, and the grid lines GL extending horizontally are also referred to as horizontal grid lines GL2.

[0019] The ECU 10 calculates a score S indicating the likelihood of object recognition for each grid G. In this case, for example, a recognition model that has previously learned a plurality of recognition targets is used to calculate the score S for an object H present in the image. The recognition model may be in any form, but for example, deep learning is used. The score S may be expressed as the probability that each object corresponds to a type such as a person, a bicycle, a four-wheeled vehicle, etc.

[0020] However, if the rectangular area E is positioned so that it is biased toward one of the grid lines GL in the grid G, such as straddling a grid line GL, only a portion of the object will be included in one grid G, lowering the score S calculated for each grid G. In this case, although the object itself can be recognized in the image, there is a concern that the score S will be unduly low and object recognition will be impossible.

[0021] Therefore, in this embodiment, the score S is increased based on the position of the rectangular area E relative to the grid line GL. Specifically, it is determined that the rectangular area E is present in a predetermined offset position relative to any of the grid lines GL in the grid G, and if it is determined that the rectangular area E is present in an offset position, a correction value for the score S is calculated based on the position of the rectangular area E, and the score S is increased based on the correction value. The method for determining the offset position will be described below.

[0022] FIG. 4 shows two rectangular areas E in a grid G, each of which has a different position relative to the vertical grid line GL1. In this embodiment, it is determined whether or not the rectangular area E exists at a predetermined deviation position in each grid G ​​based on the distance L from the perpendicular bisector of the rectangular area E to each grid line GL. In the case of FIG. 4, the distance L from the perpendicular bisector extending vertically in each rectangular area E to the vertical grid line GL1 is calculated, and it is determined whether or not the rectangular area E exists at a predetermined deviation position based on whether or not the distance L is shorter than a predetermined distance threshold Lth. The perpendicular bisector corresponds to the "center" of the rectangular area E. The distance threshold Lth may be set based on the width of the rectangular area E, and in the case of FIG. 4, it is set to a value narrower than 1 / 2 of the width of the rectangular area E in the short side direction.

[0023] In the case of (A) in FIG. 4, since the distance L is longer than a predetermined distance threshold Lth, it is determined that the rectangular area E is not biased relative to the vertical grid line GL1 and is not located at a predetermined offset position. In other words, it is determined that the rectangular area E does not straddle the vertical grid line GL1 and is entirely contained within one grid G, or that even if part of the rectangular area E protrudes within one grid G, it is only a small portion of it. In this case, the score S is not unduly reduced. Therefore, the score S is not corrected to increase.

[0024] On the other hand, in the case of (B) of FIG. 4, since the distance L is shorter than the predetermined distance threshold Lth, the rectangular area E is determined to be biased with respect to the vertical grid line GL1 and to be present at a predetermined offset position. In other words, it is determined that the rectangular area E straddles the vertical grid line GL1 and most of the rectangular area E protrudes from the grid G. In this case, there is a concern that the score S may be unduly reduced. Therefore, a correction is made to increase the score S.

[0025] Although not shown in the figure, the method of determining the deviation position of the rectangular area E with respect to the horizontal grid line GL2 is similar. In this case, the distance threshold Lth of the vertical distance L is set based on the width of the long side (vertical direction) of the rectangular area E, and is set to a value narrower than half the width in the long side direction, for example. The determination of whether the rectangular area E is at a predetermined deviation position with respect to the grid line GL is made by determining whether the center of the rectangular area E is deviated due to being biased (close) to one of the left and right vertical grid lines GL1, or whether the center is deviated due to being biased (close) to one of the upper and lower horizontal grid lines GL2.

[0026] 5 is a diagram showing four patterns of the position of rectangular areas E relative to the vertical grid line GL1 and the horizontal grid line GL2 of each grid G. Here, the four rectangular areas E are classified as rectangular areas E1, E2, E3, and E4 based on the result of deviation judgment relative to the vertical grid line GL1 and the result of deviation judgment relative to the horizontal grid line GL2.

[0027] The rectangular area E1 is a rectangular area E that is not displaced relative to the vertical and horizontal grid lines GL1 and GL2. The object in this rectangular area E1 is entirely contained within one grid G, so there is a low possibility that the score S will be calculated as an underestimate. Therefore, there is no need to increase the score S, and the correction value is set to "1".

[0028] The rectangular area E2 is a rectangular area E that is offset from the vertical grid line GL1, but not offset from the horizontal grid line GL2. The rectangular area E3 is a rectangular area E that is not offset from the vertical grid line GL1, but is offset from the horizontal grid line GL2. The objects in each of the rectangular areas E2 and E3 partially extend beyond one grid G, and the score S may be calculated as an underestimate. Therefore, a correction value greater than 1 is set to increase the score S.

[0029] Furthermore, comparing rectangular area E2 and E3, rectangular area E2 has a deviation in the short side direction, while rectangular area E3 has a deviation in the long side direction. In this case, it is advisable to set a larger correction value for rectangular area E2 than for rectangular area E3. In other words, when there is a deviation in the short side direction of rectangular area E, the ratio of the area within the grid to the total area of ​​rectangular area E is smaller than when there is a deviation in the long side direction of rectangular area E, so it is considered that rectangular area E2 tends to have a lower score S than rectangular area E3. Therefore, in this embodiment, the correction value for rectangular area E2 is set to a value larger than the correction value for rectangular area E3.

[0030] The rectangular area E4 is a rectangular area E that is offset from the vertical and horizontal grid lines GL1 and GL2. The object in this rectangular area E4 may be calculated with a smaller score S than the rectangular areas E2 and E3. Therefore, a larger correction value is set for the rectangular area E4 than for the rectangular areas E2 and E3.

[0031] 5, when a rectangular area E exists across multiple grids G, a correction value for the score S may be set based on the number of grids that the rectangular area E spans. Specifically, when the rectangular area E spans a large number of grids, the correction value for the score S is set to be larger than when the number of grids G is small. For example, the rectangular area E4 spans a larger number of grids than the rectangular areas E2 and E3, so the correction value is set to be larger than the rectangular areas E2 and E3.

[0032] In this embodiment, a first grid G1 and a second grid G2 having different sizes are set in the image, and a score S is calculated for each of the grids G1 and G2. Specifically, as shown in Fig. 6, a first grid G1 and a second grid G2 obtained by integrating two first grids G1 in the vertical and horizontal directions of the image are set as the grid G ​​of the image. Then, a score S is calculated for each of the grids G1 and G2, and a deviation judgment of the rectangular area E is performed for each of the grids G1 and G2.

[0033] 6, the grid lines GL of the first grid G1 are indicated by dashed lines, and the grid lines GL of the second grid G2 are indicated by solid lines. Note that the grid lines GL of the first grid G1 and the grid lines GL of the second grid G2 are indicated by solid lines in their common portions.

[0034] In this case, for each of the grids G1 and G2, it is determined whether or not the rectangular area E is in a deviated position with respect to the grid line GL as described in Fig. 5. In other words, for each of the grids G1 and G2, it is determined whether the rectangular area E falls into any of the following categories (1) to (4). (1) The rectangular area E is not offset from the vertical and horizontal grid lines GL1 and GL2. (2) The rectangular area E is offset from the vertical grid line GL1 and is not offset from the horizontal grid line GL2. (3) The rectangular area E is not offset from the vertical grid line GL1, and is offset from the horizontal grid line GL2. (4) The rectangular area E is offset from the vertical and horizontal grid lines GL1 and GL2.

[0035] For each of the grids G1 and G2, it is determined whether it falls under any of the above (1) to (4), and the determination results for each of the grids G1 and G2 are combined to determine whether the rectangular area E falls under any of the following (11) to (19). Note that in this example, the rectangular area E is assumed to be any of the rectangular areas E11 to E19 shown in FIG. 6.

[0036] (11) The rectangular area E11 is not offset from the vertical and horizontal grid lines GL1 and GL2 in the first grid G1, and is not offset from the vertical and horizontal grid lines GL1 and GL2 in the second grid G2.

[0037] (12) The rectangular area E12 is offset from the vertical grid line GL1 of the vertical and horizontal grid lines GL1 and GL2 in the first grid G1, and is not offset from the vertical and horizontal grid lines GL1 and GL2 in the second grid G2.

[0038] (13) The rectangular area E13 is offset from the vertical grid line GL1 of the vertical and horizontal grid lines GL1 and GL2 of the first grid G1, and is also offset from the vertical grid line GL1 of the vertical and horizontal grid lines GL1 and GL2 of the second grid G2.

[0039] (14) The rectangular area E14 is offset from the horizontal grid line GL2 of the vertical and horizontal grid lines GL1 and GL2 in the first grid G1, and is not offset from the vertical and horizontal grid lines GL1 and GL2 in the second grid G2.

[0040] (15) The rectangular area E15 is offset from the vertical and horizontal grid lines GL1 and GL2 in the first grid G1, and is not offset from the vertical and horizontal grid lines GL1 and GL2 in the second grid G2.

[0041] (16) The rectangular area E16 is offset from the vertical and horizontal grid lines GL1 and GL2 in the first grid G1, and is offset from the vertical grid line GL1 of the vertical and horizontal grid lines GL1 and GL2 in the second grid G2.

[0042] (17) The rectangular area E17 is offset from the horizontal grid line GL2 of the vertical and horizontal grid lines GL1 and GL2 in the first grid G1, and is also offset from the horizontal grid line GL2 of the vertical and horizontal grid lines GL1 and GL2 in the second grid G2.

[0043] (18) The rectangular area E18 is offset from the vertical and horizontal grid lines GL1, GL2 in the first grid G1, and is offset from the horizontal grid line GL2 of the vertical and horizontal grid lines GL1, GL2 in the second grid G2.

[0044] (19) The rectangular area E19 is offset from the vertical and horizontal grid lines GL1 and GL2 in the first grid G1, and is also offset from the vertical and horizontal grid lines GL1 and GL2 in the second grid G2.

[0045] A correction value for the score S is determined for each of the rectangular areas E11 to E19 in FIG. 6, and a list of the correction values ​​is shown in FIG. 7. The correction values ​​shown in FIG. 7 are determined as correction magnifications by which the score S is multiplied. These correction values ​​may be stored in the memory of the ECU 10.

[0046] The correction values ​​shown in Fig. 7 are determined according to which of the nine rectangular areas E11 to E19 shown in Fig. 6 the rectangular area E corresponds to, and a value of 1.0 or greater than 1.0 is set for each of them. The correction values ​​for each of the rectangular areas E11 to E19 are explained below. Note that the correction values ​​in Fig. 7 are merely examples and can be changed as appropriate.

[0047] First, the rectangular area E11 is not displaced from the vertical and horizontal grid lines GL1 and GL2 in the first grid G1 and the second grid G2, so the scores S calculated for each grid G1 and G2 are not too small. Therefore, there is no need to increase the score S, and the correction value is 1.0.

[0048] In addition, except for rectangular area E11, rectangular area E is deviated from either the vertical or horizontal grid lines GL1, GL2 of each grid G1, G2, and correction values ​​are determined individually according to the state of deviation from each of the grid lines GL1, GL2.

[0049] In this case, when the rectangular area E spans multiple grids G, if the number of grids that the rectangular area E is deemed to span is large, the correction value is larger than if the number of grids is small.

[0050] For example, comparing rectangular areas E12 and E13, rectangular area E12 is offset only with respect to the vertical grid line GL1 of the first grid G1 among the grids G1 and G2, whereas rectangular area E13 is offset with respect to the vertical grid line GL1 of the first grid G1 and the vertical grid line GL1 of the second grid G2. The number of grids that rectangular area E12 is considered to straddle is two for the first grid G1, and the number of grids that rectangular area E13 is considered to straddle is two for each of the grids G1 and G2, totaling four. Therefore, "the correction value of rectangular area E12 < the correction value of rectangular area E13".

[0051] Similarly, in comparison between rectangular areas E13, E16, and E19, in each of grids G1 and G2, the number of grids that rectangular area E13 considers to straddle is a total of 4, the number of grids that rectangular area E16 considers to straddle is a total of 6, and the number of grids that rectangular area E19 considers to straddle is a total of 8. Therefore, "correction value of rectangular area E13 < correction value of rectangular area E16 < correction value of rectangular area E19".

[0052] In addition, when the rectangular area E is a rectangular area, the correction value is larger in the first case in which it is determined that the rectangular area E is deviated from the vertical grid line GL1 in the short side direction of the rectangular area E than in the second case in which it is determined that the rectangular area E is deviated from the horizontal grid line GL2 in the long side direction of the rectangular area E.

[0053] That is, for example, comparing rectangular areas E12 and E14, both rectangular areas E12 and E14 are the same in that they are both offset from the grid line GL only in the first grid G1, but they differ in that the rectangular area E12 is offset from the vertical grid line GL1 of the first grid G1, whereas the rectangular area E14 is offset from the horizontal grid line GL2 of the first grid G1. In this case, the rectangular area E12 is offset from the vertical grid line GL1 in the short side direction, while the rectangular area E14 is offset from the vertical grid line GL1 in the long side direction. Therefore, "the correction value of the rectangular area E12 is greater than the correction value of the rectangular area E14."

[0054] For the same reason, the correction values ​​of the rectangular areas E13 and E17 are such that "the correction value of the rectangular area E13 is greater than the correction value of the rectangular area E17." Also, for the same reason, the correction values ​​of the rectangular areas E16 and E18 are such that "the correction value of the rectangular area E16 is greater than the correction value of the rectangular area E18."

[0055] In the rectangular areas E14 and E17, the rectangular area E is not displaced from the vertical grid line GL1 in the short side direction of the rectangular area E in both grids G1 and G2. Therefore, regardless of the presence or absence of displacement in the long side direction, correction of the score S is not necessary, and the correction value is set to 1.0.

[0056] In addition, in the rectangular area E15, since the rectangular area E is contained within the second grid G2, object recognition is possible based on the score S calculated in the second grid G2 regardless of the deviation state of the rectangular area E in the first grid G1. Therefore, it is deemed unnecessary to correct the score S, and the correction value is set to 1.0.

[0057] Next, the control of the object recognition process executed by the ECU 10 will be described with reference to the flowchart of Fig. 8. The process shown in Fig. 8 is repeatedly executed every time an image is acquired from the imaging device 22.

[0058] When the object recognition process starts, in step S11, it is determined whether or not an object to be recognized exists in the image. If an object to be recognized exists, the process proceeds to step S12, where the area in which the object exists is detected as a rectangular area E. The process of step S12 corresponds to the "detection unit, detection step."

[0059] In step S13, a score S is calculated for each grid G ​​for objects in the rectangular area E. At this time, a score S for an object detected in the first grid G1 and a score S for an object detected in the second grid G2 are calculated.

[0060] In step S14, a deviation determination is performed for the rectangular area E with respect to the vertical and horizontal grid lines GL1 and GL2 of the first grid G1. This determines which of the four rectangular areas E1 to E4 described in FIG. 5 the rectangular area E corresponds to in the first grid G1.

[0061] In step S15, deviation determination is performed for the rectangular area E with respect to the vertical and horizontal grid lines GL1 and GL2 of the second grid G2. This determines which of the four rectangular areas E1 to E4 described in Fig. 5 the rectangular area E corresponds to in the second grid G2. The processing in steps S14 and S15 corresponds to a "deviation determination unit."

[0062] Then, in step S16, a correction value for the score S is calculated based on the result of the deviation judgment of the rectangular area E in the first grid G1 and the result of the deviation judgment of the rectangular area E in the second grid G2. At this time, it is determined which of the nine rectangular areas E11 to E19 shown in Fig. 6 the rectangular area E corresponds to based on the result of the deviation judgment in each of the grids G1 and G2. Then, using the relationship in Fig. 7, a correction value is calculated based on which of "E11 to E19" the rectangular area E corresponds to.

[0063] In step S17, the scores S calculated for the grids G1 and G2 are added together, and the score S is corrected by the correction value calculated in step S16. The process of step S17 corresponds to "correction unit, correction step".

[0064] In step S18, the object is recognized based on the corrected score S. Specifically, it is determined whether or not the score S is greater than a predetermined score threshold Sth. If the score S is greater than the score threshold Sth, the type of the object is recognized based on the score S.

[0065] According to the present embodiment described above in detail, the following effects can be obtained.

[0066] In an image, a score S is calculated for each grid G, and when recognizing an object based on the score S, the score S is corrected based on the position of the rectangular area E relative to the grid line GL of each grid G. As a result, even if the score S for each grid G ​​is calculated to be inappropriately low when only a portion of an object is captured in each grid G, the score S can be corrected to an appropriate value, and the object can be recognized.

[0067] Specifically, it is determined whether the rectangular area E exists at a predetermined offset position with respect to the grid line GL in each grid G, and the score S is increased based on the determination result. This makes it possible to appropriately correct the score S based on the positional relationship between the rectangular area E and the grid G ​​when the rectangular area E exists near the grid line GL, for example, when the rectangular area E straddles the grid line GL.

[0068] More specifically, whether or not the rectangular area E is located at a predetermined deviation position in the grid G ​​is determined based on the distance L from the center of the rectangular area E relative to the grid line GL in each grid G. This makes it possible to properly determine the deviation state of the rectangular area E relative to the grid line GL, i.e., whether the rectangular area E straddles the grid line GL, and thus improves the accuracy of correction of the score S.

[0069] In recognizing an object, it is determined whether or not the rectangular area E is offset from the grid line GL in the short side direction, and whether or not the rectangular area E is offset from the grid line GL in the long side direction. Then, a first case in which it is determined that the rectangular area E is offset from the grid line GL in the short side direction is compared with a second case in which it is determined that the rectangular area E is offset from the grid line GL in the long side direction, and the correction value of the score S is made larger in the first case. This makes it possible to properly recognize an object even if the object has a vertically or horizontally long shape.

[0070] In a situation where the rectangular area E exists across multiple grids G, when the number of grids G that are deemed to be spanned by the rectangular area E is large, the correction value of the score S is made larger than when the number of grids G that are deemed to be spanned by the rectangular area E is small. This makes it possible to appropriately correct the score S when the rectangular area E spans multiple grids G.

[0071] Since the size of an object to be recognized in an image is unknown, it is considered to recognize the object using multiple grids G of different sizes. Therefore, a score S is calculated for each of the first grid G1 and the second grid G2, and the object is recognized based on the score S calculated for the same object in the first grid G1 and the second grid G2. In this case, depending on the position of the rectangular area E on the image, the rectangular area E may be deviated from both the grid lines GL of the first grid G1 and the grid lines GL of the second grid G2, or may be deviated from only one of them, and it is considered that the correction content of the score S will differ depending on these situations.

[0072] In this regard, the score S is increased based on the result of the deviation judgment of the rectangular area E in the first grid G1 and the result of the deviation judgment of the rectangular area E in the second grid G2. This allows the score S to be corrected more finely than when the score is corrected according to the position of the rectangular area E in one type of grid G, thereby improving the object recognition accuracy.

[0073] In the first grid G1, when it is determined that the rectangular area E is offset from the vertical grid line GL1 and also offset from the horizontal grid line GL2, and in the second grid G2, when it is determined that the rectangular area E is not offset from the vertical grid line GL1 and also not offset from the horizontal grid line GL2 (for example, when the rectangular area E is within the second grid G2 like the rectangular area E15 in FIG. 6), the score S is not corrected. This makes it possible to prevent the score S from being incorrectly corrected, which would cause problems in object recognition.

[0074] (Other embodiments) The above embodiment may be modified, for example, as follows.

[0075] In the above embodiment, the deviation of the rectangular area E in the grid G ​​is determined based on the distance from the perpendicular bisector of the rectangular area E to each grid line GL, but this may be changed. For example, the deviation of the rectangular area E in the grid G ​​may be determined based on the shortest distance from the center of the rectangular area E to each grid line GL.

[0076] Also, a configuration may be adopted in which it is determined whether the rectangular area E is in a predetermined deviation position or not depending on whether the rectangular area E straddles a grid line GL. Also, a configuration may be adopted in which it is determined whether the rectangular area E is in a predetermined deviation position or not depending on the area ratio of the rectangular area E included in the grid G.

[0077] In the above embodiment, the grid G ​​is determined to determine whether the rectangular area E is displaced from the vertical grid line GL1 and whether the rectangular area E is displaced from the horizontal grid line GL2. However, this may be changed to a determination made only for the grid lines GL extending in one direction. In this case, if the rectangular area E is a rectangular area, it is preferable to determine whether the rectangular area E is displaced from the grid lines GL extending in the short side direction, i.e., in the long side direction.

[0078] The second grid G2 is not limited to that in the above embodiment, as long as it is formed by integrating multiple first grids G1 in at least one of the vertical and horizontal directions of the image. For example, it may be formed by integrating multiple first grids G1 only in the vertical direction of the image, or may be formed by integrating multiple first grids G1 only in the horizontal direction of the image.

[0079] In the above embodiment, two types of grids G with different sizes are used to recognize an object, but the type of grid G ​​used for object recognition may be one type or three or more types. Note that the length of one side of the second grid G2 is preferably a power of two of the length of one side of the first grid G1.

[0080] In the above embodiment, the scores S calculated for the same object in the first grid G1 and the second grid G2 are added together, and then the added score S is corrected. However, this may be modified so that the scores S calculated in the first grid G1 and the second grid G2 are corrected separately, and the corrected scores S for the same object are added together.

[0081] The object recognition apparatus and the method described in the present disclosure may be realized by a special-purpose computer provided by configuring a processor and a memory programmed to execute one or more functions embodied in a computer program. Alternatively, the object recognition apparatus and the method described in the present disclosure may be realized by a special-purpose computer provided by configuring a processor with one or more dedicated hardware logic circuits. Alternatively, the object recognition apparatus and the method described in the present disclosure may be realized by one or more special-purpose computers configured by a combination of a processor and a memory programmed to execute one or more functions and a processor configured with one or more hardware logic circuits. In addition, the computer program may be stored in a computer-readable non-transitory tangible recording medium as instructions executed by the computer. [Explanation of symbols]

[0082] 10...ECU, 22...imaging device.

Claims

1. An object recognition device (10) that calculates a score indicating a likelihood of object recognition for each rectangular grid divided vertically and horizontally in an image captured by an imaging device (22) and recognizes an object based on the score, a detection unit that detects an area in the image where an object exists as a rectangular area whose vertical and horizontal sides are parallel to the grid; a correction unit that corrects the score based on a position of the rectangular area with respect to a grid line that divides the grid; An object recognition device comprising:

2. a deviation determination unit that determines whether the rectangular area is present at a predetermined deviation position relative to any grid line in the grid, The object recognition device according to claim 1 , wherein the correction unit increases the score when it is determined that the rectangular area exists at the deviation position.

3. The object recognition device according to claim 2 , wherein the deviation determination unit determines whether or not the rectangular area in the grid is present at the deviation position based on a distance from the grid line to a center of the rectangular area in the grid.

4. the deviation determination unit is capable of determining whether the rectangular area is present at the deviation position with respect to the grid lines extending in a vertical direction of the image, and determining whether the rectangular area is present at the deviation position with respect to the grid lines extending in a horizontal direction of the image, 4. The object recognition device according to claim 2 or 3, wherein, in a situation where the rectangular area is a rectangular area having short sides and long sides, the correction unit compares a first case in which it is determined that the rectangular area exists at the deviated position with respect to the grid lines in a vertical or horizontal direction in a short side direction of the rectangular area with a second case in which it is determined that the rectangular area exists at the deviated position with respect to the grid lines in a vertical or horizontal direction in a long side direction of the rectangular area, and increases a correction value for increasing the score in the first case.

5. The object recognition device according to any one of claims 2 to 4, wherein the correction unit, in a situation in which the rectangular area exists across a plurality of the grids, increases a correction value for increasing the score when a number of the grids that the rectangular area is deemed to span are large, compared to a case in which the number of the grids is small.

6. an object recognition device including, as the grid, a first grid and a second grid formed by integrating a plurality of the first grids in at least one of a vertical direction and a horizontal direction of the image, calculating the score for each of the first grid and the second grid, and recognizing a same object based on the score calculated for the object in the first grid and the second grid, the deviation determination unit determines whether or not the rectangular area is present at the deviation position with respect to the grid line in the first grid, and determines whether or not the rectangular area is present at the deviation position with respect to the grid line in the second grid, The object recognition device according to any one of claims 2 to 5, wherein the correction unit increases and corrects the score based on a result of deviation judgment of the rectangular area in the first grid and a result of deviation judgment of the rectangular area in the second grid.

7. the second grid is an aggregate of a plurality of the first grids in a vertical direction and a horizontal direction of the image, the deviation determination unit determines whether or not the rectangular area is present at the deviation position with respect to the grid lines extending in the vertical direction in the first grid, and determines whether or not the rectangular area is present at the deviation position with respect to the grid lines extending in the horizontal direction, while determining whether or not the rectangular area is present at the deviation position with respect to the grid lines extending in the vertical direction in the second grid, and determines whether or not the rectangular area is present at the deviation position with respect to the grid lines extending in the horizontal direction, 7. The object recognition device according to claim 6, wherein the correction unit does not correct the score when it is determined that, in the first grid, the rectangular area is present at the deviation position with respect to the grid lines extending in the vertical direction and that the rectangular area is present at the deviation position with respect to the grid lines extending in the horizontal direction, and when it is determined that, in the second grid, the rectangular area is not present at the deviation position with respect to the grid lines extending in the vertical direction and that the rectangular area is not present at the deviation position with respect to the grid lines extending in the horizontal direction.

8. A computer-executable program for calculating a score indicating a likelihood of object recognition for each rectangular grid divided vertically and horizontally in an image captured by an imaging device (22), and recognizing an object based on the score, a detection step (S13) of detecting an area in the image where an object exists as a rectangular area whose vertical and horizontal sides are parallel to the grid; a correction step (S23) of correcting the score based on a position of the rectangular area relative to a grid line that divides the grid; Programs including.

Images