Vehicle position recognition system and vehicle position recognition method

Abstract

To provide a technique which allows a vehicle position to be properly recognized even when a camera shoots reflected light reflected from a vehicle body.SOLUTION: A vehicle position recognition system according to the present invention executes, based on an image input from a camera for shooting a parking space in a predetermined direction, calculating a vehicle position of a target vehicle parked by automatic valley parking at the parking space, acquiring, from a database, a possible reflection area on a vehicle body of the target vehicle specified based on vehicle kind information of the target vehicle and a predicted position of the target vehicle, assimilating the possible reflection area on the vehicle body of the target vehicle with peripheral area surrounding the possible reflection area in the image input from the camera, specifying the vehicle position of the target vehicle by image recognition of the image with the possible reflection area assimilated with the peripheral area, and updating the prediction position based on the vehicle position of the target vehicle specified by the image recognition.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present disclosure relates to a technology for recognizing the position of a vehicle. [Background technology]

[0002] Patent Document 1 discloses a method for inspecting surface defects on an object that specularly reflects light. In this inspection method, a first image is captured by receiving first reflected light from a first illumination light reflected from the object, and a second image is captured by receiving second reflected light from a second illumination light reflected from the object. Then, pixels having a light intensity equal to or greater than a predetermined threshold, i.e., areas where the light source is reflected, are identified in each of the first and second images. The identified pixels in the first image are removed and interpolated with corresponding pixels in the second image, and the identified pixels in the second image are removed and interpolated with corresponding pixels of the first pixels. Through this process, surface defects present in areas where the light source is reflected are detected. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2011-080932 Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, in automated valet parking, it is being considered to measure the vehicle's position by photographing the vehicle with an infrastructure camera and then performing image recognition on the captured images using AI. One problem that has arisen is reflected light from the vehicle body. AI image recognition extracts features from the image, and it is expected that areas where light is reflected will have particularly strong features. In other words, reflected light is a disturbance in image recognition, and can reduce the accuracy of vehicle recognition or even lead to false detection.

[0005] One object of the present disclosure is to provide a technology that can appropriately recognize the position of a vehicle even when a camera captures light reflected from the vehicle body. [Means for solving the problem]

[0006] A first aspect of the present disclosure relates to a system for recognizing the position of a vehicle. The vehicle position recognition system includes a camera that captures an image of a parking space from a predetermined direction, a storage device that stores a database in which reflective areas on the body of a target vehicle parked in the parking space by automatic valet parking, where light reflections may be observed when viewed from the predetermined direction, are registered for each vehicle model and for each position of the target vehicle in the parking space, and a computer that calculates the position of the target vehicle in the parking space based on an image input from the camera. The computer is configured to acquire vehicle model information of the target vehicle, acquire a predicted position of the target vehicle in the parking space, acquire from the database the reflective area on the body of the target vehicle identified by the vehicle model information and the predicted position, merge the reflective area on the body of the target vehicle into a surrounding area surrounding the reflective area in the image input from the camera, identify the position of the target vehicle by image recognition of the image in which the reflective area has been merged into the surrounding area, and update the predicted position based on the position of the target vehicle identified by image recognition.

[0007] A second aspect of the present disclosure has the same features as the first aspect, plus the following: Blending the reflective area into the surrounding area includes painting the reflective area with the same color as the surrounding area.

[0008] A third aspect of the present disclosure has the same features as the first aspect, but further includes the following: integrating the potential reflectivity area into the surrounding area includes detecting two edges on a scan line within the potential reflectivity area, and filling pixels between the edges with the color of pixels outside the edges.

[0009] A fourth aspect of the present disclosure has the following feature in addition to the first aspect: The possible reflection areas are registered in the database for each type of target vehicle, each position of the target vehicle in the parking space, and each time period during which the target vehicle is parked.

[0010] A fifth aspect of the present disclosure has the same features as the first aspect, but further includes the following: The vehicle position of the target vehicle in the parking space in the database is defined along one or more approach lines that are predicted when the target vehicle enters the parking space.

[0011] A sixth aspect of the present disclosure relates to a method for recognizing a vehicle position. The vehicle position recognition method includes: calculating a vehicle position of a target vehicle to be parked in a parking space by automatic valet parking based on an image input from a camera that photographs the parking space from a predetermined direction; acquiring vehicle model information of the target vehicle; acquiring a predicted position of the target vehicle in the parking space; acquiring a possible reflective area on the body of the target vehicle identified by the vehicle model information and the predicted position from a database in which a possible reflective area on the vehicle body where light reflection may be observed when the target vehicle is viewed from the predetermined direction is registered for each vehicle model of the target vehicle and for each position of the target vehicle in the parking space; assimilating the possible reflective area on the body of the target vehicle into a surrounding area surrounding the possible reflective area in the image input from the camera; identifying the vehicle position of the target vehicle by image recognition of the image in which the possible reflective area has been assimilated into the surrounding area; and updating the predicted position based on the vehicle position of the target vehicle identified by image recognition. [Effects of the Invention]

[0012] According to the present disclosure, by removing light reflected by the body of the target vehicle through image processing, deterioration of vehicle position recognition accuracy and false detection can be reduced. In particular, by identifying a possible reflection area and limiting processing to that area, it is possible to prevent adverse effects on feature extraction from other areas where reflection does not occur. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a diagram for explaining an overview of a vehicle position recognition system according to an embodiment; [Figure 2] 1 is a diagram for explaining an overview of a vehicle position recognition system according to an embodiment; [Figure 3] FIG. 2 is a diagram for explaining a specific example of a first example of a vehicle position recognition system according to an embodiment. [Figure 4] FIG. 4 is a diagram for explaining a specific example of a second example of the vehicle position recognition system according to the embodiment. [Figure 5] 1 is a block diagram showing an example of the configuration of a vehicle position recognition system according to an embodiment; [Figure 6] 4 is a flowchart illustrating an example of processing of the vehicle position recognition system according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] A vehicle location recognition system and a vehicle location recognition method according to an embodiment of the present disclosure will be described with reference to the accompanying drawings. Elements common to the various drawings will be designated by the same reference numerals, and redundant description will be omitted.

[0015] 1. Overview FIG. 1 is a diagram illustrating an overview of a vehicle location recognition system 1 according to an embodiment. As shown in FIG. 1(A), the vehicle location recognition system 1 includes a camera 4 and a management device 10. Vehicles parked in parking spaces 3 in a parking lot 2 include a target vehicle 5 and another vehicle 6. The target vehicle 5 is a vehicle that is scheduled to be parked in the parking space 3, and the other vehicle 6 is a vehicle that is already parked in the parking space 3. These vehicles have an automatic valet parking function. An example of a vehicle that has an automatic valet parking function is an AVP (Automated Valet Parking) vehicle. An AVP vehicle may be an automatic driving vehicle or a manually driving vehicle.

[0016] Camera 4 is an infrastructure camera installed in parking lot 2, and captures images of parking space 3 from a predetermined direction. The predetermined direction means a direction set so that the angle of view of camera 4 includes parking space 3. Camera 4 may also have a zoom function so that it can capture a close-up image of the area around parking space 3.

[0017] The management device 10 is a device that performs parking management for the target vehicle 5 and other vehicles 6. Specifically, the management device 10 instructs the target vehicle 5 to park in a predetermined parking space 3. Then, based on images input from the camera 4, the management device 10 calculates the vehicle position of the target vehicle 5 parked in the predetermined parking space 3. Furthermore, based on the time-series information of the calculated vehicle position of the target vehicle 5, the management device 10 estimates the predicted position of the target vehicle 5 at the next time.

[0018] Here, as shown in FIG. 1(B), consider a case where a lighting device 7 is installed in a parking lot 2. In this case, when the target vehicle 5 is viewed from a predetermined direction, reflection of light 9A emitted from the lighting device 7 may be observed on the body of the target vehicle 5. The area on the body of the target vehicle 5 where reflection of light 9A may be observed (hereinafter referred to as the possible reflection area RPA) differs depending on the model of the target vehicle 5 and the vehicle position of the target vehicle 5.

[0019] Furthermore, as shown in FIG. 1(B), consider a case where a window 8 is installed in the parking lot 2. In this case, when the target vehicle 5 is viewed from a predetermined direction, the reflection of light 9B incident from the window 8 may be observed on the body of the target vehicle 5. The area on the body of the target vehicle 5 where the reflection of light 9B may be observed (as above, referred to as the possible reflection area RPA) differs depending on the type of vehicle 5 and the position of the target vehicle 5, and further differs depending on the time of day.

[0020] More specifically, if a window 8 is installed in the parking lot 2 and it is nighttime (first condition), or if a window 8 is not installed in the parking lot 2 (second condition), the reflection-possible area RPA is made up of only the portion that reflects light 9A. On the other hand, if a window 8 is installed in the parking lot 2 and it is daytime (third condition), the reflection-possible area RPA is made up of the portion that reflects light 9A and the portion that reflects light 9B.

[0021] Here, consider a case where the vehicle model of the target vehicle 5 is a predetermined vehicle model and the vehicle position of the target vehicle 5 is a predetermined position. In this case, the reflection potential area RPA on the body of the target vehicle 5 under the first and second conditions changes, for example, as shown in FIG. 1(C).

[0022] In the example shown in Fig. 1(C), the position and size on the body of the target vehicle 5 at which the reflection potential area RPA is observed are the same at each of times T1, T2, and T3. Fig. 1(C) shows images 20 of the target vehicle 5 taken by the camera 4 from a predetermined direction at each of times T1, T2, and T3.

[0023] On the other hand, the possible reflection area RPA on the body of the target vehicle 5 under the third condition changes as shown in Fig. 1(D). In the example shown in Fig. 1(D), the position and size on the body of the target vehicle 5 at which the possible reflection area RPA is observed are different at each of time T1, time T2, and time T3. However, it can be assumed that at the same time, the position and size on the body of the target vehicle 5 at which the possible reflection area RPA is observed are approximately the same. Note that Fig. 1(D) shows images 20 of the target vehicle 5 taken by the camera 4 from a predetermined direction at each of time T1, time T2, and time T3.

[0024] Therefore, for the first and second conditions, a possible reflection area RPA that may be observed on the body of the target vehicle 5 is determined for each model of the target vehicle 5 and for each position of the target vehicle 5 in the parking space 3. Furthermore, for the third condition, a possible reflection area RPA that may be observed on the body of the target vehicle 5 is determined for each model of the target vehicle 5, for each position of the target vehicle 5 in the parking space 3, and for each time period during which the target vehicle 5 is parked.

[0025] Information about these possible reflection areas RPA is registered in a database. The information about the possible reflection areas RPA registered in the database is expressed as information about the coordinate points of the upper left and lower right of a frame that surrounds the reflected portion of light on the image 20, as shown in, for example, Fig. 1(C) or Fig. 1(D). The database is registered in, for example, the management device 10.

[0026] FIG. 2 is a diagram illustrating an overview of the vehicle position of the target vehicle 5 identified when the possible reflection area RPA is removed and when it is not removed in the image recognition of the target vehicle 5. Specifically, as shown in FIG. 2(A), if the possible reflection area RPA is not removed in the image recognition of the target vehicle 5, the vehicle position of the target vehicle 5 determined by image recognition may be a position far from the actual position. In this case, the estimation result of the predicted position of the target vehicle 5 is also affected. Therefore, the next time the image recognition of the target vehicle 5 is performed, the image recognition of the target vehicle 5 is performed based on the possible reflection area RPA that corresponds to an inappropriate predicted position of the target vehicle 5. Therefore, there is a risk that the recognition accuracy of the vehicle position of the target vehicle 5 may deteriorate.

[0027] On the other hand, as shown in FIG. 2(B), if the possible reflection area RPA is removed in the image recognition of the target vehicle 5, it is expected that the vehicle position of the target vehicle 5 determined by image recognition will be the same as the actual position. In this case, the estimation result of the predicted position of the target vehicle 5 may also be appropriate. Therefore, the next time the target vehicle 5 is image recognized, the image recognition of the target vehicle 5 is performed based on the possible reflection area RPA that corresponds to the appropriate predicted position of the target vehicle 5. Therefore, the recognition accuracy of the vehicle position of the target vehicle 5 is improved, and the target vehicle 5 is parked in the parking space 3 while maintaining high recognition accuracy.

[0028] According to the vehicle position recognition system 1 according to the embodiment, possible reflection areas RPA that may be observed on the body of the target vehicle 5 are appropriately removed by image recognition. Two specific examples of methods for removing possible reflection areas RPA will be described below.

[0029] 2. Specific examples 2-1. First example 3 is a diagram for explaining a specific example of the first example of the vehicle position recognition system 1 according to the embodiment. In the specific example of the first example of the vehicle position recognition system 1 according to the embodiment, as shown in FIG. 3, the reflection possible area RPA is filled in with the same color as the surrounding area SA surrounding the reflection possible area RPA.

[0030] 3, the vehicle position recognition system 1 sets the luminance value of a pixel IP in the possible reflection area RPA in the image 20 to the luminance value of a pixel OP in the surrounding area SA in the image 20. In this case, for example, the luminance value at the coordinate value of the pixel OP that is closest to the coordinate value of the pixel IP is selected as the luminance value of the pixel OP that is set to the luminance value of the pixel IP.

[0031] 2-2. Second example 4 is a diagram for explaining a specific example of the second example of the vehicle position recognition system 1 according to the embodiment. In the specific example of the second example of the vehicle position recognition system 1 according to the embodiment, as shown in FIG. 4, the color of the reflected light portion in the reflection possible area RPA in the image 20 is filled in with the color of the other portion.

[0032] Specifically, as shown in Fig. 4, the vehicle position recognition system 1 scans the possible reflection area RPA in the image 20 with scanning lines LN (LN1, LN2, ..., LN6). The scanning line LN refers to a line that measures the brightness values ​​of pixels while moving horizontally. The scanning target of the scanning line LN may be limited to the possible reflection area RPA, or may be the entire image 20.

[0033] Then, the vehicle position recognition system 1 detects two edges on the scanning line LN scanned within the reflection potential area RPA. The two edges include a rising edge that transitions from a low brightness value to a high brightness value and a falling edge that transitions from a high brightness value to a low brightness value. In the example shown in FIG. 4, a rising edge is detected at pixel IP1 on the scanning line LN5, and a falling edge is detected at pixel IP2 on the scanning line LN5. Note that each of pixel IP1 and pixel IP2 may be a set of multiple pixels IP.

[0034] Furthermore, the vehicle position recognition system 1 fills in the pixel IP between the two detected edges with the color of the pixel OP outside the edges and in the surrounding area SA. Specifically, the vehicle position recognition system 1 sets the luminance value of the pixel IP between the two edges to the luminance value of the pixel OP outside the edges. In the example shown in FIG. 4, the luminance value of the pixel IP between the rising edge detected in pixel IP1 and the falling edge detected in pixel IP2 is set to the luminance value of the pixel OP outside the edges. In this case, for example, the luminance value at the coordinate value of the pixel OP outside the edges that is closest to the coordinate value of the pixel IP between the two edges is selected as the luminance value of the pixel IP between the two edges.

[0035] The process of setting the brightness value of the pixel IP between two edges to the brightness value of the pixel OP outside the edges may be performed only when the brightness value of the pixel IP between the two edges is equal to or greater than a threshold value, thereby reducing the processing load of the vehicle position recognition system 1.

[0036] 2-3.Effects In this way, in the vehicle position recognition system 1 according to the embodiment, the brightness value of the pixel IP in the reflection-possible area RPA is set to the brightness value of the pixel OP in the surrounding area SA. This allows the reflection-possible area RPA to be assimilated into the surrounding area SA. Therefore, it becomes possible to remove the reflection-possible area RPA.

[0037] Furthermore, by removing light reflected by the body of the target vehicle 5 through image processing, deterioration in the recognition accuracy of the vehicle position and false detections are reduced. In particular, by identifying the possible reflection area RPA and limiting processing to that area, it is possible to prevent adverse effects on the extraction of features from other parts where reflections do not occur.

[0038] 3.Configuration example 5 is a block diagram showing an example of the configuration of the management device 10 in the vehicle position recognition system 1 according to the embodiment. The management device 10 includes a computer 11 and a communication device 30. The computer 11 executes various processes. The computer 11 also stores a database 12 necessary for executing the various processes. Information on possible reflection areas RPA is registered in the database 12. Specifically, the possible reflection areas RPA for each vehicle type of the target vehicle 5 and for each vehicle position of the target vehicle 5 in the parking space 3 are registered in the database 12.

[0039] The information stored in computer 11 also includes a vehicle position recognition program (not shown). The vehicle position recognition program is a computer program executed by computer 11. The functions of computer 11 are realized by computer 11 executing the vehicle position recognition program.

[0040] The communication device 13 is a device that communicates with at least the target vehicle 5. The communication device 13 receives vehicle type information of the target vehicle 5 from the target vehicle 5.

[0041] 4. Processing example FIG. 6 is a flowchart showing an example of processing by the management device 10 in the vehicle position recognition system 1 according to the embodiment.

[0042] In step S100, the management device 10 acquires the image 20 from the camera 4. After that, the process proceeds to step S110.

[0043] In step S110, the management device 10 acquires various information. Then, the process proceeds to step S120. The various information includes vehicle type information of the target vehicle 5 and information on the possible reflection area RPA at the predicted position of the target vehicle 5. The vehicle type information of the target vehicle 5 is acquired from the target vehicle 5 via the communication device 30. Information on the possible reflection area RPA corresponding to each vehicle type of the target vehicle 5 and each vehicle position of the target vehicle 5 is acquired from the database 12.

[0044] In step S120, the management device 10 assimilates the possible reflection area RPA into the surrounding area SA for the image 20 obtained in step S100. Then, the process proceeds to step S130. Examples of the process of assimilation of the possible reflection area RPA into the surrounding area SA include the first and second examples described above.

[0045] In step S130, the management device 10 performs image recognition on the image 20 in which the reflection possible area RPA has been assimilated into the surrounding area SA. Thereafter, the process proceeds to step S140.

[0046] In step S140, the management device 10 identifies the vehicle position of the target vehicle 5 based on the image recognition information. After that, the process proceeds to step S150.

[0047] In step S150, the management device 10 updates the predicted position of the target vehicle 5 based on the vehicle position of the target vehicle 5 identified by image recognition.

[0048] 5. Other embodiments The vehicle position of the target vehicle 5 in the parking space 3 in the database 12 may be defined along one or more approach lines predicted when the target vehicle 5 enters the parking space 3. This makes it possible to reduce the amount of data that needs to be registered in advance in the database 12. [Explanation of symbols]

[0049] 1...Vehicle position recognition system, 2...Parking lot, 3...Parking space, 4...Camera, 5...Target vehicle, 6...Other vehicles, 7...Lighting device, 8...Window, 9A, 9B...Light, 10...Management device, 11...Computer, 12...Database, 20...Image, 30...Communication device, RPA...Possible reflective area, SA...Surrounding area, IP, OP...Pixel, LN...Scanning line

Claims

1. a camera that photographs the parking space from a predetermined direction; a storage device that stores a database in which reflective areas on a vehicle body where light reflection may be observed when a target vehicle parked in the parking space by automatic valet parking is viewed from the predetermined direction are registered for each vehicle type of the target vehicle and for each vehicle position of the target vehicle in the parking space; a computer that calculates the vehicle position of the target vehicle in the parking space based on the image input from the camera; The computer acquiring vehicle type information of the target vehicle; obtaining a predicted position of the target vehicle in the parking space; acquiring from the database a possible reflection area on the body of the target vehicle identified by the vehicle type information and the predicted position; In the image input from the camera, the potential reflection area on the body of the target vehicle is assimilated into a surrounding area surrounding the potential reflection area; determining the vehicle location by image recognition of an image in which the potential reflecting area is integrated into the surrounding area; updating the predicted position based on the vehicle position determined by the image recognition; Blending the potential reflecting area into the surrounding area includes painting the potential reflecting area with the same color as the surrounding area. A vehicle position recognition system characterized by:

2. A camera that photographs a parking space from a predetermined direction; a storage device that stores a database in which reflective areas on a vehicle body where light reflection may be observed when a target vehicle parked in the parking space by automatic valet parking is viewed from the predetermined direction are registered for each vehicle type of the target vehicle and for each vehicle position of the target vehicle in the parking space; a computer that calculates the vehicle position of the target vehicle in the parking space based on the image input from the camera; The computer acquiring vehicle type information of the target vehicle; obtaining a predicted position of the target vehicle in the parking space; acquiring from the database a possible reflection area on the body of the target vehicle identified by the vehicle type information and the predicted position; In the image input from the camera, the potential reflection area on the body of the target vehicle is assimilated into a surrounding area surrounding the potential reflection area; determining the vehicle location by image recognition of an image in which the potential reflecting area is integrated into the surrounding area; updating the predicted position based on the vehicle position determined by the image recognition; Integrating the reflective area into the surrounding area includes: detecting two edges on a scan line within the potentially reflective area; and filling the pixels between the edges with the color of the pixels outside the edges. A vehicle position recognition system characterized by:

3. 2. The vehicle position recognition system according to claim 1, The reflection possible area is registered in the database for each type of the target vehicle, for each vehicle position of the target vehicle in the parking space, and for each time period during which the target vehicle is parked. A vehicle position recognition system characterized by:

4. Calculating a vehicle position of a target vehicle to be parked in a parking space by automatic valet parking based on an image input from a camera photographing the parking space from a predetermined direction; acquiring vehicle type information of the target vehicle; obtaining a predicted position of the target vehicle in the parking space; acquiring a possible reflection area on the vehicle body of the target vehicle, which is identified by the vehicle type information and the predicted position, from a database in which a possible reflection area on the vehicle body where light reflection may be observed when the target vehicle is viewed from the predetermined direction is registered for each vehicle type of the target vehicle and for each vehicle position of the target vehicle in the parking space; In the image input from the camera, the potential reflection area on the body of the target vehicle is assimilated into a surrounding area surrounding the potential reflection area; determining the vehicle location by image recognition of an image in which the potential reflecting area is integrated into the surrounding area; updating the predicted position based on the vehicle position identified by the image recognition; Including, Blending the potential reflecting area into the surrounding area includes painting the potential reflecting area with the same color as the surrounding area. A vehicle position recognition method comprising:

5. Calculating the vehicle position of a target vehicle to be parked in a parking space by automatic valet parking based on an image input from a camera photographing the parking space from a predetermined direction; acquiring vehicle type information of the target vehicle; obtaining a predicted position of the target vehicle in the parking space; acquiring a possible reflection area on the vehicle body of the target vehicle, which is identified by the vehicle type information and the predicted position, from a database in which a possible reflection area on the vehicle body where light reflection may be observed when the target vehicle is viewed from the predetermined direction is registered for each vehicle type of the target vehicle and for each vehicle position of the target vehicle in the parking space; In the image input from the camera, the potential reflection area on the body of the target vehicle is assimilated into a surrounding area surrounding the potential reflection area; determining the vehicle location by image recognition of an image in which the potential reflecting area is integrated into the surrounding area; updating the predicted position based on the vehicle position identified by the image recognition; Including, Integrating the reflective area into the surrounding area includes: detecting two edges on a scan line within the potentially reflective area; and filling the pixels between the edges with the color of the pixels outside the edges. A vehicle position recognition method comprising:

Images