Driving environment determination device, vehicle, driving environment determination method, and program

Abstract

A travel environment determination device (100) comprises an image acquisition unit (105), an analysis unit (106), and a determination unit (107). The image acquisition unit (105) acquires a captured image that is obtained by imaging performed by an imaging device installed in a vehicle. The analysis unit (106) performs analysis processing with respect to the captured image and identifies a first region which corresponds to the sky in the captured image. The determination unit (107) determines, on the basis of the first region, whether or not the vehicle is inside a structure.

Description

Driving environment determination device, vehicle, driving environment determination method, and recording medium

[0001] The present invention relates to a driving environment determination device, a vehicle, a driving environment determination method, and a recording medium.

[0002] The vehicle described in Patent Document 1 includes a camera and a detection device for tunnel detection. The camera described in Patent Document 1 is arranged in front of the vehicle and is capable of capturing at least one image area around the vehicle. The image area includes a plurality of pixels. The tunnel detection device described in Patent Document 1 is designed to determine the average brightness of the at least one image area. The tunnel detection device also includes a device for characteristic detection, which can be used to obtain the brightness difference between different pixels. Thus, the tunnel detection device can detect a characteristic in the at least one image area that is characterized by a sudden change in brightness.

[0003] The automatic headlight system described in Patent Document 2 is mounted on a vehicle, captures an image of the vehicle's driving environment (mainly the scenery ahead) while the vehicle is moving, and automatically controls the on / off of the headlights based on the captured image. This automatic headlight system uses a camera with a special type of wide-angle lens to obtain an image with a wider vertical range (angle of view). A prism is also installed below the windshield inside the vehicle, and an image of the interior of the vehicle is captured in the lower region of the captured image. This results in an image with a sufficiently wide range, from the high sky above the vehicle in front of it (an area where buildings and the like are unlikely to be captured) to the interior of the vehicle below.

[0004] Then, four regions are set in the captured image: a forward sky brightness judgment region, a forward mid-air brightness judgment region, a forward distant brightness judgment region, and a vehicle interior brightness judgment region. The brightness of each judgment region is judged, and based on the judgment results, it is determined how the vehicle lights should be controlled.

[0005] The object candidate region detection device described in Patent Document 3 detects, as an object candidate region, a region in which a specific object may exist from an input image captured by a camera. This object candidate region detection device includes a reference pattern storage means, a background region division means, a determination region extraction means, a reference pattern selection means, and a detection means.

[0006] The reference pattern storage means described in Patent Document 3 stores a plurality of reference patterns that differ for road areas and non-road areas in the background of the input image. The background area division means described in Patent Document 3 divides the background of the input image currently captured by the camera into road areas and non-road areas. The determined area extraction means described in Patent Document 3 extracts a determined area from the input image currently captured by the camera.

[0007] The reference pattern selection means described in Patent Document 3 selects a reference pattern from a plurality of reference patterns that corresponds to the background area from which the determined area is cut out, that is, a road area or an area outside the road. The detection means described in Patent Document 3 detects object candidate areas from the determined area by comparing the selected reference pattern with the determined area.

[0008] Non-Patent Document 1 describes region recognition (also called region segmentation), which is one of the techniques for image recognition. Region recognition is a technique that takes an image as input and estimates the type of subject represented in each region included in the image.

[0009] JP-T-2014-517388A JP-A-2009-255722 JP-A-2007-328630

[0010] S. Schulter et al. , “Learning to Look around Objects for Top-View Representations of Outdoor Scenes”, ECCV, 2018, pp. 787-802

[0011] The technology described in Patent Document 1 detects tunnels based on differences in pixel brightness. However, if the presence of a vehicle in a tunnel is detected based solely on pixel brightness, there is a risk of the vehicle being mistakenly detected as being in a tunnel because the camera's shooting environment varies.

[0012] The technology described in Patent Document 2 recognizes whether a vehicle is in a tunnel based on the brightness of each determination area in a captured image. However, because the brightness of a determination area is a value obtained based on the luminance of pixels, the technology described in Patent Document 2 may also erroneously recognize that a vehicle is in a tunnel, as with the technology described in Patent Document 1. Furthermore, the technology described in Patent Document 2 requires a camera with a special type of wide-angle lens, and it may be difficult to recognize that a vehicle is in a tunnel using an image captured using a camera with a general angle of view.

[0013] Patent Literature 3 describes dividing an input image captured by a current camera into a road area and an area outside the road with respect to the background. Non-Patent Literature 1 also describes an example of area recognition. However, neither Patent Literature 3 nor Non-Patent Literature 1 discloses a technique for determining whether a vehicle is inside a structure such as a tunnel.

[0014] In view of the above-mentioned problems, one example of the object of the present invention is to provide a driving environment determination device, a vehicle, a driving environment determination method, and a recording medium that solve the problem of accurately determining whether a vehicle is inside a structure.

[0015] According to one aspect of the present invention, there is provided a driving environment judgment device comprising: an image acquisition means for acquiring a captured image obtained by an imaging device installed in a vehicle; an analysis means for performing an analysis process on the captured image to identify a first area in the captured image that is an area corresponding to the sky; and a judgment means for determining whether the vehicle is within a structure based on the first area.

[0016] According to one aspect of the present invention, there is provided a vehicle including the driving environment determination device and the imaging device that is installed in the vehicle and captures images to generate the captured image.

[0017] According to one aspect of the present invention, a driving environment determination method is provided in which a computer acquires a captured image obtained by a photographing device installed in a vehicle, performs an analysis process on the captured image to identify a first area in the captured image that corresponds to the sky, and determines whether the vehicle is inside a structure based on the first area.

[0018] According to one aspect of the present invention, a recording medium is provided having recorded thereon a program for causing a computer to acquire an image obtained by an imaging device installed in a vehicle, perform an analysis process on the image, identify a first area in the image that corresponds to the sky, and determine whether the vehicle is within a structure based on the first area.

[0019] According to one aspect of the present invention, it is possible to provide a driving environment determination device, a vehicle, a driving environment determination method, and a recording medium that solve the problem of accurately determining whether a vehicle is inside a structure.

[0020] 1 is a diagram showing an overview of a driving environment determination device according to embodiment 1. FIG. 2 is a diagram showing an overview of a vehicle according to embodiment 1. FIG. 3 is a flowchart showing an overview of driving environment determination processing according to embodiment 1. FIG. 4 is a diagram showing a detailed configuration example of a vehicle according to embodiment 1. FIG. 5 is a diagram showing an example of a physical configuration of a driving environment determination device according to embodiment 1. FIG. 6 is a flowchart showing an example of driving environment determination processing according to embodiment 1. FIG. 7 is a diagram showing a first example of a captured image. FIG. 8 is a diagram showing a second example of a captured image. FIG. 9 is a diagram showing an example of a functional configuration of a driving environment determination device according to embodiment 2. FIG. 10 is a flowchart showing an example of driving environment determination processing according to embodiment 2. FIG. 11 is a flowchart showing a detailed example of determination processing according to embodiment 2.

[0021] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, like components are designated by like reference numerals, and the description thereof will be omitted as appropriate.

[0022] 1 is a diagram showing an overview of a driving environment determination device 100 according to embodiment 1. The driving environment determination device 100 includes an image acquisition unit 105, an analysis unit 106, and a determination unit 107.

[0023] The image acquisition unit 105 acquires a captured image obtained by an imaging device installed in the vehicle. The analysis unit 106 performs an analysis process on the captured image to identify a first region in the captured image that corresponds to the sky. The determination unit 107 determines whether the vehicle is inside a structure based on the first region.

[0024] According to this driving environment determination device 100, it is possible to accurately determine whether or not the vehicle is inside a structure.

[0025] 2 is a diagram showing an overview of a vehicle 120 according to embodiment 1. The vehicle 120 includes the traveling environment determination device 100 and an image capturing device 121. The image capturing device 121 is installed in the vehicle and captures images to generate captured images.

[0026] With this vehicle 120, it is possible to accurately determine whether or not the vehicle 120 is inside a structure.

[0027] FIG. 3 is a flowchart showing an outline of the driving environment determination process according to the first embodiment.

[0028] The image acquisition unit 105 acquires a captured image obtained by capturing an image using the imaging device 121 installed on the vehicle 120 (step S101). The analysis unit 106 performs an analysis process on the captured image to identify a first region, which is a region in the captured image that corresponds to the sky (step S102). The determination unit 107 determines whether the vehicle 120 is inside a structure based on the first region (step S103).

[0029] According to this driving environment determination method, it is possible to accurately determine whether or not the vehicle is inside a structure.

[0030] (Details) A detailed example of the vehicle 120 according to the first embodiment will be described below.

[0031] 4 is a diagram showing a detailed configuration example of the vehicle 120 according to this embodiment. The vehicle 120 is, for example, a passenger car, a truck, a bus, etc. The vehicle 120 may also be a motorcycle, a bicycle, etc.

[0032] The vehicle 120 according to this embodiment includes an imaging device 121, a vehicle control device 122, and a driving environment determination device 100. The imaging device 121, the vehicle control device 122, and the driving environment determination device 100 are connected to each other via a communication line configured by a wired or wireless connection or a combination thereof so as to be able to transmit and receive information to and from each other.

[0033] The image capturing device 121 is, for example, a camera, a terminal device such as a smartphone with an image capturing function, or the like. The image capturing device 121 is installed on the vehicle 120 so as to capture images of the area around the vehicle 120. FIG. 4 shows an example in which the image capturing device 121 is installed so as to capture images of the area in front of the vehicle 120. The image capturing device 121 according to this embodiment generates a captured image of the area in front of the vehicle 120. The angle of view of the image capturing device 121 may be the same as that of a camera typically mounted on a vehicle or a camera typically provided in a terminal device, and may be, for example, 80 degrees to 110 degrees. However, the angle of view of the image capturing device 121 is not limited to this.

[0034] The imaging device 121 is not limited to being installed in front of the vehicle 120, but may be installed to capture an image of the rear or side of the vehicle 120, for example, and generate a captured image of the rear or side of the vehicle 120. Also, while Fig. 4 shows an example in which the imaging device 121 is installed inside the cabin of the vehicle 120, the installation location of the imaging device 121 is not limited to being inside the cabin as long as it is installed in the vehicle 120.

[0035] The vehicle control device 122 is an ECU (Electronic Control Unit) or the like. The vehicle control device 122 controls the vehicle 120.

[0036] For example, if the vehicle 120 has an automatic driving function, the vehicle control device 122 uses map information, current position information, etc. to control the driving of the vehicle 120. The vehicle control device 122 acquires the current position information using, for example, a Global Positioning System (GPS).

[0037] For example, if the vehicle 120 has an automatic lighting function that controls exterior lights that emit light to the outside of the vehicle 120, the vehicle control device 122 controls the turning on and off of the exterior lights. The exterior lights may be, for example, one or more of headlights HL, tail lamps (also called tail lights or taillights) TL, and sidelights (not shown).

[0038] (Example of Functional Configuration of Driving Environment Determination Device 100) The driving environment determination device 100 according to this embodiment is a device for determining the environment in which the vehicle 120 is driving. The driving environment determination device 100 determines, for example, whether the driving environment of the vehicle 120 is inside a tunnel provided on an expressway, a public road, or the like, or whether the driving environment is inside a structure such as an indoor parking lot.

[0039] As described above with reference to FIG. 1, the driving environment determination device 100 functionally includes an image acquisition unit 105, an analysis unit 106, and a determination unit 107.

[0040] The image acquisition unit 105 acquires the captured image generated by the image capturing device 121 via a communication line from the image capturing device 121 .

[0041] The analysis unit 106 performs an analysis process on the captured image acquired by the image acquisition unit 105 to identify various areas included in the captured image.

[0042] The analysis process performed by the analysis unit 106 to identify various regions included in the captured image may, for example, be region recognition (also referred to as region division, segmentation, etc.), which is one of the techniques for image recognition. Region recognition is a technique that takes an image as input and estimates the type of subject represented in each region included in the image. An example of such region recognition is the technique described in Non-Patent Document 1.

[0043] Furthermore, the analysis unit 106 may identify various regions included in the captured image using a learning model that has been trained by machine learning. In this case, the analysis unit 106 identifies various regions using, for example, a learning model that inputs a captured image and outputs region information for dividing the captured image into regions. In this case, when training the learning model, supervised learning may be performed using captured images in which the type of subject is assigned to each region of the captured image as training data.

[0044] Generally, the roads on which the vehicle 120 travels include various types of roads, such as expressways and general roads. For example, general roads often have pedestrian sidewalks and roadside trees near the road, whereas expressways rarely have pedestrian sidewalks or roadside trees near the road. In this way, the subjects included in the captured image differ depending on the attributes of the road. Therefore, the analysis unit 106 may store multiple learned learning models according to the attributes of the road.

[0045] The analysis unit 106 may then identify various areas included in the captured image, such as the first area, using a learning model, among the multiple learning models, that corresponds to the attributes of the road on which the vehicle 120 is traveling. In this case, the type of road on which the vehicle 120 is traveling may be acquired, for example, from the vehicle control device 122. The vehicle control device 122 may determine the type of road from, for example, position information and map information of the vehicle 120, may determine the type of road from the traveling speed of the vehicle 120, or may determine the type of road from the results of analyzing the captured image.

[0046] Using such a technique, the analysis unit 106 identifies, for example, a first region. As described above, the first region is a region corresponding to the sky in the captured image.

[0047] Furthermore, for example, the area identified by the analysis unit 106 includes a reference area. The reference area is an area above a position related to the road in the captured image. In this embodiment, the reference area is a first area and a second area (described later) of the area above the position related to the road in the captured image.

[0048] The position related to the road is, for example, the position of the top edge of the road in the captured image, which is identified as, for example, the position of the vanishing point of the road in the captured image.

[0049] To find the vanishing point, the analysis unit 106 identifies lines in the captured image that correspond to lines extending parallel to the actual road on which the vehicle 120 is traveling. Lines extending parallel to the actual road include, for example, lines corresponding to both side edges of the road, white lines on the road, yellow lines on the road, etc. The analysis unit 106 then uses the identified lines to find the vanishing point of the road in the captured image.

[0050] Furthermore, for example, the analysis unit 106 identifies a second region, which is a region corresponding to a predetermined type of subject other than the sky, in the captured image.

[0051] The second region may include, for example, a region corresponding to at least one of an obstacle and a structure. The obstacle may include other vehicles around the vehicle 120. The structure may include at least one of a tunnel and an indoor parking lot. The structure may further include a building around the road.

[0052] The second area is not limited to obstacles and structures, but may also include, for example, one or more of roads, people, traffic lights, white lines, yellow lines, poles (street lights), motorcycles, signs, stop lines, crosswalks, parking lots (parking spaces on the side of the road), road paint, sidewalks, driveways (vehicle paths on sidewalks connecting the roadway to facilities, etc.), railroad tracks, trees, plants, and others.

[0053] The determination unit 107 determines whether or not the vehicle 120 is inside a structure based on the reference area and the first area identified by the analysis unit 106. The structure inside which the vehicle 120 travels is typically a tunnel, a building or structure with an indoor parking lot, or the like.

[0054] So far, an example of the functional configuration of the traveling environment determination device 100 according to the first embodiment has been described. From here, an example of the physical configuration of the traveling environment determination device 100 according to the first embodiment will be described.

[0055] 5 is a diagram showing an example of the physical configuration of the driving environment determination device 100 according to this embodiment. The driving environment determination device 100 physically includes, for example, a bus 1010, a processor 1020, a memory 1030, a storage device 1040, a network interface 1050, and a user interface 1060.

[0056] The bus 1010 is a data transmission path for transmitting and receiving data among the processor 1020, memory 1030, storage device 1040, network interface 1050, and user interface 1060. However, the method of connecting the processor 1020 and the like to each other is not limited to bus connection.

[0057] The processor 1020 is implemented as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit).

[0058] The memory 1030 is a main storage device realized by a RAM (Random Access Memory) or the like.

[0059] The storage device 1040 is an auxiliary storage device realized by a hard disk drive (HDD), a solid state drive (SSD), a memory card, a read only memory (ROM), or the like. The storage device 1040 stores program modules for realizing each functional unit of the driving environment determination device 100. The processor 1020 reads each of these program modules into the memory 1030 and executes them, thereby realizing each functional unit corresponding to the program module.

[0060] The network interface 1060 is an interface for connecting the driving environment determination device 100 to a communication line.

[0061] The user interface 1050 is an interface for connecting a terminal or the like that allows a user to make various settings in the driving environment determination device 100, for example.

[0062] So far, an example of the functional and physical configuration of the vehicle 120 according to the first embodiment has been described. Next, an example of the operation of the vehicle 120 according to the first embodiment will be described.

[0063] (Example of operation of vehicle 120 according to embodiment 1) Fig. 6 is a flowchart showing an example of a driving environment determination process according to this embodiment. The driving environment determination process is a process for determining the environment in which the vehicle 120 is driving. The environment determination process is executed repeatedly while the vehicle 120 is driving, for example. The imaging device 121 captures an image of the area ahead of the vehicle 120 while the vehicle 120 is driving, and generates a captured image of the captured image.

[0064] The image acquisition unit 105 acquires a photographed image generated by the photographing device 121 (step S101).

[0065] 7 and 8 each show an example of a photographed image generated by the photographing device 121. Fig. 7 shows an example of a photographed image taken by the photographing device 121 installed in the vehicle 120 traveling outside a tunnel on a highway. Fig. 8 shows an example of a photographed image taken by the photographing device 121 installed in the vehicle 120 traveling inside a tunnel on a highway.

[0066] The analysis unit 106 analyzes the captured image acquired in step S101 (step S102).

[0067] Specifically, the analysis unit 106 identifies a first region, which is a region corresponding to the sky in the captured image (see FIGS. 7 and 8).

[0068] The analysis unit 106 also identifies a reference area in the captured image. As described above, the reference area in this embodiment is the first and second areas in the captured image above a position related to the road. As described above, the second area is an area corresponding to a predetermined type of subject other than the sky, and in this case, it is assumed to be an obstacle or a structure.

[0069] When the position related to the road in the captured image is the vanishing point of the road, in the example of the captured image shown in Figures 7 and 8, the analysis unit 106 is the first and second areas of the area above the vanishing point of the road in the captured image.

[0070] In the example shown in Fig. 7, the second area includes buildings, which are structures, and other vehicles, which are obstacles. In the example shown in Fig. 8, the second area includes tunnels (inner walls), which are structures, and other vehicles, which are obstacles.

[0071] The position of the top of the road may be determined by any method other than the vanishing point of the road. For example, the analysis unit 106 may determine an area corresponding to the road as the second area, and determine the top of the road by the point located at the top of the captured image within the area corresponding to the road (see FIG. 8 ).

[0072] The determination unit 107 determines whether the vehicle 120 is inside a structure based on the reference area and the first area identified in step S102 (step S103).

[0073] Any appropriate method may be adopted as the determination method in step S103.

[0074] For example, the determination unit 107 may determine whether the vehicle 120 is inside a structure based on the proportion of the first region in the reference region. Generally, when the vehicle 120 is inside a structure, the proportion of the first region in the reference region is smaller than when the vehicle 120 is outside the structure. Therefore, for example, the determination unit 107 determines that the vehicle 120 is inside a structure when the proportion of the first region in the reference region is equal to or smaller than a predetermined threshold. Furthermore, the determination unit 107 determines that the vehicle 120 is not inside a structure when the proportion of the first region in the reference region is greater than the predetermined threshold.

[0075] For example, the determination unit 107 may determine whether the vehicle 120 is inside a structure based on whether the periphery of the first region in the reference region is surrounded by the second region. As illustrated in Figures 7 and 8, when the vehicle 120 is inside a structure, the periphery of the first region is surrounded by the second region, whereas when the vehicle 120 is not inside a structure, the periphery of the first region is not surrounded by the second region. Here, the periphery of the first region being surrounded by the second region means that the second region exists above and to the side of the first region.

[0076] Therefore, for example, when the first area in the reference area is surrounded by the second area, the determination unit 107 determines that the vehicle 120 is inside a structure. Also, when the first area in the reference area is not surrounded by the second area, the determination unit 107 determines that the vehicle 120 is not inside a structure.

[0077] The vehicle control device 122 controls the vehicle 120 based on the result of the determination in step S103 (step S104).

[0078] For example, when controlling the automatic driving of the vehicle 120, the vehicle control device 122 switches the control mode between, for example, a normal mode and an in-structure mode.

[0079] The normal mode is a control mode used outside a structure. In the normal mode, the vehicle control device 122 controls the automatic driving of the vehicle 120 using current position information obtained using, for example, a GPS.

[0080] The in-structure mode is a control mode used inside a structure. Generally, in structures such as tunnels and indoor parking lots, the accuracy of current position information obtained using GPS decreases due to diffuse reflection of radio waves. Therefore, in the in-structure mode, the vehicle control device 122 controls the autonomous driving of the vehicle 120 using current position information obtained using self-driving information including the travel distance and travel direction of the vehicle 120, instead of GPS, for example.

[0081] Furthermore, for example, during the daytime when exterior vehicle lights are not turned on outside a structure, the vehicle control device 122 turns on the exterior vehicle lights when the vehicle 120 is inside the structure and turns off the exterior vehicle lights when the vehicle 120 is outside the structure. In other words, the vehicle control device 122 turns on the exterior vehicle lights while the vehicle 120 is inside the structure.

[0082] (Actions and Effects) According to this embodiment, the driving environment determination device 100 includes an image acquisition unit 105, an analysis unit 106, and a determination unit 107. The image acquisition unit 105 acquires a captured image obtained by an imaging device installed in the vehicle. The analysis unit 106 performs an analysis process on the captured image to identify a first region, which is a region in the captured image that corresponds to the sky. The determination unit 107 determines whether or not the vehicle 120 is inside a structure based on the first region.

[0083] This allows a first area, which is an area corresponding to the sky, to be identified from the captured image, and it is possible to determine whether or not the vehicle 120 is inside a structure based on the first area. Therefore, it becomes possible to accurately determine whether or not the vehicle 120 is inside a structure.

[0084] According to the present embodiment, the analysis unit 106 further identifies a reference area in the captured image that satisfies a predetermined criterion. The determination unit 107 determines whether the vehicle 120 is inside a structure based on the reference area and the first area.

[0085] This makes it possible to determine whether the vehicle 120 is inside a structure based on the first region, which is a region corresponding to the sky in the captured image, and the reference region that satisfies a predetermined criterion. Therefore, it becomes possible to accurately determine whether the vehicle 120 is inside a structure.

[0086] According to this embodiment, the reference area is an area above a position related to the road in the captured image.

[0087] This makes it possible to determine whether the vehicle 120 is inside a structure based on the first region, which is a region corresponding to the sky in the captured image, and the reference region that satisfies a predetermined criterion. Therefore, it becomes possible to accurately determine whether the vehicle 120 is inside a structure.

[0088] The position related to the road is the position of the vanishing point of the road in the captured image.

[0089] This makes it possible to determine whether the vehicle 120 is inside a structure based on the first region, which is the region corresponding to the sky, and the reference region, which is the region above the vanishing point of the road in the captured image. Therefore, it becomes possible to accurately determine whether the vehicle 120 is inside a structure.

[0090] The analysis unit 106 performs an analysis process on the captured image to identify a second area, which is an area in the captured image that corresponds to a predetermined type of subject other than the sky, and identifies the first and second areas in the area above a position related to the road in the captured image as reference areas.

[0091] This makes it possible to determine whether the vehicle 120 is inside the structure based on the first area and the reference area that is based on the first area and the second area, thereby making it possible to accurately determine whether the vehicle 120 is inside the structure.

[0092] The second region includes a region corresponding to at least one of an obstacle and a structure.

[0093] This makes it possible to determine whether the vehicle 120 is inside the structure based on the first area and the reference area that is based on the first area and the second area, thereby making it possible to accurately determine whether the vehicle 120 is inside the structure.

[0094] The determination unit 107 determines whether the vehicle 120 is inside a structure based on the proportion of the first area in the reference area.

[0095] This makes it possible to determine whether or not the vehicle 120 is inside the structure based on the first area and the reference area. Therefore, it becomes possible to accurately determine whether or not the vehicle 120 is inside the structure.

[0096] The determination unit 107 determines whether the vehicle 120 is inside a structure based on whether the first area is surrounded by the second area in the reference area.

[0097] This makes it possible to determine whether the vehicle 120 is inside the structure based on the first area and the reference area that is based on the first area and the second area, thereby making it possible to accurately determine whether the vehicle 120 is inside the structure.

[0098] The analysis unit 106 inputs a captured image and identifies the first region using a learning model that outputs region information for dividing the captured image into regions included in the captured image.

[0099] This allows a first area, which is an area corresponding to the sky, to be identified from the captured image, and it is possible to determine whether or not the vehicle 120 is inside a structure based on the first area. Therefore, it becomes possible to accurately determine whether or not the vehicle 120 is inside a structure.

[0100] The learning model is one of a plurality of learning models corresponding to the attributes of the road. The analysis unit 106 identifies the first area using one of the plurality of learning models corresponding to the attributes of the road on which the vehicle 120 travels.

[0101] This allows the first area to be identified using a learning model according to the attributes of the road on which the vehicle 120 is traveling, making it possible to identify the first area with greater accuracy. Then, based on this accurately identified first area, it is possible to determine whether the vehicle 120 is inside a structure. Therefore, it is possible to more accurately determine whether the vehicle 120 is inside a structure.

[0102] (Modification 1) In the first embodiment, the analysis unit 106 identifies the reference region. However, the analysis unit 106 does not have to identify the reference region.

[0103] In this modification, for example, the determination unit 107 may determine whether the vehicle 120 is inside a structure based on the proportion of the first region in the entire captured image and a predetermined threshold. In detail, for example, the determination unit 107 determines that the vehicle 120 is inside a structure when the proportion of the first region in the entire captured image is equal to or smaller than the threshold, and determines that the vehicle 120 is not inside a structure when the proportion of the first region in the entire captured image is greater than the threshold.

[0104] Furthermore, for example, the determination unit 107 may determine whether the vehicle 120 is inside a structure based on whether the first area is surrounded by a second area including a road, a structure, an obstacle, etc. In detail, for example, the determination unit 107 determines that the vehicle 120 is inside a structure when the first area is surrounded by the second area in the reference area, and determines that the vehicle 120 is not inside a structure when the first area is not surrounded by the second area in the reference area.

[0105] This modification also makes it possible to identify a first area, which is an area corresponding to the sky, from the captured image and determine whether or not the vehicle 120 is inside a structure based on the first area. Therefore, it becomes possible to accurately determine whether or not the vehicle 120 is inside a structure.

[0106] (Variation 2) In the first embodiment, the example in which the analysis unit 106 identifies the second region has been described. However, the analysis unit 106 may not identify the second region, but may instead identify an area above a position related to the road in the captured image as the reference region.

[0107] In this modification, for example, the determination unit 107 may determine whether the vehicle 120 is inside a structure based on the proportion of the first area in the reference area, similar to the first embodiment.

[0108] For example, the determination unit 107 may determine whether the vehicle 120 is inside a structure based on whether the first region is surrounded by a reference region. Here, the fact that the first region is surrounded by a reference region means that a second region exists above and to the side of the first region. In detail, for example, the determination unit 107 may determine that the vehicle 120 is inside a structure when the first region is surrounded by a reference region, and may determine that the vehicle 120 is not inside a structure when the first region is not surrounded by a reference region.

[0109] This modification also makes it possible to identify a first area, which is an area corresponding to the sky, from the captured image and determine whether or not the vehicle 120 is inside a structure based on the first area. Therefore, it becomes possible to accurately determine whether or not the vehicle 120 is inside a structure.

[0110] Second Embodiment In the first embodiment, an example has been described in which it is determined whether or not the vehicle 120 is inside a structure using one captured image. However, it may also be determined whether or not the vehicle 120 is inside a structure using a time series of captured images, i.e., a plurality of captured images. In this embodiment, an example will be described in which it is determined whether or not the vehicle 120 is inside a structure using a time series of captured images.

[0111] The vehicle according to the second embodiment includes the driving environment determination device 200 according to the first embodiment instead of the driving environment determination device 100 according to the first embodiment. Except for this point, the vehicle according to this embodiment may be configured similarly to the vehicle according to the first embodiment.

[0112] 9 is a diagram showing an example of the functional configuration of the driving environment determination device 200 according to this embodiment. The driving environment determination device 200 functionally includes an image acquisition unit 205, an analysis unit 206, and a determination unit 207.

[0113] The image acquisition unit 205 acquires time-series captured images from the image capturing device 121 .

[0114] The analysis unit 206 performs an analysis process on each of the time-series captured images acquired by the image acquisition unit 205, and identifies various regions included in each captured image. The analysis unit 206 identifies, for example, a first region, a second region, and a reference region.

[0115] The determination unit 207 determines whether or not the vehicle is inside a structure based on the reference area and the first area identified by the analysis unit 206, similar to the determination unit 107 according to the first embodiment.

[0116] As shown in FIG. 9, the determination unit 207 according to this embodiment includes a first processing unit 207a and a second processing unit 207b.

[0117] The first processing unit 207a determines whether each of the captured images in the time series is an image inside a structure, based on the first region identified in each captured image by the analysis unit 206. An image inside a structure is an image captured inside a structure.

[0118] The second processing unit 207b determines whether or not the vehicle is inside a structure based on the determination results of the first processing unit 207a for each of the captured images in time series.

[0119] The image acquisition unit 205, the analysis unit 206, and the determination unit 207 may be configured generally similarly to the image acquisition unit 105, the analysis unit 106, and the determination unit 107 according to the first embodiment, except for the points described above.

[0120] The driving environment determining device 200 according to this embodiment may be physically configured in the same manner as the driving environment determining device 100 according to the first embodiment.

[0121] 10 is a flowchart showing an example of the driving environment determination process according to this embodiment. The driving environment determination process according to this embodiment includes steps S201 to S103 that replace steps S101 to S103 of the driving environment determination process according to the first embodiment.

[0122] The image acquisition unit 205 acquires a plurality of captured images generated by the image capturing device 121 (step S201).

[0123] The analysis unit 206 analyzes each of the captured images acquired in step S201 (step S202). The content of the analysis process performed on each of the captured images in step S202 may be the same as the analysis process performed on the captured images in step S102 according to the first embodiment.

[0124] The determination unit 207 determines whether the vehicle 120 is inside a structure based on the reference area and the first area identified in step S202 (step S203).

[0125] FIG. 11 is a flowchart showing a detailed example of the determination process (step S203) according to this embodiment.

[0126] The first processing unit 207a determines whether each of the time-series captured images is an image inside a structure, based on the first region identified for each captured image in step S202 (step S203a).

[0127] The second processing unit 207b determines whether or not the vehicle is inside the structure based on the determination result in step S203a for each of the captured images in time series (step S203b).

[0128] In detail, for example, the second processing unit 207b determines that the vehicle is inside the structure when, among a predetermined number of consecutive captured images, a number of captured images equal to or greater than a predetermined threshold are determined to be images inside the structure.Furthermore, the second processing unit 207b determines that the vehicle is not inside the structure when, among a predetermined number of consecutive captured images, a number of captured images less than the threshold are determined to be images inside the structure.

[0129] For example, there is a possibility that some of the predetermined number of captured images may be erroneously determined to be images inside a structure in step S203a. In this embodiment, whether or not a vehicle is inside a structure is determined based on whether or not the number of captured images determined to be images inside a structure among a predetermined number of temporally consecutive captured images is equal to or greater than a threshold. This makes it possible to correctly determine whether or not a vehicle is inside a structure even if an erroneous determination is made in step S203a for some of the captured images.

[0130] (Operations and Effects) According to this embodiment, the image acquisition unit 205 acquires photographed images in time series. The analysis unit 206 performs an analysis process on each of the photographed images in time series to identify a first region in each photographed image.

[0131] The determination unit 207 includes a first processing unit 207a and a second processing unit 207b. The first processing unit 207a determines whether each of the time-series captured images is an interior image of a structure captured inside the structure, based on the first region identified in each captured image. The second processing unit 207b determines whether a vehicle is inside the structure, based on the determination result of the first processing unit 207a regarding each of the time-series captured images.

[0132] As a result, even if an erroneous determination is made in step S203a for some of the captured images, as described above, it is possible to correctly determine whether or not the vehicle is inside the structure. Therefore, it is possible to accurately determine whether or not the vehicle is inside the structure.

[0133] Although the embodiments of the present invention have been described above with reference to the drawings, these are merely examples of the present invention, and various other configurations can also be adopted.

[0134] In addition, although the flowcharts used in the above description show multiple steps (processes) in a sequential order, the order of steps executed in each embodiment is not limited to the order shown. In each embodiment, the order of the steps shown in the drawings can be changed as long as it does not cause any problems in terms of content. Furthermore, the above-described embodiments can be combined as long as the content is not contradictory.

[0135] Some or all of the above embodiments may be described as, but are not limited to, the following supplementary notes. 1. A driving environment determination device comprising: image acquisition means for acquiring a captured image obtained by capturing an image with a capturing device installed in a vehicle; analysis means for performing an analysis process on the captured image to identify a first area that is an area corresponding to the sky in the captured image; and determination means for determining whether or not the vehicle is inside a structure based on the first area. 2. The driving environment determination device described in 1., wherein the analysis means further identifies a reference area that meets predetermined criteria in the captured image, and the determination means determines whether or not the vehicle is inside a structure based on the reference area and the first area. 3. The driving environment determination device described in 2., wherein the reference area is an area above a position related to a road in the captured image. 4. The driving environment determination device described in 3., wherein the position related to the road is the position of a vanishing point of a road in the captured image. 5. The driving environment determination device according to any one of 2. to 4., wherein the analysis means performs an analysis process on the captured image to identify a second area in the captured image that is an area corresponding to a predetermined type of subject other than the sky, and identifies the first area and the second area in an area above a position related to a road in the captured image as the reference area. 6. The driving environment determination device according to 5., wherein the second area includes an area corresponding to at least one of an obstacle and a structure. 7. The driving environment determination device according to any one of 2. to 6., wherein the determination means determines whether or not the vehicle is inside a structure based on the proportion of the first area in the reference area. 8. The driving environment determination device according to 5. or 6., wherein the determination means determines whether or not the vehicle is inside a structure based on whether the first area is surrounded by the second area in the reference area. 9. The analysis means identifies the first area using a learning model that inputs the captured image and outputs area information for dividing the captured image into areas included in the captured image. 10. The driving environment determination device according to claim 9, wherein:10. The driving environment determination device described in any one of 1. to 10., wherein the learning model is one of a plurality of learning models according to the attributes of a road, and the analysis means identifies the first area using one of the plurality of learning models according to the attributes of the road on which the vehicle is traveling. 11. The image acquisition means acquires the captured images in time series, and the analysis means performs an analysis process on each of the captured images in the time series to identify the first area in each of the captured images, and the determination means includes: first processing means that determines whether each of the captured images in the time series is an image inside a structure captured within a structure based on the first area identified in each of the captured images, and second processing means that determines whether the vehicle is inside a structure based on the determination result of the first processing means for each of the captured images in the time series. 12. A vehicle comprising the driving environment determination device described in any one of 1. to 11., and the imaging device that is installed on the vehicle and generates the captured images by capturing images. 13. The vehicle described in 12. further comprises vehicle control means installed on the vehicle and controlling the vehicle. 14. A driving environment determination method in which a computer acquires a photographed image obtained by photographing with a photographing device installed on the vehicle, performs an analysis process on the photographed image to identify a first area in the photographed image that is an area corresponding to the sky, and determines whether or not the vehicle is inside a structure based on the first area. 15. A recording medium having recorded thereon a program for causing a computer to acquire a photographed image obtained by photographing with a photographing device installed on the vehicle, perform an analysis process on the photographed image to identify a first area in the photographed image that is an area corresponding to the sky, and determine whether or not the vehicle is inside a structure based on the first area. 16. A program for causing a computer to acquire a photographed image obtained by photographing with a photographing device installed on the vehicle, perform an analysis process on the photographed image to identify a first area in the photographed image that is an area corresponding to the sky, and determine whether or not the vehicle is inside a structure based on the first area.

[0136] 100, 200 Traveling environment determination device 105, 205 Image acquisition unit 106, 206 Analysis unit 107, 207 Determination unit 120 Vehicle 121 Photography device 122 Vehicle control device 207a First processing unit 207b Second processing unit

Claims

1. an image acquisition means for acquiring an image captured by an image capturing device installed in the vehicle; an analysis means for performing an analysis process on the captured image to identify a first region in the captured image, the first region being a region corresponding to the sky; a determination means for determining whether the vehicle is within a structure based on the first area. Driving environment judgment device.

2. The analysis means further identifies a reference area in the captured image that satisfies a predetermined criterion, The determination means determines whether the vehicle is within a structure based on the reference area and the first area. The driving environment determination device according to claim 1 .

3. The reference area is an area above a position related to a road in the captured image. The driving environment determination device according to claim 2.

4. The position related to the road is the position of the vanishing point of the road in the captured image. The driving environment determination device according to claim 3.

5. The analysis means performs an analysis process on the captured image to identify a second area in the captured image that corresponds to a predetermined type of subject other than the sky, and identifies the first area and the second area in the captured image above a position related to a road as the reference area. The driving environment determination device according to any one of claims 2 to 4.

6. The second region includes a region corresponding to at least one of an obstacle and a structure. The driving environment determination device according to claim 5.

7. The determination means determines whether the vehicle is within a structure based on a ratio of the first area to the reference area. The driving environment determination device according to any one of claims 2 to 4.

8. The driving environment determination device according to any one of claims 1 to 4, The photographing device is installed in the vehicle and takes photographs to generate the photographed image. vehicle.

9. The computer Acquire a photographed image obtained by photographing with a photographing device installed in the vehicle; performing an analysis process on the captured image to identify a first region in the captured image that corresponds to the sky; Determining whether the vehicle is within a structure based on the first area A method for determining driving environment.

10. On the computer, Acquire a photographed image obtained by photographing with a photographing device installed in the vehicle; performing an analysis process on the captured image to identify a first region in the captured image that corresponds to the sky; A program for causing the computer to execute a process of determining whether the vehicle is within a structure based on the first area.