Traveling lane identifying apparatus, traveling lane identifying method, traveling lane identifying program, lane match determination apparatus, lane match determination method, and lane match determination program

Abstract

To provide a traveling lane identifying apparatus, and the like, capable of improving the accuracy of identifying a traveling lane.SOLUTION: In a traveling lane identifying system which implements a traveling lane identifying apparatus, a server includes: a reference time duration determination unit which determines time duration for referring to camera images acquired from a mobile body based on a state amount of the mobile body, and extracts a plurality of camera images according to the time duration; a lane marking detection unit which generates lane-marking detection results from the extracted camera images; an imaginary lane marking detection unit which generates, for each of the extracted camera images, as imaginary lane-marking detection results, lane-marking detection results which were not generated in the camera images at a designated time and date but generated in the camera images at another time and date; a lane marking reliability calculation unit which calculates reliability for the lane-marking detection results and reliability for the imaginary lane-marking detection results; and a traveling lane identifying unit which identifies a traveling lane on the basis of the reliability for each of the lane-marking detection results and the imaginary lane-marking detection results.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to a driving lane identification device, a driving lane identification method, and a driving lane identification program that identify the lane in which a moving body is traveling based on camera images of the area around the moving body captured by a camera mounted on the moving body, as well as a driving lane matching determination device, a driving lane matching determination method, and a driving lane matching determination program that determine whether or not a moving body is traveling in the same lane based on camera images within a close range of the same road captured by a camera mounted on the moving body. [Background technology]

[0002] In navigation systems that provide route guidance, or driving assistance systems such as autonomous driving systems and Advanced Driver Assistance Systems (ADAS), accurate detection of the positional relationship between a vehicle and the lane in which the vehicle is traveling is required in order to assist the driver of the vehicle or to control the autonomous driving of the vehicle.

[0003] Conventionally, a technique has been disclosed in which a lane marking that cannot be detected due to the presence of a preceding vehicle is estimated as a virtual lane marking based on the position of a lane marking that has been detected in the past (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2018 / 012179 Summary of the Invention [Problem to be solved by the invention]

[0005] In Patent Document 1, when a vehicle is detected in a location where a lane marking is assumed to exist, a virtual lane marking is estimated based on the position of a previously detected lane marking, but the reliability of the estimated virtual lane marking cannot be corrected to a high level. As a result, lane markings are not detected in situations where there are many vehicles in the vicinity, and the driving lane may be erroneously determined.

[0006] The present disclosure has been made to solve such problems, and aims to provide a driving lane identification device, a driving lane identification method, and a driving lane identification program that can improve the accuracy of identifying the driving lane, as well as a driving lane match determination device, a driving lane match determination method, and a driving lane match determination program. [Means for solving the problem]

[0007] In order to solve the above problems, the driving lane identification device disclosed herein includes a reference time width determination unit that determines a time width for referencing camera images obtained from the moving body at a specified date and time for identifying the driving lane of the moving body and at a time before, after, or before and after the specified date and time based on the state quantities of the moving body, and extracts multiple camera images according to the determined time width; a lane line detection unit that generates lane line detection results that are candidates for lane lines of a road including the driving lane of the moving body from each camera image extracted by the reference time width determination unit; a virtual lane line detection unit that generates, as virtual lane line detection results, lane line detection results that were not generated in the camera image at the specified date and time but were generated in the camera image at another time, for each camera image extracted by the reference time width determination unit; a lane line reliability calculation unit that calculates the reliability of the lane line detection results generated by the lane line detection unit and the reliability of the virtual lane line detection results generated by the virtual lane line detection unit; and a driving lane identification unit that identifies the driving lane based on the reliability of each of the lane line detection results and the virtual lane line detection results. Effect of the Invention

[0008] According to the present disclosure, it is possible to improve the accuracy of identifying the driving lane. [Brief description of the drawings]

[0009] [Figure 1] 1 is a block diagram showing an example of the configuration of a driving lane identification system according to a first embodiment. [Diagram 2] FIG. 2 is a block diagram showing an example of a configuration of a server according to the first embodiment. [Diagram 3] 4 is a diagram for explaining the operation of a reference duration determination unit according to the first embodiment. FIG. [Figure 4] 4 is a diagram for explaining the operation of a virtual lane line detection unit according to the first embodiment. FIG. [Diagram 5] 5 is a flowchart showing an example of the operation of the server according to the first embodiment. [Figure 6] FIG. 11 is a block diagram showing an example of a configuration of a server according to the second embodiment. [Figure 7] 13 is a diagram for explaining the operation of a vehicle detection time reliability correction unit according to the second embodiment. FIG. [Figure 8] 11 is a flowchart showing an example of the operation of the server according to the second embodiment. [Figure 9] FIG. 11 is a block diagram showing an example of a configuration of a server according to the third embodiment. [Figure 10] 13 is a diagram for explaining the operation of a road surface area boundary line detection unit according to the third embodiment. FIG. [Figure 11] FIG. 11 is a block diagram showing an example of the configuration of a driving lane agreement determination system according to a fourth embodiment. [Figure 12] FIG. 13 is a block diagram showing an example of a configuration of a server according to the fourth embodiment. [Figure 13] 13 is a flowchart showing an example of the operation of the server according to the fourth embodiment. [Figure 14] FIG. 13 is a block diagram showing an example of a configuration of a server according to the fifth embodiment. [Figure 15] 13 is a diagram for explaining the operation of a match determination impossible unit at the time of vehicle detection according to the fifth embodiment. FIG. [Figure 16] 13 is a flowchart showing an example of the operation of the server according to the fifth embodiment. [Figure 17]FIG. 23 is a block diagram showing an example of the configuration of a moving body according to a sixth embodiment. [Figure 18] FIG. 23 is a block diagram showing an example of the configuration of a moving body according to a seventh embodiment. [Figure 19] FIG. 23 is a block diagram showing an example of the configuration of a moving body according to a seventh embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] <Embodiment 1> <Configuration> 1 is a block diagram showing an example of a configuration of a driving lane identification system 1000 according to the first embodiment. The driving lane identification system 1000 realizes a driving lane identification device according to the first embodiment.

[0011] 1, the driving lane identification system 1000 includes a server 101 and a moving body 102. The moving body 102 is composed of one or more units. In the present disclosure, the moving body 102 will be described as a vehicle.

[0012] FIG. 2 is a block diagram showing an example of the configuration of the server 101. As shown in FIG.

[0013] 2, the server 101 includes a processor 100, a storage device 40, a communication interface 20, and a communicator 30. The processor 100 includes a reference time width determination unit 1, a lane line detection unit 2, a virtual lane line detection unit 3, a lane line reliability calculation unit 4, and a driving lane identification unit 5. The server 101 is a computer server, such as an edge server or a cloud server.

[0014] The processor 100 is connected to other hardware via a signal line and controls the other hardware. The processor 100 is an IC (Integrated Circuit) for executing instructions written in a program and performing processes such as data transfer, calculation, processing, control, and management. The processor 100 has an arithmetic circuit, and a register and cache memory in which instructions and information are stored. The processor 100 is specifically a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or a GPU (Graphics Processing Unit).

[0015] The storage device 40 is a random access memory (RAM), a hard disk drive (HDD), or a solid state drive (SSD). The storage device 40 may be a portable storage medium such as a secure digital (SD, registered trademark) memory card, a compact flash (CF, registered trademark), a NAND flash, a flexible disk, an optical disk, a compact disk, a Blu-ray (registered trademark) disk, or a digital versatile disk (DVD).

[0016] The communicator 30 is a device including a receiver for receiving data from the mobile object 102 and a transmitter for transmitting data to the mobile object 102. Specifically, the communication interface 20 is a communication chip or a network interface card (NIC). The communication interface 20 uses a communication protocol such as Dedicated Short Range Communication (DSRC) dedicated to vehicle communication or IEEE802.11p. The communication interface 20 may also use a cellular network such as Long Term Evolution (LTE, registered trademark) or a fifth generation mobile communication system (5G), or a wireless local area network (LAN) such as Bluetooth (registered trademark) or IEEE802.11a / b / g / n / ac.

[0017] Note that the server 101 and the mobile object 102 may not be communicatively connected. In this case, the sensor information accumulated in a storage device (not shown) provided in the mobile object 102 needs to be manually stored in the storage device 40 of the server 101.

[0018] The storage device 40 stores a mobile sensor information management table 6 in addition to the programs to be executed by the processor 100.

[0019] The mobile sensor information management table 6 acquires and stores GNSS (Global Navigation Satellite System) sensor information and camera sensor information from the mobile object 102 via the communication device 30 and the communication interface 20. The GNSS sensor information includes a mobile object identification number for identifying the mobile object 102, position information of the mobile object 102, accuracy information when the GNSS sensor information was acquired, acquisition date and time of the GNSS sensor information, the speed of the mobile object 102, and the direction of the mobile object 102. The position information includes at least latitude and longitude, and also includes altitude if it is possible to acquire it. The camera sensor information includes a mobile object identification number for identifying the mobile object 102, acquisition date and time of the camera sensor information, and a camera image. It is assumed that the camera image is processed so that the shooting range and image size are the same.

[0020] The reference time span determination unit 1 sets an arbitrary designated date and time at which it is desired to identify the travel lane of the mobile body 102. Then, in order to identify the travel lane of the mobile body 102 by referring to multiple camera images among those captured before and after the designated date and time by a camera mounted on the same mobile body 102, the reference time span determination unit 1 extracts multiple camera images to be referenced from the mobile body sensor information management table 6. Specifically, the reference time span determination unit 1 acquires the speed and camera images at the designated date and time from the mobile body sensor information management table 6, and extracts multiple camera images to be referenced according to the acquired speed.

[0021] For example, assume that one camera image taken after a specified date and time and three camera images taken before the specified date and time are to be referenced. In Fig. 3, a1 indicates the time axis, and each of times t1 to t7 is assumed to be at an interval of td1 seconds. The camera images are acquired at each of times t1 to t7. The specified date and time is assumed to be t2.

[0022] A2 indicates a position coordinate axis, and a camera image at position p1 is acquired at time t1. Similarly, a camera image at position p2 is acquired at time t2, a camera image at position p3 is acquired at time t4, and a camera image at position p4 is acquired at time t6. The positions p1 to p4 are spaced apart by dd1. If the speed of the moving object 102 at time t1 is s1, and the speed of the moving object 102 at times t2, t4, and t6 is less than s1 / 2, the camera images to be referred to are the camera images acquired at times t1, t2, t4, and t6.

[0023] If the speed of the moving object 102 at time t5 is smaller than S1 / 3 and the speed of the moving object 102 at time t7 is larger than S1 / 3 and smaller than S1 / 2, the camera images to be referenced may be those at times t1, t2, t5, and t7.

[0024] The reference time width determination unit 1 may determine the time width for referring to the camera image based on the moving distance of the moving body 102 or the amount of change in the orientation of the moving body 102, instead of the speed of the moving body 102, and extract the camera image at the determined time width. When determining the time width for referring to the camera image based on the moving distance of the moving body 102, the time width may be determined so that the moving distance is a predetermined interval. Also, when determining the time width for referring to the camera image based on the amount of change in the orientation of the moving body 102, the time width may be determined so that the amount of change in the orientation is a predetermined amount of change. In this way, the time width for referring to the camera image can be determined based on the state quantity of the moving body 102 including the speed, moving distance, or amount of change in the orientation of the moving body 102.

[0025] The lane line detection unit 2 extracts edge points for each camera image extracted by the reference time width determination unit 1, and extracts straight lines using a straight line detection method such as the Hough transform.The lane line detection unit 2 then calculates a vanishing point from the extracted straight lines, and extracts only the straight lines that point to the calculated vanishing point.The lane line detection unit 2 then performs clustering based on the slope of the lines for the multiple straight lines that point to the vanishing point, and generates a lane line detection result from the slope and vanishing point that indicate the characteristics of each cluster by performing statistical processing such as taking the average for each cluster.

[0026] In addition, the lane marking detection unit 2 may input each camera image extracted by the reference time width determination unit 1 to a classifier that has been pre-trained to identify the type of object represented by each pixel, extract edge points from the results of extracting pixels identified as lane marks, and extract straight lines using a straight line detection method such as a Hough transform.

[0027] The virtual lane line detection unit 3 assigns the same identification number to identical lane lines in the lane line detection results of each camera image generated by the lane line detection unit 2. Methods for determining whether or not they are the same section lines include a method of determining that they are the same lane line when the difference in the slope of the lane line is within a threshold, or a method of setting an arbitrary horizontal axis in the camera image and determining that they are the same lane line when the difference in the position of the lane line that intersects with the horizontal axis is within a threshold.

[0028] The virtual lane line detection unit 3 generates, as a virtual lane line detection result, a lane line detection result that is not detected in a camera image at the specified date and time, but is detected in a camera image at any other time (other time).

[0029] 4, the camera images extracted by the reference time width determination unit 1 are P100 to P103, and the camera image at the specified date and time is P102. The lane line detection results generated from the camera image P100 are L101 to L104, the lane line detection results generated from the camera image P101 are L112 to L114, the lane line detection results generated from the camera image P102 are L122 and L123, and the lane line detection results generated from the camera image P103 are L132 to L134.

[0030] The virtual lane line detection unit 3 assigns the lane line detection result L101 as the same lane line identification number n101, assigns the lane line detection results L102, L112, L122, and L132 as the same lane line identification number n102, assigns the lane line detection results L103, L113, L123, and L133 as the same lane line identification number n103, and assigns the lane line detection results L104, L114, and L134 as the same lane line identification number n104.

[0031] The virtual lane line detection unit 3 then generates as virtual lane lines those lane lines corresponding to lane line identification numbers n101, n104 that have not been generated in the camera image P102. For the shape of the virtual lane lines corresponding to lane line identification numbers n101, n104, the lane line detection result of a camera image associated with position coordinates closer to the position coordinates associated with the camera image P102 may be used. Alternatively, the lane line detection result of a camera image captured after (in the future) the camera image at the specified date and time may be used preferentially.

[0032] The lane line reliability calculation unit 4 calculates the reliability of the lane line detection result generated by the lane line detection unit 2 and the reliability of the virtual lane line detection result generated by the virtual lane line detection unit 3.

[0033] Specifically, the lane line reliability calculation unit 4 calculates the reliability according to the number of lane line detection results to which the same lane line identification number is assigned. For example, in the example of Fig. 4, if a reliability of "0.25" is added for each lane line detection result so that the sum of the reliability is "1.0" when lane line detection results are generated in all camera images P100 to P103, the reliability of lane line identification number n101 will be "0.25", the reliability of lane line identification number n102 will be "1.0", the reliability of lane line identification number n103 will be "1.0", and the reliability of lane line identification number n104 will be "0.75".

[0034] The lane line reliability calculation unit 4 may perform weighting so as to increase the reliability added to camera images taken after the camera image at the specified date and time. For example, when calculating reliability using four camera images, a reliability of "0.2" may be added to the lane line detection result of the camera image taken before the specified date and time, a reliability of "0.25" may be added to the lane line detection result of the camera image taken at the specified date and time, and a reliability of "0.35" may be added to the lane line detection result of the camera image taken after the specified date and time.

[0035] The driving lane identification unit 5 regards the lane detection result and the virtual lane detection result having a reliability equal to or higher than a threshold as a lane, based on the reliability of each of the lane detection results and the virtual lane detection results calculated by the lane reliability calculation unit 4. The driving lane identification unit 5 then identifies which lane the moving object 102 is traveling in from the left or right side of the road. Specifically, the driving lane identification unit 5 determines how many lane lines are detected in both the left and right directions from the center of the camera image, and based on this, identifies which lane the moving object 102 is traveling in from the left or right side of the road.

[0036] In the example of Figure 4, the driving lane identification unit 5 adds a reliability of "0.25" for each lane detection result, and when it considers lane detection results and virtual lane detection results with a reliability of "0.6" or higher to be lane lines, it considers lane line identification numbers n102, n103, and n104 to be lane lines, and determines that the moving object 102 is traveling in the first lane from the left, that is, between lane line identification numbers n102 and n103.

[0037] In addition, the driving lane identification system 1000 may be configured as part of a map generation system that generates map information from mobile sensor information collected by performing statistical processing on data collected in the same driving lane, or a map change point detection system that detects change points on a map.

[0038] The functions of the reference time width determination unit 1, the lane line detection unit 2, the virtual lane line detection unit 3, the lane line reliability calculation unit 4, and the driving lane identification unit 5 described above are realized by a processing circuit. That is, the driving lane identification device includes a processing circuit for determining a time width for referring to a camera image, generating a lane line detection result, generating a virtual lane line detection result, calculating the reliability of each of the lane line detection result and the virtual lane line detection result, and identifying the driving lane based on the reliability. The processing circuit may be dedicated hardware, or may be a processor (also called a CPU, central processing unit, processing unit, arithmetic unit, microprocessor, microcomputer, or DSP (Digital Signal Processor)) that executes a program stored in a memory.

[0039] When the processing circuit is a dedicated hardware, the processing circuit corresponds to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination of these. Each function of the reference time width determination unit 1, the lane marking detection unit 2, the virtual lane marking detection unit 3, the lane marking reliability calculation unit 4, and the driving lane identification unit 5 may be realized by a processing circuit, or each function may be realized together by a single processing circuit.

[0040] When the processing circuit is the processor 100, the functions of the reference time width determination unit 1, the lane line detection unit 2, the virtual lane line detection unit 3, the lane line reliability calculation unit 4, and the driving lane identification unit 5 are realized by software, firmware, or a combination of software and firmware. The software or firmware is written as a program and stored in the storage device 40. The processor 100 realizes each function by reading and executing the program recorded in the storage device 40. That is, the driving lane identification device includes a storage device 40 for storing a program that results in the execution of a step of determining a time width for referring to a camera image, a step of generating a lane line detection result, a step of generating a virtual lane line detection result, a step of calculating the reliability of each of the lane line detection result and the virtual lane line detection result, and a step of identifying a driving lane based on the reliability. In addition, it can be said that these programs cause a computer to execute the procedures or methods of the reference time width determination unit 1, the lane line detection unit 2, the virtual lane line detection unit 3, the lane line reliability calculation unit 4, and the driving lane identification unit 5.

[0041] In addition, with regard to each function of the reference time width determination unit 1, the lane line detection unit 2, the virtual lane line detection unit 3, the lane line reliability calculation unit 4, and the driving lane identification unit 5, some functions may be realized by dedicated hardware and other functions may be realized by software or firmware.

[0042] Thus, the processing circuitry may implement the functions described above through hardware, software, firmware, or a combination of these.

[0043] The above describes the hardware configuration of the driving lane identification device of embodiment 1, but the hardware configurations of the driving lane identification devices of embodiments 2 and 3, and the driving lane agreement determination device of embodiments 4 to 7, which will be described below, are also similar.

[0044] <Operation> 5 is a flowchart showing an example of the operation of the server 101 (driving lane identification device). The operation of the driving lane identification device according to the first embodiment corresponds to the operation of the driving lane identification method or the driving lane identification system according to the first embodiment. Moreover, the operation of the driving lane identification device according to the first embodiment corresponds to the processing of the driving lane identification program according to the first embodiment.

[0045] In step S101, the reference time duration determination unit 1 determines and extracts, from the mobile sensor information management table 6 in the storage device 40, a plurality of camera images to be referenced.

[0046] In step S102, the lane marking detection unit 2 generates lane marking detection results for the multiple camera images extracted by the reference time width determination unit 1.

[0047] In step S103, the virtual lane line detection unit 3 identifies identical lane lines between the camera images from the lane line detection results generated by the lane line detection unit 2, and generates a virtual lane line detection result.

[0048] In step S104, the lane marking reliability calculation unit 4 calculates the reliability of the lane marking detection result and the reliability of the virtual lane marking detection result.

[0049] In step S105, the driving lane identification unit 5 detects the lane markings based on the reliability of each of the lane marking detection result and the virtual lane marking detection result.

[0050] In step S106, the driving lane identification unit 5 identifies the driving lane of the moving object 102 based on the detected lane markings.

[0051] The result of identifying the driving lane may be stored in the storage device 40 together with the date and time (specified date and time) and the position coordinates.

[0052] In addition, when identifying the driving lane for multiple camera images, the lane marking detection results generated in step S102 may be saved, and when identifying the driving lane for other camera images by performing the processing of step S102 for the same camera images, the saved lane marking detection results may be used.

[0053] <Effects> As described above, the driving lane identification device according to the first embodiment determines the time width for referencing camera images taken around a specified date and time based on the amount of change in the speed, distance traveled, or orientation of a moving object, so that when identifying the lane line detection results generated from each camera image as the same lane line, it is possible to suppress the change in position and angle of the lane line. Therefore, the lane lines can be correctly identified and associated, improving the accuracy of identifying the driving lane.

[0054] <Embodiment 2> <Configuration> 6 is a block diagram showing an example of the configuration of the server 101 (driving lane identification device) according to embodiment 2. In the driving lane identification system including the server 101 according to embodiment 2, the same components as those in the driving lane identification system 1000 according to embodiment 1 (see FIGS. 1 and 2) are given the same reference numerals, and duplicated detailed descriptions are omitted.

[0055] In the server 101 according to the second embodiment, when a vehicle is detected at the position of the virtual lane line detection result, the lane line cannot be detected because it temporarily overlaps with the vehicle, but the lane line is likely to exist, and a correction is made to increase the reliability of the virtual lane line detection result, which is different from the server 101 according to the first embodiment. Specifically, the server 101 according to the second embodiment is characterized by including a vehicle detection reliability correction unit 7.

[0056] The virtual lane line detection unit 3 identifies identical lane lines based on the lane line detection results of each camera image generated by the lane line detection unit 2, and generates a lane line detection result as a virtual lane line detection result that is not detected in the camera image at the specified date and time but is detected in the camera image at any other time (other time).

[0057] The vehicle detection reliability correction unit 7 detects a vehicle region where vehicles exist around the moving object from the camera image extracted by the reference time width determination unit 1. For example, the vehicle detection reliability correction unit 7 inputs the camera image extracted by the reference time width determination unit 1 to a classifier that has been trained in advance to identify the type of object represented by each pixel, and extracts an image that has been identified as a vehicle. Alternatively, the vehicle detection reliability correction unit 7 inputs the camera image extracted by the reference time width determination unit 1 to a classifier that has been trained in advance to identify a shape corresponding to the vehicle region, and calculates the shape corresponding to the vehicle region.

[0058] Then, when the virtual lane line detection result generated by the virtual lane line detection unit 3 overlaps with the vehicle area detected by the vehicle detection reliability correction unit 7, the vehicle detection reliability correction unit 7 corrects the reliability calculated by the lane line reliability calculation unit 4.

[0059] For example, in FIG. 7, in camera image P102, a virtual lane line detection result for lane line identification number n104 is generated based on the shape of lane line detection result L134, and vehicle areas C200 and C201 are detected. When adding a reliability of "0.25" for one lane line detection result, lane line reliability calculation unit 4 calculates the reliability of lane line identification number n104 as "0.5" from lane line detection results L104 and L134 of camera images P100 and P103. When vehicle areas C200 to C204 are detected in camera images P101 and P102 at positions overlapping with the virtual lane line detection result generated by virtual lane line detection unit 3 based on lane line detection results L104 and L134, vehicle detection reliability correction unit 7 adds a reliability smaller than the reliability for the lane line detection result, for example, a reliability of "0.2", and sets the reliability of lane line identification number n104 to "0.9".

[0060] The determination of whether the virtual lane marking detection result and the vehicle area overlap may be made by determining that there is an overlap when a range in the virtual lane marking detection result that is equal to or greater than a threshold is covered by the vehicle area. Alternatively, an arbitrary horizontal axis may be set, and an intersection point between the horizontal axis and the virtual lane marking detection result and a section where the horizontal axis and the vehicle area intersect may be calculated, and an overlap may be determined when the intersection point is included in the calculated section.

[0061] The driving lane identification unit 5 regards the lane detection results and virtual lane detection results having a reliability equal to or higher than a threshold as lane marks, based on the reliability corrected by the vehicle detection reliability correction unit 7. The driving lane identification unit 5 then identifies which lane from the left or right side of the road the moving object 102 is traveling in.

[0062] <Operation> Fig. 8 is a flowchart showing an example of the operation of the server 101 (traffic lane identification device). Note that the processes other than step S201 in Fig. 8 are the same as those in Fig. 5, and therefore will not be described here. The process of step S201 will be described below.

[0063] In step S201, the vehicle detection reliability correction unit 7 detects a vehicle area where vehicles exist around the moving object in the camera image extracted by the reference time width determination unit 1. Then, when the virtual lane line detection result generated by the virtual lane line detection unit 3 overlaps with the vehicle area detected by the vehicle detection reliability correction unit 7, the vehicle detection reliability correction unit 7 corrects the reliability calculated by the lane line reliability calculation unit 4.

[0064] <Effects> As described above, the driving lane identification device according to the second embodiment makes it easier to detect the lane markings by correcting the reliability of the lane markings that are temporarily detected or not detected due to the influence of vehicles around the moving object. This makes it possible to improve the accuracy of identifying the driving lane.

[0065] <Embodiment 3> <Configuration> 9 is a block diagram showing an example of the configuration of the server 101 (driving lane identification device) according to embodiment 3. In the driving lane identification system including the server 101 according to embodiment 3, the same components as those in the driving lane identification system 1000 according to embodiment 1 (see FIGS. 1 and 2) are given the same reference numerals, and duplicated detailed descriptions are omitted.

[0066] The server 101 according to the third embodiment differs from the server 101 according to the first embodiment in that a straight line directed toward a vanishing point is detected from the road surface shape, and the straight line is added as a deemed lane line to the lane line detection result generated by the lane line detection unit 2. Specifically, the server 101 according to the third embodiment is characterized by including a road surface area boundary detection unit 8.

[0067] The road surface area boundary detection unit 8 extracts road surface boundaries or road surface shapes created by approximating road surface boundaries from the camera images extracted by the reference time width determination unit 1. For example, the road surface area boundary detection unit 8 may input the camera images extracted by the reference time width determination unit 1 to a classifier that has been trained in advance to identify the type of object represented by each pixel, and generate road surface boundaries or road surface shapes from the results of extracting pixels identified as road surface areas. Specifically, semantic segmentation is performed by machine learning to color the pixels of the road surface and the lane markings. This makes it possible to extract the boundaries of areas that are colored as road surfaces even if they are not divided by lane marks. The road surface area boundary detection unit 8 detects straight lines that are directed toward the vanishing point calculated by the lane marking detection unit 2 from the straight lines that make up the extracted road surface shape. Then, the road surface area boundary detection unit 8 extracts straight lines that intersect with the minimum and maximum values ​​of any horizontal axis from the detected straight lines as deemed lane marks.

[0068] 10, P300 is a camera image extracted by the reference time width determination unit 1, A301-A303 are road surface shapes, and L311-L313 are lane line detection results generated by the lane line detection unit 2. The road surface area boundary detection unit 8 extracts straight lines L301-L306 that extend from the road surface shapes A301-A303 toward the vanishing points of L311-L313. Then, the road surface area boundary detection unit 8 extracts the straight lines L301, L306 that are the minimum (closest position from the left end of the camera image P300) and maximum (farthest position from the left end of the camera image P300) of the positions where an arbitrary horizontal axis X300 intersects with the extracted straight lines L301-L306 as deemed lane lines.

[0069] The lane line detection unit 2 combines the extracted lane line detection result with the deemed lane line extracted by the road surface area boundary detection unit 8, identifies the same lane line detection result from their inclinations, and adds the deemed lane line as a lane line detection result if it is not included in the lane line detection result. In the example of Fig. 10, the lane line detection unit 2 compares the inclinations of the deemed lane lines L301, L306 with the lane line detection results L311-L313, identifies that the deemed lane line L306 and the lane line detection result L313 are the same lane line detection result, and adds the deemed lane line L301 as a lane line detection result.

[0070] <Operation> The operation of the server 101 (driving lane identification device) according to the embodiment 3 is similar to the operation (see FIG. 5) of the server 101 (driving lane identification device) according to the embodiment 1. Here, the differences from the embodiment 1 will be described.

[0071] 5, the road surface area boundary detection unit 8 extracts deemed lane lines from the road surface shape. The lane line detection unit 2 distinguishes between the deemed lane lines extracted by the road surface area boundary detection unit 8 and the lane line detection results extracted by the lane line detection unit 2, and if there is a deemed lane line that is not included in the lane line detection results, adds the deemed lane line as a lane line detection result.

[0072] <Effects> As described above, according to the driving lane identification device of embodiment 3, since the dividing line is extracted from the road surface shape, it is possible to improve the accuracy of identifying the driving lane even when the dividing line is difficult to detect due to fading or the like.

[0073] <Fourth embodiment> <Configuration> 11 is a block diagram showing an example of a configuration of a driving lane agreement determination system 1001 according to the embodiment 4. The driving lane agreement determination system 1001 realizes a driving lane agreement determination device according to the embodiment 4.

[0074] As shown in Fig. 11, the driving lane matching determination system 1001 includes a server 101 and a moving object 102. The moving object 102 is made up of one or more vehicles.

[0075] 12 is a block diagram showing an example of the configuration of the server 101 (driving lane agreement determination device). Note that in the driving lane agreement determination system including the server 101 according to the fourth embodiment, the same components as those in the driving lane identification system 1000 according to the first embodiment (see FIGS. 1 and 2) are given the same reference numerals, and overlapping detailed descriptions are omitted.

[0076] The server 101 according to the fourth embodiment differs from the server 101 according to the first embodiment in that it determines whether a specified camera image is a camera image taken while traveling in the same traveling lane. Specifically, the server 101 according to the fourth embodiment is characterized by including a match determination target image extraction unit 9 and a traveling lane match determination unit 10.

[0077] The match determination target image extraction unit 9 extracts from the mobile sensor information management table 6 a plurality of camera images to be subjected to determination as to whether or not they are camera images taken while traveling along the same travel lane.

[0078] In addition, the matching determination target image extraction unit 9 may extract camera images from the mobile sensor information management table 6 that were taken while traveling on the same road and are associated with position coordinates within a threshold range, based on the position information or orientation of the mobile body 102.

[0079] In addition, the accuracy information of the extracted position coordinates may be acquired, and if the extracted position coordinates include highly accurate position coordinates, the camera image may not be extracted, and in other cases, the camera image may be extracted as a processing target of the driving lane agreement determination system. Specifically, when the position coordinates acquired by the DGNSS positioning and the RTK positioning are extracted, the camera image may be excluded from the processing target as a driving lane agreement determination based on distance comparison.

[0080] The driving lane matching determination system 1001 may be configured as part of a map generation system that generates map information from collected mobile sensor information by performing statistical processing on data collected in the same driving lane, or a map change point detection system that detects change points on a map.

[0081] The reference time range determination unit 1 receives as input a plurality of camera images extracted by the match determination target image extraction unit 9, and extracts from the mobile sensor information management table 6 camera images taken by a camera mounted on the same mobile body 102 before and after the reference time (the time when the camera images extracted by the match determination target image extraction unit 9 were taken).

[0082] The driving lane matching determination unit 10 considers the lane line detection results and virtual lane line detection results having a reliability above a threshold value as lane lines, based on the reliability of each of the lane line detection results and virtual lane line detection results calculated by the lane line reliability calculation unit 4 for each camera image extracted by the matching determination target image extraction unit 9.

[0083] Then, the driving lane coincidence determination unit 10 identifies whether the detected lane markings are the same between the camera images extracted by the coincidence determination target image extraction unit 9. As a method for determining whether the lane markings are the same, a method for determining that the lane markings are the same when the difference in the inclination of the lane markings is within a threshold value, or a method for determining that the lane markings are the same when the difference in the position of the lane markings intersecting with the horizontal axis is within a threshold value, is included. The driving lane coincidence determination unit 10 generates a driving lane coincidence determination result that indicates "agreement" when the lane markings detected between the camera images all coincide as the same lane markings, and "mismatch" otherwise. The driving lane coincidence determination result is incorporated, for example, into a map generation system. In this case, when data of the same lane is collected and statistically processed, only data determined to be "agreement" in the driving lane coincidence determination device according to the fourth embodiment is extracted.

[0084] <Operation> The operation of the server 101 (driving lane coincidence determination device) according to the fourth embodiment is characterized by steps S101, S301, and S302, compared with the operation of the server 101 (driving lane identification device) according to the first embodiment (see FIG. 5), with reference to the flowchart in FIG. 13. Steps S101, S301, and S302 will be described below. Other processes are similar to those in the operation of FIG. 5.

[0085] In step S101, the match determination target image extraction unit 9 extracts multiple camera images to be used for determining whether they are camera images taken while traveling in the same traveling lane from the mobile body sensor information management table 6. The reference time width determination unit 1 receives the multiple camera images extracted by the match determination target image extraction unit 9 and extracts camera images taken by a camera mounted on the same mobile body 102 around the reference time from the mobile body sensor information management table 6.

[0086] In step S301, the driving lane match determination unit 10 identifies identical lane markings between the camera images extracted by the match determination target image extraction unit 9.

[0087] In step S302, the driving lane match determination unit 10 generates a driving lane match determination result for the camera images extracted by the match determination target image extraction unit 9 from the results of identifying identical lane lines between the camera images.

[0088] <Effects> As described above, the driving lane coincidence determination device according to the fourth embodiment determines the camera image to be referenced when generating the virtual lane line detection result based on the amount of change in the speed, distance traveled, or orientation of the moving object, so that it is possible to suppress changes in the position and angle of the lane line when identifying the lane line detection results generated from each camera image as the same lane. Therefore, since the lane lines can be correctly identified and associated, it is possible to improve the accuracy of determining whether the camera images were taken while traveling in the same driving lane.

[0089] <Fifth Preferred Embodiment> <Configuration> 14 is a block diagram showing an example of the configuration of the server 101 (driving lane agreement determination device) according to embodiment 5. In the driving lane agreement determination system including the server 101 according to embodiment 5, the same components as those in the driving lane agreement determination system 1001 according to embodiment 4 (see FIGS. 11 and 12) are given the same reference numerals, and duplicated detailed descriptions will be omitted.

[0090] The server 101 according to the fifth embodiment differs from the server 101 according to the fourth embodiment in that, when determining whether the camera images were taken while driving in the same driving lane, the detected lane markings are compared between the camera images extracted by the match determination target image extraction unit 9, and if a lane marking detected in one camera image is not detected in another camera image and a vehicle is detected at the position of the lane marking in the other camera image, a match determination is impossible and a driving lane match determination result is generated. Specifically, the server 101 according to the fifth embodiment is characterized by including a match determination impossible unit 11 when a vehicle is detected.

[0091] Using the result of the driving lane agreement determination unit 10's discrimination of whether the lane lines are the same between the camera images, the vehicle detection time match determination inability unit 11 generates a driving lane agreement determination result indicating that agreement is infeasible when the lane lines detected between the camera images do not match as the same lane line, and when a lane line detected in one camera image is not detected in another camera image and a vehicle is detected at the position of the lane line in the other camera image. At this time, the vehicle detection time match determination infeasible unit 11 detects a vehicle area in which vehicles exist around the moving object in the camera images extracted by the reference time width determination unit 1.

[0092] 15, P400 and P401 are camera images extracted by match determination target image extraction unit 9, L400 to L404 are lane markings detected by driving lane match determination unit 10, and C400 and C401 are vehicle areas detected by vehicle detection time match determination impossible unit 11. When lane marking L402 is not detected in camera image P401 but would overlap with vehicle areas C400 and C401 if drawn on camera image P401, vehicle detection time match determination impossible unit 11 generates a driving lane match determination result indicating that match determination is impossible.

[0093] <Operation> The operation of the server 101 (driving lane consistency determination device) according to the fifth embodiment is the same as the operation of the server 101 (driving lane consistency determination device) according to the fourth embodiment (see FIG. 13) with reference to the flowchart in FIG. 16. Step S401 will be described below.

[0094] In step S401, the vehicle detection match incompatibility determination unit 11 uses the results of the lane matching determination unit 10's identification of whether or not the lane lines are the same between the camera images to check whether or not a vehicle has been detected at the position of the lane line in the other camera image when the lane lines detected between the camera images do not match as the same lane line and a lane line detected in one camera image is not detected in the other camera image.

[0095] <Effects> From the above, according to the driving lane coincidence determination device of the fifth embodiment, when the lane markings detected in the respective camera images do not match as the same lane marking, and when a lane marking detected in one camera image is not detected in another camera image, and a vehicle is detected at the position of the lane marking in the other camera image, it is highly likely that the vehicle cannot see the lane marking, so a driving lane coincidence determination result is generated indicating that it is not possible to determine whether the lane markings match or not (match determination impossible). This reduces the possibility of erroneous determination of the driving lane.

[0096] <Sixth embodiment> Fig. 17 is a block diagram showing an example of the configuration of the moving object 102 according to the embodiment 6. In the moving object 102 according to the embodiment 6, the same components as those in the server 101 according to the embodiment 1 (see Fig. 2) are denoted by the same reference numerals, and the detailed description thereof will be omitted.

[0097] The moving body 102 according to the sixth embodiment differs from the first to third embodiments in that the moving body 102 alone realizes a driving lane identification system. Specifically, the moving body 102 according to the sixth embodiment includes a storage device 40, a mobile body sensor interface 50, a mobile body sensor 60, and a processor 100. The storage device 40 and the processor 100 are the same as the storage device 40 and the processor 100 shown in FIG. 2. Furthermore, the operation of the processor 100 is the same as that of the processor 100 shown in FIG. 2.

[0098] The mobile sensor information management table 6 receives GNSS sensor information and camera sensor information from the mobile sensor 60 via the mobile sensor interface 50 and stores them.

[0099] In the above, the processor 100 shown in Fig. 17 has the same configuration as the processor 100 shown in Fig. 2, but this is not limited to the above. The processor 100 shown in Fig. 17 may have the same configuration as the processor 100 shown in Fig. 6 or 9.

[0100] <Effects> From the above, the mobile body 102 (driving lane identification device) according to the sixth embodiment makes it possible for the mobile body 102 to identify the driving lane by using sensor information collected by itself. In addition, since the time width for referencing camera images taken around a specified date and time is determined based on the amount of change in the speed, travel distance, or orientation of the mobile body, it is possible to suppress the change in position and angle of the lane line when identifying the lane line detection results generated from each camera image as the same lane line. Therefore, the lane lines can be correctly identified and associated, improving the accuracy of identifying the driving lane.

[0101] <Embodiment 7> Fig. 18 is a block diagram showing an example of the configuration of the moving object 102 according to the embodiment 7. In the moving object 102 according to the embodiment 7, the same components as those in the server 101 according to the embodiment 4 (see Fig. 12) are denoted by the same reference numerals, and the detailed description thereof will be omitted.

[0102] The moving body 102 according to the seventh embodiment differs from the fourth and fifth embodiments in that the moving body 102 alone realizes a driving lane agreement determination system. Specifically, the moving body 102 according to the seventh embodiment includes a storage device 40, a mobile body sensor interface 50, a mobile body sensor 60, and a processor 100. The storage device 40 and the processor 100 are the same as the storage device 40 and the processor 100 shown in FIG. 12. Also, the operation of the processor 100 is the same as the processor 100 shown in FIG. 12.

[0103] The mobile sensor information management table 6 receives GNSS sensor information and camera sensor information from the mobile sensor 60 via the mobile sensor interface 50 and stores them.

[0104] The match determination target image extraction unit 9 extracts from the mobile sensor information management table 6 a plurality of camera images to be subjected to determination as to whether or not they are camera images taken while traveling along the same travel lane.

[0105] 19, the road map 12 in which a camera image of the front of the moving body 102 acquired at the time of map data measurement at any position coordinate is stored may be saved in the storage device 40. In this case, the match determination target image extraction unit 9 may extract a camera image stored in the road map 12 and a camera image acquired at a position coordinate within a threshold range from the position coordinate at which the camera image was acquired (a camera image captured by a camera mounted on the moving body 102) as targets for determining whether they are camera images captured while traveling in the same driving lane.

[0106] In the above, the processor 100 shown in Figures 18 and 19 has the same configuration as the processor 100 shown in Figure 12, but this is not limited to the above. The processor 100 shown in Figures 18 and 19 may have the same configuration as the processor 100 shown in Figure 14.

[0107] <Effects> From the above, the moving body 102 (driving lane coincidence determination device) according to the seventh embodiment can use sensor information collected by the moving body 102 itself to generate a driving lane coincidence determination result that determines whether or not the lane markings detected between the camera images extracted by the coincidence determination target image extraction unit 9 coincide as the same lane marking. By making the determination using a camera image captured when the moving body 102 traveled in the past or a camera image stored in a map, the result can be utilized for estimating the position of the moving body, detecting change points on a road map, or for statistical processing when generating a road map.

[0108] In addition, because the camera images to be referenced when generating virtual lane line detection results are determined based on the amount of change in the speed, distance traveled, or orientation of the moving object, it is possible to suppress changes in the position and angle of the lane line when identifying the lane line detection results generated from each camera image as the same lane. Therefore, since the lane lines can be correctly identified and associated, it is possible to improve the accuracy of determining whether the camera images were taken while traveling in the same driving lane.

[0109] Within the scope of the present disclosure, the embodiments can be freely combined, and each embodiment can be modified or omitted as appropriate.

[0110] <Additional Notes> Various aspects of the present disclosure are summarized below as appendices.

[0111] (Appendix 1) a reference time span determination unit that determines a time span for referencing camera images acquired from the moving object at a designated date and time for identifying a lane in which the moving object is traveling and at a time before, after, or both of the designated date and time based on a state quantity of the moving object, and extracts a plurality of camera images according to the determined time span; a lane marking detection unit that generates a lane marking detection result that is a candidate for a lane marking of a road including a travel lane of the moving object from each of the camera images extracted by the reference time width determination unit; a virtual lane line detection unit that generates, for each of the camera images extracted by the reference time width determination unit, as a virtual lane line detection result, the lane line detection result that was not generated in the camera image at the specified date and time but was generated in the camera image at another time; a lane line reliability calculation unit that calculates a reliability of the lane line detection result generated by the lane line detection unit and a reliability of the virtual lane line detection result generated by the virtual lane line detection unit; a lane identification unit that identifies the lane based on a reliability of each of the lane marking detection result and the virtual lane marking detection result; A driving lane identification device comprising: (Appendix 2) 2. The driving lane identification device according to claim 1, wherein the state quantity of the moving body is a speed of the moving body, a moving distance of the moving body, or an amount of change in orientation of the moving body. (Appendix 3) The lane detection unit inputs each of the camera images extracted by the reference time width determination unit to a classifier that has been trained in advance to identify the type of object represented by each pixel, and generates the lane detection result from the result of extracting pixels identified as lane lines. (Appendix 4) the virtual lane line detection unit generates, for each of the camera images extracted by the reference time width determination unit, the lane line detection result that is not generated in some camera images and is generated in other camera images, as the virtual lane line detection result; The driving lane identification device according to any one of appendix 1 to 3, further comprising a vehicle detection reliability correction unit that detects a vehicle area in which vehicles exist around the moving body for each of the camera images extracted by the reference time width determination unit, and corrects reliability for each of the lane detection result and the virtual lane line detection result when the vehicle area overlaps with the virtual lane line detection result. (Appendix 5) A road surface area boundary detection unit extracts a road surface shape from each of the camera images extracted by the reference time width determination unit, and extracts, as deemed lane lines, straight lines that are directed toward a vanishing point calculated by the lane line detection unit when generating the lane line detection result, from among the straight lines that constitute the road surface shape, A driving lane identification device described in any one of Appendices 1 to 4, wherein the lane detection unit identifies identical lane extraction results based on the lane detection result and the deemed lane, and if the deemed lane is not included in the lane detection result, adds the deemed lane as the lane detection result. (Appendix 6) The driving lane identification device according to any one of appendix 1 to 5, wherein the driving lane identification device is mounted on a moving body. (Appendix 7) a matching determination target image extraction unit that extracts a plurality of camera images to be subjected to determination as to whether or not the camera images are camera images captured by a camera mounted on the moving object while the moving object is traveling in the same traveling lane; a reference time span determination unit that determines, for each of the camera images extracted by the match determination target image extraction unit, a time span for referencing camera images captured by the camera mounted on the same moving object at times before and after the extracted camera image based on a state quantity of the moving object, and extracts a plurality of camera images according to the determined time span; a lane marking detection unit that generates a lane marking detection result that is a candidate for a lane marking of a road including a travel lane of the moving object from each of the camera images extracted by the reference time width determination unit; a virtual lane line detection unit that generates, for each of the camera images extracted by the reference time width determination unit, as a virtual lane line detection result, the lane line detection result that was not generated in the camera image extracted by the match determination target image extraction unit but was generated in a camera image at another time; a lane line reliability calculation unit that calculates a reliability of the lane line detection result generated by the lane line detection unit and a reliability of the virtual lane line detection result generated by the virtual lane line detection unit; a driving lane coincidence determination unit that detects lane lines in each of the camera images extracted by the reference time width determination unit based on reliability of each of the lane line detection result and the virtual lane line detection result, and generates a result of determining whether the lane lines are identical between each of the camera images extracted by the coincidence determination target image extraction unit as a driving lane coincidence determination result; A driving lane matching determination device comprising: (Appendix 8) The driving lane consistency determination device according to claim 7, wherein the matching determination target image extraction unit extracts the multiple camera images which are camera images of the same road and which are associated with positions within a predetermined range. (Appendix 9) The driving lane matching determination device described in Appendix 7 or 8, further comprising a vehicle detection time matching determination inability unit that generates the driving lane matching determination result indicating that a match cannot be determined when the matching determination target image extraction unit determines that the lane lines in each of the camera images are not identical and when the lane line detected in one camera image is not detected in another camera image, and a vehicle around the moving body is present at a position corresponding to the lane line in the other camera image. (Appendix 10) The driving lane consistency determination device according to any one of appendixes 7 to 9, wherein the driving lane consistency determination device is mounted on a moving body. (Appendix 11) determining a time range for referring to camera images acquired from the moving body at a designated date and time for identifying a travel lane of the moving body and at a time before, after, or both of the designated date and time based on a state quantity of the moving body, and extracting a plurality of camera images according to the determined time range; generating a lane marking detection result, which is a candidate for a lane marking of a road including a lane of the moving object, from each of the extracted camera images; For each of the extracted camera images, the lane line detection result that was not generated in the camera image at the specified date and time but was generated in the camera image at another time is generated as a virtual lane line detection result; Calculating a reliability of the lane marking detection result and a reliability of the virtual lane marking detection result; The method for identifying a driving lane includes identifying the driving lane based on a reliability of each of the lane marking detection result and the virtual lane marking detection result. (Appendix 12) extracting a plurality of camera images to be subjected to a determination as to whether or not the camera images were taken by a camera mounted on the moving body while the moving body was traveling in the same travel lane; For each of the camera images extracted as the object to be determined, a time width for referring to camera images taken by the camera mounted on the same moving body at a time before and after the extracted camera image is determined based on a state quantity of the moving body, and a plurality of camera images are extracted according to the determined time width; generating a lane marking detection result, which is a candidate for a lane marking of a road including a travel lane of the moving object, from each of the camera images extracted according to the time width; For each of the camera images extracted according to the time width, the lane line detection result that is not generated in the camera image extracted as the object to be determined but is generated in the camera image at another time is generated as a virtual lane line detection result; Calculating a reliability of the lane marking detection result and a reliability of the virtual lane marking detection result; A method for determining whether or not a lane is matched, the method detecting lane marks in each of the camera images extracted according to the time width based on the reliability of each of the lane marking detection results and the virtual lane marking detection results, and generating a result of determining whether or not the lane marks are identical between each of the camera images extracted as the objects to be determined as a lane marking match determination result. (Appendix 13) A driving lane identification program for causing a computer to function as a driving lane identification device, determining a time range for referring to camera images acquired from the moving object at a designated date and time for identifying a lane in which the moving object is traveling and at a time before, after, or both before and after the designated date and time based on a state quantity of the moving object, and extracting a plurality of camera images according to the determined time range; generating, from each of the extracted camera images, a lane marking detection result that is a candidate for a lane marking of a road including a lane of the moving object; generating, for each of the extracted camera images, the lane line detection result that was not generated in the camera image at the specified date and time but was generated in the camera image at another time, as a virtual lane line detection result; A step of calculating a reliability of the lane marking detection result and a reliability of the virtual lane marking detection result; identifying the driving lane based on a reliability of each of the lane marking detection result and the virtual lane marking detection result; A driving lane identification program that causes the computer to execute the above steps. (Appendix 14) A driving lane coincidence determination program for causing a computer to function as a driving lane coincidence determination device, A step of extracting a plurality of camera images to be subjected to a determination as to whether or not the camera images are captured by a camera mounted on the moving object while the moving object is traveling in the same travel lane; a step of determining, for each of the camera images extracted as the object to be determined, a time span for referring to camera images taken by the camera mounted on the same moving body at a time before and after the extracted camera image based on a state quantity of the moving body, and extracting a plurality of camera images according to the determined time span; generating a lane marking detection result, which is a candidate for a lane marking of a road including a travel lane of the moving object, from each of the camera images extracted according to the time width; generating, as a virtual lane line detection result, the lane line detection result that is not generated in the camera image extracted as the object to be determined and that is generated in the camera image at another time, for each of the camera images extracted according to the time width; A step of calculating a reliability of the lane marking detection result and a reliability of the virtual lane marking detection result; a step of detecting lane markings in each of the camera images extracted according to the time width based on a reliability of each of the lane marking detection result and the virtual lane marking detection result, and generating a result of determining whether or not the lane markings are identical between each of the camera images extracted as the subject of the determination as a driving lane coincidence determination result; The driving lane matching determination program causes the computer to execute the above steps. [Explanation of symbols]

[0112] 1 reference time width determination unit, 2 lane line detection unit, 3 virtual lane line detection unit, 4 lane line reliability calculation unit, 5 driving lane identification unit, 6 mobile sensor information management table, 7 reliability correction unit when vehicle is detected, 8 road surface area boundary detection unit, 9 image extraction unit for matching judgment, 10 driving lane match judgment unit, 11 unit for determining whether or not a match can be judged when a vehicle is detected, 12 road map, 20 communication interface, 30 communication device, 40 storage device, 50 mobile sensor interface, 60 mobile sensor, 100 processor, 101 server, 102 mobile unit, 1000 driving lane identification system, 1001 driving lane match judgment system.

Claims

1. a reference time span determination unit that determines a time span for referencing camera images acquired from the moving object at a designated date and time for identifying a lane in which the moving object is traveling and at a time before, after, or both of the designated date and time based on a state quantity of the moving object, and extracts a plurality of camera images according to the determined time span; a lane marking detection unit that generates a lane marking detection result that is a candidate for a lane marking of a road including a travel lane of the moving object from each of the camera images extracted by the reference time width determination unit; a virtual lane line detection unit that generates, for each of the camera images extracted by the reference time width determination unit, as a virtual lane line detection result, the lane line detection result that was not generated in the camera image at the specified date and time but was generated in the camera image at another time; a lane line reliability calculation unit that calculates a reliability of the lane line detection result generated by the lane line detection unit and a reliability of the virtual lane line detection result generated by the virtual lane line detection unit; a lane identification unit that identifies the lane based on a reliability of each of the lane marking detection result and the virtual lane marking detection result; A driving lane identification device comprising:

2. The driving lane identification device according to claim 1 , wherein the state quantity of the moving body is a speed of the moving body, a moving distance of the moving body, or an amount of change in orientation of the moving body.

3. 3. The driving lane identification device according to claim 1, wherein the lane detection unit inputs each of the camera images extracted by the reference time width determination unit to a classifier that has been trained in advance to identify the type of object represented by each pixel, and generates the lane detection result from the result of extracting pixels identified as lane lines.

4. the virtual lane line detection unit generates, for each of the camera images extracted by the reference time width determination unit, the lane line detection result that is not generated in some camera images and is generated in other camera images, as the virtual lane line detection result; 3. The driving lane identification device according to claim 1, further comprising a vehicle detection reliability correction unit that detects a vehicle area in which vehicles exist around the moving body for each of the camera images extracted by the reference time width determination unit, and corrects reliability for each of the lane detection result and the virtual lane detection result when the vehicle area overlaps with the virtual lane detection result.

5. A road surface area boundary detection unit extracts a road surface shape from each of the camera images extracted by the reference time width determination unit, and extracts, as deemed lane lines, straight lines that are directed toward a vanishing point calculated by the lane line detection unit when generating the lane line detection result, from among the straight lines that constitute the road surface shape, The lane identification device according to claim 1 or 2, wherein the lane detection unit identifies identical lane extraction results based on the lane detection result and the deemed lane, and if the deemed lane is not included in the lane detection result, adds the deemed lane as the lane detection result.

6. The driving lane identification device according to claim 1 , wherein the driving lane identification device is mounted on a moving body.

7. a matching determination target image extraction unit that extracts a plurality of camera images to be subjected to determination as to whether or not the camera images are camera images captured by a camera mounted on the moving object while the moving object is traveling in the same traveling lane; a reference time span determination unit that determines, for each of the camera images extracted by the match determination target image extraction unit, a time span for referencing camera images captured by the camera mounted on the same moving object at times before and after the extracted camera image based on a state quantity of the moving object, and extracts a plurality of camera images according to the determined time span; a lane marking detection unit that generates a lane marking detection result that is a candidate for a lane marking of a road including a travel lane of the moving object from each of the camera images extracted by the reference time width determination unit; a virtual lane line detection unit that generates, for each of the camera images extracted by the reference time width determination unit, as a virtual lane line detection result, the lane line detection result that was not generated in the camera image extracted by the match determination target image extraction unit but was generated in a camera image at another time; a lane line reliability calculation unit that calculates a reliability of the lane line detection result generated by the lane line detection unit and a reliability of the virtual lane line detection result generated by the virtual lane line detection unit; a driving lane coincidence determination unit that detects lane lines in each of the camera images extracted by the reference time width determination unit based on reliability of each of the lane line detection result and the virtual lane line detection result, and generates a result of determining whether the lane lines are identical between each of the camera images extracted by the coincidence determination target image extraction unit as a driving lane coincidence determination result; A driving lane matching determination device comprising:

8. 8. The driving lane consistency determination device according to claim 7, wherein the match determination target image extraction unit extracts the plurality of camera images which are camera images of the same road and which are associated with positions within a predetermined range.

9. The driving lane matching determination device of claim 7 or 8, further comprising a vehicle detection time matching inability determination unit that generates the driving lane matching determination result indicating that a match cannot be determined when the matching determination target image extraction unit determines that the lane matching lines in each of the camera images are not identical and when the lane matching line detected in one camera image is not detected in another camera image, and a vehicle is present around the moving body at a position corresponding to the lane matching line in the other camera image.

10. The driving lane consistency determination device according to claim 7 , wherein the driving lane consistency determination device is mounted on a moving body.

11. determining a time range for referring to camera images acquired from the moving body at a designated date and time for identifying a travel lane of the moving body and at a time before, after, or both of the designated date and time based on a state quantity of the moving body, and extracting a plurality of camera images according to the determined time range; generating a lane marking detection result, which is a candidate for a lane marking of a road including a lane of the moving object, from each of the extracted camera images; For each of the extracted camera images, the lane line detection result that was not generated in the camera image at the specified date and time but was generated in the camera image at another time is generated as a virtual lane line detection result; Calculating a reliability of the lane marking detection result and a reliability of the virtual lane marking detection result; The method for identifying a driving lane includes identifying the driving lane based on a reliability of each of the lane marking detection result and the virtual lane marking detection result.

12. extracting a plurality of camera images to be subjected to a determination as to whether or not the camera images were taken by a camera mounted on the moving body while the moving body was traveling in the same travel lane; For each of the camera images extracted as the object to be determined, a time width for referring to camera images taken by the camera mounted on the same moving body at a time before and after the extracted camera image is determined based on a state quantity of the moving body, and a plurality of camera images are extracted according to the determined time width; generating a lane marking detection result, which is a candidate for a lane marking of a road including a travel lane of the moving object, from each of the camera images extracted according to the time width; For each of the camera images extracted according to the time width, the lane line detection result that is not generated in the camera image extracted as the object to be determined but is generated in the camera image at another time is generated as a virtual lane line detection result; Calculating a reliability of the lane marking detection result and a reliability of the virtual lane marking detection result; A method for determining whether or not a lane is matched, the method detecting lane marks in each of the camera images extracted according to the time width based on the reliability of each of the lane marking detection results and the virtual lane marking detection results, and generating a result of determining whether or not the lane marks are identical between each of the camera images extracted as the objects to be determined as a lane marking match determination result.

13. A driving lane identification program for causing a computer to function as a driving lane identification device, determining a time range for referring to camera images acquired from the moving object at a designated date and time for identifying a lane in which the moving object is traveling and at a time before, after, or both before and after the designated date and time based on a state quantity of the moving object, and extracting a plurality of camera images according to the determined time range; generating, from each of the extracted camera images, a lane marking detection result that is a candidate for a lane marking of a road including a lane of the moving object; generating, for each of the extracted camera images, the lane marking detection results that were not generated in the camera images at the specified date and time but were generated in the camera images at other times, as virtual lane marking detection results; A step of calculating a reliability of the lane marking detection result and a reliability of the virtual lane marking detection result; identifying the driving lane based on a reliability of each of the lane marking detection result and the virtual lane marking detection result; A driving lane identification program that causes the computer to execute the above steps.

14. A driving lane coincidence determination program for causing a computer to function as a driving lane coincidence determination device, A step of extracting a plurality of camera images to be subjected to a determination as to whether or not the camera images are captured by a camera mounted on the moving object while the moving object is traveling in the same travel lane; a step of determining, for each of the camera images extracted as the object to be determined, a time span for referring to camera images taken by the camera mounted on the same moving body at a time before and after the extracted camera image based on a state quantity of the moving body, and extracting a plurality of camera images according to the determined time span; generating a lane marking detection result, which is a candidate for a lane marking of a road including a travel lane of the moving object, from each of the camera images extracted according to the time width; generating, as a virtual lane line detection result, the lane line detection result that is not generated in the camera image extracted as the object to be determined and that is generated in the camera image at another time, for each of the camera images extracted according to the time width; A step of calculating a reliability of the lane marking detection result and a reliability of the virtual lane marking detection result; a step of detecting lane markings in each of the camera images extracted according to the time width based on a reliability of each of the lane marking detection result and the virtual lane marking detection result, and generating a result of determining whether or not the lane markings are identical between each of the camera images extracted as the subject of the determination as a driving lane coincidence determination result; The driving lane matching determination program causes the computer to execute the above steps.

Images