Simulated road generation system, simulated road generation method and program

Abstract

A simulated road generation system (10) comprises an acquisition unit (11), a detection unit (12), an estimation unit (13), and a generation unit (14). The acquisition unit (11) acquires an image indicating the surroundings of a vehicle, and location information of the vehicle. The detection unit (12) detects demarcation lines of a road from the image. On the basis of the location information of the vehicle and the demarcation lines in the image, the estimation unit (13) estimates road parameters including at least one of the number of lanes included in a prescribed region in real space, the length of the lanes, the width of the lanes, and the curvature of curves. The generation unit (14) generates a simulated road on the basis of the road parameters.

Description

[Technical field]

[0001] The present disclosure relates to a simulated road generation system, a simulated road generation method, and a non-transitory computer-readable medium, and more particularly to a simulated road generation system, a simulated road generation method, and a non-transitory computer-readable medium that reproduce roads. [Background technology]

[0002] In recent years, roads have been virtually reproduced in various situations. For example, roads have been reproduced on a simulator to study the movement of vehicles on the road. In addition, risk factors have been detected by comparing images of the surroundings of a vehicle in a real space with a road virtually reproduced by a simulator.

[0003] Here, a method for virtually reproducing a road based on map information has been proposed. For example, Patent Document 1 describes a method for generating a map information image when viewed from the same direction as an image captured by an on-board camera, using three-dimensional map information that represents the contour lines of the road. As a method for generating three-dimensional map information, Patent Document 2, for example, discloses a three-dimensional map generation system that generates three-dimensional map information based on measurement data of a measurement vehicle. This three-dimensional map generation system discloses that three-dimensional map information based on measurement data of a measurement vehicle is interpolated based on measurement data of another measurement vehicle traveling near a defective area. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-164611 A [Patent Document 2] International Publication No. 2019 / 107367 Summary of the Invention [Problem to be solved by the invention]

[0005] However, when reproducing roads using the method described in the above-mentioned Patent Document 1, it is necessary to first obtain 3D map information, calculate road parameters required for reproducing the road from the 3D map information or extract 3D point cloud data, and input these into a simulator, which requires time and cost. Also, when obtaining 3D map information using the method described in the above-mentioned Patent Document 2, in order to generate the 3D map information by measurement, it is necessary to install expensive laser scanner cameras on multiple vehicles and take pictures while driving, which requires more time and cost. Therefore, there is a problem in that it requires time and cost to virtually reproduce roads.

[0006] In view of the above-mentioned problems, an object of the present disclosure is to provide a simulated road generation system, a simulated road generation method, and a non-transitory computer-readable medium that can easily reproduce roads. [Means for solving the problem]

[0007] A simulated road generation system according to one aspect of the present disclosure includes: An acquisition means for acquiring an image showing the surroundings of a vehicle and position information of the vehicle; A detection means for detecting road markings from the image; an estimation means for estimating road parameters including at least one of the number of lanes, the length of a lane, the width of a lane, and a curve curvature of a lane included in a predetermined area in real space based on the position information of the vehicle and the lane markings in the image; a generating means for generating a simulated road based on the road parameters; Equipped with.

[0008] A method for generating a simulated road according to one aspect of the present disclosure includes: an acquisition stage for acquiring an image showing the surroundings of a vehicle and position information of said vehicle; a detection step of detecting road dividing lines from the image; an estimation step of estimating road parameters including at least one of the number of lanes, the length of a lane, the width of a lane, and a curve curvature of a lane included in a predetermined area in a real space based on the position information of the vehicle and the lane markings in the image; generating a simulated road based on the road parameters; Equipped with.

[0009] A non-transitory computer-readable medium according to one aspect of the present disclosure stores a program. The program is an acquisition process for acquiring an image showing the surroundings of a vehicle and position information of the vehicle; A detection process for detecting road markings from the image; an estimation process for estimating road parameters including at least one of the number of lanes, the length of lanes, the width of lanes, and the curvature of a curve, of lanes included in a predetermined area in real space, based on the position information of the vehicle and the lane markings in the image; A generation process for generating a simulated road based on the road parameters; to be executed by the computer. Effect of the Invention

[0010] The present disclosure makes it possible to provide a simulated road generation system, a simulated road generation method, and a non-transitory computer-readable medium that can easily reproduce roads. [Brief description of the drawings]

[0011] [Figure 1] 1 is a block diagram showing a configuration of a simulated road generation system according to a first embodiment. [Diagram 2] 1 is a flowchart showing the flow of a simulation road generating method according to a first embodiment. [Diagram 3] 1 is a block diagram showing a configuration of a system to which a simulated road generating device can be applied. [Figure 4] FIG. 11 is a diagram for explaining a method for generating a simulated road according to a second embodiment. [Diagram 5]FIG. 11 is a diagram for explaining a method for generating a simulated road according to a second embodiment. [Figure 6] FIG. 11 is a diagram for explaining a method for generating a simulated road according to a second embodiment. [Figure 7] FIG. 11 is a diagram for explaining a method for generating a simulated road according to a second embodiment. [Figure 8] FIG. 11 is a diagram for explaining a method for generating a simulated road according to a second embodiment. [Figure 9] FIG. 11 is a diagram for explaining a method for generating a simulated road according to a second embodiment. [Figure 10] 10 is a flowchart showing an example of the flow of a simulated road generating method according to a second embodiment. [Figure 11] FIG. 11 is a diagram showing an example of a surrounding image according to the second embodiment. [Figure 12] FIG. 11 is a block diagram showing a configuration of a system according to a third embodiment. [Figure 13] FIG. 11 is a diagram for explaining a method for generating a simulated road according to a third embodiment. [Figure 14] FIG. 11 is a diagram for explaining a method for generating a simulated road according to a third embodiment. [Figure 15] FIG. 11 is a diagram for explaining a method for generating a simulated road according to a third embodiment. [Figure 16] 13 is a flowchart showing an example of the flow of a simulation road generating method according to a third embodiment. [Figure 17] FIG. 13 is a block diagram showing a configuration of a system according to a fourth embodiment. [Figure 18] 13 is a flowchart showing an example of the flow of a determination method according to the fourth embodiment. [Figure 19] FIG. 1 is a diagram showing an example of the configuration of a computer that can be used as a simulated road generating device and a vehicle device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, the present disclosure will be described through embodiments, but the disclosure according to the claims is not limited to the following embodiments. In addition, all of the configurations described in the embodiments are not necessarily essential as means for solving the problems. In each drawing, the same elements are given the same reference numerals, and duplicated explanations are omitted as necessary.

[0013] <Embodiment 1> First, a first embodiment of the present disclosure will be described. Fig. 1 is a block diagram showing a configuration of a simulated road generation system 10 according to the first embodiment.

[0014] The simulated road generation system 10 is a computer system that generates a simulated road on a simulator. A simulated road is a road that is a virtual reproduction of a road in real space. A simulator is a device that has a function of virtually reproducing a road, and is realized by software, hardware, or a combination of these. The simulated road generation system 10 includes an acquisition unit 11, a detection unit 12, an estimation unit 13, and a generation unit 14.

[0015] The acquisition unit 11, also called an acquisition means, acquires a surrounding image showing the surroundings of the vehicle and position information of the vehicle. The surrounding image is a two-dimensional image, and includes an image area capturing at least the road dividing lines around the vehicle. The surrounding image may be a captured image of the scenery around the vehicle captured by an on-board camera (not shown), or may be an image in which a predetermined image such as a guide line is superimposed on the captured image. The acquisition unit 11 supplies the acquired surrounding image to the detection unit 12, and supplies the acquired vehicle position information to the estimation unit 13.

[0016] The detection unit 12, also called a detection means, detects road markings from the surrounding image. Markings are lines that are formed on the road surface in white or yellow, for example. Markings include center lines, lane boundaries, and roadside strips. The detection unit 12 supplies information on the markings to the estimation unit 13.

[0017] The estimation unit 13 is also called an estimation means, and estimates road parameters based on the vehicle's position information and the dividing line in the surrounding image. The road parameters include individual road parameters in a predetermined area. The individual road parameters indicate at least one of the number of lanes, the lane length, the lane width, and the curve curvature, and are estimated based on at least the dividing line. For example, the road parameters may be information that associates the individual road parameters with the position information of the predetermined area. The position information of the predetermined area is estimated based on at least the vehicle's position information. Note that the "lane length" is the length of the lane in the predetermined area along the vehicle's traveling direction, and the "lane width" is the length of the lane in the predetermined area along the direction perpendicular to the vehicle's traveling direction. The estimation unit 13 supplies the road parameters to the generation unit 14.

[0018] The generator 14 is also called a generating means, and generates a simulated road on the simulator based on road parameters. The simulated road may be model data showing a three-dimensional shape of the road. The simulated road may also be model data showing a two-dimensional shape of the road when viewed from a predetermined point. The model data is CAD (Computer-Aided Design) data. The simulated road may also show an image generated based on model data.

[0019] 2 is a flowchart showing the flow of the simulated road generating method according to the first embodiment. First, the acquisition unit 11 of the simulated road generating system 10 acquires a surrounding image showing the surroundings of the vehicle and the position information of the vehicle (step S10). Next, the detection unit 12 detects the road dividing lines from the surrounding image (step S11). Next, the estimation unit 13 estimates road parameters based on the vehicle position information and the dividing lines (step S12). Next, the generation unit 14 generates a simulated road on the simulator based on the road parameters (step S13).

[0020] In this way, the simulated road generation system 10 according to the first embodiment automatically generates road parameters required for reproducing a road based on the image of the vehicle's surroundings and the vehicle's position information, and inputs the road parameters to the simulator. Therefore, the simulated road generation system 10 can easily reproduce a road without the need to obtain map information, calculate parameters, and input them. When reproducing a wide-area road, the amount of map information to be obtained and the amount of parameters to be calculated and input are enormous, but the simulated road generation system 10 has a more significant effect of reducing the effort and cost.

[0021] In addition, the image of the surroundings of the vehicle is a two-dimensional image, which can be easily acquired and contains a small amount of information, and therefore requires less effort and cost than using three-dimensional data. Therefore, the simulated road generation system 10 can reproduce roads more easily.

[0022] <Embodiment 2> Next, a second embodiment of the present disclosure will be described. In the second embodiment, a simulated road generation system (hereinafter, referred to as a simulated road generation device) generates a simulated road mainly for a vehicle to virtually run on a simulator. Fig. 3 is a block diagram showing a configuration of a system 1 to which the simulated road generation device can be applied.

[0023] The system 1 is a computer system that generates a simulated road on a simulator and performs various simulations using the simulated road. The system 1 includes a vehicle device 200 and a simulated road generating device 110, which are communicatively connected via a network (not shown). The network includes various networks such as the Internet, a wide area network (WAN), a local area network, and a mobile communication network, a dedicated line, or a combination of these. The network is a wireless communication line, but may also be a wired communication line.

[0024] (Vehicle device 200) The vehicle device 200 is a computer device mounted on a vehicle traveling on a road. The vehicle device 200 may function as a drive recorder. The vehicle device 200 acquires a photographed image of the scenery around the vehicle while traveling, and transmits the photographed image or an image generated based on the photographed image as a surrounding image to the simulated road generating device 110 via a network. The photographed image is a two-dimensional image, for example, an RGB image, and includes at least an image area of ​​a road dividing line. In the second embodiment, the photographed image that becomes the surrounding image or the basis of the surrounding image is an image photographed by a front camera. However, the photographed image that becomes the surrounding image or the basis of the surrounding image may be an image photographed by a rear camera or a side camera instead. In addition, a part of the photographed image that becomes the surrounding image or the basis of the surrounding image may be an image photographed by a front camera, and a part of the photographed image may be an image photographed by a rear camera or a side camera. In addition, the surrounding image generated based on the photographed image is, for example, an image in which a predetermined guide line is superimposed on the photographed image. The guide lines indicate the outer edge of an area defined by a predetermined width and depth located at a predetermined distance from the vehicle in real space when projected onto the image. In other words, the length and position of the guide lines in the surrounding image are known in real space. In the following, the surrounding image will be described as an image in which a predetermined guide line is superimposed on a captured image. The vehicle device 200 also acquires vehicle speed and position information and transmits it to the simulated road generation device 110 via the network. The vehicle device 200 has a communication unit 201, a camera 202, an image generation unit 203, and a vehicle information acquisition unit 204.

[0025] The camera 202 is mounted at an arbitrary position on the vehicle, and captures the scenery around the vehicle while it is traveling, to generate captured images. In the second embodiment, the camera 202 is a front camera that captures the scenery in front of the vehicle, but it may be a rear camera that captures the scenery behind the vehicle. The camera 202 generates captured images at, for example, 10 frames per second (10 fps), and supplies the generated captured images to the image generation unit 203 every 1 / 10th of a second.

[0026] The image generating unit 203 generates an image in which a guide line of a predetermined size is superimposed at a predetermined position on the captured image, as a surrounding image.

[0027] The vehicle information acquisition unit 204 acquires vehicle position information and vehicle speed information (vehicle speed information). The vehicle position information includes at least longitude and latitude, and may also include altitude in addition to these. Specifically, the vehicle information acquisition unit 204 acquires the vehicle position information from a satellite positioning system such as a Global Navigation Satellite System (GNSS), for example, a Global Positioning System (GPS). The vehicle information acquisition unit 204 is connected to a positioning information receiving unit (not shown) that receives positioning information for measuring the vehicle's position, and acquires vehicle position information from the positioning information receiving unit. In this case, the vehicle position information is sometimes called GPS information. The vehicle information acquisition unit 204 acquires the vehicle position information at a predetermined interval, for example, every second. The vehicle position information is not limited to GPS information, and may be a relative position from an object whose position information is known. The vehicle information acquisition unit 204 is also connected to the vehicle's ECU (Electronic Control Unit) via an in-vehicle communication bus such as CAN (Controller Area Network), and acquires vehicle speed information from the ECU.

[0028] The communication unit 201 is a communication interface with the network. The communication unit 201 transmits the surrounding image, and the vehicle position information, moving speed (vehicle speed) information, and the shooting time when the captured image that is the basis of the surrounding image was captured, as vehicle data to the simulated road generating device 110 via the network. Hereinafter, the shooting time of the captured image that is the basis of the surrounding image may be simply referred to as the shooting time of the surrounding image.

[0029] (Simulated road generation device 110) The simulated road generating device 110 is a computer device that generates a simulated road based on a surrounding image, vehicle position information, and vehicle speed information. In the second embodiment, the simulated road is a model that reproduces the two-dimensional or three-dimensional shape of a road in a predetermined area in real space. In other words, the simulated road includes a simulated lane that reproduces the shape of a lane included in a predetermined area in real space in two or three dimensions. The simulated road generating device 110 includes an acquisition unit 111, a detection unit 112, an estimation unit 113, a generation unit 114, a storage unit 115, and a display unit 116.

[0030] The acquisition unit 111 is an example of the above-mentioned acquisition unit 11. The acquisition unit 111 includes a communication interface with a network, and acquires vehicle data including a surrounding image, vehicle position information, vehicle speed information, and image capture time from the vehicle device 200 via the network. The acquisition unit 111 supplies the acquired vehicle data to the detection unit 112 and the estimation unit 113. The acquisition unit 111 may store the acquired vehicle data in the storage unit 115.

[0031] The detection unit 112 is an example of the detection unit 12 described above. The detection unit 112 acquires vehicle data from the acquisition unit 111 or the storage unit 115, and detects at least road markings from a surrounding image included in the vehicle data by image recognition. The detection unit 112 may detect other road signs and other vehicles in addition to the markings. The detection unit 112 supplies information on the detected road markings to the estimation unit 113.

[0032] The estimation unit 113 is an example of the estimation unit 13 described above. The estimation unit 113 generates road parameters based on the vehicle data acquired from the acquisition unit 111 or the storage unit 115 and information on the lane markings of the road detected by the detection unit 112. Specifically, the estimation unit 113 identifies a divided image area in the surrounding image that corresponds to the divided area and is defined based on the lane markings for each divided area formed when a lane in the real space is divided into a predetermined division length. The division length is determined based on the vehicle speed. As an example, the divided area is an area of ​​one lane whose length in the length direction (the traveling direction of the vehicle) is the division length. As an example, the divided image area may be a substantially rectangular area whose length direction has a divided image length corresponding to the division length and is defined along the lane markings. The number of divided image areas differs depending on the number of lanes in the surrounding image. For example, if the lane markings for three lanes are included in the surrounding image, the estimation unit 113 identifies three divided image areas. Therefore, the number of identified divided image areas includes information on the number of lanes.

[0033] The estimation unit 113 estimates divided road parameters for each divided image area based on the divided image area. The divided road parameters are individual road parameters indicating at least one of the lane length, lane width, and curve curvature of the divided area. For example, the estimation unit 113 performs projective transformation of the divided image area onto a predetermined two-dimensional plane so that the divided image area becomes a top view of the road when viewed from above in real space, and calculates the length, width, and curve curvature of the divided image area after projective transformation as the divided road parameters. The estimation unit 113 also estimates position information of the divided area based on vehicle position information for each divided image area. At this time, when the simulated road generating device 110 generates a three-dimensional simulated road, the estimation unit 113 may estimate the altitude of the divided area based on the altitude included in the vehicle position information, and add the estimated altitude to the position information. The estimation unit 113 generates road parameters including the divided road parameters, the position information of the divided area, and the number of divided image areas based on these. For example, the estimation unit 113 generates information in which divided road parameters are associated with position information of the divided area for each divided image area (i.e., for each divided area) as road parameters. In other words, the estimation unit 113 generates road parameters based on the divided road parameters, the position information of the vehicle when the surrounding image was captured, and the number of divided image areas.

[0034] The estimation unit 113 supplies the generated road parameters to the generation unit 114.

[0035] The generator 114 is an example of the generator 14. The generator 114 inputs road parameters to a simulator and generates a simulated road on the simulator based on the road parameters. The generator 114 causes the display unit 116 to display the generated simulated road.

[0036] The storage unit 115 is a storage device that stores information necessary for the simulation road generation process.

[0037] The display unit 116 is a display device that displays a simulated road.

[0038] Next, a specific example of the method for generating a simulated road will be described. Figures 4 to 9 are diagrams for explaining the method for generating a simulated road according to the second embodiment.

[0039] First, FIG. 4 shows a surrounding image 600 acquired by the simulated road generating device 110 from the vehicle device 200. The surrounding image 600 includes the road division line L and other vehicles 400-403. The other vehicles 400-403 traveling ahead of the vehicle cover the distant division lines, so that the distant division lines cannot be seen. In addition, the guide line 300 is superimposed on the surrounding image 600. The guide line 300 indicates, for example, a rectangular area with a width of 3.5 m and a depth of 8 m in the real space projected onto the image space. In other words, the width X0 of the guide line 300 corresponds to a width of 3.5 m in the real space, and the depth Y0 of the guide line 300 corresponds to a depth of 8 m in the real space.

[0040] Here, FIG. 5 shows a top view image 700 of a lane viewed from above in real space when the surrounding image 600 is captured. In this figure, for the sake of explanation, the lane markings and other vehicles are omitted. Points P1, P2, P3, and P4 in this figure indicate the positions of the vehicle when acquiring position information. Since the vehicle acquires position information at a predetermined interval Δt, the distance between each point is expressed by the vehicle speed v and the interval Δt. For example, when the vehicle speed at point P1 is v1, the distance between points P1 and P2 is expressed by r12≒v1·Δt. Also, when the vehicle speed at point P2 is v2, the distance between points P2 and P3 is expressed by r23≒v2·Δt.

[0041] As shown in this figure, the vehicle is located at P1 when the surrounding image 600 is captured. At this time, the estimation unit 113 defines three areas surrounded by thick lines having a division length Y1 in the longitudinal direction with a point P2 as divided areas 500C, 500R, and 500L.

[0042] For example, the division length Y1 is expressed as follows using the vehicle speed and interval Δt when the vehicle is located at points P1 and P2: Y1=(v1·Δt+v2·Δt) / 2=(v1+v2)·Δt / 2

[0043] Here, an image 600a shown in Fig. 6 indicates divided image areas 501C, 501R, and 501L identified from the surrounding image 600 by the estimation unit 113. The divided image areas 501C, 501R, and 501L are image areas corresponding to the divided areas 500C, 500R, and 500L in the real space shown in Fig. 5. Therefore, a depth Y1' of the divided image area 501C corresponds to a division length Y1 in the real space.

[0044] There are two methods for the estimation unit 113 to calculate the length of the depth Y1' of the divided image area 501C in the surrounding image corresponding to the division length Y1 in real space and the position of the divided image area 501C in the surrounding image. The first method is to use the guide line 300 whose size and position in real space are known. In this case, the estimation unit 113 specifies the depth Y' of the divided image area 501C corresponding to the divided area 500C and the position of the divided image area 501C in the surrounding image based on the size (for example, width Xo, depth Yo) and position of the guide line 300 in the surrounding image. The second method is to use the image area of ​​another vehicle in the surrounding image. The size of the other vehicle in real space is known for each vehicle type. Therefore, the estimation unit 113 specifies the depth Y' of the divided image area 501C corresponding to the divided area 500C and the position of the divided image area 501C in the surrounding image based on the size and position of the image area of ​​the other vehicle in the surrounding image. In addition, in FIG. 6, the width of the divided image area 501C is determined based on the dividing line in the surrounding image detected by the detection unit 112. In the case where the dividing line is a broken line as in FIG. 6, the estimation unit 113 may specify the line connecting the adjacent dividing lines as a pair of sides that define the width of the approximately rectangular area in the divided image area 501C, and specify the divided image area 501C. Then, when there are lanes on the right and left sides of the divided image area 501C in the surrounding image, the estimation unit 113 specifies the divided image areas 501R and 501L on the right and left sides of the divided image area 501C based on the dividing line. In addition, when there are four or more lanes, that is, when there are lanes on the outside (opposite side to the divided image area 501C) of the divided image areas 501R and 501L in the surrounding image, the estimation unit 113 may also specify the outer divided image area based on the dividing line.

[0045] The length of the outer edge, the width of the outer edge, the angle formed by the outer edge, the curvature of the curve, etc. of the identified divided image areas 501C, 501R, 501L correspond to the divided road parameters of the divided image areas. Specifically, first, the estimation unit 113 performs projective transformation of the identified divided image area onto a predetermined two-dimensional plane, for example, a plane as viewed from above as shown in Fig. 5. The estimation unit 113 then measures the length of the outer edge (i.e., the length of the lane), the width of the outer edge (i.e., the width of the lane), the angle formed by the outer edge, the curvature of the curve, etc. of the divided image area after projective transformation as the divided road parameters of the divided image area.

[0046] From the divided road parameters corresponding to one divided image area estimated in this way, one lane block into which one lane is divided can be generated on the simulator. Where to place the generated lane block on the simulator can be determined based on the position information of the vehicle at the time of capturing the surrounding image. Therefore, the estimation unit 113 generates road parameters for each divided image area based on the divided road parameters and the position information of the vehicle at the time of capturing the surrounding image corresponding to the divided image area. More specifically, the estimation unit 113 may generate information including the divided road parameters and the position information of the lane block determined based on the position information of the vehicle as the road parameters for each divided image area. Note that, when generating a three-dimensional simulated road on the simulator, the estimation unit 113 may include altitude information of the lane block determined based on the altitude of the vehicle in the position information of the block. This allows the simulated road generating device 110 to easily generate a simulated road without using map information.

[0047] Next, a surrounding image 620 captured when the vehicle was located at point P3 is shown in Fig. 7. As shown in this figure, the lane makes a large rightward curve. The surrounding image 620 also includes the lane markings L and guide lines 300 of the road.

[0048] 8 shows an image 700a of a top view of a lane in real space when viewed from above at the time of capturing the surrounding image 620. The estimation unit 113 defines an area surrounded by a thick line having a division length Y3 in the longitudinal direction, centered on the point P4, as a divided area 520C, and defines the adjacent areas as divided areas 520R and 520L.

[0049] For example, the division length Y3 is expressed as follows using the vehicle speed and interval Δt when the vehicle is located at points P3 and P4. Y3=(v3·Δt+v4·Δt) / 2=(v3+v4)·Δt / 2 Incidentally, v3 is the vehicle speed at point P3, and v4 is the vehicle speed at point P4.

[0050] Here, the image 620a shown in FIG. 9 shows the divided image areas 521C, 521R, and 521L identified from the surrounding image 620 by the estimation unit 113. The divided image areas 521C, 521R, and 521L are image areas corresponding to the divided areas 520C, 520R, and 520L in the real space shown in FIG. 8. Therefore, the depth Y3' of the divided image area 521C corresponds to the division length Y3 in the real space. The estimation unit 113 estimates the divided road parameters for the divided image areas 521C, 521R, and 521L as well as the divided image areas 501C, 501R, and 501L. Then, the estimation unit 113 generates road parameters based on the divided road parameters of each divided image area and the position information of the vehicle at the time when the surrounding image corresponding to the divided image area was photographed.

[0051] In this way, the simulated road generating device 110 generates divided road parameters for each divided area obtained by dividing the lane, and generates road parameters from the divided road parameters. As a result, even if information on distant lane lines cannot be seen in the surrounding image 600 due to the presence of other vehicles, road parameters can be generated from information on lane lines in other surrounding images 620 that correspond to the lane lines that cannot be seen.

[0052] 10 is a flowchart showing an example of the flow of the simulated road generating method according to the embodiment 2. First, the simulated road generating device 110 repeats the processes shown in steps S20 to S25 for each surrounding image, starting from the surrounding image captured earliest. Note that the surrounding images are captured at intervals Δt at which the vehicle device 200 acquires vehicle position information.

[0053] First, the detection unit 112 of the simulated road generating device 110 acquires the surrounding image, the vehicle position information, and the vehicle speed information from the acquisition unit 111 or the storage unit 115 (step S20). Then, the detection unit 112 detects the lane markings from the surrounding image (step S21). Then, the estimation unit 113 determines the division length of the divided area defined in the real space based on the vehicle speed information and the interval Δt (step S22). Next, the estimation unit 113 identifies the divided image area in the surrounding image that has a length corresponding to the division length and is defined based on the lane markings (step S23). Note that the above-mentioned step S21 may be performed in parallel with step S23. That is, the estimation unit 113 may detect the lane markings from a predetermined image area in the surrounding image that has a length corresponding to the division length, and identify the divided image area so that it is defined by the lane markings. Next, the estimation unit 113 performs projective transformation of the divided image area onto a predetermined two-dimensional plane (step S24). Then, the estimation unit 113 generates divided road parameters based on the shapes of the divided image regions that have been subjected to projective transformation (step S25).

[0054] In step S26, the estimation unit 113 generates road parameters based on the divided road parameters of each divided image area and the vehicle position information at the time of capturing the surrounding image corresponding to the divided image area. Then, in step S27, the generation unit 114 inputs the generated road parameters to the simulator and generates a simulated road on the simulator. At this time, the generation unit 114 may display the simulated road on the display unit 116. For example, the generation unit 114 may display an image similar to the image 700a of FIG. 8 as the simulated road on the display unit 116.

[0055] The generated simulated road may be used for vehicle running simulation. For example, the simulated road generating device 110 may virtually reproduce the running of the host vehicle on the simulated road generated on the simulator based on the history of GPS information when the host vehicle ran on an actual road. This allows the behavior of the host vehicle to be grasped and analyzed in detail. For example, the behavior of the host vehicle can be grasped from an angle different from that of a real camera, and danger can be estimated. In addition, the simulated road generating device 110 may virtually reproduce the running of other vehicles on the simulated road on the simulator based on the detection result of other vehicles in the surrounding image. This allows the relationship between the host vehicle and other vehicles, and even the relationship between the other vehicles, to be grasped and analyzed in detail. In other words, the running simulation allows the information such as the road shape, the position of the host vehicle, and other vehicles during running to be precisely captured, and the surrounding situation at that moment to be virtually reproduced. The running simulation may be used, for example, for vehicle development. As an example, when verifying the situation when a malfunction occurs in a vehicle under development (e.g., an autonomous vehicle), verification and testing costs can be reduced by verifying the situation under the same conditions on a simulator and conducting experiments after correction.

[0056] The road parameters include information on the number of divided image regions, i.e., information on the number of lanes, and therefore also information on sections where the number of lanes increases or decreases. 11 is a diagram showing an example of a surrounding image 690 according to the second embodiment. The surrounding image 690 includes an area A where the number of lanes is increased. The simulated road generating device 110 executes the above-mentioned simulated road generating method using surrounding images captured around the capture time of the surrounding image 690, thereby generating road parameters that reflect the section where the number of lanes is increased.

[0057] In this way, the simulated road generating device 110 according to the second embodiment automatically generates road parameters required for reproducing a road based on the image of the surroundings of the vehicle, the vehicle's position information, and the vehicle speed information, and inputs the road parameters to the simulator. Therefore, the simulated road generating device 110 can easily reproduce a road without the need to obtain map information, calculate parameters, and input them. When reproducing a wide-area road, the amount of map information to be obtained and the amount of parameters to be calculated and input are enormous, but the simulated road generating device 110 has a more significant effect of reducing the effort and cost.

[0058] The image of the surroundings of the vehicle is a two-dimensional image, which can be easily acquired and contains a small amount of information, and therefore requires less effort and cost than three-dimensional data. Therefore, the simulated road generation device 110 can more easily reproduce roads.

[0059] In addition, the simulated road generating device 110 uses surrounding images captured by an on-board camera, so it can reproduce roads precisely at the vehicle level. By driving a vehicle on the reproduced road, it is possible to perform a precise driving simulation. This can reduce the development costs of the vehicle.

[0060] <Embodiment 3> Next, a third embodiment of the present disclosure will be described. In the third embodiment, the simulated road generating device is characterized in that road parameters corresponding to a predetermined image area in a surrounding image captured at any time point are corrected based on a surrounding image captured after that time point. The simulated road generated by the corrected road parameters may be used primarily to detect dangerous events related to other vehicles from surrounding images captured in real space around the vehicle.

[0061] FIG. 12 is a block diagram showing a configuration of a system 2 according to a third embodiment. The system 2 is a computer system that generates a simulated road corresponding to a predetermined image area included in a surrounding image captured at an arbitrary time. In the following, the system 2 is described as generating a simulated road to detect a dangerous event related to another vehicle, but the purpose is not limited to this. The dangerous event related to another vehicle may be dangerous driving of the other vehicle, such as dangerous cutting in, meandering, or tailgating. The system 2 has a configuration basically similar to that of the system 1, but includes a simulated road generating device 120 instead of the simulated road generating device 110.

[0062] The simulated road generating device 120 includes an acquisition unit 121 , a detection unit 122 , an estimation unit 123 , a generation unit 124 , a storage unit 125 , and a display unit 126 .

[0063] The acquisition unit 121 is an example of the acquisition unit 11 described above, and corresponds to the acquisition unit 111 described above. The acquisition unit 121 includes a communication interface with a network, and acquires vehicle data including a surrounding image, vehicle position information, vehicle speed information, and shooting time from the vehicle device 200 via the network. Here, among the surrounding images captured continuously, a surrounding image captured at an arbitrary time and targeted for risk event detection is called a first frame, and a surrounding image captured after the first frame is called a second frame. The acquisition unit 121 acquires a plurality of frames including the first and second frames, and vehicle position information, vehicle speed information, and shooting time when each frame was captured, as vehicle data. Note that, when a frame (surrounding image) is generated based on a captured image, the shooting time refers to the shooting time of the captured image that is the basis of the frame.

[0064] The detection unit 122 is an example of the detection unit 12 described above, and corresponds to the detection unit 112 described above. The detection unit 122 detects the dividing line of the road for the first frame. Then, the detection unit 122 supplies information on the dividing line for the first frame to the estimation unit 123. Here, the detection unit 122 determines whether or not there is a blocked area in which the dividing line is not detected in the first frame, that is, the dividing line is blocked by an obstacle. The obstacle may be another vehicle or a road wall. When the detection unit 122 determines that there is a blocked area, the detection unit 122 detects the dividing line of the road for the second frame that is photographed near the blocked area in the real space and captures the blocked area. Then, the detection unit 122 supplies information on the dividing line for the second frame to the estimation unit 123.

[0065] The estimation unit 123 is an example of the estimation unit 13 described above, and corresponds to the estimation unit 113 described above. The estimation unit 123 estimates a first road parameter based on at least information on the division line of the second frame and position information of the vehicle at the time of photographing each of the first and second frames. The first road parameter is a road parameter of the area included in the first frame. The first road parameter may be a road parameter of the non-occluded area and the occluded area included in the first frame. The first road parameter may also be a road parameter of only the occluded area included in the first frame, that is, a road parameter included in the first frame but not including the road parameter of the non-occluded area. Here, when the first road parameter is a road parameter of the non-occluded area and the occluded area included in the first frame, the following processing may be performed. For example, first, the estimation unit 123 estimates a second road parameter, which is a road parameter of the occluded area, for the second frame based on the position information of the vehicle at the time of photographing and the division line. Specifically, the estimation unit 123 estimates individual road parameters of the shielded area included in the second frame based on the dividing line of the area corresponding to the shielded area in the second frame. The estimation unit 123 generates second road parameters based on the individual road parameters of the shielded area and the position information of the vehicle at the time of shooting the second frame. Furthermore, the estimation unit 123 estimates first road parameters, which are road parameters of the area included in the first frame, for the first frame based on the position information of the vehicle at the time of shooting and the dividing line. Then, the estimation unit 123 corrects the first road parameters based on the second road parameters. The correction may be rephrased as expansion or interpolation. Furthermore, when the first road parameters are road parameters of only the shielded area included in the first frame, the estimation unit 123 generates road parameters used when projecting the shielded area included in the second frame onto the first frame as the first road parameters. In this case, the estimation unit 123 generates the first road parameters based on the individual road parameters of the blocked area included in the second frame and the position information of the vehicle when the first and second frames were captured.

[0066] The generating unit 124 is an example of the generating unit 14 described above, and inputs the generated first road parameters to a simulator, generates dividing lines in a predetermined space on the simulator based on the first road parameters, and generates a generated simulated road. For example, the generating unit 124 performs projective transformation of the lanes generated based on the first road parameters onto the field of view of the first frame. This makes it possible to generate a simulated road that would be visible if there were no obstacles in the field of view of the first frame. Then, the generating unit 124 may generate a superimposed image in which the lanes generated based on the first road parameters are superimposed on the first frame as the simulated road.

[0067] The storage unit 125 has the same function as the above-mentioned storage unit 115. Moreover, the display unit 126 has the same function as the above-mentioned display unit .

[0068] 13 to 15 are diagrams for explaining a method for generating a simulated road according to the third embodiment. In the following, a case will be explained in which the first road parameters are road parameters of the non-occluded area and the occluded area included in the first frame. An image 600b shown in FIG. 13 shows the non-occluded area EA and the occluded area SA when the surrounding image 600 shown in FIG. 4 is the first frame. In this figure, the presence of the other vehicle 400, the other vehicle 401, and the road wall of the curved road occludes the distant division line, forming the occluded area SA. For example, the estimation unit 123 specifies the non-occluded area EA excluding the occluded area SA for the first frame as the above-mentioned divided image area. The divided image area is demarcated by the division line. In this figure, the divided image area is specified only for the lane on which the vehicle is traveling, but may be specified for each lane. Then, the estimation unit 123 performs projective transformation of the divided image area onto a predetermined two-dimensional plane, and calculates the length, width, and curve curvature of the lane of the projectively transformed divided image area as individual road parameters of the non-obstructed area EA. The predetermined two-dimensional plane may be a plane when the lane is viewed from above in real space as shown in Fig. 5, or may be the same as the plane showing the field of view of the first frame. The estimation unit 123 then provisionally calculates, as the first road parameters, information that associates the calculated individual road parameters of the non-obstructed area EA with position information of the vehicle at the time of photographing the first frame.

[0069] The estimation unit 123 then estimates the distance from the vehicle to the shielded area SA using the guide line 300, the size and position of which are known. The size and position of another vehicle included in the surrounding image 600 may be used to estimate the distance from the vehicle to the shielded area SA. The estimation unit 123 estimates the position information of the vehicle when the vicinity of the shielded area SA (i.e., the second frame) was photographed based on the position information of the vehicle when the surrounding image 600 was photographed and the distance from the vehicle to the shielded area SA. Alternatively, the estimation unit 123 estimates the photographing time when the second frame was photographed based on the vehicle speed information of the vehicle when the surrounding image 600 was photographed and the distance from the vehicle to the shielded area SA. In this way, the estimation unit 123 can identify the second frame.

[0070] The image 620b shown in FIG. 14 shows an area SA' corresponding to the shielded area SA in the case where the surrounding image 620 shown in FIG. 7 is the second frame. The estimation unit 123 specifies the area SA' corresponding to the shielded area SA in the second frame as the divided image area described above. In this case, the divided image area is also defined by the dividing line. In this figure, the divided image area is specified only for the lane on which the vehicle is traveling, but it may be specified for each lane. The estimation unit 123 performs projective transformation of the divided image area onto a two-dimensional plane similar to the case where the first road parameter is provisionally calculated, and calculates the length, width, and curve curvature of the lane of the projectively transformed divided image area as individual road parameters of the shielded area SA. Then, the estimation unit 123 calculates information in which the calculated individual road parameters of the shielded area SA are associated with the position information of the vehicle at the time of photographing the second frame as the second road parameter.

[0071] Then, the estimation unit 123 corrects the first road parameters based on the second road parameters. For example, the estimation unit 123 corrects the first road parameters by integrating the second road parameters associated with the vehicle position information when the second frame was captured with the first road parameters associated with the vehicle position information when the first frame was captured.

[0072] The image 700b shown in FIG. 15 shows a simulated road in which lanes generated using the modified first road parameters are projected onto the field of view of the surrounding image 600, which is the first frame. Specifically, the generation unit 124 performs projective transformation of the lanes generated using the modified first road parameters onto the field of view of the first frame, and superimposes the projectively transformed lanes onto the first frame. This makes it possible to generate a simulated road that would be visible if there were no obstacles in the field of view of the first frame. Note that if the two-dimensional plane used when calculating the first and second road parameters is the same as the plane showing the field of view of the first frame, the projective transformation of the lanes may be omitted.

[0073] 15, that is, according to the simulated road, it can be seen that the other vehicle 401 is traveling across the lane. It can also be seen that the other vehicle 400 is traveling on the left side of the lane.

[0074] FIG. 16 is a flowchart showing an example of the flow of the simulated road generating method according to the third embodiment. First, the detection unit 122 of the simulated road generating device 120 acquires the surrounding image at the shooting time t=t0, the vehicle position information, and the vehicle speed information from the acquisition unit 121 or the storage unit 125 (step S30). The surrounding image at the shooting time t=t0 is the first frame. Then, the detection unit 122 detects the lane markings from the first frame (step S31). Next, the estimation unit 123 identifies an unobstructed area in the first frame, and provisionally estimates the first road parameters based on the lane markings in the unobstructed area and the vehicle position information at the time of shooting the first frame (step S32). Next, the estimation unit 123 determines whether or not there is an obstructed area in the first frame where the lane markings have not been detected (step S33). If the estimation unit 123 does not determine that there is an obstructed area (No in step S33), the process proceeds to step S39.

[0075] On the other hand, when it is determined that there is a blocked area (Yes in step S33), the estimation unit 123 estimates the shooting time t=t1 (>t0) of the surrounding image that captured the vicinity of the blocked area using the vehicle speed information (step S34). The surrounding image that captured the vicinity of the blocked area is the second frame. The detection unit 122 acquires the second frame, which is the surrounding image at the shooting time t=t1, and the position information of the vehicle from the acquisition unit 121 or the storage unit 125 (step S35). Then, the detection unit 122 detects the division lines for the second frame as well as the first frame (step S36). Next, the estimation unit 123 estimates the second road parameters based on the division lines of the area of ​​the second frame that corresponds to the blocked area and the position information of the vehicle at the time of capturing the second frame (step S37). Next, the estimation unit 123 corrects the first road parameters based on the second road parameters (step S38). The generator 124 then generates lanes using the corrected first road parameters, and generates simulated lanes by projecting the lanes onto the field of view of the surrounding image 600, which is the first frame (step S39). The generator 124 then superimposes the simulated lanes onto the first frame, thereby generating a superimposed image (step S40).

[0076] In the above description, the first road parameters are road parameters of the non-occluded area and the blocked area included in the first frame. However, if the first road parameters are road parameters of the blocked area included in the first frame, the estimation unit 123 first calculates individual road parameters of the blocked area from the dividing line of the blocked area included in the second frame. Then, the estimation unit 123 determines a position in the first frame where the simulated lane corresponding to the blocked area is to be superimposed based on the position information of the vehicle when the first and second frames are photographed. The estimation unit 123 generates first road parameters including the individual road parameters of the blocked area included in the second frame and information on the superimposition position. Then, the generation unit 124 generates the simulated lane in the blocked area based on the first road parameters, performs projective transformation on the field of view of the first frame, and superimposes the projectively transformed simulated lane corresponding to the blocked area on the superimposition position of the first frame. Therefore, in this case, the generation and superimposition of the simulated lane corresponding to the non-occluded area are omitted.

[0077] In this way, the third embodiment can achieve the same effects as the second embodiment. Furthermore, according to the third embodiment, only the information that is lacking in the target frame is acquired from future frames and applied, so that the necessary information can be generated efficiently. Therefore, the processing speed is improved.

[0078] The simulated road generating device 120 may transmit data of the generated simulated road to the vehicle device 200. Then, the vehicle device 200, which receives the data of the simulated road, may display the simulated road on a display unit (not shown). This allows the vehicle occupant of the vehicle device 200 to easily understand the road conditions a predetermined time ago. Alternatively, the vehicle device 200 may store the received data of the simulated road in an external storage device (not shown). Then, a person related to the vehicle device 200 can easily provide information on incidents and accidents by providing the data of the simulated road stored in the external storage device to a police agency or an insurance company.

[0079] <Embodiment 4> Next, a fourth embodiment of the present disclosure will be described. In the fourth embodiment, a simulated road generating device uses a simulated road to detect risky events related to other vehicles from surrounding images captured around a vehicle in real space.

[0080] 17 is a block diagram showing the configuration of a system 2a according to the fourth embodiment. The system 2a has basically the same configuration and functions as the system 2, but includes a simulated road generating device 120a instead of the simulated road generating device 120. The simulated road generating device 120a includes a determination unit 127 and an output unit 128 in addition to the configuration of the simulated road generating device 120.

[0081] The determination unit 127 detects a dangerous event related to another vehicle based on the simulated road generated by the generation unit 124. For example, a dangerous event related to another vehicle is dangerous driving of the other vehicle, such as dangerous cutting in, meandering, or tailgating, and the determination unit 127 determines whether or not the other vehicle is driving dangerously based on the image areas of the lanes and the other vehicle included in the superimposed image.

[0082] The output unit 128 outputs the result of the determination made by the determination unit 127. The output unit 128 may be configured integrally with the display unit 126.

[0083] FIG. 18 is a flowchart showing the flow of the determination method according to the fourth embodiment. First, the determination unit 127 of the simulated road generating device 120a acquires a superimposed image generated as a simulated road by the generating unit 124, and identifies an image area of ​​another vehicle included in the superimposed image (step S50). Then, the determination unit 127 determines whether or not a dangerous event related to the other vehicle exists based on the position of the lane and the position of the image area of ​​the other vehicle included in the superimposed image (step S51). At this time, the determination unit 127 may determine the type of the dangerous event in addition to the presence or absence of the dangerous event. The determination unit 127 may detect the movement of the other vehicle based on the position of the lane and the position of the image area of ​​the other vehicle in the superimposed image generated for a plurality of frames, and determine the presence or absence of a dangerous event related to the other vehicle. This makes it possible to detect dangerous cutting in, meandering, or tailgating by the other vehicle. Then, the output unit 128 outputs the determination result in a predetermined format (step S52).

[0084] The output unit 128 may provide information on dangerous events related to other vehicles by transmitting the information to a computer (not shown) of a police agency or an insurance company via a network.

[0085] In this way, the simulated road generating device 120a according to the fourth embodiment can detect dangerous events related to other vehicles using the simulated road. The simulated road generating device 120a may transmit data of a superimposed image in which the dangerous event is detected to the vehicle device 200 in response to detecting the dangerous event. The vehicle device 200 that receives the data of the superimposed image may display the superimposed image on a display unit (not shown). This allows the occupants of the vehicle device 200 to easily grasp the dangerous event that occurred a predetermined time ago. Alternatively, the vehicle device 200 may store the received data of the superimposed image in an external storage device (not shown). Then, a person related to the vehicle device 200 can view the data stored in the external storage device. Overlay Image By providing this data to police agencies and insurance companies, it will be easier to provide information about incidents and accidents.

[0086] Next, the physical configuration of the simulated road generating device and the vehicle device will be described. Fig. 18 is a diagram showing an example of the configuration of a computer that can be used as the simulated road generating device and the vehicle device. The computer 1000 has a processor 1010, a storage unit 1020, a ROM (Read Only Memory) 1030, a RAM (Random Access Memory) 1040, a communication interface (IF: Interface) 1050, and a user interface 1060.

[0087] The communication interface 1050 is an interface for connecting the computer 1000 to a communication network via a wired communication means or a wireless communication means, etc. The user interface 1060 includes a display unit such as a display, and also includes an input unit such as a keyboard, a mouse, and a touch panel.

[0088] The storage unit 1020 is an auxiliary storage device that can hold various data. The storage unit 1020 does not necessarily need to be a part of the computer 1000, and may be an external storage device or a cloud storage connected to the computer 1000 via a network.

[0089] The ROM 1030 is a non-volatile storage device. For example, a semiconductor storage device with a relatively small capacity, such as a flash memory, is used for the ROM 1030. The programs executed by the processor 1010 may be stored in the storage unit 1020 or the ROM 1030. The storage unit 1020 or the ROM 1030 stores various programs for implementing the functions of the simulated road generating device and each unit in the vehicle device, for example.

[0090] The above program can be stored using various types of non-transitory computer-readable media and can be provided to the computer 1000. The non-transitory computer-readable media includes various types of tangible storage media. Examples of the non-transitory computer-readable media include magnetic recording media such as a flexible disk, a magnetic tape, or a hard disk; magneto-optical recording media such as a magneto-optical disk; optical disk media such as a compact disc (CD) or a digital versatile disk (DVD); and mask ROM, PROM (programmable ROM), etc. ROM), EPROM (erasable PROM), flash ROM, or RAM, etc. The program may be provided to the computer using various types of temporary computer-readable media. Examples of the temporary computer-readable medium include an electric signal, an optical signal, and an electromagnetic wave. The temporary computer-readable medium may provide the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.

[0091] The RAM 1040 is a volatile storage device. The RAM 1040 may include various types of semiconductor memory such as a dynamic random access memory (DRAM) or a static random access memory (SRAM). A semiconductor memory device is used. The RAM 1040 can be used as an internal buffer for temporarily storing data, etc. The processor 1010 loads a program stored in the storage unit 1020 or the ROM 1030 into the RAM 1040 and executes it. The processor 1010 is a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The processor 1010 may execute a program to realize the functions of each unit in the simulated road generation device and the vehicle device. The processor 1010 may have an internal buffer that can temporarily store data, etc.

[0092] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present disclosure within the scope of the invention. For example, in the second to fourth embodiments, the acquisition units 111 and 121 acquire vehicle data from the vehicle device 200 via a network. However, instead of this, the acquisition units 111 and 121 may acquire vehicle data from an external storage device (not shown) that stores vehicle data. The external storage device may be a recording medium such as an SD card or USB storage.

[0093] A part or all of the above-described embodiments can be described as, but is not limited to, the following supplementary notes. (Appendix 1) An acquisition means for acquiring an image showing the surroundings of a vehicle and position information of the vehicle; A detection means for detecting road markings from the image; an estimation means for estimating road parameters including at least one of the number of lanes, the length of a lane, the width of a lane, and a curve curvature of a lane included in a predetermined area in real space based on the position information of the vehicle and the lane markings in the image; a generating means for generating a simulated road based on the road parameters; A simulated road generation system comprising: (Appendix 2) The estimation means includes: For each divided area formed when a lane is divided into a predetermined length in real space, a divided image area is identified in the image that corresponds to the divided area and is defined based on the lane markings; Estimating divided road parameters based on the divided image regions; A road parameter is generated based on the divided road parameter and the position information of the vehicle at the time of capturing the image. 2. A simulated road generation system as described in appendix 1. (Appendix 3) The estimation means determines the predetermined length based on a speed of the vehicle. 3. A simulated road generation system as described in appendix 2. (Appendix 4) the image is an image in which a guide line, the length and position of which are known in real space, is superimposed on an image captured by a camera of the vehicle; The estimation means identifies the divided image area in the image based on the guide line in the image. 4. The simulated road generation system according to claim 2 or 3. (Appendix 5) The estimation means identifies the divided image area in the image based on a size and a position of an image area of ​​another vehicle included in the image. 4. The simulated road generation system according to claim 2 or 3. (Appendix 6) the acquisition means acquires a first frame, a second frame captured after the first frame, and position information of the vehicle at the time when each of the first and second frames was captured; The detection means detects a road dividing line for each of the first and second frames; When there is an obstructed area in which no lane markings are detected in the first frame, the estimation means estimates a first road parameter of the area included in the first frame based on the lane markings in the second frame in which the obstructed area is photographed and on position information of the vehicle at the times when each of the first and second frames is photographed. 2. A simulated road generation system as described in appendix 1. (Appendix 7) The estimation means includes: estimating a first road parameter of an area included in the first frame based on position information of the vehicle at the time of photographing the first frame and the lane marking; Estimating a second road parameter of the occluded area based on position information of the vehicle at the time of capturing the second frame and the lane marking; Modifying the first road parameters based on the second road parameters; The generating means generates a simulated road based on the modified first road parameters. 7. A simulated road generation system according to claim 6. (Appendix 8) The generating means generates, as the simulated road, an image in which a simulated lane generated based on the first road parameters is superimposed on the first frame. 8. The simulated road generation system according to claim 6 or 7. (Appendix 9) A determination unit is provided for determining whether or not a dangerous event related to the other vehicle exists based on an image area of ​​the simulated lane and the other vehicle included in the simulated road. 9. The simulated road generation system according to claim 8. (Appendix 10) an acquisition stage for acquiring an image showing the surroundings of a vehicle and position information of said vehicle; a detection step of detecting road dividing lines from the image; an estimation step of estimating road parameters including at least one of the number of lanes, the length of a lane, the width of a lane, and a curve curvature of a lane included in a predetermined area in a real space based on the position information of the vehicle and the lane markings in the image; generating a simulated road based on the road parameters; A method for generating a simulated road comprising the steps of: (Appendix 11) an acquisition process for acquiring an image showing the surroundings of a vehicle and position information of the vehicle; A detection process for detecting road markings from the image; an estimation process for estimating road parameters including at least one of the number of lanes, the length of lanes, the width of lanes, and the curvature of a curve, of lanes included in a predetermined area in real space, based on the position information of the vehicle and the lane markings in the image; A generation process for generating a simulated road based on the road parameters; A non-transitory computer-readable medium that stores a program for causing a computer to execute the program. [Explanation of symbols]

[0094] 1,2,2a system 10, 110, 120, 120a Simulation road generation system (simulation road generation device) 11,111,121 Acquisition Department 12,112,122 Detector 13,113,123 Estimation part 14,114,124 generation part 115,125 Storage section 116,126 Display section 127 Judgment section 128 Output section 200 Vehicle equipment 201 Communications Department 202 Camera 203 Image Generation Unit 204 Vehicle Information Acquisition Department 300 Guide Line 400, 401, 402, 403 Other vehicles 500,520 divided areas 501,521 Segmented Image Region 600,620,690 Surrounding images 600a,600b,620a,620b Images 700,700a,700b Images 1000 Computers 1010 Processor 1020 Storage section 1030 ROM 1040 RAM 1050 Communication Interface 1060 User Interface A Lane Increase Area L Lot Line

Claims

1. An acquisition means for acquiring an image showing the surroundings of a vehicle and position information of the vehicle; a detection means for detecting a lane marking formed on a road surface in real space from the image; an estimation means for estimating individual road parameters indicating at least one of the number of lanes, the length of the lanes, the width of the lanes and the curvature of the curve of the lanes included in a predetermined area in real space based on the lane markings, and road parameters associating the individual road parameters with vehicle position information; a generating means for generating a simulated road based on the road parameters; Equipped with The estimation means includes: For each divided area formed when a lane is divided into a predetermined length in real space, a divided image area is identified in the image that corresponds to the divided area and is defined based on the lane markings; estimating divided road parameters of the divided regions based on the divided image regions, the divided road parameters including at least one of a lane length, a lane width, and a curve curvature; generating the road parameters based on the divided road parameters, position information of the vehicle at the time of capturing the image, and the number of divided image areas; The road parameters include information on the number of lanes corresponding to the number of divided image regions. A simulated road generation system.

2. The estimation means determines the predetermined length based on a speed of the vehicle. The simulated road generation system according to claim 1 .

3. the image is an image in which a guide line, the length and position of which are known in real space, is superimposed on an image captured by a camera of the vehicle; The estimation means identifies the divided image area in the image based on the guide line in the image.

3. The simulated road generating system according to claim 1 or 2.

4. The estimation means identifies the divided image area in the image based on a size and a position of an image area of ​​another vehicle included in the image.

3. The simulated road generating system according to claim 1 or 2.

5. The computer an acquisition stage for acquiring an image showing the surroundings of a vehicle and position information of said vehicle; a detection step of detecting a lane marking formed on a road surface in a real space from the image; an estimation step of estimating individual road parameters indicating at least one of the number of lanes, the length of a lane, the width of a lane, and a curve curvature of a lane included in a predetermined area in real space based on the lane markings, and road parameters associating the individual road parameters with vehicle position information; generating a simulated road based on the road parameters; Run In the estimation step, For each divided area formed when a lane is divided into a predetermined length in real space, a divided image area is identified in the image that corresponds to the divided area and is defined based on the lane markings; Estimating divided road parameters of the divided areas based on the divided image areas, the divided road parameters including at least one of a lane length, a lane width, and a curve curvature; generating the road parameters based on the divided road parameters, position information of the vehicle at the time of capturing the image, and the number of divided image areas; The road parameters include information on the number of lanes corresponding to the number of divided image regions. Simulated road generation method.

6. an acquisition process for acquiring an image showing the surroundings of a vehicle and position information of the vehicle; A detection process for detecting a lane marking formed on a road surface in a real space from the image; an estimation process for estimating individual road parameters indicating at least one of the number of lanes, the length of the lanes, the width of the lanes, and the curvature of the curve of the lanes included in a predetermined area in real space based on the lane markings, and road parameters that associate the individual road parameters with vehicle position information; A generation process for generating a simulated road based on the road parameters; on the computer, In the estimation process, For each divided area formed when a lane is divided into a predetermined length in real space, a divided image area is identified in the image that corresponds to the divided area and is defined based on the lane markings; Estimating divided road parameters of the divided areas based on the divided image areas, the divided road parameters including at least one of a lane length, a lane width, and a curve curvature; generating the road parameters based on the divided road parameters, position information of the vehicle at the time of capturing the image, and the number of divided image areas; The road parameters include information on the number of lanes corresponding to the number of divided image regions. program.

Images