Processing device, processing method, and recording medium

Abstract

A processing device according to the present disclosure comprises: an acquisition unit that acquires vehicle trajectory data or position data measured by a fixed-point sensor; and an estimation unit that estimates a lane region on the basis of the trajectory data or the position data.

Description

Processing device, processing method, and recording medium

[0001] The present disclosure relates to a processing device, a processing method, a program, and a recording medium.

[0002] A technology related to the present disclosure is disclosed in Patent Document 1. The technology disclosed in Patent Document 1 detects white lines on a road by image analysis, and identifies the lane in which a vehicle is traveling based on the detection result of the white lines.

[0003] JP 2009-26164 A

[0004] There is a need for a technology that can estimate lane areas based on data measured by a fixed-point sensor. For example, a method such as that disclosed in Patent Literature 1 can be considered, in which white lines on a road are detected from an image generated by a camera (a fixed-point sensor) and the lane areas are estimated based on the detection results of the white lines.

[0005] However, this method cannot be used when white lines cannot be detected from data measured by fixed sensors. For example, white lines may have faded due to aging or other factors. Furthermore, when fixed sensors are located far from the white lines, obstacles can easily obscure the white lines. Furthermore, the white lines may become unclear in the image due to the influence of light (sunlight, vehicle headlights, etc.). For these and other reasons, white lines may not be detected from data measured by fixed sensors.

[0006] In view of the above-mentioned problems, an example of an objective of the present disclosure is to provide a new technique for estimating lane areas based on data measured by fixed-point sensors.

[0007] According to the present disclosure, a processing device is provided that includes: an acquisition means for acquiring vehicle trajectory data or position data measured by a fixed-point sensor; and an estimation means for estimating a lane area based on the trajectory data or the position data.

[0008] The present disclosure also provides a processing method in which one or more computers acquire vehicle trajectory data or position data measured by fixed sensors, and estimate lane areas based on the trajectory data or the position data.

[0009] Furthermore, according to the present disclosure, a program is provided that causes a computer to function as: an acquisition means that acquires vehicle trajectory data or position data measured by a fixed-point sensor; and an estimation means that estimates a lane area based on the trajectory data or the position data.

[0010] According to one aspect of the present disclosure, a new technique for estimating lane areas based on data measured by fixed sensors is realized.

[0011] FIG. 1 is a diagram illustrating an example of a functional block diagram of a processing device according to the present disclosure. FIG. 2 is a flowchart illustrating an example of a processing flow of a processing device according to the present disclosure. FIG. 3 is a diagram illustrating an example of a hardware configuration of a processing device according to the present disclosure. FIG. 4 is a flowchart illustrating another example of a processing flow of a processing device according to the present disclosure. FIG. 5 is a diagram illustrating another example of a processing flow of a processing device according to the present disclosure. FIG. 6 is a diagram illustrating another example of a processing flow of a processing device according to the present disclosure. FIG. 7 is a diagram illustrating another example of a processing flow of a processing device according to the present disclosure. FIG. 8 is a diagram illustrating another example of a processing flow of a processing device according to the present disclosure. FIG. 9 is a diagram illustrating another example of a processing flow of a processing device according to the present disclosure.

[0012] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In this disclosure, the drawings relate to one or more embodiments. In all drawings, similar components are designated by similar reference numerals, and descriptions thereof will be omitted as appropriate.

[0013] <<First Embodiment>> Fig. 1 is a functional block diagram showing an overview of a processing device 10. Fig. 2 is a flowchart showing an example of the flow of processing executed by the processing device 10.

[0014] 1, the processing device 10 includes an acquisition unit 11 and an estimation unit 12. These functional units execute the process of the flowchart in FIG.

[0015] In S10, the acquisition unit 11 acquires vehicle trajectory data or position data measured by fixed point sensors. In S11, the estimation unit 12 estimates lane areas based on the trajectory data or position data acquired in S10.

[0016] In this way, the processing device 10 estimates lane areas based on vehicle trajectory data or position data measured by fixed sensors. Vehicles travel along lanes. For this reason, the vehicle movement trajectory indicated by trajectory data measured by fixed sensors tends to follow the lanes. Furthermore, the vehicle positions indicated by vehicle position data measured by fixed sensors tend to be scattered along the lanes. The processing device 10 estimates lane areas based on the vehicle movement trajectories and positions that tend to follow this pattern.

[0017] In the case of such a processing device 10, it is sufficient to detect the movement trajectory and position of a vehicle on the road, and there is no need to detect white lines on the road by image analysis, etc. Such a processing device 10 realizes a new technology for estimating lane areas based on data measured by fixed-point sensors.

[0018] Furthermore, with the processing device 10, after a fixed sensor is installed, lane areas can be estimated automatically or semi-automatically based on data measured by the fixed sensor after installation. Furthermore, with the processing device 10, even after the orientation, position, or measurement conditions (zoom, etc.) of the fixed sensor are changed, lane areas can be estimated automatically or semi-automatically based on data measured by the fixed sensor after the change. This processing device 10 avoids the tedious task of having workers register lane areas within the measurement range of each fixed sensor each time a fixed sensor is installed or the orientation, position, measurement conditions, etc. are changed. For example, when installing fixed sensors in locations that require temporary monitoring, such as construction sites, the installation and settings of fixed sensors may be frequently changed. In this case, it would be tedious if workers had to register lane areas each time. The processing device 10 can alleviate this inconvenience.

[0019] <<Second Embodiment>>

[0020] <Overview> The processing apparatus 10 of the second embodiment is a specific embodiment of the configuration of the processing apparatus 10 of the first embodiment. The processing apparatus 10 will be described in detail below.

[0021] <Hardware Configuration> First, an example of the hardware configuration of the processing device 10 will be described. Each functional unit of the processing device 10 is realized by any combination of hardware and software. Those skilled in the art will understand that there are various variations in the realization method and device. The software includes programs that are pre-loaded when the device is shipped, and programs downloaded from recording media such as CDs (Compact Discs) or servers on the Internet.

[0022] FIG. 3 is a block diagram illustrating an example of the hardware configuration of a processing device 10. As shown in FIG. 3, the processing device 10 has a processor 1A, a memory 2A, an input / output interface 3A, a peripheral circuit 4A, and a bus 5A. The peripheral circuit 4A includes various modules. The processing device 10 does not necessarily have to have the peripheral circuit 4A. Note that the processing device 10 may be composed of multiple devices that are physically and / or logically separated. In this case, each of the multiple devices may have the above hardware configuration.

[0023] The bus 5A is a data transmission path for the processor 1A, memory 2A, peripheral circuit 4A, and input / output interface 3A to mutually transmit and receive data. The processor 1A is, for example, a central processing unit (CPU) or a graphics processing unit (GPU). The memory 2A is, for example, a random access memory (RAM) or a read-only memory (ROM). The input / output interface 3A includes interfaces for acquiring information from input devices, external devices, external servers, external sensors, cameras, etc., and interfaces for outputting information to output devices, external devices, external servers, etc. The input / output interface 3A also includes an interface for connecting to a communication network such as the Internet. Examples of input devices include a keyboard, mouse, microphone, physical buttons, and touch panel. Examples of output devices include a display, projection device, speaker, printer, and mailer. The processor 1A can issue commands to each module and perform calculations based on the results of those calculations.

[0024] <Functional Configuration> Next, a detailed description will be given of the functional configuration of the processing device 10. Fig. 1 shows an example of a functional block diagram of the processing device 10. As shown in the figure, the processing device 10 has an acquisition unit 11 and an estimation unit 12.

[0025] The acquisition unit 11 and the estimation unit 12 execute the processes S1 to S5 in Fig. 4. The acquisition unit 11 executes S1 to S3. Then, the estimation unit 12 executes S4 and S5. An overview of each process is as follows.

[0026] Data acquisition process S1: The acquisition unit 11 acquires trajectory data. Extraction process S2: The acquisition unit 11 extracts trajectory data suitable for extracting lane areas from the acquired trajectory data. Determination process S3: The acquisition unit 11 determines whether the processes S1 and S2 executed up to that point satisfy predetermined conditions. The acquisition unit 11 then repeats S1 and S2 until the predetermined conditions are satisfied. If it is determined that the predetermined conditions are satisfied, the process proceeds to S4. By performing the process of S3, it is possible to proceed to the processes from S4 onwards after a sufficient amount of trajectory data has been extracted in S2. Lane area estimation process S4: The estimation unit 12 estimates lane areas based on the trajectory data extracted up to that point in S2. Lane area confirmation process S5: The estimation unit 12 determines whether the lane areas are confirmed by the estimation result of S4. If not confirmed, the process returns to S1 and the same processes are repeated. By performing the processing of S5, the estimation result of S4 is not determined as the lane area as it is, but rather the estimation result can be determined as the lane area at the stage when a reliable estimation result is obtained in S4.

[0027] Each process will be described in detail below.

[0028] "Data Acquisition Process S1 (FIG. 4)" The acquisition unit 11 acquires vehicle trajectory data measured by fixed point sensors.

[0029] A "vehicle" is a vehicle that travels on the road. Examples of vehicles include, but are not limited to, four-wheeled vehicles, large vehicles, and motorcycles.

[0030] The "trajectory data" indicates the movement trajectory of the vehicle within the measurement range of the fixed sensor. In one example, the trajectory data indicates the movement trajectory of the vehicle's reference point. In this embodiment, the reference point of the vehicle is the point obtained by lowering the approximate center of the left-right direction (vehicle width direction) of the vehicle toward the direction of travel of the vehicle to the ground. For example, the reference point of the vehicle may simply be the center of the base of a rectangular area in which the vehicle detected in the image is captured. Note that the reference point of the vehicle is not limited to the one shown here. Other locations on the vehicle may also be used as the reference point of the vehicle. Even in such a modified example, the effects of this embodiment can be achieved by appropriately adjusting the processing of S4. Details will be described later. Furthermore, a driver may be detected using person detection technology, and the movement trajectory of the detected person (driver) may be used as a pseudo-trajectory of the vehicle's movement.

[0031] The trajectory data may be, for example, data indicating the movement trajectory of the vehicle as a set of coordinates in a two-dimensional coordinate system set in an image generated by a camera (fixed-point sensor). Alternatively, the trajectory data may be data indicating the movement trajectory of the vehicle as a set of vehicle positions (coordinates in a three-dimensional coordinate system) in a three-dimensional space detected by a position detection sensor (such as Lidar).

[0032] A "fixed point sensor" is a sensor capable of detecting the movement trajectory and position of a vehicle traveling on a road. For example, the fixed point sensor may be a camera. The camera may detect visible light and create an image, or may detect other light such as infrared light, ultraviolet light, or X-rays and create an image. Alternatively, the fixed point sensor may be a position detection sensor (such as Lidar) that transmits electromagnetic waves and receives reflected waves to detect the position of an object. Alternatively, the position detection sensor (fixed point sensor) may be an infrared sensor. For example, an infrared sensor may detect a heat source (such as a person, an engine, a tire, or a battery) and detect the position and movement trajectory of the heat source. By using an infrared sensor, trajectory and position information can be obtained even in areas that are not visible to the naked eye. Alternatively, the fixed point sensor may be a combination of at least two of these sensors.

[0033] Trajectory data indicating the vehicle's movement trajectory is generated based on data measured by such fixed sensors. The generation of trajectory data based on data measured by fixed sensors can be achieved using any widely known technology. In one example, a vehicle is detected in a video image using image analysis. A vehicle may also be detected using technology such as deep learning. The detected vehicle is then tracked in the video image. Trajectory data for the vehicle is then generated by generating a history of the position of the vehicle's reference point in the image. The vehicle's reference point may be detected using features of the vehicle's appearance, or by other methods. For example, the center of the base of a rectangular area in which the vehicle detected in the image is captured may be used as a simple reference point for the vehicle. Note that the example of generating trajectory data shown here is merely an example and is not limited to this.

[0034] A fixed-point sensor is a sensor that stays at a certain position for a certain period of time or more and senses data (images and detection results from a position detection sensor such as Lidar) of vehicles traveling on the road from that position.

[0035] For example, a fixed sensor may be a sensor that is installed at a location where it can measure data on vehicles traveling on the road and remains at that location. In this example, the fixed sensor may be installed at that location for a relatively long period (e.g., several years) and acquire data on that location over the long term. Alternatively, the fixed sensor may be installed at that location for a relatively short period (e.g., several days to several months) and acquire data on that location over the short term.

[0036] Such fixed-point sensors may be able to change at least one of the orientation, position, and measurement conditions (zoom, etc.) automatically or in response to an operator's operation. Such a function is preferable because it widens the range that can be measured with a single fixed-point sensor.

[0037] Alternatively, the fixed sensor may be a sensor installed on a mobile object. In this case, the sensor installed on the mobile object performs measurements when the mobile object is stopped. When the mobile object is stopped, the position of the sensor installed on the mobile object does not change. Therefore, the sensor installed on the mobile object functions as a fixed sensor. The mobile object may be an airborne vehicle such as a drone, or a landborne vehicle such as a motorcycle or automobile. The mobile object may also be a robot equipped with a means of locomotion. The means of locomotion may be a mechanism for moving on land or a mechanism for moving in the air. In either case, the robot may employ any well-known locomotion mechanism. For example, the means of locomotion may be a bipedal mechanism or a mechanism for walking on three or more legs. The means of locomotion may be a mechanism equipped with wheels or crawlers, or may be any other special mechanism. The robot may also be an airborne vehicle for moving in the air, such as a drone.

[0038] The acquisition unit 11 may acquire data measured by a fixed-point sensor (images generated by a camera or detection results from a position detection sensor such as Lidar) and analyze the data to generate trajectory data. In this example, the data measured by the fixed-point sensor is input to the processing device 10 by any means. The input may be performed by real-time processing or batch processing. For example, the processing device 10 and the fixed-point sensor may be connected so as to be able to communicate with each other. The fixed-point sensor may then transmit the measured data to the processing device 10. Alternatively, the data measured by the fixed-point sensor may be stored in any storage device. The storage device may be provided in the fixed-point sensor or in an external device connected so as to be able to communicate with the fixed-point sensor. The data stored in the storage device may then be input to the processing device 10 by any means and at any timing.

[0039] Alternatively, a calculation device different from the processing device 10 may acquire data measured by a fixed-point sensor, analyze the data, and generate trajectory data. The trajectory data generated by the calculation device may then be input to the processing device 10 by any means. The acquisition of data by the calculation device, the generation of trajectory data, and the input of the trajectory data to the processing device 10 may be performed by real-time processing or batch processing. In this example, the data measured by the fixed-point sensor is input to the calculation device by any means. This input is achieved by means similar to the means for inputting the data measured by the fixed-point sensor to the processing device 10 described above.

[0040] "Extraction process S2 (FIG. 4)" The acquisition unit 11 extracts trajectory data suitable for extracting lane areas from the trajectory data acquired in S1. Specifically, the acquisition unit 11 extracts trajectory data from the trajectory data acquired in S1, the trajectory data of which at least one of length and direction satisfies a predetermined condition. The predetermined condition can be defined based on the "length of the trajectory" or the "angle between the trajectory (e.g., the trajectory of a partial section) and a reference direction," etc. The predetermined condition here may be, for example, one of the following conditions, or a condition in which both are connected by a predetermined logical operator. Note that these predetermined conditions are merely examples and are not limited to these.

[0041] The length of the trajectory is equal to or greater than a threshold value. There is no section where the angle between the trajectory and the reference direction is equal to or greater than a threshold value. The threshold value is a value that is set in advance.

[0042] By using the "length of the trajectory" as a condition, it is possible to extract trajectory data whose trajectory length is equal to or greater than a threshold, and to exclude noise data whose trajectory length is less than the threshold.

[0043] Furthermore, by using the "angle between the trajectory and the reference direction" as a condition, it is possible to extract trajectories in which lane changes did not occur and exclude trajectories in which lane changes occurred. The reference direction is the same as or close to the direction in which the lane extends (the difference in direction is less than or equal to a threshold). A trajectory that has a section where the angle with the reference direction is greater than or equal to a threshold can be considered to be a trajectory in which a lane change occurred.

[0044] The reference direction may be set by an operator. Alternatively, the acquisition unit 11 may calculate the reference direction by statistically processing the trajectory data acquired in S1. Usually, the number of vehicles that change lanes is considered to be smaller than the number of vehicles that do not change lanes. For this reason, the acquisition unit 11 may, for example, take a majority vote on the extension direction of each piece of trajectory data acquired in S1 and determine the most common extension direction as the reference direction.

[0045] The acquisition unit 11 may delete the trajectory data that was not extracted in S2 from the trajectory data acquired in S1.

[0046] "Determination process S3 (FIG. 4)" The acquisition unit 11 determines whether the processes S1 and S2 executed up to that point satisfy a predetermined condition. The acquisition unit 11 then repeats S1 and S2 until the predetermined condition is satisfied. If it is determined that the predetermined condition is satisfied, the process proceeds to S4.

[0047] As described above, the predetermined condition here is a condition for advancing the process to S4. In order to improve the accuracy of estimation from S4 onwards, it is preferable to perform the processes from S4 onwards after a certain amount of trajectory data has been extracted in S2. The predetermined condition has contents that make it possible to realize such a preferred form. The predetermined condition here may be any one of the following, or may be a condition in which two or more of the following are connected by a predetermined logical operator:

[0048] The number of trajectory data (number of vehicle movement trajectories) extracted in S2 up to that point is equal to or greater than a threshold value. The duration of execution of the processes of S1 and S2 is equal to or greater than a threshold value. The number of trajectory data (number of vehicle movement trajectories) passing through at least one area set within the measurement range of the fixed sensor is equal to or greater than a threshold value. Note that the threshold value is a value that is set in advance. The "at least one area set within the measurement range of the fixed sensor" can be set in advance by an operator, for example.

[0049] "Lane Area Estimation Process S4 (Fig. 4)" The lane area estimation process S4 includes four processes (S4-1 to S4-4) shown in Fig. 5. By executing these four processes, the estimation unit 12 estimates a lane area based on the trajectory data extracted in S2 up to that point.

[0050] Specifically, by executing the processes of S4-1 to S4-3, the estimation unit 12 calculates a line indicating the tendency of the vehicle's movement trajectory as a lane reference line based on the trajectory data extracted in S2 up to that point. As described above, the trajectory data in this embodiment indicates the movement trajectory of a point that is approximately the center of the vehicle's left-right direction in the traveling direction and is lowered to the ground. Therefore, the lane reference line is a line that indicates the tendency of the movement trajectory of a point that is approximately the center of the vehicle's left-right direction in the traveling direction and is lowered to the ground.

[0051] The estimation unit 12 then executes the process of S4-4 to estimate the lane area from the lane reference lines calculated in S4-1 to S4-3. Each process will be described in detail below.

[0052] "Calculation of lane candidate points S4-1 (FIG. 5)" First, the estimation unit 12 determines a reference direction within the measurement range of the fixed point sensor. The definition of the reference direction and the method for setting it have been described above, so a description thereof will be omitted here.

[0053] An example is shown in Figure 6. Figure 6 shows an image captured by a camera (fixed-point sensor). Also shown are the movement trajectories VL of multiple vehicles indicated by the trajectory data extracted in S2. The up-down direction of this image is defined as the reference direction. In this embodiment, this reference direction is referred to as the y-axis direction. Note that in the image of Figure 6, the up-down direction of the image is defined as the reference direction (y-axis direction), but depending on the position and orientation of the camera, a diagonal direction or left-right direction of the image may also be defined as the reference direction (y-axis direction).

[0054] Then, the estimation unit 12 determines a plurality of coordinates in the reference direction at predetermined intervals, as shown in FIG. 6. ０ , y １ , y ２ ...) are multiple coordinates of the reference direction determined at predetermined intervals.

[0055] The value of the predetermined interval is a design factor. The estimation unit 12 may determine multiple y coordinates at predetermined intervals across the entire measurement range of the fixed sensor (across the entire image in the case of FIG. 6 ). Alternatively, the estimation unit 12 may determine multiple y coordinates at predetermined intervals in a portion of the measurement range of the fixed sensor. This portion is, for example, an area where the vehicle movement trajectory VL indicated by the trajectory data extracted in S2 exists. In the example of FIG. 6 , the vehicle movement trajectory VL does not extend toward the upper edge of the image. The estimation unit 12 may exclude such an area where the vehicle movement trajectory VL does not extend from the area for which the estimation unit 12 determines multiple y coordinates at predetermined intervals.

[0056] Next, the estimation unit 12 estimates a plurality of y coordinates (y ０ , y １ , y ２ , etc.), the position through which the vehicle movement trajectory VL indicated by the trajectory data extracted in S2 passes. That is, the estimation unit 12 calculates a position through which the vehicle movement trajectory VL indicated by the trajectory data extracted in S2 passes through a plurality of y coordinates (y ０ , y １ , y ２ In each of the points (...), the intersection of a line where y = each y coordinate and the vehicle movement trajectory VL indicated by the trajectory data extracted in S2 is calculated. When an x-axis perpendicular to the y-axis is defined on the image, multiple y-coordinates (y ０ , y １ , y ２ In each of the above, {x １ , x ２ , . . .}, a set of x coordinates indicating positions through which the vehicle movement trajectory VL passes is obtained.

[0057] "Calculation of the number of lanes S4-2 (FIG. 5)" First, the estimation unit 12 calculates a plurality of y coordinates (y ０ , y １ , y ２ ...) to extract the y coordinate used to calculate the number of lanes.

[0058] As shown in FIG. 7, a plurality of y coordinates (y ０ , y １ , y ２In each of the above cases, the number of intersections between the line where y = each y coordinate and the vehicle movement trajectory VL indicated by the trajectory data extracted in S2 (the number of positions through which the vehicle movement trajectory VL passes) may be different. In the example of FIG. 7, the coordinate y ０ The number of intersections in the coordinate y １ The number of intersections is 3, and the coordinate y ２ The number of intersections is 1. In S4-1, a plurality of y coordinates (y ０ , y １ , y ２ The number of elements in the set of x coordinates calculated in each of the above steps is the number of intersections.

[0059] The estimation unit 12 estimates a plurality of y coordinates (y ０ , y １ , y ２ . . ) where the number of intersections is equal to or greater than a threshold is extracted as the y coordinate to be used for calculating the number of lanes.

[0060] The threshold value here may be determined in advance. Alternatively, the threshold value here may be determined based on the number of trajectory data extracted in S2. For example, the estimation unit 12 may set the threshold value here to a value obtained by multiplying the number of trajectory data extracted in S2 by a predetermined coefficient greater than 0 and equal to or less than 1. The predetermined coefficient is a predetermined value.

[0061] By this process, y coordinates that are passed through (intersected with) less by the vehicle movement trajectory VL indicated by the trajectory data extracted in S2 can be excluded from the coordinates used to calculate the number of lanes.

[0062] Next, for each y coordinate extracted in the above process (hereinafter, sometimes simply referred to as an "extracted y coordinate"), the estimation unit 12 can calculate a histogram of the positions (x coordinates) of the intersections between a line, where y = each y coordinate, and the vehicle movement trajectory VL indicated by the trajectory data extracted in S2, as shown in Figure 8(A). The horizontal axis of Figure 8(A) is the x coordinate, and the vertical axis is the number of intersections.

[0063] Furthermore, the estimation unit 12 can calculate a probability density function for the position where the vehicle passes for each extracted y coordinate, as shown in FIG. 8B. For example, the estimation unit 12 can calculate the probability density function using kernel density estimation. The "position where the vehicle passes" is the position (x coordinate) of the intersection between a line where y = each y coordinate and the vehicle movement trajectory VL indicated by the trajectory data extracted in S2. The horizontal axis of FIG. 8B represents the x coordinate, and the vertical axis represents density. Note that the estimation unit 12 can treat all values ​​below the threshold r as "0," as shown in FIG. 9. This can reduce the influence of the passing position (intersection position) of the vehicle movement trajectory VL, which rarely occurs due to lane changes, etc.

[0064] Next, as shown in FIG. 10A, the estimation unit 12 calculates the number of peaks and the positions (x coordinates) of the peaks in the probability density function for each extracted y coordinate. A peak is a position where a maximum value is obtained. In FIG. 10A, two peaks P １ and P ２ The probability density function of FIG. 10A is calculated by the coordinate y ４ 10B is a probability density function relating to the position where the vehicle passes through. In FIG. 10B, two peaks P calculated using the probability density function of FIG. 10A are １ and P ２ This shows the location of.

[0065] After calculating the number of peaks for each extracted y-coordinate, the estimation unit 12 calculates the number of lanes based on the number of peaks for each extracted y-coordinate. For example, the estimation unit 12 takes a majority vote and calculates the number of peaks with the largest number as the number of lanes.

[0066] "Combining S4-3 (FIG. 5)" The estimation unit 12 combines the peak positions for each y coordinate calculated in S4-2 along the y-axis direction to generate lines for the number of lanes calculated in S4-2.

[0067] The estimation unit 12 may determine the peak position for the y coordinate not extracted in S4-2 by the method described in S4-2. The estimation unit 12 may also use the peak position calculated for the y coordinate not extracted in S4-2 as a target for the combining process. That is, the estimation unit 12 may determine the peak position for the y coordinate not extracted in S4-2 by the method described in S4-2. ０ , y １ , y ２ ...), the peak position may be determined using the method described in S4-2, and the peak positions for each y coordinate may be combined along the y-axis direction to generate lines for the number of lanes calculated in S4-2.

[0068] Hereinafter, the line generated in this manner will be referred to as the “lane reference line.” Furthermore, hereinafter, the peak position determined for each y coordinate by the method described in S4-2 may be simply referred to as the “peak position for each y coordinate.”

[0069] There are various methods for combining the peak positions for each y coordinate along the y-axis direction, but one example will be described below with reference to FIGS.

[0070] First, the estimation unit 12 determines the y coordinate to be the start position of the bond. In one example, among the multiple y coordinates with the largest number of peaks, the y coordinate that is located in the middle in the arrangement along the reference direction can be set as the start position of the bond. In the example of FIG. 11, y ｎ , y ｎ＋１ , y ｎ＋２ , y ｎ＋８ , y ｎ＋９ Among these five y coordinates, the coordinate y ｎ＋２ In the example of FIG. ｎ＋２ is determined as the bond start position. By determining the bond start position in this manner, the following effects can be obtained.

[0071] Generally, in many angles of view, the actual number of lanes (the most frequent peaks) is distributed mostly near the center of the image. In other words, the y coordinates with the most frequent peaks tend to be concentrated near the center of the image. For example, if multiple y coordinates at a predetermined interval are used, ０ or y２０ If " is set, y １０ Near, for example, y ８ or y １３ This is because the number of peaks increases in the downward direction of the image (y ０ ) the vehicle will appear larger, and the upward (y ２０ ) the vehicle appears smaller, which reduces the accuracy of vehicle detection.

[0072] In this way, in many cases, the most common peaks are near the center of the image (in the above example, y １０ In rare cases, it may be near the top of the image (for example, y １９ ) or near the bottom (in the above example, for example, y ３ ) may appear isolated in a certain area. If the above-mentioned joining is started from the y-coordinate where such a peak appears isolated, a situation may arise in which it is not possible to find a joining destination for some peaks due to the joining cost. As a result, the number of lines generated by the above-mentioned joining may be less than the actual number of lanes. By determining the starting position of the joining as described above, it is more likely that the y-coordinate in the center of the image, where the peaks are expected to be concentrated, can be used as the starting position. As a result, the above-mentioned inconvenience can be suppressed.

[0073] The y-coordinate of the joining start position may be determined by other methods that can set the y-coordinate of the central part of the image where the most peaks are expected to be concentrated as the starting position. The y-coordinate of the joining start position may also be determined by other methods, although such an effect may not be obtained. Even when other methods are used, the lane area can be estimated with a certain degree of accuracy.

[0074] The estimation unit 12 determines the y-coordinate as the starting position for the joining, and then joins the peak positions for each y-coordinate along the y-axis direction (a direction parallel to the reference direction) to generate lines for the number of lanes calculated in S4-2.

[0075] Specifically, the estimation unit 12 can perform the following processes (1) to (8) in this order: (1) The estimation unit 12 selects one lane as the processing target. (2) The estimation unit 12 selects one peak position as the processing target. (3) The estimation unit 12 extracts, as a combination candidate, a peak position whose cost for combining with the peak position selected in (2) is within the maximum cost. (4) The estimation unit 12 narrows down the combination candidates by extracting combination candidates that satisfy predetermined conditions from the combination candidates extracted in (3). (5) The estimation unit 12 determines one of the combination candidates extracted in (4) as the peak position to be combined with the peak position selected as the processing target in (2). (6) The estimation unit 12 combines the peak positions to generate (extend) a line. (7) The estimation unit 12 returns to (1), selects a new lane as the processing target, and executes (2) to (6). (8) After (7), the estimation unit 12 returns to (1), selects a new lane as the processing target, and executes (2) to (6).

[0076] Each process will be described in detail below.

[0077] (1) "The estimation unit 12 selects one lane as the processing target." If the number of lanes calculated in S4-2 is Q, the estimation unit 12 selects one of the Q lanes as the processing target. Then, the estimation unit 12 generates a lane reference line corresponding to the selected lane by the following process.

[0078] (2) "The estimation unit 12 selects one peak position to be processed." When performing the combination of peak positions for the first time in correspondence with the lane selected in (1), one of the peak positions existing at the y coordinate as the starting position of the combination is selected by any means and made the peak position the processing target.

[0079] As will be described below, the processes from (2) onward are repeatedly executed. Therefore, there are cases where the process of combining peak positions corresponding to the lanes selected in (1) is executed at least once, and a line is generated, and then (2) is executed. In this case, the estimation unit 12 selects the peak position at the beginning of the line generated up to that point as the processing target.

[0080] The "peak position at the beginning of a line" is the beginning (end) of a line generated by combining the peak positions up to that point. In FIG. 11, when peak positions are combined to extend the line downward in the figure, the lower end of the line becomes the peak position at the beginning of the line. On the other hand, in FIG. 11, when peak positions are combined to extend the line upward in the figure, the upper end of the line becomes the peak position at the beginning of the line. Note that before the combining begins (before the line is generated), the peak position at the y coordinate that is the starting position of the combining is treated as the peak position at the beginning of the line.

[0081] After selecting one peak position to be processed, the estimation unit 12 calculates the maximum cost that can be combined for the selected peak positions based on the following formulas (1) and (2). Formula (1) is a formula for calculating the cost for combining two peak positions, and formula (2) is a formula for calculating the maximum cost.

[0082]

[0083]

[0084] (3) "The estimation unit 12 extracts, as a combination candidate, a peak position whose cost for combining with the peak position selected in (2) is within the maximum cost." This process will be described with reference to FIG. 12. １ is the peak position selected in (2). Figure 12 shows the line (P ０ and P １ The line connecting the two points is generated.

[0085] In this case, P １ is the peak position at the beginning of the line, which is the peak position selected in (2) as described above. ０ is the position of the peak immediately before that. The maximum combinable cost (max_cost in equation (2)) obtained in (2) above shows the combinable range in FIG. 12. In FIG. 12, P selected in (2) １ When the peak position where the cost to combine with P is within the maximum cost is extracted, ２ and P ３As shown in equation (2), the peak position P １ and the previous peak position P ０ By adding a constant to the cost of combining and determining the maximum cost, the range of possible combinations can be narrowed as the line extends.

[0086] (4) "The estimation unit 12 narrows down the combination candidates by extracting combination candidates that satisfy a predetermined condition from the combination candidates extracted in (3)." The predetermined condition here may be one of the following conditions, or may be a condition in which both are connected by a predetermined logical operator. Note that these predetermined conditions are merely examples and are not limited to these. - The peak position is not already combined with another peak position. - The cost of combining with the peak position at the beginning of the line corresponding to the lane being processed is smaller than the cost of combining with the peak position at the beginning of the line corresponding to another lane.

[0087] (5) "The estimation unit 12 determines one of the bond candidates extracted in (4) as the peak position to be combined with the peak position of the processing target selected in (2)." The estimation unit 12 determines, for example, from the bond candidates extracted in (4), the bond candidate that has the smallest cost for combining with the peak position of the processing target selected in (2) as the peak position to be combined with the peak position of the processing target selected in (2).

[0088] (6) "The estimation unit 12 generates (extends) a line by joining the peak positions." The estimation unit 12 joins the peak position to be processed selected in (2) with the joining candidate determined in (5) to generate (extend) a line. Then, the estimation unit 12 sets the joining candidate determined in (5) as the leading peak position of the line.

[0089] (7) "The estimation unit 12 returns to (1), selects a new lane as the processing target, and executes (2) to (6)." If the number of lanes calculated in S4-2 is Q, the estimation unit 12 selects all of the Q lanes and repeats the process until it executes (2) to (6) for all lanes. The estimation unit 12 selects a new lane as the processing target from among the lanes that have not yet been selected. In this way, it is possible to proceed with the joining for all lanes in parallel. Then, after executing (2) to (6) for all lanes, the estimation unit 12 proceeds to (8).

[0090] (8) "After (7), the estimation unit 12 returns to (1), selects a new lane as the processing target, and executes (2) to (6)." After selecting all lanes as the processing target in (7), the estimation unit 12 executes the process of (8). In such (8), the estimation unit 12 resets the lane selection history in (7) immediately before (8) and selects a new lane from scratch. That is, the estimation unit 12 may newly select a lane that was previously selected in (7). The estimation unit 12 repeats the processes of (1) to (7) until the leading positions of all lanes are no longer updated.

[0091] Here, the characteristics of the combination in the above processes (1) to (8) will be explained using FIG. 13. As shown in FIG. 13, two or more peak positions are not combined to one peak position. The two peak positions with the smallest combination cost are combined. Furthermore, peak positions that are separated by a distance in the y-axis direction by a predetermined value or more can also be combined. That is, as shown in the figure, ｎ＋１ The peak position and y ｎ＋２ Not only the peak positions of adjacent y coordinates such as the peak positions of ｎ＋１ The peak position and y ｎ＋３ Peak positions on the y coordinate that are separated by a predetermined value or more may be directly combined, such as the peak positions in (a) and (b). Also, even if the number of lanes and the peaks are the same, they will not be combined if the combining cost is high.

[0092] The estimation unit 12 may perform a process of removing noise data after generating a lane reference line for each lane by the processes (1) to (8) above. For example, the estimation unit 12 may delete lane reference lines that satisfy conditions such as a length that is equal to or less than a threshold or a length in the y-axis direction that is equal to or less than a threshold.

[0093] As shown in FIG. 14, the three lane reference lines EL １ ~EL ３ By the removal process, the estimation unit 12 generates the lane reference line EL ３ Then, as shown in FIG. 15, the lane reference line EL after removing the noise data is １ and EL ２ can be output as the processing result.

[0094] For example, there is a possibility that a larger number of lane reference lines than the actual number of lanes may be generated due to some malfunction. When a larger number of lane reference lines than the actual number of lanes are generated by this process, noise data can be removed.

[0095] "Lane Area Estimation S4-4 (FIG. 5)" The estimation unit 12 estimates the lane area based on the lane reference lines generated in S4-3. Specifically, the estimation unit 12 determines the boundary in the width direction of the lane based on the lane reference lines. This processing by the estimation unit 12 will be explained using FIG. 16. In S4-3, the two lane reference lines EL １ and EL ２ is generated.

[0096] When a plurality of lane reference lines are detected, the estimation unit 12 estimates that the position where the adjacent lane reference lines are divided at a predetermined ratio is the boundary between the adjacent lanes.

[0097] First, the estimation unit 12 can identify a location where an adjacent lane reference line exists. The location can be identified by various methods. For example, the estimation unit 12 can identify a location where an adjacent lane reference line EL １The estimation unit 12 then generates a line by extending any point in the line perpendicular to the reference direction. The estimation unit 12 then identifies a location where another lane reference line intersects with the line as a location where an adjacent lane reference line exists.

[0098] Next, the estimation unit 12 estimates the position at the identified location where the adjacent lane reference lines are divided by a predetermined ratio as the boundary of the adjacent lanes. The predetermined ratio is determined according to the setting of the "vehicle reference point" described above. The vehicle reference point in this embodiment is the point where the "approximate center" in the left-right direction of the vehicle is lowered to the ground in the direction of travel of the vehicle. In other words, it is the point where the width direction of the vehicle is divided at a 1:1 ratio. In this embodiment, the predetermined ratio is 1:1. That is, the estimation unit 12 estimates the midpoint between the adjacent lane reference lines as the boundary of the adjacent lanes, as shown in FIG. 16. By this processing, the boundary BL shown in the figure is obtained. １ The reference point of the vehicle is determined as E １ Against E ２ When the predetermined ratio is E ２ Against E １ is.

[0099] In addition, in a location where there is no adjacent lane reference line, the estimation unit 12 can estimate the boundary using, for example, the following method.

[0100] First, the boundary BL in FIG. ２ and BL ３ The process of estimating the lane reference line EL will be described. ２ From the perspective of the boundary BL ２ In this case, the estimation unit 12 determines whether there is another lane reference line on the side of the lane reference line EL. ２ The boundary BL on the opposite side １ Based on the distance to the boundary BL ２ can be estimated.

[0101] That is, the estimation unit 12 estimates the lane reference line EL ２ From the boundary BL １ The estimation unit 12 then calculates the distance D to the lane reference line EL ２ From the boundary BL １ The position at a distance D in the opposite direction is the boundary BL ２Note that due to the perspective effect, the distance D may differ for each coordinate in the reference direction on the image. For this reason, the distance D is calculated for each coordinate in the reference direction, and the boundary BL is estimated using the distance D for each coordinate in the reference direction. ２ It is preferable to calculate the boundary BL ３ is the boundary BL ２ It can be estimated using the same method.

[0102] Next, the boundary BL in FIG. ４ and BL ５ The process of estimating the lane reference line EL will be described. ２ From the perspective of the boundary BL ４ There are no other lane reference lines on the side of the lane reference line EL. ２ From the perspective of the boundary BL ４ In such a case, the estimation unit 12 uses the distance D calculated at another coordinate in the reference direction to estimate the boundary BL ４ That is, the estimation unit 12 estimates the lane reference line EL ２ From the perspective of the boundary BL ４ The position at a distance D in the direction of ４ As described above, the distance D may differ for each coordinate in the reference direction. ４ The boundary BL is calculated by using the distance D calculated at the point closest to the point where the boundary BL is to be estimated (the point closest in the y direction). ４ It is preferable to estimate the boundary BL ５ is the boundary BL ４ It can be estimated using the same method.

[0103] "Lane Area Confirmation Process S5 (FIG. 4)" The estimation unit 12 determines whether to confirm the lane area based on the estimation result of S4. If not, the process returns to S1 and the same process is repeated. By performing the process of S5, the estimation result of S4 is not confirmed as the lane area as it is, but rather, once a reliable estimation result is obtained in S4, the estimation result can be confirmed as the lane area. As a result, it becomes possible to output a highly reliable result.

[0104] As described above, the acquisition unit 11 continues to acquire trajectory data, and the estimation unit 12 repeatedly executes estimation of lane areas based on the accumulated trajectory data.

[0105] After estimating a lane area in the nth processing, the estimation unit 12 determines whether each estimated lane area is identical to the lane area estimated in the (n-1)th processing. Specifically, the estimation unit 12 determines the difference between each lane area estimated in the nth processing and each lane area estimated in the (n-1)th processing. If the difference between the lane area estimated in the (n-1)th processing and the lane area estimated in the nth processing is equal to or smaller than a reference value, the estimation unit 12 determines that the lane area estimated in the (n-1)th processing and the lane area estimated in the nth processing are identical.

[0106] The estimation unit 12 performs the above-mentioned identity determination each time a lane area is estimated in each processing. If the lane areas are determined to be identical a predetermined number of times or more in succession, the estimation unit 12 confirms the lane areas estimated during that time as the estimation result. In other words, the estimation unit 12 confirms the lane areas that have been determined to be identical a predetermined number of times or more in succession.

[0107] For example, the estimation unit 12 can execute, as S5, three processes (S5-1 to S5-3) shown in Fig. 17. Note that the estimation unit 12 may determine the identity of the lane areas described above using a method different from the process described below.

[0108] "Comparison with previous lane area S5-1 (FIG. 17)" The estimation unit 12 determines whether the lane area estimated in the (n-1)th processing is the same as the lane area estimated in the nth processing. If the difference between the lane area estimated in the (n-1)th processing and the lane area estimated in the nth processing is equal to or less than a predetermined standard, the estimation unit 12 determines that the lane area estimated in the (n-1)th processing is the same as the lane area estimated in the nth processing. There are various methods for calculating the difference between two lane areas, but one example will be described below. Hereinafter, the lane area estimated in the (n-1)th processing may be referred to as the "(n-1)th lane area," and the lane area estimated in the nth processing may be referred to as the "nth lane area."

[0109] First, the estimation unit 12 calculates a three-dimensional probability density function based on the constituent points on the lane reference line used to estimate the (n-1)th lane area. For example, the estimation unit 12 can calculate a three-dimensional probability density function as shown in FIG. 18 by kernel density estimation (KDE).

[0110] The estimation unit 12 generates data for generating a three-dimensional probability density function from data on the group of constituent points on the lane reference line used to estimate the (n-1)th lane area, and can calculate the three-dimensional probability density function based on the generated data for generating the three-dimensional probability density function.

[0111] The data for generating the three-dimensional probability density function is random data that approximates the data of the group of constituent points on the lane reference line used to estimate the (n-1)th lane area. Such data for generating the three-dimensional probability density function can be generated by various methods. For example, the estimation unit 12 generates the i-th data (x ｉ ,y i ) as a processing target. Then, the estimation unit 12 randomly generates data that is similar to the selected data based on the following equation (3).

[0112]

[0113] b and α are predetermined values. b is the coordinate y when a vehicle is detected by image analysis, for example. i Alternatively, the variance range may be calculated from the average width of the base of a rectangular area (area where a vehicle is present) detected in the vicinity of the vehicle.

[0114] The estimation unit 12 performs this process for all i and generates a set of randomly generated data corresponding to each piece of data on the constituent points on the lane reference line used to estimate the (n-1)th lane area. This set of data becomes data on the constituent points that approximates the data on the constituent points on the lane reference line used to estimate the (n-1)th lane area. The estimation unit 12 repeats this process a predetermined number of times (e.g., 100 times) to generate a predetermined number of pieces of data on the constituent points that approximates the data on the lane reference line used to estimate the (n-1)th lane area. Then, the estimation unit 12 calculates a three-dimensional probability density function based on the data for the predetermined number of times.

[0115] The estimation unit 12 generates data for generating a three-dimensional probability density function from data on the group of constituent points on the lane reference line used to estimate the (n-1)th lane area, for example, as described above. Then, the estimation unit 12 inputs each of the constituent points on the lane reference line used to estimate the nth lane area into the three-dimensional probability density function, and calculates the probability (occurrence rate) of the value input for each constituent point.

[0116] If the number of component points whose calculated probabilities are equal to or greater than the threshold is equal to or greater than a reference value, the estimation unit 12 determines that the (n-1)th lane area and the nth lane area are the same. On the other hand, if the number of component points whose calculated probabilities are equal to or greater than the threshold is less than the reference value, the estimation unit 12 determines that the (n-1)th lane area and the nth lane area are different. The threshold and reference value here are predetermined values.

[0117] "Lane area determination S5-2 (Figure 17)" When the estimation unit 12 determines that the (n-1)th lane area and the nth lane area are the same for a predetermined number of consecutive times or more, it determines the lane area estimated during that time as the estimation result.

[0118] In the condition "when the (n-1)th lane area and the nth lane area are judged to be the same for a predetermined number of consecutive times or more," "judged to be the same" may also be "judged to be the same in S5-1."

[0119] As another example, the estimation unit 12 may perform a more detailed determination of identity in addition to the determination of identity in S5-1. In addition, the "determined to be identical" in the condition "when the (n-1)th lane area and the nth lane area are determined to be identical for a predetermined number of consecutive times or more" may be "determined to be identical in this detailed determination of identity" or "determined to be identical in both this detailed determination of identity and in S5-1."

[0120] The estimation unit 12 can perform a detailed determination of identity based on, for example, the following formula (4): If the condition of the following formula (4) is satisfied, the estimation unit 12 determines that the (n-1)th lane area and the nth lane area are identical.

[0121]

[0122] NewLane x ｉ is the x-coordinate of the i-th point on the lane reference line used to estimate the n-th lane area. ｉ is the i-th x coordinate in the group of constituent points on the lane reference line used to estimate the (n-1)-th lane area. ｉ and OldLane x ｉ are the same y-coordinate (y ｉ ) is the x-coordinate corresponding to

[0123] ImageShape X is the width of the image in the x-axis direction, and M is a predetermined value, such as 0.9.

[0124] "Post-processing S5-3 (FIG. 17)" When the estimation unit 12 determines one lane area in S5-2, it deletes the trajectory data used to estimate the determined lane area from the trajectory data extracted in S2. The trajectory data used to estimate the lane area is the trajectory data extracted in S2 before the lane area is determined and used to calculate the lane reference line of the lane area in S4.

[0125] When there are multiple lanes in the road, multiple lane areas may be determined simultaneously, or the lane areas may be determined one by one. When the lane areas are determined one by one, steps S1 to S5 in FIG. 4 are repeated after one lane area is determined until another lane area is determined.

[0126] Under these assumptions, if the above deletion is performed and there is no trajectory data related to one lane area (a state in which it is not included in the extraction result of S2), an incorrect result may be calculated when calculating the probability density function of S4-2 based on the trajectory data extracted in S2.

[0127] Therefore, to prevent this problem, the estimation unit 12 can delete trajectory data related to one lane area and then create dummy data related to that lane area. Thereafter, the estimation unit 12 can calculate the probability density function in S4-2 using the dummy data related to that lane area and the accumulated trajectory data related to other lane areas (trajectory data extracted in S2). The estimation unit 12 can create the dummy data, for example, using the following method.

[0128] The dummy data is random data that approximates the data of the group of constituent points on the lane reference line used to estimate the confirmed lane area. Such dummy data can be generated by various methods. For example, the estimation unit 12 generates the i-th data (x ｉ ,y i ) as a processing target. Then, the estimation unit 12 randomly generates data that is approximate to the selected data based on the following equation (5).

[0129]

[0130] β is a predetermined value. The estimation unit 12 performs this process for all i to generate a set of randomly generated data corresponding to each piece of data on the constituent point group on the lane reference line used to estimate the confirmed lane area. This set of data becomes data on the constituent point group that is approximate to the data on the constituent point group on the lane reference line used to estimate the confirmed lane area. The estimation unit 12 repeats this process a predetermined number of times (e.g., 10 times) to generate dummy data for the predetermined number of times.

[0131] "Effects" The processing device 10 of this embodiment achieves the same effects as the processing device 10 of the first embodiment. In addition, the processing device 10 of this embodiment, which estimates lane areas using the characteristic processing described above, can estimate lane areas with higher accuracy.

[0132] <<Third Embodiment>> The processing device 10 of this embodiment acquires vehicle position data instead of vehicle trajectory data. The processing device 10 then estimates lane areas based on the vehicle position data. This will be described in detail below.

[0133] In this embodiment, the acquisition unit 11 and the estimation unit 12 execute the processes of S1, S4, and S5 in FIG. 19. The acquisition unit 11 executes S1. Then, the estimation unit 12 executes S4 and S5. The configuration of this embodiment differs from the second embodiment in that it does not have S2 and S3. An overview of each process is as follows.

[0134] Data acquisition process S1: The acquisition unit 11 acquires position data. Lane area estimation process S4: The estimation unit 12 estimates the lane area based on the position data acquired in S1 up to that point. Lane area confirmation process S5: The estimation unit 12 determines whether the lane area is confirmed by the estimation result of S4. If not confirmed, the process returns to S1 and the same process is repeated. By performing the process of S5, the estimation result of S4 is not confirmed as the lane area as it is, but rather the estimation result can be confirmed as the lane area once a reliable estimation result is obtained in S4. As a result, it is possible to output a highly reliable result.

[0135] "Data Acquisition Process S1 (FIG. 19)" The acquisition unit 11 acquires vehicle position data measured by fixed point sensors. The concepts of the vehicle and fixed point sensors are as explained in the second embodiment.

[0136] The "position data" indicates the position of the vehicle within the measurement range of the fixed sensor. More specifically, the position data indicates the position of the reference point of the vehicle. In this embodiment, the reference point of the vehicle is the point where the approximate center in the left-right direction of the vehicle is lowered to the ground in the direction of travel of the vehicle. As explained in the second embodiment, other points on the vehicle may also be used as the reference point.

[0137] The position data may be data indicating the position of the vehicle using coordinates in a two-dimensional coordinate system set in an image generated by a camera (fixed-point sensor), or may be data indicating the position of the vehicle in a three-dimensional space (coordinates in a three-dimensional coordinate system) detected by a position detection sensor (such as Lidar).

[0138] Position data indicating the position of a vehicle is generated based on data measured by a fixed-point sensor (images or detection results from a position detection sensor such as Lidar). The generation of position data based on data measured by a fixed-point sensor can be achieved using any well-known technology. In one example, a vehicle and a reference point for the vehicle are detected for each image using image analysis. Then, the position of the reference point for the vehicle detected for each image is calculated within the image to generate vehicle position data. Vehicles may be detected using technologies such as deep learning. Furthermore, the reference point for the vehicle may be detected using features of the vehicle's appearance, or by other methods. For example, the center of the base of a rectangular area in which the vehicle detected in the image is captured may be used as a simple reference point for the vehicle. It is not necessary to link the position data of the same vehicle across multiple images; the position of the reference point for the vehicle may simply be detected for each image and the results may be accumulated. The example shown here is merely an example, and is not limiting.

[0139] The acquisition unit 11 may acquire data measured by a fixed-point sensor (images generated by a camera or detection results from a position detection sensor such as Lidar) and analyze the data to generate position data. In this example, the data measured by the fixed-point sensor is input to the processing device 10 by any means. The input may be performed by real-time processing or batch processing. For example, the processing device 10 and the fixed-point sensor may be connected so as to be able to communicate with each other. The fixed-point sensor may then transmit the measured data to the processing device 10. Alternatively, the data measured by the fixed-point sensor may be stored in any storage device. The storage device may be provided in the fixed-point sensor or in an external device connected so as to be able to communicate with the fixed-point sensor. The data stored in the storage device may then be input to the processing device 10 by any means and at any timing.

[0140] Alternatively, a calculation device different from the processing device 10 may acquire data measured by a fixed-point sensor, analyze the data, and generate position data. The position data generated by the calculation device may then be input to the processing device 10 by any means. The acquisition of data by the calculation device, the generation of position data, and the input of the position data to the processing device 10 may be performed by real-time processing or batch processing. In this example, the data measured by the fixed-point sensor is input to the calculation device by any means. This input is achieved by means similar to the means for inputting the data measured by the fixed-point sensor to the processing device 10 described above.

[0141] "Lane Area Estimation Process S4 (FIG. 19)" The estimation unit 12 estimates the lane area based on the vehicle position data. The estimation unit 12 estimates the lane area based on a plurality of y coordinates (y ０ , y １ , y ２ The method of generating the probability density function for each of the points (...) differs from that of the second embodiment. The processing after generating the probability density function is the same as that of the second embodiment. The method of generating the probability density function of this embodiment will be described below.

[0142] The estimation unit 12 calculates a three-dimensional probability density function based on the vehicle position data as shown in Fig. 20(A) . For example, the estimation unit 12 can calculate the three-dimensional probability density function as shown in Fig. 20(A) by kernel density estimation.

[0143] This three-dimensional probability density function is used to calculate multiple y coordinates (y ０ , y １ , y ２ , . . . ), the probability of occurrence of vehicle position data at each position (x coordinate) in the width direction of the lane is indicated. ０ , y １ , y ２ . . ), the peak positions and the number of peaks of such a three-dimensional probability density function are calculated (FIG. 20B).

[0144] Other configurations of the processing apparatus 10 of this embodiment are similar to those of the first and second embodiments.

[0145] The processing device 10 of this embodiment achieves the same effects as those of the first and second embodiments. Furthermore, the processing device 10 of this embodiment can estimate lane areas using vehicle position data, rather than vehicle trajectory data. Thus, the processing device 10 of this embodiment can estimate lane areas based on data measured by fixed-point sensors using a new method that is partially different from that of the second embodiment.

[0146] <<Modifications>> <Modification 1> The estimation unit 12 estimates the “lane reference line” using the methods of the first to third embodiments (using a plurality of y coordinates at a predetermined interval (y ０ , y １ , y ２ The calculation may be performed using a method different from the method using a probability density function for each lane (...). For example, the estimation unit 12 may calculate the lane reference line as a line indicated by data obtained by averaging the trajectory data. If it is expected that the trajectory data to be processed includes trajectory data for multiple lanes, the estimation unit 12 may group the trajectory data based on positions (x-coordinate positions) close to each other and calculate the lane reference line by averaging each group. The grouping is achieved using a clustering technique or the like.

[0147] <Variation 2> The estimation unit 12 may identify the traveling direction of the vehicle for each estimated lane area. For example, the estimation unit 12 identifies the direction in which the vehicle detected in each estimated lane area moves over time based on data measured by a fixed-point sensor. Then, the estimation unit 12 sets the identified direction as the traveling direction of the vehicle in each lane area.

[0148] <Variation 3> The processing device 10 can perform various processes based on the estimation results of the estimation unit 12. For example, the processing device 10 can detect targets such as vehicles that frequently change lanes or vehicles that travel across multiple lanes based on the estimation results of the estimation unit 12 and data measured by fixed-point sensors. Furthermore, the processing device 10 can measure the traffic conditions (e.g., congestion) and traffic volume for each lane based on the estimation results of the estimation unit 12 and data measured by fixed-point sensors. These measurements can be realized using any widely known technology.

[0149] <Variation 4> The processing device 10 can output the estimation result of the estimation unit 12. For example, the processing device 10 may output an image in which information indicating the lane area estimated by the estimation unit 12 is superimposed on an image generated by a camera (fixed-point sensor). Alternatively, the processing device 10 may output an image in which information indicating the lane area estimated by the estimation unit 12 is superimposed on an image indicating the position of an object in three-dimensional space detected by a position detection sensor (fixed-point sensor) such as Lidar. The processing device 10 may output the image in which information indicating the lane area estimated by the estimation unit 12 is superimposed via an output device such as a display or a projection device. Alternatively, the processing device 10 may transmit the image in which information indicating the lane area estimated by the estimation unit 12 is superimposed to an external device.

[0150] <Modification 5> The processing device 10 may perform processing to remove motorcycle data (trajectory data / position data) from the acquired vehicle data (trajectory data / position data). The processing device 10 may then estimate lane areas based on the vehicle data (trajectory data / position data) from which the motorcycle data (trajectory data / position data) has been removed. Examples of motorcycles include motorcycles, electric kick scooters, and bicycles.

[0151] As described above, the processing device 10 estimates lane areas based on the vehicle's travel trajectory and position while traveling. However, the travel trajectory and position while traveling of a motorcycle tend to differ from the travel trajectory and position while traveling of a four-wheeled vehicle or a large vehicle. Specifically, four-wheeled vehicles and large vehicles, which are wider than a motorcycle, tend to travel approximately in the center of the lane. However, because a motorcycle has a wider range of travelable positions, it tends to travel at the edge of the lane or approximately in the center of the lane. Removing such motorcycle trajectory data and position data improves the accuracy of lane area estimation. Whether acquired vehicle data (trajectory data / position data) is motorcycle data can be determined by processing (e.g., image analysis) based on data generated by a fixed-point sensor. This determination processing may be performed by the processing device 10 or another device.

[0152] <Modification 6> The processing device 10 may extract data (trajectory data / position data) of a predetermined vehicle from the acquired vehicle data (trajectory data / position data). Then, the processing device 10 may estimate a lane area based on the extracted vehicle data (trajectory data / position data).

[0153] The predetermined vehicle may be a vehicle for which a dedicated lane or a priority lane exists. For example, the predetermined vehicle may be a bus, a large vehicle, a taxi, a bicycle, etc. Data (trajectory data / position data) of such a predetermined vehicle is suitable for estimating the dedicated lane or priority lane for each vehicle. By extracting data (trajectory data / position data) of such a predetermined vehicle and estimating lane areas based on the extracted data (trajectory data / position data) of the predetermined vehicle, the dedicated lane or priority lane for each vehicle can be estimated with high accuracy.

[0154] Whether the acquired vehicle data (trajectory data / position data) is data of a specific vehicle can be determined by processing (image analysis, etc.) based on data generated by a fixed-point sensor. The determination processing may be performed by the processing device 10 or another device.

[0155] <<Usage Scenarios>> The processing device 10 described in the first to third embodiments and the modified examples can be used in various situations. An example will be described below. Note that the examples given here are merely examples, and the usage scenarios of the processing device 10 are not limited to the examples given here.

[0156] In one example, the processing device 10 can notify a vehicle of the lane area estimation result. The processing device 10 can notify a vehicle traveling on a road of the lane area estimation result via any communication means, such as road-to-vehicle communication. The vehicle can use the lane area estimation result received from the processing device 10 to control autonomous driving.

[0157] In another example, the processing device 10 can estimate each parking space (lane area) based on vehicle trajectory data or position data measured by fixed sensors installed in the parking lot. The processing device 10 can then notify vehicles located in the parking lot of the estimated results of each parking space (lane area) via any communication means, such as road-to-vehicle communication. The vehicles can use the estimated results of each parking space (lane area) received from the processing device 10 to control automatic parking. Alternatively, the vehicle may notify the driver of the estimated results of each parking space (lane area) via an output device such as a display. In parking lots, the white lines separating parking spaces may fade and become difficult to see due to aging or other reasons. By using the processing device 10 of this embodiment, the inconvenience in such cases can be alleviated.

[0158] In another example, when the processing device 10 detects an accident or the like within the monitoring range of the fixed sensor, the processing device 10 can notify a predetermined notification destination, including information about the accident lane. Also, in a modified example in which the direction of the traveling lane is additionally detected, the processing device 10 can detect a wrong-way driving vehicle and a lane where there is a risk of wrong-way driving, and notify the detected vehicle and its location information.

[0159] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0160] In addition, in the flowcharts used in the above description, multiple steps (processes) are described in order. However, the order of the steps performed in each embodiment is not limited to the order described. In each embodiment, the order of the steps shown in the drawings can be changed as long as it does not cause any problems in terms of the content.

[0161] Some or all of the above embodiments can be described as, but are not limited to, the following supplementary notes. 1. A processing device having: acquisition means for acquiring vehicle trajectory data or position data measured by a fixed-point sensor; and estimation means for estimating lane areas based on the trajectory data or the position data. 2. The processing device according to 1, wherein the estimation means calculates a probability density function based on the trajectory data or the position data, and estimates lane areas based on the probability density function. 3. The processing device according to 1 or 2, wherein the estimation means calculates a probability density function relating to a position where the vehicle will pass at each of a plurality of coordinates in a reference direction, combines peak positions of the probability density function calculated at each of the plurality of coordinates in the reference direction to generate lane reference lines, and estimates lane areas based on the lane reference lines. 4. The processing device according to 3, wherein the estimation means calculates the number of lanes based on the number of peaks of the probability density function calculated at each of the plurality of coordinates in the reference direction, and calculates the lane reference lines for the calculated number of lanes. 5. 6. The processing device according to any one of 1 to 5, wherein the estimation means, when a plurality of lane reference lines have been detected, estimates a position that divides the reference lines of adjacent lanes at a predetermined ratio as a boundary between the adjacent lanes. 7. The processing device according to any one of 1 to 5, wherein the acquisition means continues to acquire the trajectory data or the position data, and the estimation means repeatedly estimates a lane area based on the accumulated trajectory data or the position data, and when a difference between the lane area estimated in the (n-1)th processing and the lane area estimated in the nth processing is equal to or less than a reference value, determines that the lane area estimated in the (n-1)th processing and the lane area estimated in the nth processing are the same, and when they are determined to be the same a predetermined number of times or more consecutively, confirms the lane area estimated during that time as the estimation result.7. The processing device according to 2, wherein the estimation means, when there are multiple lanes for which lane areas are to be estimated, confirms the estimation result of the lane area for one lane, deletes the accumulated trajectory data or position data for that lane, creates dummy data for the trajectory data or position data for that lane, and thereafter calculates the probability density function using the dummy data for that lane and the accumulated trajectory data or position data for other lanes. 8. The processing device according to any of 1 to 7, wherein the acquisition means acquires trajectory data of which at least one of length and direction satisfies a predetermined condition. 9. A processing method in which one or more computers acquire trajectory data or position data of a vehicle measured by a fixed sensor, and estimates a lane area based on the trajectory data or the position data. 10. A program that causes a computer to function as: acquisition means for acquiring trajectory data or position data of a vehicle measured by a fixed sensor; and estimation means for estimating a lane area based on the trajectory data or the position data.

[0162] Some or all of Supplements 2 to 8 that are dependent on the processing device of Supplement 1 described above may also be dependent on the processing method of Supplement 9 and the program of Supplement 10 in the same dependent relationship as Supplement 1 and Supplements 2 to 8. Furthermore, within the scope of each of the above-described embodiments, some or all of the configurations described as Supplements can be realized in various hardware, software, various recording means for recording software, or systems.

[0163] This application claims priority based on Japanese Patent Application No. 2024-010838, filed January 29, 2024, the disclosure of which is incorporated herein in its entirety by reference.

[0164] 10 Processing device 11 Acquisition unit 12 Estimation unit 1A Processor 2A Memory 3A Input / output I / F 4A Peripheral circuit 5A Bus

Claims

1. A processing device having: an acquisition means for acquiring vehicle trajectory data or position data measured by a fixed-point sensor; and an estimation means for estimating a lane area based on the trajectory data or the position data.

2. The processing device according to claim 1, wherein the estimation means calculates a probability density function based on the trajectory data or the position data, and estimates a lane area based on the probability density function.

3. A processing device as described in claim 1 or 2, wherein the estimation means calculates a probability density function relating to the position where the vehicle will pass at each of multiple coordinates in the reference direction, combines the peak positions of the probability density function calculated at each of the multiple coordinates in the reference direction to generate a lane reference line, and estimates the lane area based on the lane reference line.

4. The processing device according to claim 3, wherein the estimation means calculates the number of lanes based on the number of peaks of the probability density function calculated at each of the multiple coordinates in the reference direction, and calculates the reference lines of the lanes for the calculated number of lanes.

5. A processing device as described in claim 4, wherein the estimation means, when detecting a plurality of lane reference lines, estimates the position where the reference lines of adjacent lanes are divided at a predetermined ratio as the boundary between the adjacent lanes.

6. A processing device according to any one of claims 1 to 5, wherein the acquisition means continues to acquire the trajectory data or the position data, the estimation means repeatedly estimates the lane area based on the accumulated trajectory data or the position data, and if the difference between the lane area estimated in the (n-1)th processing and the lane area estimated in the nth processing is equal to or less than a reference value, determines that the lane area estimated in the (n-1)th processing and the lane area estimated in the nth processing are the same, and if they are determined to be the same for a predetermined number of consecutive times or more, confirms the lane area estimated during that time as the estimation result.

7. The processing device of claim 2, wherein when there are multiple lanes for which lane areas are to be estimated, the estimation means, upon confirming the estimation result of the lane area for one lane, deletes the accumulated trajectory data or position data for that lane, creates dummy data for the trajectory data or position data for that lane, and thereafter calculates the probability density function using the dummy data for that lane and the accumulated trajectory data or position data for other lanes.

8. A processing device according to any one of claims 1 to 7, wherein the acquisition means acquires the trajectory data in which at least one of the length and direction satisfies a predetermined condition.

9. A processing method in which one or more computers acquire vehicle trajectory data or position data measured by fixed sensors, and estimate lane areas based on the trajectory data or the position data.

10. The processing method according to claim 9, wherein the estimation of the lane area comprises calculating a probability density function based on the trajectory data or the position data, and estimating the lane area based on the probability density function.

11. A processing method as described in claim 9 or 10, wherein the estimation of the lane area comprises calculating a probability density function relating to the position where the vehicle will pass at each of a plurality of coordinates in a reference direction, combining the peak positions of the probability density functions calculated at each of the plurality of coordinates in the reference direction to generate a lane reference line, and estimating the lane area based on the lane reference line.

12. A processing method according to claim 11, wherein in estimating the lane area, the number of lanes is calculated based on the number of peaks of the probability density function calculated at each of the multiple coordinates in the reference direction, and reference lines for the lanes are calculated for the calculated number of lanes.

13. A processing method as described in claim 12, wherein, in estimating the lane area, if multiple reference lines of the lane are detected, the position dividing the reference lines of adjacent lanes at a predetermined ratio is estimated to be the boundary of the adjacent lanes.

14. A processing method according to any one of claims 9 to 13, further comprising: continuing to acquire the trajectory data or the position data; and repeatedly estimating the lane area based on the accumulated trajectory data or the position data; if the difference between the lane area estimated in the (n-1)th processing and the lane area estimated in the nth processing is equal to or less than a reference value, determining that the lane area estimated in the (n-1)th processing and the lane area estimated in the nth processing are the same; and if they are determined to be the same for a predetermined number of consecutive times or more, confirming the lane area estimated during that time as the estimation result.

15. A recording medium having recorded thereon a program that causes a computer to function as: an acquisition means that acquires vehicle trajectory data or position data measured by a fixed-point sensor; and an estimation means that estimates lane areas based on the trajectory data or the position data.

16. A recording medium according to claim 15, wherein the estimation means calculates a probability density function based on the trajectory data or the position data, and estimates a lane area based on the probability density function.

17. A recording medium as described in claim 15 or 16, wherein the estimation means calculates a probability density function relating to the position where the vehicle will pass at each of multiple coordinates in the reference direction, combines the peak positions of the probability density function calculated at each of the multiple coordinates in the reference direction to generate a lane reference line, and estimates a lane area based on the lane reference line.

18. A recording medium according to claim 17, wherein the estimation means calculates the number of lanes based on the number of peaks of the probability density function calculated at each of the multiple coordinates in the reference direction, and calculates reference lines for the lanes for the calculated number of lanes.

19. A recording medium according to claim 18, wherein the estimation means, when detecting a plurality of lane reference lines, estimates the position where the reference lines of adjacent lanes are divided at a predetermined ratio as the boundary between the adjacent lanes.

20. A recording medium described in any one of claims 15 to 19, wherein the acquisition means continues to acquire the trajectory data or the position data, and the estimation means repeatedly estimates the lane area based on the accumulated trajectory data or the position data, and if the difference between the lane area estimated in the (n-1)th processing and the lane area estimated in the nth processing is below a reference value, determines that the lane area estimated in the (n-1)th processing and the lane area estimated in the nth processing are the same, and if they are determined to be the same for a predetermined number of consecutive times or more, confirms the lane area estimated during that time as the estimation result.

Images