Self-localization method, self-localization device, and program

Abstract

To provide a self-position estimation method or the like capable of estimating a self-position more accurately than before even if an original road state is changed by fallen snow etc.SOLUTION: A self-position estimation method for estimating a position on a map of an observation body having an observation portion includes a calculation step of calculating a ratio of an area of a landmark relative to reference map information showing a position of the landmark on a road in a predetermined area stored in a storage portion and a ratio of the landmark relative to observation information that includes information on the road and is acquired by the observation portion that the observation body has, a calculation step of calculating a degree of matching showing a degree of matching between an area showing the landmark included in a reference map image and an area showing the landmark included in the observation information from the two ratios, and an estimation step of estimating a position of the observation body on the basis of the degree of matching.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a self-position estimation method, a self-position estimation device, and a program.

Background Art

[0002] In the field of autonomous driving and the like, technologies related to self-position estimation of a traveling vehicle have been proposed. In Patent Document 1, mapping parameters are determined based on a first feature point acquired by LiDAR (Light Detection and Ranging) and a second feature point in a map database, and when the difference between the first feature point and the second feature point is within a predetermined value, it is determined that the accuracy check for the first feature point has succeeded. A method for calibrating external parameters of an on-board sensor including such a step is disclosed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the above conventional technology, there is a problem that it is difficult to accurately perform self-position estimation when the state of the road that should originally exist is changed due to snow accumulation or the like.

[0005] Therefore, the present invention provides a self-position estimation method and the like that can perform self-position estimation more accurately than before even when the state of the road that should originally exist is changed due to snow accumulation or the like.

Means for Solving the Problems

[0006] A self-position estimation method according to one aspect of the present invention is a self-position estimation method for estimating the position of an observation body equipped with an observation unit on a map, comprising: a calculation step of calculating the ratio of the area of ​​a road landmark to reference map information indicating the position of a road landmark in a predetermined area stored in a memory unit, and the ratio of the area of ​​the landmark to observation information acquired by the observation unit including the road information; a calculation step of calculating a matching degree from the two ratios to represent the degree to which the area showing the landmark included in the reference map image and the area showing the landmark included in the observation information coincide; and an estimation step of estimating the position of the observation body based on the matching degree.

[0007] Furthermore, a program according to one aspect of the present invention is a program for causing a computer to execute a self-localization method according to one aspect of the present invention.

[0008] Furthermore, a self-position estimation device according to one aspect of the present invention is a self-position estimation device that estimates the position of an observation body equipped with an observation unit on a map, comprising: a calculation unit that calculates the ratio of the area of ​​a road landmark to reference map information indicating the position of a road landmark in a predetermined area stored in a memory unit, and the ratio of the area of ​​the landmark to observation information acquired by the observation unit; a calculation unit that calculates a matching degree representing the degree to which the area indicating the landmark included in the reference map image and the area indicating the landmark included in the observation information coincide, based on the two ratios; and an estimation unit that estimates the position of the observation body based on the calculated matching degree. [Effects of the Invention]

[0009] A self-position estimation method according to one aspect of the present invention can perform self-position estimation more accurately than conventional methods, even when the road conditions have been altered due to snow accumulation or other factors. [Brief explanation of the drawing]

[0010] [Figure 1]Figure 1 is a block diagram showing the configuration of the self-position estimation device in the embodiment. [Figure 2] Figure 2 is a flowchart illustrating the general operation of the self-localization method in the embodiment. [Figure 3] Figure 3 shows an example of calculating a correlation between a generated map image and a reference map image. [Figure 4] Figure 4 shows an example of calculating the ratio from a generated map image and a reference map image. [Figure 5] Figure 5 is a graph showing the ratio of the first ratio to the second ratio. [Figure 6] Figure 6 is a flowchart showing the operation of rejecting the self-localization result when the matching degree of the self-localization method in the embodiment is low. [Figure 7] Figure 7 is a flowchart illustrating the detailed operation of the self-position estimation device in the embodiment. [Modes for carrying out the invention]

[0011] (Embodiment) In this embodiment, a self-position estimation method and self-position estimation device, etc., that can perform self-position estimation more accurately than conventional methods, even when the road conditions have been altered due to snow accumulation or the like, will be described.

[0012] [composition] First, the configuration of the self-position estimation device in the embodiment will be described. Figure 1 is a block diagram showing the configuration of the self-position estimation device 1 in the embodiment. The self-position estimation device 1 comprises a control unit 10 and a storage unit 20. The control unit 10 comprises a calculation unit 30, a calculation unit 31, an estimation unit 32, a rejection unit 33, and an adoption unit 34. The self-position estimation device 1 is configured to communicate with the observation object 80 through a communication unit (not shown in the diagram), etc.

[0013] The self-position estimation device 1 estimates the location of the observation object 80 on the map by comparing a generated map image, which is generated based on observation information acquired as data by the observation unit of the observation object 80, with a reference map image, which is map information stored in the memory unit 20.

[0014] Here, the observation body 80 is equipped with a sensor as an observation unit, and acquires observation information, including road information, using this sensor. Examples of sensors include a camera as an imaging unit and a LiDAR as a distance detection unit, and the observation information includes image information, video information, distance information between the observation body 80 and the object being observed, etc. The road information includes at least landmarks that serve as markers for the road. The landmarks are most preferably white lines on the road, but are not limited to these, and may also be guardrails, road curbs, or signs, etc. Furthermore, the landmarks may be a combination of at least two selected from white lines, guardrails, curbs, and signs. The road information may also include the shape and width of the road.

[0015] The generated map image is an orthorectified image of a bird's-eye view map image generated in real time based on observational information acquired by the sensors of observation device 80. Alternatively, the bird's-eye view map image before orthorectification may be used as the generated map image. The generated map image is a concrete example of the observational information.

[0016] The reference map image also includes at least landmarks related to roads, similar to the above. The reference map image may be stored in the storage unit 20 in advance, or may be acquired from the outside and stored in the storage unit 20. The reference map image is three-dimensional geospatial information of a road and its surroundings. For example, the reference image is observation information including road information acquired by a sensor previously, or observation information including road information created separately in advance, a dynamic map created for autonomous driving, and the like. Further, the reference image may be high-precision three-dimensional geospatial information (basic map information) in which the position of the host vehicle related to a road and its surroundings can be specified at the lane level. The reference map image may further be a map with various additional map information (for example, traffic control information including dynamic information such as accident and construction information in addition to static information such as speed limits) mounted thereon in order to support autonomous driving and the like on top of the high-precision three-dimensional geospatial information.

[0017] The control unit 10 consists of a CPU (Central Processing Unit), a processor, a semiconductor element, and the like.

[0018] The storage unit 20 consists of a memory or the like. The storage unit 20 stores a reference map image including the positions of landmarks such as white lines of a road, observation information acquired by sensors included in the observation body 80, a generated map image indicating road information generated from data acquired by sensors included in the observation body 80, and the matching degree calculated by the calculation unit 31 and the like.

[0019] The calculation unit 30 calculates the ratio of the area indicating the landmark in a predetermined area included in the reference map image and the ratio of the area indicating the landmark in the area included in the generated map image and corresponding to the predetermined area of the reference map image. Note that the area corresponding to the predetermined area may be an area in the generated map image indicating substantially the same position as the position indicated on the reference map image. When the scales of the reference map image and the generated map image are the same, it may be an area in the generated map image indicating the same position as the position indicated on the reference map image and having the same size as the area on the reference map image.

[0020] The calculation unit 31 calculates a matching degree from the two ratios mentioned above, which represents the degree to which the area indicating landmarks included in the reference map image matches the area indicating landmarks included in the generated map image.

[0021] The estimation unit 32 estimates the position of the observed object 80 based on the calculated matching degree.

[0022] The rejection unit 33 may instruct the estimation unit 32 not to use the reference map image to estimate the position of the observation object 80 on the road map when the estimation unit 32 is estimating the position of the observation object 80.

[0023] If the position is estimated by the estimation unit 32, the selection unit 34 instructs the estimation unit 32 to use the above-mentioned reference map image to estimate the position of the observation object 80 on the road map.

[0024] In other words, if the matching degree is satisfactory, the estimation unit 32 uses the position of the reference map image to estimate the position of the observation object 80 on the road map.

[0025] [Operation] Next, we will describe the general operation of the self-position estimation device 1. Figure 2 is a flowchart illustrating the general operation of the self-position estimation method in the embodiment.

[0026] The self-position estimation device 1 is a device having sensors such as a camera, and for example, the device may be a self-propelled vehicle. The self-position estimation device 1 estimates the position of the observation object 80 in real time by comparing a generated map image generated from data acquired by the sensors of the observation object 80 with a reference map image that is stored in advance.

[0027] First, the calculation unit 30 calculates the proportion of areas indicating landmarks within a predetermined region included in the reference map image stored in the storage unit 20, and the proportion of areas indicating landmarks included in the observation information within the region corresponding to the predetermined region (step S10). The predetermined region is, for example, a 64m x 64m area on the map.

[0028] The ratios calculated above are the ratio of the area showing landmarks to the entire reference map image (hereinafter referred to as the first ratio) and the ratio of the area showing landmarks to the entire generated map image (hereinafter referred to as the second ratio).

[0029] In this case, the reference map image and the generated map image have the same dimensions. The calculation unit 30 may further divide the reference map image in a predetermined region and the observation information in the region corresponding to the predetermined region into multiple smaller sub-regions, and for each of the multiple sub-regions, identify the reference map image and the observation information based on the correlation between the reference map image and the observation information, and then perform the above calculation.

[0030] Next, the calculation unit 31 calculates the degree of matching between the area indicating landmarks included in the reference map image and the area indicating landmarks included in the generated map image based on the calculated ratio (step S11). The degree of matching is the degree to which the area indicating landmarks included in the reference map image and the area indicating landmarks included in the generated map image coincide. For example, the degree of matching may be determined by the ratio of the second ratio to the first ratio. The closer the values ​​of the first ratio and the second ratio are, the higher the degree of matching. That is, the closer the ratio of the first ratio to the second ratio is to 1, the higher the degree of matching.

[0031] Next, the estimation unit 32 estimates the position of the observed object 80 based on the degree of matching (step S12). The degree of matching can be considered high if the above ratio is within the threshold range.

[0032] [Processing details] Next, we will explain the method for determining the degree of matching of the self-localization method in the embodiment. Figure 3 shows an example of calculating the ratio using the generated map image and the reference map image.

[0033] Figure 3(a) shows an image illustrating the white line 40a on a road 50a included in a reference map image representing the shape of the road. A map image from which only the white line 40a has been extracted may be used as the reference map image. The reference map image may also be generated by deep learning-based pattern recognition, extracting only the region of the white line 40a from the map image.

[0034] Figure 3(b) shows a generated map image representing the shape of the road, etc., which is generated from data acquired by the sensors on the observation device 80, and in which the white line 40b on the road 50b is shown.

[0035] Figure 3(c) shows the correlation between the reference map image shown in Figure 3(a) and the generated map image shown in Figure 3(b). In Figure 3(c), the degree of agreement between the reference map image shown in Figure 3(a) and the generated map image shown in Figure 3(b) increases, from black to white. The peak position in Figure 3(c) (the part with the highest correlation between Figure 3(a) and Figure 3(b), i.e., the white area) is considered to be the most likely position of the observed object 80.

[0036] The calculation unit 30, by shifting the relative position of Figure 3(b) to Figure 3(a), makes the above determination at the position with the highest degree of agreement. In other words, the calculation unit 30 determines that the position of the observed object 80 is most likely when the two images, the reference map image and the generated map image, are superimposed at the peak position with the highest correlation.

[0037] Figure 4 shows an example of determining the degree of matching using a generated map image and a reference map image. Figure 4(a) is a generated map image showing the shape of a road. In the generated map image of Figure 4(a), a white line 40a on the road is shown. The generated map image of Figure 4(a) may be binarized. For example, the image shown in Figure 4(e) is a binarized version of the image shown in Figure 4(a). In Figure 4(e), the white line 40a on the road 50a is shown in white.

[0038] Next, the calculation unit 30 generates the expanded white line map image shown in Figure 4(b). The expanded white line map image is a map that emphasizes the areas of the white lines shown in the reference map image that indicates the positions of the white lines. The expanded white line map image is an image generated by cutting out a section at a position that has a high correlation with the white line 40a in the generated map image. In Figure 4(b), the white line 40b, which has been cut out at a position that has a high correlation with the white line 40a in the generated map image, is shown as the expanded white line map image.

[0039] Then, as shown in Figure 4(c), the calculation unit 30 generates a white line mask image by masking the white line 40a. Here, the masking of the white line 40a may be a process that extracts only the white line portion from the image. Note that masking is not required. Then, as shown in Figure 4(d), the white line mask image is binarized. For example, the white line region may be binarized to white and the other regions to black.

[0040] The calculation unit 30 then calculates the number of pixels N in the region of the generated map image where the brightness value is higher than a predetermined threshold (for example, a white region). RefLaneHigh The calculation unit calculates the number of pixels N in the area where the brightness value of the generated map image is below a predetermined threshold (for example, the area of ​​the white line 40a, see Figure 4(d)). Subsequently, the calculation unit 30 calculates the number of pixels N in the area where the brightness value of the generated map image is below a predetermined threshold (for example, the black area). RefLaneLow The calculation is performed. The region where the brightness value is below a predetermined threshold is, for example, the region of the black area 60 (see Figure 4(d)).

[0041] Next, the calculation unit 30 calculates the ratio of the region where the brightness value is higher than a predetermined threshold to the region where the brightness value is below a predetermined threshold.

[0042] Similarly, the calculation unit 30 performs the same processing on the expanded white line map image. That is, the calculation unit 30 calculates the number of pixels N in the expanded white line map image where the brightness value is higher than a predetermined threshold (for example, white areas). RefLaneHigh The calculation unit 30 then calculates the number of pixels N in the region where the brightness value of the expanded white line map image is below a predetermined threshold (for example, the region of the white line 40a). RefLaneLow This calculates the value. A region where the brightness value is below a predetermined threshold is, for example, a black region.

[0043] Based on the above, the first ratio, which is the proportion of the area showing landmarks (white lines) relative to the entire reference map image, can be determined. Furthermore, the second ratio, which is the proportion of the area showing landmarks (white lines) relative to the entire generated map image, can be determined.

[0044] Figure 5 is a graph showing the ratio of the second ratio to the first ratio. The ratio is expressed as second ratio ÷ first ratio. The data 70a shown in Figure 5(a) is a line graph showing the ratio of the second ratio to the first ratio on sunny days. Lines 71 and 72 represent examples of threshold values ​​for the ratio of the second ratio to the first ratio. Note that the average value in Figure 5(a) is less than or equal to 1 because the white line observed by the sensor includes factors that cause it to become smaller over time (such as fading of the white line due to aging).

[0045] Furthermore, the data 70b shown in Figure 5(b) is a line graph showing the ratio of the second ratio to the first ratio for rainy days, and the data 70c shown in Figure 5(c) is a line graph showing the ratio of the second ratio to the first ratio for snowy days.

[0046] When the ratio of the second ratio to the first ratio (second ratio / first ratio) is within the threshold range, the estimation unit 32 determines that the matching degree is high and it is qualified. The upper limit b of the threshold is the straight line 71 in FIG. 5, and the lower limit a of the threshold is the straight line 72 in FIG. 5. When a < ratio < b, the matching degree is high. Here, the ratio is a value such as a ratio calculated by a predetermined calculation formula calculated by the calculation unit 31.

[0047] And when the adoption unit 34 determines that the matching degree is high and it is qualified, it instructs the estimation unit 32 to use the position of the reference land map image at that time as the position of the observation object 80 on the road map for estimation. In that case, the estimation unit 32 estimates the position of the observation object 80 on the road map based on the matching degree calculated by the calculation unit 31.

[0048] Also, when the ratio is not within the threshold range (a < ratio < b), the estimation unit 32 determines that the matching degree is low and it is unqualified. Then, the rejection unit 33 instructs the estimation unit 32 not to estimate the position of the generated map image as the position of the observation object 80 on the road map. In that case, the estimation unit 32 does not estimate the position of the observation object 80 on the road map based on the matching degree calculated by the calculation unit 31. For example, the estimation unit 32 does not estimate a place where the matching degree is low (outside the threshold range) on the road map as the position where the observation object 80 exists.

[0049] Next, the operation when the self-position is not estimated will be described. FIG. 6 is a flowchart showing an operation of rejecting the self-position estimation result when the matching degree is low.

[0050] First, the estimation unit 32 determines whether the ratio is within the threshold range (step S20). Here, the ratio is the ratio of the ratio of the pixels indicating the white line in the reference land map image to the ratio of the pixels indicating the white line in the generated map image, and it is determined whether the ratio is within the threshold range.

[0051] If the estimation unit 32 determines that the ratio of the second ratio to the first ratio (second ratio / first ratio) is outside the threshold range (No in step S20), the rejection unit 33 instructs the estimation unit 32 not to use the location of the reference map image to estimate the location of the observation body 80 on the road map (step S21). In other words, when the ratio of the second ratio to the first ratio is outside the threshold range, the estimation unit 32 does not make the estimation that the observation body 80 is located at the location of the reference map image.

[0052] If the estimation unit 32 determines that the ratio of the second ratio to the first ratio is within the threshold range (Yes in step S20), the selection unit 34 instructs the estimation unit 32 to use the position of the reference map image at that time to estimate the position of the observation object 80 on the road map (step S22). That is, the estimation unit 32 estimates that the observation object 80 is located in a place where the ratio of the second ratio to the first ratio is within the threshold range. Alternatively, in step S21, the reference map image may be updated using only the information of the area indicating a landmark where the ratio of the second ratio to the first ratio calculated in step S11 is within the threshold range.

[0053] Note that the values ​​of a and b (lower and upper thresholds) cannot be determined precisely due to factors such as road location and other conditions. However, if the ratio differs by approximately 1.3 to 2, it can be considered outside the threshold. In that case, a can be set to 0.5 to 0.7 and b to 1.3 to 2.0.

[0054] Furthermore, the ratio may also be the ratio of the first ratio to the second ratio. Also, although the ratio is shown as the proportion of the landmark's area, i.e., the proportion of its area, other parameters such as the width or length of the landmark may also be used.

[0055] Next, we will explain the details of the operation of the self-position estimation device 1. Figure 7 is a flowchart showing the detailed operation of the self-position estimation device 1 in the embodiment.

[0056] First, the calculation unit 30 extracts a 64m x 64m area from the reference map image (step S30). The calculation unit 30 uses a 40,000 pixel area of ​​the reference map image as the processing area. Here, the calculation unit 30 may use a reference map image stored in the memory unit 20, or it may use a reference map image generated from a road map image by deep learning after extracting a 64m x 64m area from a general road map image.

[0057] Furthermore, the calculation unit 30 may generate a reference map image showing the location of the white line 40a (see Figures 3 and 4) from images previously acquired by sensors (such as cameras) on the observation body 80. Alternatively, the calculation unit 30 may learn the correct data for the area showing the white line 40a included in the road map using a learning algorithm such as deep learning, and generate a reference map image based on the learning algorithm.

[0058] Next, the calculation unit 30 extracts a 24m x 24m area from the generated map image (step S31). The calculation unit 30 uses the 24m x 24m area of ​​the generated map image, which was generated from data acquired by sensors on the observation device 80 such as a camera, as the processing area.

[0059] Next, the calculation unit 30 moves a 200-pixel square area within the processing target area one pixel at a time vertically or horizontally, and processes each area. The calculation unit 30 repeatedly performs the process described in step 32a a total of 40,000 times within the processing target area (step S32).

[0060] For example, the calculation unit 30 sets i (an integer) to 0 and performs the process of step 32a, described later, for the range 0 ≤ i ≤ 39999. During this time, the integer i is incremented by 1 each time the process of step 32a is completed.

[0061] In step 32a, the correlation between the reference map image and the generated map image is calculated each time (step 32a). In step 33, similar to Figure 3, the location where the reference map image and the generated map image have a high degree of agreement is found. At that location, the next step S34 is performed. In other words, the calculation unit 31 determines the best location by considering the location of the pixel with the highest correlation value among the correlations calculated in step S32a for each of the 40,000 200x24m regions as having the highest correlation (step S33). In other words, the calculation unit 31 determines the best location as the 200x200 pixel region with the highest correlation among the 24m x 24m processing area.

[0062] Next, the calculation unit 30 calculates a first ratio of the area showing white lines included in the reference map image and a second ratio of the area showing white lines included in the generated map image in an area of ​​the same size as a predetermined area (step S34). For example, as shown in Figure 4, the calculation unit 30 binarizes the white line areas included in the reference map image and the generated map image and calculates the ratios of the white line areas included in the reference map image and the generated map image as the first ratio and the second ratio, respectively.

[0063] Here, the first ratio is the ratio of the number of pixels in the white line region included in the processing area to the total number of pixels in the processing area in the reference map image. The second ratio is the ratio of the number of pixels in the white line region included in the processing area to the total number of pixels in the processing area in the generated map image.

[0064] Next, the calculation unit 31 calculates the ratio of the first ratio to the second ratio and determines the degree of matching (step S35). The calculation unit 31 calculates the ratio of the second ratio to the first ratio.

[0065] Next, the calculation unit 31 determines whether the ratio of the best positions is within the threshold range (step S36). In other words, the calculation unit 31 determines whether the matching degree of the best positions is closer to 1 than the threshold.

[0066] If the calculation unit 31 determines that the ratio at the best position is within the threshold range (Yes in step S36, high matching degree), the estimation unit 32 estimates the best position as the position of the observation object 80 (step S37). In this case, the selection unit 34 selects the best position as the estimated position of the observation object 80.

[0067] If the calculation unit 31 determines that the ratio of the best positions is outside the threshold range (No in step S36), the estimation unit 32 does not estimate the best position as the position of the observation object 80 (step S38). In other words, if the estimation unit 32 determines that the ratio of the best positions is outside the threshold range, the rejection unit 34 does not adopt the best position for estimating the position of the observation object 80.

[0068] Although the self-position estimation device 1 and self-position estimation method are described as being applied to a reference map image of a road that does not have elevation information, they may also be applied to an elevation map that does have elevation information. By calculating the degree of matching of the white line area using the self-position estimation method in the embodiment and determining the degree of matching, it is possible to detect that the shape of the road has changed due to snow accumulation or damage in areas where the degree of matching is low.

[0069] [Effects, etc.] A self-position estimation method in one aspect of the present disclosure is a self-position estimation method for estimating the position of an observation body 80 that observes a road on a road map, and includes: a calculation step (step S10) that calculates the ratio of the area showing the white line 40a included in a reference map image showing the position of the white line 40a of a road 50a in a predetermined area, and the ratio of the area showing the white line 50b included in a generated map image showing the shape of a road 50b generated from data acquired by a sensor equipped on the observation body 80 in an area of ​​the same size as the predetermined area; a calculation step (step S11) that calculates a degree of matching from the above ratio, which represents the degree of agreement between the area showing the white line 40a included in the reference map image and the area showing the white line 40b included in the generated map image; and an estimation step (step S12) that estimates the position of the observation body 80 based on the calculated degree of matching.

[0070] As a result, the self-position estimation method in one aspect of this disclosure can determine whether or not to estimate the position of the observation body 80 based on the degree of matching. Therefore, the estimation method in one aspect of this disclosure can perform self-position estimation more accurately than conventional methods, even when the road conditions have been altered due to snow accumulation or other factors.

[0071] Furthermore, for example, a self-localization method in one aspect of this disclosure further includes an adoption step (step S22) used to estimate the position of the observation body 80 on a road map when the degree of matching is high and passes the test.

[0072] As a result, the self-localization method in one aspect of this disclosure can estimate the position of the observation body 80 at a location where the degree of agreement between the shape of the white line 40b of the observed road 50b and the shape of the white line 40a on the reference map image is high.

[0073] Furthermore, for example, in a self-localization method according to one aspect of the present disclosure, the calculation step calculates a first ratio, which is the ratio of the area showing the white line included in a reference map image showing the position of the white line 40a of a road 50a in a predetermined area, and a second ratio, which is the ratio of the area showing the white line 50b included in a generated map image in an area of ​​the same size as the predetermined area. The calculation step calculates the ratio of the first ratio to the second ratio, and the calculation step determines that the degree of matching is high if the ratio of the first ratio to the second ratio is within a threshold range.

[0074] As a result, the self-localization method in one aspect of this disclosure can accurately find locations where the shapes of the white lines 40a and 40b match by comparing the proportion occupied by the white line 40a on road 50a with the proportion occupied by the white line 50b on road 50b.

[0075] Furthermore, for example, in a self-localization method according to one aspect of this disclosure, the calculation step involves the camera of the observation body 80 generating a reference map image showing the position of the white line 40a from previously acquired images.

[0076] As a result, the self-localization method in one aspect of this disclosure can generate a reference map image in real time from data acquired by sensors, etc.

[0077] Furthermore, for example, in a self-localization method according to one aspect of this disclosure, the calculation step involves generating a reference map image using deep learning.

[0078] As a result, the self-localization method in one aspect of this disclosure can automatically generate a reference map image with high accuracy from data acquired by sensors, etc.

[0079] Furthermore, in a self-localization method according to one aspect of the present disclosure, in the calculation step, a reference map image in a predetermined area and a generated map image in an area of ​​the same size as the predetermined area are further divided into a plurality of smaller sub-regions, a correlation is calculated for each of the plurality of sub-regions, and in the calculation step, the correlation between the reference map image and the observation information is calculated for each of the plurality of sub-regions.

[0080] As a result, the self-localization method in one aspect of this disclosure can estimate the position of the observation object 80 with fine granularity.

[0081] Furthermore, the program in one aspect of this disclosure may be a program that causes a computer to execute the self-localization method in one aspect of this disclosure.

[0082] As a result, a program in one aspect of this disclosure can achieve the same effect as the self-localization method described above.

[0083] Furthermore, a self-position estimation device 1 in one aspect of the present disclosure is a self-position estimation device 1 that estimates the position of an observation body 80 that observes roads on a road map, and comprises: a calculation unit 30 that calculates the ratio of the area showing white lines 40a to a reference map image showing the position of white lines of road 50a in a predetermined area, and the ratio of the area showing white lines 40b to a generated map image showing the shape of road 50b generated from data acquired by sensors on the observation body 80 in an area of ​​the same size as the predetermined area; a calculation unit 31 that calculates a matching degree representing the degree to which the area showing white lines 40a included in the reference map image and the area showing white lines 40b included in the generated map image match from the ratio; and an estimation unit 32 that estimates the position of the observation body 80 based on the calculated matching degree.

[0084] As a result, the self-position estimation device 1 in one aspect of this disclosure can achieve the same effects as the self-position estimation method described above.

[0085] [Other embodiments] Although embodiments have been described above, the present invention is not limited to the embodiments described above.

[0086] For example, in the above embodiment, the ratio of white lines in the reference map image and the generated map image was used as the degree of matching, but the determination of the degree of matching is not limited to the above method. For example, the degree of matching may be calculated using the absolute value of pixels showing white lines in the reference map image and the observation map image as observation information, or the degree of matching may be determined by whether or not the difference in the absolute value of pixels showing white lines in the reference map image and the observation map image is within a threshold.

[0087] Furthermore, in the above embodiment, the processing performed by a specific processing unit may be performed by another processing unit. Also, the order of multiple processing units may be changed, or multiple processing units may be executed in parallel.

[0088] Furthermore, in the above embodiment, each component may be realized by executing a software program suitable for each component. Each component may also be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0089] Furthermore, each component may be implemented by hardware. Each component may also be a circuit (or integrated circuit). These circuits may form a single circuit as a whole, or they may be separate circuits. Also, each of these circuits may be a general-purpose circuit or a dedicated circuit.

[0090] Furthermore, general or specific embodiments of the present invention may be implemented as a system, apparatus, method, integrated circuit, computer program, or recording medium such as a computer-readable CD-ROM. Alternatively, they may be implemented as any combination of a system, apparatus, method, integrated circuit, computer program, and recording medium.

[0091] For example, the present invention may be implemented as a terminal according to the above embodiment, or as a system equivalent to a terminal. Furthermore, the present invention may be implemented as a self-localization method, as a program for causing a computer to execute a self-localization method, or as a computer-readable non-temporary recording medium on which such a program is recorded. The program includes an application program for operating a general-purpose mobile terminal as the mobile terminal according to the above embodiment.

[0092] Furthermore, although the self-position estimation device 1 was implemented by a single device in the above embodiment, it may be implemented by multiple devices. Also, if the self-position estimation device 1 is implemented by multiple devices, the components of the stimulus output system described in the above embodiment may be distributed among the multiple devices in any way.

[0093] Furthermore, the present invention also includes forms obtained by applying various modifications to each embodiment that a person skilled in the art could conceive, or forms realized by arbitrarily combining the components and functions of each embodiment without departing from the spirit of the present invention. [Explanation of Symbols]

[0094] 1 Self-position estimation device 10 Control Unit 20 Memory section 30 Calculation section 31 Calculation Section 32 Estimation part 33 Rejection Section 34 Recruitment Department 40a, 40b White lines 50a, 50b road 60 black area 70a, 70b, 70c data 71 straight line 80 Observers

Claims

1. A self-position estimation method for estimating the position on a map of an observation body equipped with an observation unit, A calculation step of calculating the ratio of the area of ​​the landmark to reference map information indicating the location of the road landmark in a predetermined area stored in the memory unit, and the ratio of the area of ​​the landmark to observation information acquired by the observation unit, which includes the road information. A calculation step to calculate a matching degree, which represents the degree to which the area indicating the landmark included in the reference map image and the area indicating the landmark included in the observation information coincide, based on the two aforementioned ratios. Includes an estimation step of estimating the position of the observed object based on the degree of matching, Self-localization method.

2. If the estimation step is successful, the following steps are included: an adoption step in which the position of the reference map image is used to estimate the position of the observed object on the map; The self-localization method according to claim 1.

3. In the calculation step, a first ratio is calculated, which is the ratio of the area showing the landmark included in the reference map image that shows the location of the road landmark in the predetermined area, and a second ratio is calculated, which is the ratio of the area showing the landmark included in the observation information in the area corresponding to the predetermined area. In the calculation step described above, the ratio of the first ratio to the second ratio is calculated, In the estimation step, if the ratio of the first ratio to the second ratio is within a predetermined threshold range, it is determined to be a pass. The self-localization method according to claim 2.

4. In the calculation step, the reference map image showing the location of the landmark is generated from the image acquired by the imaging unit of the observation device. The self-localization method according to claim 1 or 2.

5. In the calculation step described above, the reference map image is generated using deep learning. The self-localization method according to claim 4.

6. In the calculation step, the reference map image in the predetermined region and the observation information in the region corresponding to the predetermined region are further divided into a plurality of smaller sub-regions, and for each of the plurality of sub-regions, the reference map image and the observation information are identified from the correlation between the reference map image and the observation information, and the calculation step is performed. The self-localization method according to claim 1 or 2.

7. In the estimation step described above, when the position of the observed object is estimated, the reference map image is updated using only the information of the region indicating the landmark. The self-localization method according to claim 1 or 2.

8. A program for causing a computer to execute the self-localization method described in claim 1.

9. A self-position estimation device that estimates the position of an observation body equipped with an observation unit on a map, A calculation unit calculates the ratio of the area of ​​the landmark to reference map information indicating the location of road landmarks in a predetermined area stored in the memory unit, and the ratio of the area of ​​the landmark to observation information acquired by the observation unit. A calculation unit calculates a matching degree that represents the degree to which the area indicating the landmark included in the reference map image and the area indicating the landmark included in the observation information coincide, based on the two aforementioned ratios. The system includes an estimation unit that estimates the position of the observed object based on the calculated matching degree, Self-location estimation device.

Images