Information processing device, information processing method, and information processing program

Abstract

To provide an information processor, a method for processing information, and an information processing program that can estimate the reliability of a position.SOLUTION: The information processor includes: an acquisition unit for acquiring image information based on imaging of the outside in a vehicle and positional information on where the vehicle is travelling; and an estimation unit for estimating the reliability of the travelling position according to a scenery recorded in the image information on the basis of a learned model based on learning the scenery recorded in the image and a label put on the scenery.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] Conventionally, there is a driving assistance device that assists a vehicle traveling on a road. The driving assistance device acquires the position (autonomous position) of the vehicle based on the vehicle's traveling direction and traveling speed. The driving assistance device also selects some artificial satellites from among a plurality of artificial satellites based on celestial information, and acquires a position (heteronomic position) measured based on satellite signals transmitted from the selected artificial satellites. Here, the celestial information is incorporated into road map information and includes information indicating the azimuth and elevation angles at which features are located as viewed from the road. The driving assistance device corrects accumulated errors occurring in the autonomous position based on the heteronomic position. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2006-267004 Summary of the Invention [Problem to be solved by the invention]

[0004] High-precision maps, such as road maps, are sometimes created using heteronomous positioning. The accuracy of heteronomous positioning can vary depending on factors such as the reception strength of satellite signals and the number of satellites used for positioning. For this reason, it is necessary to estimate the reliability of identifying the location of a vehicle (measurement vehicle) used to create a road map.

[0005] As described above, the driving assistance device described in Patent Document 1 uses a technology for correcting cumulative errors in autonomous position based on heteronomous position. Because the driving assistance device aims to correct the autonomous position, it is unable to apply any technical processing when the accuracy of the heteronomous position used in creating road maps, etc., is relatively poor, making it impossible to create more accurate road maps. In other words, the driving assistance device described in Patent Document 1 cannot be used, for example, in creating road maps.

[0006] The present disclosure provides an information processing device, an information processing method, and an information processing program that are capable of estimating the reliability of a position. [Means for solving the problem]

[0007] An information processing device of one embodiment includes an acquisition unit that acquires image information based on an image of the outside captured by the vehicle and position information regarding the vehicle's driving position, and an estimation unit that estimates the reliability of the driving position corresponding to the scenery recorded in the image information based on a trained model that is based on learning the scenery recorded in the image and the label assigned to the scenery. [Effects of the Invention]

[0008] According to one aspect, the reliability of a driving position according to the scenery recorded in image information can be estimated based on a trained model. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an information processing device according to an embodiment. [Figure 2] FIG. 1 is a block diagram illustrating an information processing device according to an embodiment. [Figure 3] FIG. 1 is a diagram illustrating an example of a road including an underpass section. [Figure 4] FIG. 10 is a diagram illustrating an example of correspondence information. [Figure 5]1 is a flowchart illustrating an information processing method according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] An embodiment will be described below.

[0011] [Overview of information processing device 1] First, an overview of an information processing device 1 according to an embodiment will be described. FIG. 1 is a diagram illustrating an information processing device 1 according to an embodiment.

[0012] Introduction: For example, in generating high-precision map information, the positional accuracy of the travel trajectory of a vehicle (measurement vehicle) equipped with a measuring instrument is an important factor. The travel position of the vehicle may be determined using, for example, GNSS, etc., and in areas where the reception environment of satellite signals is unstable, such as urban areas and mountainous regions, it is required to effectively and efficiently correct the travel trajectory (travel position). However, when the driving trajectory (driving position) of a vehicle or the like needs to be corrected, the map creator may not be able to determine the degree and range of deviation of the driving trajectory (driving position). Furthermore, the state of vegetation around the road varies depending on the season in which the vehicle is traveling, and depending on the timing of the vehicle's travel, large vehicles may be traveling alongside the vehicle, interfering with positioning. In light of these circumstances, rather than correcting the driving position through trial and error on the part of the map creator, the information processing device 1 narrows down the range in which the accuracy of the driving position is expected to deteriorate, and then corrects the driving position.

[0013] That is, the information processing device 1 of the present disclosure may be configured as, for example, an estimation device that estimates the reliability of the traveling position of a mobile object based on position information. The information processing device 1 may be configured as, for example, a correction device that corrects the traveling position of a mobile object based on position information. The information processing device 1 may be configured as, for example, a creation device that creates a road map based on the traveling position. The information processing device 1 is not limited to the device of the example described above, and may be configured as various devices. The information processing device 1 may be a computer such as a server, a desktop, a laptop, a tablet, or a smartphone.

[0014] The information processing device 1 acquires, for example, image information and position information. The image information may be generated by, for example, capturing an image of an external scene from a moving object. The image information may be generated by, for example, capturing an image of the moving object's direction of travel by the imaging unit 101 disposed in the moving object, thereby recording a scene including an area above the moving object. Alternatively, the image information may be generated by, for example, capturing an image of the area above the moving object by the imaging unit 101, thereby recording a scene including an area above the moving object. The location information may be, for example, information about the traveling position of the mobile object. For example, the location information may be information about the location acquired when the mobile object moves from a departure point to a destination.

[0015] A mobile object (e.g., a vehicle 100) may acquire location information by performing positioning based on receiving, for example, satellite signals (radio signals) using GNSS (artificial satellites 111) or radio signals emitted from an external antenna 112 (e.g., multiple antennas at a radio signal station, an antenna using Bluetooth (registered trademark), etc.) located outside the mobile object. Furthermore, in the case of an external antenna 112 capable of identifying a location, e.g., an access point capable of emitting radio signals over a relatively short distance, the mobile object may estimate the position of the external antenna 112 (e.g., access point) as its own position.

[0016] Furthermore, the mobile body may estimate its own position based on measurements made by an on-board inertial sensor 103 mounted on the mobile body. The on-board inertial sensor 103 may be, for example, an inertial measurement unit (IMU) including various physical quantity sensors such as a gyro sensor and an acceleration sensor.

[0017] In the present disclosure, location information acquired using radio signals may be referred to as radio wave location information, etc. Similarly, location information acquired using the measurement results of the inertial sensor 103 may be referred to as self-location information, etc.

[0018] A mobile object (for example, the vehicle 100) may transmit, for example, position information and image information to the information processing device 1 or the server 200. In this case, the mobile object may transmit, for example, the position information and the image information in association with each other to the outside. Here, the association may be in a form that allows identification of the position information and the image information acquired during one trip when the mobile object moves from the departure point to the destination. The server 200 may, for example, accumulate position information and image information and transmit the position information and image information to the information processing device 1. In addition, the mobile body and server 200 are not limited to transmitting the location information and image information to the information processing device 1 via a communication line or the like, but may store the location information and image information in an external memory and input the location information and image information to the information processing device 1 via the external memory.

[0019] The mobile body may be, for example, an automobile, a motorcycle, a light vehicle, or a mobile terminal such as a smartphone, a tablet, a smartwatch, or a laptop. In the following, a "vehicle" will be described as an example of the mobile body. Furthermore, the "vehicle" in the present disclosure may be synonymous with the mobile body described above or may be a concept that includes the mobile body.

[0020] The information processing device 1 estimates the reliability of location information based on, for example, image information and a trained model. The trained model may be, for example, a model generated based on learning a scene recorded in an image and a label assigned to the scene. That is, the information processing device 1 may input image information to the trained model and estimate the reliability of location information based on the scene recorded in the image information.

[0021] The information processing device 1 may correct the position information according to the reliability obtained based on the above-described estimation, for example. Furthermore, the information processing device 1 may create a road map based on, for example, the position information and the corrected position information.

[0022] [Details of information processing device 1] Next, the information processing device 1 according to an embodiment will be described in detail. FIG. 2 is a block diagram illustrating an information processing device 1 according to an embodiment. FIG. 3 is a diagram illustrating an example of a road including an underpass section.

[0023] The information processing device 1 includes, for example, a communication unit 21, a storage unit 22, a display unit 23, and a control unit 11. The communication unit 21, the storage unit 22, and the display unit 23 may be an embodiment of an output unit. The control unit 11 includes, for example, an acquisition unit 12, an estimation unit 13 (a first estimation unit 131 and a second estimation unit 132), a correction unit 14, a creation unit 15, and an output control unit 16. The control unit 11 may be configured, for example, by an arithmetic processing device of the information processing device 1. The control unit 11 (for example, an arithmetic processing device) may realize the functions of each unit (for example, the acquisition unit 12, the estimation unit 13 (the first estimation unit 131 and the second estimation unit 132), the correction unit 14, the creation unit 15, and the output control unit 16) by, for example, appropriately reading and executing various programs stored in the storage unit 22.

[0024] The communication unit 21 is, for example, a communication interface capable of transmitting and receiving various information to and from devices (external devices) outside the information processing device 1. Here, examples of the external devices may be the vehicle 100, the server 200, and a user terminal (not shown), etc.

[0025] The storage unit 22 may store, for example, various information and programs. Examples of the storage unit 22 may include a memory, a solid state drive, a hard disk drive, etc. Note that the storage unit 22 may also be, for example, a storage area or a server on a cloud.

[0026] The display unit 23 is a display capable of displaying, for example, various characters, symbols, images, and the like.

[0027] The acquisition unit 12 acquires image information based on an image of the outside captured by the vehicle 100, and position information related to the traveling position of the vehicle 100. The acquisition unit 12 may acquire the image information and position information from the outside via the communication unit 21, for example. In this case, the outside may be, for example, the vehicle 100 and the server 200. Alternatively, for example, if the image information and position information are stored in an external memory (not shown), the acquisition unit 12 may acquire the image information and position information from the external memory when the external memory is inserted into an interface (not shown) of the information processing device 1.

[0028] The image based on the image information may be, for example, a moving image or a still image. If the image information is a still image, a plurality of still images may be generated by capturing images at predetermined intervals in the vehicle 100. The predetermined intervals may be, for example, at predetermined intervals.

[0029] In this case, the acquisition unit 12 may acquire radio wave location information and self-location information as the location information. As in the example described above, the radio wave location information may be location information based on receiving a radio wave signal by the on-board receiver 102 of the vehicle 100 (see FIG. 1). The radio wave signal may be, for example, a satellite signal using GNSS, or a signal emitted from an external antenna 112 (for example, multiple antennas such as a radio signal station, a Bluetooth antenna, an access point, etc.). Similarly, as in the example described above, the self-location information may be location information acquired based on the measurement results of the on-board inertial sensor 103.

[0030] For example, the estimation unit 13 may classify scenes (scenery) based on image information (driving images) captured by the vehicle 100 and estimate the accuracy of radio wave location information of the shooting position (quality of the radio wave signal reception environment). That is, the estimation unit 13 estimates the reliability of the driving position corresponding to the scenery recorded in the image information based on a trained model that is based on learning the scenery recorded in the image and the labels assigned to the scenery. The estimation unit 13 may estimate the reliability of the driving position based on the radio wave location information.

[0031] As an example, when vehicle 100 travels in a built-up area, a tunnel, or the like, the reception conditions for radio signals (satellite signals) using GNSS become relatively poor, and therefore the accuracy of the position acquired by positioning is considered to become relatively poor. As a specific example, as shown in Fig. 3, when vehicle 100 travels on road 300, the reception conditions for radio signals (satellite signals) using GNSS become poorer when vehicle 100 travels in section 301 under an elevated road than when vehicle 100 travels in section 302 that is not under an elevated road. The reliability of vehicle 100's travel position 303 in section 301 under an elevated road is considered to be relatively lower than that in section 302 that is not under an elevated road. Similarly, as an example, when the vehicle 100 is traveling along the sea on flat ground, the reception conditions for radio signals (satellite signals) using GNSS are relatively good, and the accuracy of the position obtained by positioning is considered to be relatively good. In such a case, the reliability of the traveling position of the vehicle 100 is considered to be relatively high. Therefore, the estimation unit 13 may estimate the reliability of the vehicle 100's traveling position based on the radio wave location information in accordance with the scenery recorded in the image information, based on the image information acquired by the acquisition unit 12 and the learned model.

[0032] The trained model may be generated by learning in a learning unit (not shown) that is a function of the control unit 11 of the information processing device 1 (or a learning unit arranged outside the control unit 11). Alternatively, the trained model may be generated by learning in a learning device (not shown) or the like that is external to the information processing device 1. When the trained model is generated outside the information processing device 1, the control unit 11 may acquire the trained model from the outside via the communication unit 21 or via an external memory or the like (not shown). The trained model may be stored in, for example, the storage unit 22 or the like. The estimation unit 13 may acquire the trained model described above.

[0033] For example, when classifying scenes (scenery), the estimation unit 13 may use image recognition (for example, EfficientNet, a deep learning model) to add annotations such as cityscape, mountainous area, underpass, tree cover, large vehicles running side by side, etc., to classify the scenes (scenery) recorded in the image information acquired by the acquisition unit 12. The estimation unit 13 may set the position reliability of the scene (scenery) according to the annotations added to each image. As a specific example, the estimation unit 13 may set the position reliability of the estimated scene (scenery) with respect to the traveling trajectory (traveling position) of the vehicle 100, and specify a range in which the numerical value of the reliability is equal to or less than a threshold. In this case, the estimation unit 13 may visualize the specified range (for example, a section where the reliability of the traveling position is relatively low).

[0034] The above-described estimation unit 13 may include, for example, a first estimation unit 131 and a second estimation unit 132. The first estimation unit 131 may estimate, for example, by image recognition, the scenery (the classification content of the scenery) of the image information acquired by the acquisition unit 12. In this case, the first estimation unit 131 may estimate the classification content of the scenery recorded in the image information acquired by the acquisition unit 12 based on a trained model that has learned the scenery and the classification content of the scenery as a label. During the learning stage when generating a trained model, one or more labels may be assigned to a single scene. The classification content of the landscape as a label may be, for example, a feature for estimating the landscape, or a feature for classifying the landscape. That is, the first estimation unit 131 may estimate the landscape recorded in the image information based on, for example, a trained model that has learned the landscape and the feature for estimating the landscape. Alternatively, the first estimation unit 131 may estimate the classification of the landscape recorded in the image information based on a trained model that has learned the landscape and the feature for classifying the landscape.

[0035] The classification of the scenery estimated by the first estimation unit 131 may be, for example, a classification of the surrounding conditions of the location where the vehicle 100 is traveling, such as whether the scenery recorded in the image information is a mountainous area or a seaside area, whether the area above the vehicle 100 is covered with trees (tree obstruction), whether it is under an overpass or in a city with buildings, or whether a large vehicle is traveling alongside the vehicle 100 (host vehicle). The first estimation unit 131 inputs the image information acquired by the acquisition unit 12 into a trained model and estimates the scenery (scenery classification) recorded in the image information. In this case, the first estimation unit 131 may estimate, for example, multiple scenery (scenery classifications). As an example, the first estimation unit 131 may estimate that the scenery (scenery classification) recorded in the image information is a seaside and mountainous area, a mountainous area and obstructed by trees, or an underpass and city with buildings. It should be noted that the first estimation unit 131 may estimate, for example, not only two landscapes (landscape classifications) as described above, but also three or more landscapes (landscape classifications).

[0036] For example, when the acquisition unit 12 acquires a video as image information, the first estimation unit 131 (estimation unit 13) may estimate the classification of a landscape recorded in the frame based on all frames constituting the video or a portion of frames constituting the video and the trained model. When a portion of frames is used, the first estimation unit 131 (estimation unit 13) may, for example, extract frames from the video at predetermined timings and estimate the classification of the landscape based on the extracted frames and the trained model. The predetermined timings may be, for example, every predetermined time. When a portion of frames is used, the first estimation unit 131 (estimation unit 13) can reduce the processing load of estimation compared to when all frames constituting the video are used.

[0037] For example, even when the acquisition unit 12 acquires multiple still images as image information, the first estimation unit 131 (estimation unit 13) may estimate the classification content of the scenery recorded in the frame based on all of the multiple still images or some of the multiple still images and the trained model, as in the case of the video described above.

[0038] FIG. 4 is a diagram illustrating an example of the correspondence information.

[0039] The second estimation unit 132 may estimate the reliability of the traveling position for each image based on the image information. That is, the second estimation unit 132 estimates the reliability of the traveling position based on the position information acquired by the acquisition unit 12 based on the classification of the scenery estimated by the first estimation unit 131. For example, if the scenery (scenery classification) estimated by the first estimation unit 131 is one in which the radio signal reception conditions are relatively poor, or if there is a possibility that the radio signal reception conditions will become relatively poor, the second estimation unit 132 may estimate that the reliability of the traveling position based on the radio signal position information acquired based on the radio signal is relatively low. Alternatively, for example, if the scenery (scenery classification) estimated by the first estimation unit 131 is one in which the radio signal reception conditions are relatively good, or if there is a possibility that the radio signal reception conditions will become relatively good, the second estimation unit 132 may estimate that the reliability of the traveling position based on the radio signal position information acquired based on the radio signal is relatively high.

[0040] In this case, the second estimation unit 132 may estimate the reliability of the driving position based on the position information acquired by the acquisition unit 12, based on correspondence information that associates the classification content of the scenery with the reliability of the driving position. The correspondence information may be the relationship between the scenery (classification of the scenery) and the reliability of the driving position when the scenery (classification of the scenery) occurs. 4, the correspondence information may be information that associates a category of scenery with a score indicating the reliability of the category. The second estimation unit 132 may estimate a score indicating the reliability of the traveling position based on the correspondence information.

[0041] For example, if the scenery (scenery classification) estimated by the first estimation unit 131 is one in which the reception conditions for the radio signal are relatively poor, or if there is a possibility that the reception conditions for the radio signal will become relatively poor, a relatively low value (a value indicating a relatively low reliability) may be associated with that scenery (scenery classification). Alternatively, for example, if the scenery (scenery classification) estimated by the first estimation unit 131 is one in which the reception conditions for the radio signal are relatively poor, or if there is a possibility that the reception conditions for the radio signal will become relatively poor, a relatively high value (a value indicating a relatively low reliability) may be associated with that scenery (scenery classification).

[0042] In other words, for example, if the scenery (scenery classification) estimated by the first estimation unit 131 has relatively better reception conditions for radio signals, or if there is a possibility that the reception conditions for radio signals will become relatively better, a relatively high value (a value indicating a relatively high reliability) may be associated with that scenery (scenery classification). Alternatively, for example, if the scenery (scenery classification) estimated by the first estimation unit 131 is one in which the reception conditions for the radio signal are relatively good, or if there is a possibility that the reception conditions for the radio signal will be relatively good, a relatively low value (a value indicating a relatively high reliability) may be associated with that scenery (scenery classification).

[0043] Here, the estimation unit 13 may estimate the reliability of a landscape recorded in image information based on numerical values ​​output from a trained model as possibilities for each of a plurality of landscape classification contents, for example. As a specific example, when the vehicle 100 is traveling in a mountainous area and the estimation unit 13 uses the learned model to estimate with 100% probability that the scenery recorded in the image information during the vehicle 100's travel is a mountainous area, the estimation unit 13 may use the score recorded in the corresponding information (for example, the score corresponding to the mountainous area (70 points)) as the reliability score (70 points) of the travel position. As another specific example, when the vehicle 100 is traveling in a mountainous area and the scenery recorded in the image information of the vehicle 100 traveling is estimated to be a mountainous area with a 60% probability using the trained model, the estimation unit 13 may adjust the score recorded in the correspondence information based on the 60% probability and use the adjusted score as the reliability score of the traveling location. One example of this adjustment may be an adjustment such as multiplying the probability output from the trained model by the score of the classification content of the scenery estimated using the trained model (the score of the correspondence information). In this example, the estimation unit 13 may multiply the probability (60%) by the score (70 points) to calculate 42 points as the reliability score of the traveling location.

[0044] The estimation unit 13 may estimate the reliability not only as in the example described above, but also as follows, for example. That is, the estimation unit 13 may estimate the reliability of the traveling position corresponding to the scenery recorded in the image information based on a trained model that has trained the scenery and the reliability of the traveling position as a label. The trained model is generated by, for example, learning the scenery and the reliability of the traveling position of the vehicle 100 based on the position information (radio wave position information) at the time of the scenery. During this learning, as in the case of the second estimation unit 132 described above, for example, if the reception conditions for the radio signal are relatively poor in a certain landscape, or if there is a possibility that the reception conditions for the radio signal will become relatively poor, it may be learned that the reliability of the driving position based on the radio wave position information obtained based on the radio signal is relatively low. Similarly, during this learning, for example, if the reception conditions for the radio signal are relatively good in a certain landscape, or if there is a possibility that the reception conditions for the radio signal will become relatively good, it may be learned that the reliability of the driving position based on the radio wave position information obtained based on the radio signal is relatively high. In this case, the reliability may be expressed, for example, by a score.

[0045] The trained model may be generated by training using a training unit or a training device (not shown), as described above. The control unit 11 may acquire the trained model using various methods, as described above. The trained model may be stored in the storage unit 22, for example. The estimation unit 13 may acquire the trained model described above.

[0046] For example, when the estimation unit 13 identifies a range in which the numerical value of the position reliability is equal to or less than a threshold as described above, the correction unit 14 may give a higher priority to the self-position information (e.g., position information using autonomous navigation, etc.) than the radio wave position information within the identified range, or change the weight of the self-position information (for example, by giving a higher weight to the self-position information than the radio wave position information), and correct the driving trajectory (driving position). If the position reliability is relatively high, the correction unit 14 may maintain the driving position based on the radio wave position information. If the position reliability is relatively low, the correction unit 14 may correct the driving position based on the radio wave position information. In this case, the correction unit 14 may correct the traveling trajectory (traveling position) by using, for example, various traveling trajectory (traveling position) correction algorithms. The correction unit 14 may perform the correction so as to minimize an error between the traveling trajectory (traveling position) obtained by integrating the measurement results (e.g., the traveling speed of the vehicle 100) of the inertial sensor 103 and the traveling trajectory (traveling position) based on the radio wave position information. Alternatively, the correction unit 14 may correct the traveling trajectory (traveling position) based on the radio wave position information by using, for example, image tracking and VSLAM (Visual Simultaneous Localization and Mapping) (self-location estimation).

[0047] That is, the correction unit 14 may correct the traveling position based on the position information acquired by the acquisition unit 12, based on the reliability of the traveling position estimated by the estimation unit 13. In this case, when the reliability is relatively low, the correction unit 14 may correct the traveling position with relatively low reliability. The correction unit 14 may, for example, use radio wave position information as the position information to be corrected. As an example, the correction unit 14 may correct the driving position based on the radio wave position information when the reliability corresponds to a case where the scenery (scenery classification) corresponds to at least one of underpass, cityscape with buildings, mountainous area, and tree obstruction, etc.

[0048] The correction unit 14 may correct the traveling position based on the radio wave position information according to the reliability corresponding to various kinds of scenery (scenery classification), without being limited to the above-mentioned example. That is, the correction unit 14 may correct the traveling position based on the position information acquired by the acquisition unit 12, for example, based on the relationship between the score indicating the reliability of the traveling position estimated by the second estimation unit 132 and a threshold value. The correction unit 14 may correct the traveling position based on the radio wave position information in accordance with a comparison between the score indicating the reliability and a threshold value.

[0049] As an example, when the estimation unit 13 estimates (assigns) a lower score when the reliability is low, the correction unit 14 may correct the driving position based on the radio wave position information when the score becomes equal to or less than a threshold value. Alternatively, as an example, when the estimation unit 13 estimates (assigns) a higher score when the reliability is low, the correction unit 14 may correct the driving position based on the radio wave position information when the score becomes equal to or exceeds a threshold value.

[0050] When the estimation unit 13 estimates that the reliability of the driving position is relatively low, the correction unit 14 may correct the driving position with relatively low reliability by using the driving position based on the self-position information. That is, the correction unit 14 may correct, for example, the driving position based on the radio wave position information by using the driving position based on the self-position information. In this case, the correction unit 14 may correct the driving position estimated to have a relatively low reliability or a driving position within a predetermined section including the driving position estimated to have a low reliability by using the driving position based on the self-position information.

[0051] As an example, the above-mentioned "traveling position estimated to have a relatively low reliability" may be a traveling position under an overpass, inside a tunnel, or the like. As another example, the above-mentioned "traveling position within a specified section including a traveling position estimated to have low reliability" may be a plurality of traveling positions within a section including a traveling position (first position) that is under an overpass or inside a tunnel, a traveling position up to a specified distance before the start point of that traveling position (first position), and a traveling position up to a specified distance after the end point of that traveling position (first position). As another example, the above-mentioned "traveling position within a predetermined section including a traveling position estimated to have low reliability" may be a traveling position (second position) that is under an overpass or inside a tunnel, etc., and multiple traveling positions within a link of a road network that includes the traveling position (second position). The road network may be, for example, a road map that is represented by nodes that are characteristic points of roads and links that connect adjacent nodes. In this case, the correction unit 14 may use the radio wave location information and the road network to identify which position on the road network the traveling position based on the radio wave location information corresponds to, thereby identifying the link that includes the traveling position (second position) and multiple traveling positions within the link.

[0052] For example, to correct the traveling position, the correction unit 14 may replace a traveling position (third position) estimated to have a relatively low reliability or a traveling position (traveling position based on radio wave location information) within a predetermined section including the traveling position (third position) estimated to have a low reliability with a traveling position based on the self-location information that corresponds to the traveling position (third position) or within the traveling section. For example, the correction unit 14 may replace the third position with a traveling position based on the self-location information that is closest to the third position. Alternatively, the correction unit 14 may replace the third position with a traveling position based on the self-location information that corresponds to a time closest to the time when the third position was acquired by the vehicle 100 (for example, the positioning time and the radio signal reception time, etc.). In this case, the vehicle 100 may include, for example, information on the time when the radio wave location information is acquired (for example, the positioning time and the reception time of the radio wave signal, etc.) in the radio wave location information. Similarly, the vehicle 100 may include, for example, information on the time when the vehicle 100 acquires its own location information (for example, the measurement time of the inertial measurement unit, etc.) in its own location information.

[0053] The creation unit 15 may use the travel trajectory (travel position) corrected by the correction unit 14 to update the point cloud information and the road surface orthoimage, or may use it for estimating the position of a three-dimensional object, etc.

[0054] The creation unit 15 may create a road map based on the traveling position of the vehicle 100 corrected by the correction unit 14. In this case, the creation unit 15 may update, for example, the point cloud information and road surface orthoimage information used to create the road map or the like so that they correspond to the corrected traveling position.

[0055] The point cloud information may be, for example, data generated by measurements using a 3D laser scanner or the like mounted on the vehicle 100. The vehicle 100 acquires the image information and position information described above when performing measurements using the 3D laser scanner or the like. The information processing device 1 can update the measurement position used when obtaining the point cloud information so that it corresponds to the corrected traveling position.

[0056] The road surface orthoimage information may be information about an orthoimage of the road surface, for example. The road surface orthoimage information includes, for example, position information about the position of the road. The information processing device 1 can update the position recorded in the road surface orthoimage information so that it corresponds to the corrected traveling position.

[0057] The creation unit 15 creates a road map based on, for example, at least one of point cloud information and road surface orthoimages. As an example, the creation unit 15 can create a road map using various information including road position information and feature information (shape, size, and location information of features). Furthermore, the creation unit 15 is not limited to the above example, and may create a road map by various methods using the traveling position corrected by the correction unit 14 (for example, the trajectory of the traveling position).

[0058] The output control unit 16 may control the output unit to output, for example, at least one selected from the reliability estimation result by the estimation unit 13, the travel position correction result by the correction unit 14, and a group (first group) of road maps created by the creation unit 15. Here, the output unit may be, for example, a communication unit 21, a memory unit 22, a display unit 23, etc. That is, the output control unit 16 may control the communication unit 21 to transmit at least one piece of information selected from the first group described above to an external device, etc. Here, the external device may be, for example, the vehicle 100, a server, a user terminal, etc. The user terminal is, for example, a terminal used by a user of the information processing device 1, and specific examples thereof may include a desktop, a laptop, a tablet, a smartphone, etc. The output control unit 16 may, for example, control the storage unit 22 to store at least one piece of information selected from the first group described above. The output control unit 16 may, for example, control the display unit 23 to display at least one selected from the first group described above.

[0059] [Information processing method] Next, an information processing method according to an embodiment will be described. FIG. 5 is a flowchart illustrating an information processing method according to an embodiment.

[0060] In step ST101, the acquisition unit 12 acquires image information based on an image of the outside captured by the vehicle 100, and position information relating to the traveling position of the vehicle 100. The image information may be, for example, an image of the moving body in the direction of travel, which captures a scene including the area above the moving body, or may be, for example, an image of the area above the moving body, which captures a scene including the area above the moving body. The location information may include, for example, radio wave location information and self-location information. The radio wave location information may be location information based on receiving a radio wave signal by the on-board receiver 102 of the vehicle 100. The radio wave signal may be, for example, a satellite signal using GNSS or a signal emitted from the external antenna 112. The self-location information may be location information acquired based on the measurement results of the on-board inertial sensor 103.

[0061] In step ST102, the estimation unit 13 estimates the reliability of the traveling position corresponding to the scenery recorded in the image information acquired in step ST101, based on a trained model that is based on learning the scenery recorded in the image and the label assigned to the scenery. The estimation unit 13 may estimate the reliability of the traveling position based on the radio wave location information. In this case, the estimation unit 13 (first estimation unit 131) may estimate the classification content of the landscape recorded in the image information acquired by the acquisition unit 12, for example, based on a learned model that has learned the landscape and the classification content of the landscape as a label.

[0062] In step ST103, the estimation unit 13 (second estimation unit 132) estimates the reliability of the traveling position based on the position information acquired in step ST101, based on the category of the scenery estimated in step ST102. In this case, the estimation unit 13 (second estimation unit 132) may estimate the reliability of the traveling position based on the position information acquired in step ST101, for example, based on correspondence information that associates the category of the scenery with the reliability of the traveling position. The correspondence information may be information that associates the category of the scenery with a score indicating the reliability of the category. The estimation unit 13 (second estimation unit 132) may estimate the score indicating the reliability of the traveling position based on the correspondence information.

[0063] Alternatively, as a processing step instead of the processing of the above-mentioned step ST102 and step ST103, the estimation unit 13 may estimate the reliability of the driving position according to the scenery recorded in the image information, for example, based on a trained model that has learned the scenery and the reliability of the driving position as a label.

[0064] In step ST104, the correction unit 14 may correct the traveling position based on the position information acquired in step ST101, based on the reliability of the traveling position estimated in step ST103, etc. In this case, if the reliability is relatively low, the correction unit 14 may correct the traveling position with relatively low reliability. As an example, the correction unit 14 may correct the traveling position based on the position information acquired in step ST101, based on the relationship between the score of the reliability of the traveling position estimated in step ST103, etc. and a threshold value. For example, if the reliability of the traveling position estimated in step ST103, etc. is relatively low, the correction unit 14 may correct the traveling position with relatively low reliability by using the traveling position based on the self-location information.

[0065] In step ST105, the creation unit 15 creates a road map based on the traveling position of the vehicle 100 corrected in step ST104. [Variations] In the above-described embodiment, an example has been described in which the control unit 11 includes the acquisition unit 12, the estimation unit 13 (the first estimation unit 131 and the second estimation unit 132), the correction unit 14, the creation unit 15, and the output control unit 16. However, the present disclosure is not limited to this example. That is, the control unit 11 may be configured to include any of the acquisition unit 12, the estimation unit 13 (the first estimation unit 131 and the second estimation unit 132), the correction unit 14, the creation unit 15, and the output control unit 16. As an example, the control unit 11 of the information processing device 1 may be configured to include a correction unit 14, a creation unit 15, an output control unit 16, etc. In this case, a computer (on-board device) (for example, an arithmetic processing device such as an on-board control unit) (not shown) mounted on the vehicle 100 may be configured to include an acquisition unit 12, an estimation unit 13, etc. Alternatively, as another example, the control unit 11 of the information processing device 1 may be configured to include a second estimation unit 132, a correction unit 14, a creation unit 15, an output control unit 16, etc. In this case, a computer (on-board device) (for example, an arithmetic processing device such as an on-board control unit) (not shown) mounted on the vehicle 100 may be configured to include the acquisition unit 12 and the first estimation unit 131.

[0066] This allows the vehicle 100 to estimate the classification of the scenery recorded in the image information or the reliability of the location information using an on-board device without sending the image information to the information processing device 1. Even in this case, the information processing device 1 acquires location information (for example, radio wave location information and self-location information, etc.) from the vehicle 100. That is, by sending the classification of the scenery (determination information that determines the scene at the location where the vehicle 100 is traveling) or reliability information regarding the reliability of the traveling location to the information processing device 1, the vehicle 100 can reduce the amount of communication, etc., compared to when sending image information to the information processing device 1.

[0067] The information processing device 1 can estimate the reliability of the traveling position based on, for example, the classification content of the scenery acquired from the vehicle 100. Alternatively, the information processing device 1 can correct the traveling position based on, for example, the reliability of the traveling position acquired from the vehicle 100.

[0068] Each unit of the information processing device 1 described above may be realized as a function of a computer's arithmetic processing device, etc. That is, the acquisition unit 12, the estimation unit 13 (first estimation unit 131 and second estimation unit 132), the correction unit 14, the creation unit 15, and the output control unit 16 (control unit 11) of the information processing device 1 may be realized as an acquisition function, an estimation function (first estimation function and second estimation function), a correction function, a creation function, and an output control function (control function), respectively, by a computer's arithmetic processing device, etc. The information processing program can cause a computer to realize each of the above-mentioned functions. The information processing program may be recorded on a non-transitory computer-readable recording medium, such as a memory, a solid-state drive, a hard disk drive, or an optical disk. The recording medium may also be referred to as a non-transitory computer-readable medium. Furthermore, as described above, each unit of the information processing device 1 may be realized by an arithmetic processing device of a computer or the like. The arithmetic processing device or the like is configured by, for example, an integrated circuit or the like. Therefore, each unit of the information processing device 1 may be realized as a circuit that constitutes the arithmetic processing device or the like. That is, the acquisition unit 12, the estimation unit 13 (first estimation unit 131 and second estimation unit 132), the correction unit 14, the creation unit 15, and the output control unit 16 (control unit 11) of the information processing device 1 may be realized as an acquisition circuit, an estimation circuit (first estimation circuit and second estimation circuit), a correction circuit, a creation circuit, and an output control circuit (control circuit) that constitute the arithmetic processing device of a computer or the like. Furthermore, the communication unit 21, the storage unit 22, and the display unit 23 (output unit) of the information processing device 1 may be realized as a communication function, a storage function, and a display function (output function) including the functions of an arithmetic processing device, etc. Furthermore, the communication unit 21, the storage unit 22, and the display unit 23 (output unit) of the information processing device 1 may be realized as a communication circuit, a storage circuit, and a display circuit (output circuit) by being configured, for example, by an integrated circuit, etc. Furthermore, the communication unit 21, the storage unit 22, and the display unit 23 (output unit) of the information processing device 1 may be configured as a communication device, a storage device, and a display device (output device) by being configured, for example, by being configured by a plurality of devices.

[0069] The information processing device 1 can be configured by combining one or any combination of the above-described multiple units. In this disclosure, the term "information" is used, but the term "information" can be replaced with "data" and the term "data" can be replaced with "information."

[0070] [Aspects and Effects of the Present Embodiment] Next, one aspect of this embodiment and the effects of each aspect will be described. Note that each aspect described below is an example at the time of filing, and this embodiment is not limited to the aspects described below. In other words, this embodiment is not limited to each aspect described below, and may be realized by appropriately combining each of the above-mentioned parts. Furthermore, the effects described below are only examples, and the effects achieved by each aspect are not limited to those described below. Furthermore, each aspect may, for example, achieve at least one of the effects described below.

[0071] (Aspect 1) An information processing device of one embodiment includes an acquisition unit that acquires image information based on an image of the outside of the vehicle and position information regarding the vehicle's driving position, and an estimation unit that estimates the reliability of the driving position corresponding to the scenery recorded in the image information based on a trained model that is based on learning the scenery recorded in the image and the label assigned to the scenery. This allows the information processing device to estimate the reliability of the traveling position based on the position information (radio wave position information), for example. Furthermore, since the information processing device estimates the reliability of the traveling position based on image information, more accurate estimation can be achieved. Here, for example, when performing positioning using GNSS or the like, even if the reception strength of the satellite signals is not reduced, the accuracy of the positioning relatively decreases when a relatively small number of satellite signals are received. Furthermore, when a vehicle performs positioning by receiving satellite signals, for example, it may be difficult to determine whether the received satellite signals are reliable for positioning. That is, when a vehicle receives multipath satellite signals, for example, it may be difficult to determine whether there is multipath, which satellite the satellite signals are transmitted from (for example, whether the satellite signals are transmitted from a satellite near the zenith), and whether the satellite signals are reliable for positioning. The above-mentioned concerns regarding satellite signals also apply to Bluetooth signals and the like. In this regard, in this embodiment, the information processing device can estimate the reliability of the traveling position more accurately by using image information. Furthermore, it is not possible to obtain information about the situation of, for example, obstruction by trees at a relatively high elevation angle and obstruction by buildings at a relatively low elevation angle from a road map, etc. In contrast, the information processing device can obtain the situation of obstruction by trees, buildings, etc. by using image information, and can obtain a more accurate reliability of the driving position according to the road on which the vehicle is traveling and the surrounding conditions.

[0072] (Aspect 2) In one embodiment of the information processing device, the estimation unit includes a first estimation unit that estimates the classification content of a landscape recorded in image information acquired by the acquisition unit based on a learned model that has learned the landscape and the classification content of the landscape as a label, and a second estimation unit that estimates the reliability of a driving position based on the position information acquired by the acquisition unit based on the classification content of the landscape estimated by the first estimation unit. This allows the information processing device to use the image information to more accurately estimate the reliability of the traveling position.

[0073] (Aspect 3) In one embodiment of the information processing device, the second estimation unit may estimate the reliability of the driving position based on the position information acquired by the acquisition unit based on correspondence information that associates the classification content of the scenery with the reliability of the driving position. This allows the information processing device to use the image information to more accurately estimate the reliability of the traveling position.

[0074] (Aspect 4) In one embodiment of the information processing device, the second estimation unit may estimate a score indicating the reliability of the driving position based on corresponding information that associates the classification content of the scenery with a score indicating the reliability of the classification content. This allows the information processing device to use the image information to more accurately estimate the reliability of the traveling position.

[0075] (Aspect 5) In one aspect of the information processing device, the estimation unit may estimate the reliability of the driving position corresponding to the scenery recorded in the image information based on a trained model that has learned the scenery and the reliability of the driving position as a label. This allows the information processing device to use the image information to more accurately estimate the reliability of the traveling position.

[0076] (Aspect 6) The information processing device of one embodiment may include a correction unit that corrects the traveling position based on the position information acquired by the acquisition unit based on the relationship between the reliability score of the traveling position estimated by the second estimation unit and a threshold value. This allows the information processing device to correct the running position as necessary depending on the magnitude relationship between the reliability score and the threshold value, thereby obtaining a more accurate running position (for example, a trajectory of the running position).

[0077] (Aspect 7) The information processing device according to one aspect may include a correction unit that corrects the traveling position based on the position information acquired by the acquisition unit, based on the reliability of the traveling position estimated by the estimation unit. When the information processing device obtains a relatively more accurate driving position (a driving position based on more accurate radio wave position information) (for example, when more accurate positioning can be performed), it can maintain higher position accuracy for driving positions with higher reliability by correcting only driving positions with low reliability or driving positions with low reliability. Furthermore, for example, when correcting multiple driving positions within a predetermined section (or all driving positions within the predetermined section) including a driving position with low reliability, the information processing device may not need to estimate the reliability for all driving positions based on the radio wave location information. That is, by estimating the reliability for one of all driving positions based on the radio wave location information, the information processing device can reduce the processing burden for the estimation compared to estimating the reliability for all driving positions. In this case, the information processing device (estimation unit) may estimate the reliability at, for example, a predetermined timing. The predetermined timing may be, for example, at a predetermined time interval or at a predetermined travel distance based on the radio wave location information, etc. Furthermore, the information processing device can obtain a more accurate running position (for example, a trajectory of the running position) by correcting the running position to be corrected.

[0078] (Aspect 8) In the information processing device of one aspect, when the reliability is relatively low, the correction unit may correct the traveling position with the relatively low reliability. This allows the information processing device to correct a driving position with a relatively low reliability and maintain a driving position with a relatively high reliability as a more accurate driving position based on radio wave position information.

[0079] (Aspect 9) In one embodiment of the information processing device, the acquisition unit acquires, as location information, radio wave location information based on receiving radio wave signals with the vehicle's onboard receiver and self-location information based on measurement with the vehicle's onboard inertial sensor, the estimation unit estimates the reliability of the driving location based on the radio wave location information, and if the estimation unit estimates that the reliability of the driving location is relatively low, the correction unit may correct the driving location with relatively low reliability using the driving location based on the self-location information. This allows the information processing device to correct the traveling position based on radio wave location information with a relatively low reliability to the traveling position based on self-location information with a relatively high reliability.

[0080] (Aspect 10) The information processing device according to one aspect may include a creating unit that creates a road map based on the vehicle's traveling position corrected by the correcting unit. This allows the information processing device to create a road map with higher accuracy by using the corrected driving position, etc.

[0081] (Aspect 11) In one aspect of the information processing method, a computer executes an acquisition step of acquiring image information based on an image of the outside captured by the vehicle and position information regarding the vehicle's traveling position, and an estimation step of estimating the reliability of the traveling position according to the scenery recorded in the image information based on a trained model based on learning the scenery recorded in the image and the label assigned to that scenery. As a result, the information processing method may achieve the same effect as the information processing device of the above-described aspect.

[0082] (Aspect 12) One embodiment of an information processing program enables a computer to implement an acquisition function that acquires image information based on images of the outside captured by the vehicle and location information regarding the vehicle's driving location, and an estimation function that estimates the reliability of the driving location according to the scenery recorded in the image information based on a trained model that learns the scenery recorded in the image and the labels assigned to the scenery. As a result, the information processing program may achieve the same effect as the information processing device of the above-described aspect. [Explanation of symbols]

[0083] 1. Information processing equipment 11 Control section 12 Acquisition Department 13 Estimation part 131 1st estimation part 132 Second estimation part 14 Correction unit 15 Creation Department 16 Output control section 21 Communications Department 22 Memory section 23 Display section 100 vehicles 101 Imaging unit 102 In-vehicle receiver 103 Inertial Sensor for Automotive 111 Artificial satellite (GNSS) 112 External Antenna 200 servers 300 road 301 Underpass section 302 Section not under elevated tracks 303 Running position

Claims

1. an acquisition unit that acquires image information based on an image of the outside captured by the vehicle and position information related to a traveling position of the vehicle obtained by using a Global Navigation Satellite (GNSS); and an estimation unit that estimates the reliability of the driving position according to the scenery recorded in the image information based on a trained model based on learning the scenery recorded in the image and a label that specifies the quality of the reception environment of the GNSS radio signal that is assigned to the scenery, The estimation unit is a first estimation unit that estimates a classification content of a landscape recorded in image information acquired by the acquisition unit based on the trained model that has learned the landscape and a classification content that classifies, as the label, a quality of a reception environment of a GNSS radio wave signal according to the landscape; a second estimation unit that estimates a reliability of a traveling position based on the position information acquired by the acquisition unit, based on the classification content of the scenery estimated by the first estimation unit; Equipped Information processing device.

2. The second estimation unit estimates the reliability of the traveling position based on the position information acquired by the acquisition unit, based on correspondence information that associates the classification content of the scenery with the reliability of the traveling position. The information processing device according to claim 1 .

3. The second estimation unit estimates a score indicating the reliability of the traveling position based on information in which a classification content of the scenery is associated with a score indicating the reliability of the classification content as the correspondence information. The information processing device according to claim 2 .

4. The estimation unit estimates the reliability of the traveling position according to the scenery recorded in the image information based on a trained model that has trained the scenery and the reliability of the traveling position as the label. The information processing device according to claim 1 .

5. a correction unit that corrects the traveling position based on the position information acquired by the acquisition unit based on a relationship between a score of the reliability of the traveling position estimated by the second estimation unit and a threshold value; The information processing device according to any one of claims 1 to 4.

6. A correction unit corrects a traveling position based on the position information acquired by the acquisition unit based on a reliability of the traveling position estimated by the estimation unit. The information processing device according to any one of claims 1 to 4.

7. The correction unit corrects the traveling position having a relatively low reliability when the reliability is relatively low.

7. The information processing device according to claim 5 or 6.

8. The acquisition unit acquires, as the location information, radio wave location information based on receiving a radio wave signal using GNSS with an in-vehicle receiver of the vehicle, and self-location information based on measurement with an inertial sensor of the vehicle, The estimation unit estimates a reliability of a traveling position based on radio wave position information, When the estimation unit estimates that the reliability of the traveling position is relatively low, the correction unit corrects the traveling position having the relatively low reliability by using a traveling position based on self-position information.

8. The information processing device according to claim 6 or 7.

9. A creation unit that creates a road map based on the vehicle travel position corrected by the correction unit. The information processing device according to any one of claims 5 to 8.

10. The computer an acquisition step of acquiring image information based on an image of the outside captured by the vehicle and position information related to a traveling position of the vehicle obtained by using a Global Navigation Satellite (GNSS); and an estimation step of estimating the reliability of the driving position according to the scenery recorded in the image information based on a trained model based on learning the scenery recorded in the image and the label that specifies the quality of the reception environment of the GNSS radio signal that is assigned to the scenery, and The estimation step includes: a first estimation step of estimating a classification content of a scenery recorded in image information acquired by the acquisition step based on the trained model that has learned the scenery and a classification content that classifies, as the label, the quality of a reception environment of a GNSS radio wave signal according to the scenery; a second estimation step of estimating a reliability of a traveling position based on the position information acquired in the acquisition step, based on the classification content of the scenery estimated in the first estimation step; Run Information processing methods.

11. On the computer, An acquisition function for acquiring image information based on an image of the outside taken by the vehicle and position information related to a traveling position of the vehicle obtained by using a Global Navigation Satellite (GNSS); and an estimation function for estimating the reliability of the driving position according to the scenery recorded in the image information based on a trained model based on learning the scenery recorded in the image and the label that specifies the quality of the reception environment of the GNSS radio signal that is assigned to the scenery, The estimated function is A first estimation function that estimates the classification content of a landscape recorded in image information acquired by the acquisition function based on the trained model that has learned the landscape and the classification content that classifies the reception environment of a GNSS radio signal according to the landscape as the label; and a second estimation function that estimates a reliability of a traveling position based on the position information acquired by the acquisition function, based on the classification content of the scenery estimated by the first estimation function; Make it happen Information processing program.

Images