Information processing device, information processing method, and program

Abstract

The information processing device of the present invention comprises: an acquisition unit for acquiring sensor detection information representing detection results from a sensor mounted on a vehicle; an export unit for exporting multiple indicators for the surrounding environment based on the sensor detection information acquired by the acquisition unit; and an evaluation unit for evaluating attribute information of the location where the sensor detection information was acquired based on the multiple indicators exported by the export unit.

Description

Technical Field

[0001] The present invention relates to an information processing device, an information processing method, and a program.

[0002] This application claims priority based on Tokugan Application No. 2017-118694 filed in Japan on June 16, 2017, the contents of which are incorporated herein by reference. Background Art

[0003] Among navigation devices that provide passengers with information on the route of a vehicle, there is a navigation device that provides not only information on the route to a destination but also store information and sightseeing information around the point where the vehicle is traveling (for example, Patent Document 1).

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2007-285870 Summary of the invention

[0007] Problems to be solved by the invention

[0008] However, the conventional technology does not obtain information related to the scenery provided to the passengers from the actually traveling vehicle, nor does it evaluate the beauty of the scenery around the actual vehicle traveling.

[0009] The present invention has been made in view of such a situation, and one object of the present invention is to provide an information processing device, an information processing method, and a program that can automatically index the beauty of a landscape based on information acquired from a traveling vehicle.

[0010] Means for solving problems

[0011] The information processing device according to the present invention has the following configuration.

[0012] (1): An information processing device involved in one embodiment of the present invention comprises: an acquisition unit for acquiring sensor detection information representing a detection result from a sensor mounted on a vehicle (for example, the information acquisition unit 210 of the embodiment); an export unit for exporting a plurality of indicators for a surrounding environment based on the sensor detection information acquired by the acquisition unit (for example, the indicator export unit 220 of the embodiment); and an evaluation unit for evaluating attribute information of a location where the sensor detection information was acquired based on the plurality of indicators exported by the export unit (for example, the landscape evaluation unit 230 of the embodiment).

[0013] (2) In the above-mentioned embodiment (1), the evaluation unit derives an evaluation value of the sensitivity index at the location in association with the location where the sensor detection information is obtained, based on the plurality of indexes and the vehicle information including the location information.

[0014] (3) In the above-mentioned embodiment (2), the evaluation unit derives the evaluation value of the sensitivity index in association with a time contained in a unit of a given time length.

[0015] (4) In the above-mentioned aspect (2) or (3), the device further comprises: an information providing unit that provides information related to the scenery to the vehicle based on the evaluation value.

[0016] (5): Based on the solution of (4) above, the information providing unit provides information related to the scenery to the vehicle in accordance with the association between the category of the beauty level selected by the passenger of the vehicle and the attribute information.

[0017] (6): Based on any one of the above solutions (1) to (5), the vehicle is an autonomous driving vehicle, and the vehicle performs at least one of lane changes or changes in distance relationship with other vehicles to improve the beauty evaluation of the evaluation unit.

[0018] (7): The information processing method involved in each scheme of the present invention is executed by a computer: obtaining sensor detection information representing the detection results from a sensor mounted on a vehicle, deriving multiple indicators for the surrounding environment based on the obtained sensor detection information, and evaluating the attribute information of the location where the sensor detection information was obtained based on the derived multiple indicators.

[0019] (8): A program involved in one scheme of the present invention enables a computer to execute: obtaining sensor detection information representing detection results from a sensor mounted on a vehicle, deriving multiple indicators for the surrounding environment based on the obtained sensor detection information, and evaluating attribute information of the location where the sensor detection information was obtained based on the derived multiple indicators.

[0020] (9): Based on the scheme of (1) above, the derivation unit derives a plurality of indicators representing attributes of the surrounding environment on the vehicle by inputting the sensor detection information obtained by the acquisition unit into a plurality of evaluation units defined in advance corresponding to the purpose used in the control, and the plurality of evaluation units include: a first evaluation unit that uses the sensor detection information to determine control attributes reflected in the control content of the vehicle; and a second evaluation unit that uses the sensor detection information to determine environmental attributes representing attributes of the environment for the location.

[0021] (10) In the above-mentioned embodiment (9), the first evaluation unit determines the control attribute at a higher frequency than the second evaluation unit.

[0022] (11): Based on the above-mentioned solution (9), the information processing device includes: an updating unit that updates the definition of the evaluation unit, and the information processing device can add the attribute information output from the evaluation unit.

[0023] (12): Based on the above solution (4), it includes: a data generation unit that establishes a correspondence between the sensor detection information and the multiple indicators and the position information, and selectively generates transmission data to be sent to the outside according to the contents of the multiple indicators.

[0024] (13): Based on the scheme of (12) above, it is provided with: a communication unit that receives a response signal sent back to the transmission data sent to the outside, and the information providing unit provides landscape information representing information related to the landscape around the vehicle based on either the attribute information determined by the vehicle and the updated attribute information contained in the response signal.

[0025] (14): An information processing device involved in one embodiment of the present invention comprises: an acquisition unit for acquiring sensor detection information representing a detection result from a sensor mounted on a vehicle via communication; an exporting unit for exporting a plurality of indicators of the surrounding environment of the vehicle based on the sensor detection information acquired by the acquisition unit; an evaluation unit for evaluating attribute information of a location where the sensor detection information was acquired based on the plurality of indicators exported by the exporting unit and the attribute information acquired from the vehicle; and an information providing unit for sending the evaluation result evaluated by the evaluation unit to the vehicle.

[0026] Effects of the Invention

[0027] According to (1), (7), (8), and (14), attribute information of the surrounding environment of the vehicle can be evaluated based on information of the landscape around the vehicle detected by the vehicle.

[0028] According to (2), it is possible to evaluate the perceptual index seen by the vehicle at the location where the vehicle actually travels.

[0029] According to (3), by evaluating the sensory index from the vehicle at the time when the vehicle is actually running, it is possible to evaluate the sensory index that changes in time zones and seasons.

[0030] According to (4), by providing the information of the evaluated perceptual index to the vehicle, the passenger can select a route corresponding to the perceptual index to drive.

[0031] According to (5), a passenger selects a category of scenery with a high degree of beauty and can select a route with a high degree of beauty corresponding to the category for driving.

[0032] According to (6), the autonomous driving vehicle can drive with improved scenic views by changing its own lane and its position relative to other vehicles.

[0033] According to (9), the vehicle can use the derived multiple indices for vehicle driving control and route information provision.

[0034] According to (10), the vehicle can perform driving support control of the vehicle by determining the attributes related to driving control with high frequency.

[0035] According to (11), the user can set the attribute that he wants to investigate and obtain the evaluation result.

[0036] According to (12), when there is no calculated attribute value or when the condition is not satisfied, transmission data is not generated, which can improve communication efficiency.

[0037] According to (13), in addition to the attribute information used in the vehicle determination, the attribute information updated in the external server can also be used, which can make the support of the vehicle driving control more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a diagram showing an example of the configuration of the information processing system 1 according to the embodiment.

[0039] Figure 2 1 is a diagram showing an example of the structure of the external sensing unit 110 according to the embodiment.

[0040] Figure 3 1 is a diagram showing an area around vehicle 100 detected by each sensor according to the embodiment.

[0041] Figure 4 1 is a diagram showing an example of the content of the detection data 115 generated by the object recognition device 114 according to the embodiment.

[0042] Figure 5 It is a diagram showing an example of the structure of the navigation device 120 according to the embodiment.

[0043] Figure 6 is a histogram showing the distribution of distances according to an embodiment.

[0044] Figure 7 It is a histogram showing the distribution of chroma according to the embodiment.

[0045] Figure 8 It is a diagram showing the movement of the subject in the continuously captured images according to the embodiment.

[0046] Fig. 9 This is a diagram showing an example of the content of the derived index value data 222 according to the embodiment.

[0047] Fig.10This is a diagram showing an example of the content of POI data of a place with a high degree of beauty according to the embodiment.

[0048] Fig.11 : This is a graph showing an example of the relationship between the driving data of the vehicle 100 and the final evaluation value according to the embodiment.

[0049] Fig.12 This is a flowchart showing an example of the flow of processing executed in the information processing system 1 according to the embodiment.

[0050] Fig.13 This is a diagram showing a configuration in which the autonomous driving vehicle 300 according to the embodiment is applied to the information processing system 2 .

[0051] Fig.14 1 is a diagram showing an example of a structure of an automatically driven vehicle 300 according to a modification.

[0052] Fig.15 This is a diagram showing an example of the content of data included in the control attribute.

[0053] Fig.16 This is a diagram showing an example of the content of data included in the environment attribute.

[0054] Fig.17 It is a diagram showing an example of the configuration of the external server 400 . DETAILED DESCRIPTION

[0055] Embodiments of the information processing device, the information processing method, and the program according to the present invention will be described below with reference to the drawings.

[0056] Figure 1 1 is a diagram showing an example of the structure of an information processing system 1 according to an embodiment. The information processing system 1 includes, for example, a vehicle 100 and an information processing device 200. The vehicle 100 performs wireless communication and communicates with the information processing device 200 via a network NW. The information processing device 200 obtains the detection results of the sensors mounted on the vehicle 100 and evaluates the beauty of the place where the vehicle 100 has traveled.

[0057] [vehicle]

[0058] The vehicle 100 includes, for example, an external sensing unit 110, a navigation device 120, a communication device 130, and a control unit 140. The external sensing unit 110 acquires external information using a sensor mounted on the vehicle 100 and sensing the outside.

[0059] Figure 21 is a diagram showing an example of the configuration of the external sensing unit 110. The external sensing unit 110 includes a camera 111, a radar device 112, a detector 113, and an object recognition device 114 as sensors. These sensors are used, for example, as external monitoring sensors for autonomous driving.

[0060] The camera 111 is a digital camera using a solid-state imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 111 captures the surroundings of the vehicle 100. One or more cameras 111 are installed at any position of the vehicle 100 to capture the surroundings of the vehicle 100. When capturing the front, the camera 111 is installed on the upper part of the front windshield, the back of the interior mirror, etc.

[0061] In addition, the camera 111 is installed near the rear bumper when the camera 111 is used to shoot the rear. In the case of shooting the left and right directions, the camera 111 is installed on the left and right side mirrors. The camera 111 may be a stereo camera installed on the top of the vehicle 100 to shoot 360° surrounding scenery. The camera 111 shoots the surroundings of the vehicle 100 periodically and repeatedly.

[0062] The radar device 112 radiates radio waves such as millimeter waves to the periphery of the vehicle 100, and detects radio waves (reflected waves) reflected by an object to detect at least the position (distance and direction) of the object. One or more radar devices 112 are installed at any position of the vehicle 100. The radar device 112 can detect the position and speed of the object by FMCW (Frequency Modulated Continuous Wave). In addition, a distance camera for measuring the distance can be used for measuring the distance.

[0063] The detector 113 is a LIDAR (Light Detection and Ranging or Laser Imaging Detection and Ranging) that detects the distance to an object by measuring scattered light of the irradiated light. One or more detectors 113 are installed at any position of the vehicle 100 .

[0064] Figure 31 is a diagram showing the area around the vehicle 100 detected by each sensor. (a) is the detection area of ​​the sensor detecting the front, (b) is the detection area of ​​the sensor detecting the right, (c) is the detection area of ​​the sensor detecting the rear, and (d) is the detection area of ​​the sensor detecting the left. The camera 111, radar device 112, and detector 113 mounted on the vehicle 100 can sense the front, back, left, and right directions of the vehicle 100.

[0065] The object recognition device 114 performs sensor fusion processing on part or all of the detection results of the camera 111, the radar device 112, and the detector 113 to recognize the position, type, speed, etc. of objects outside the vehicle 100. The object recognition device 114 recognizes the position, speed, acceleration, etc. of surrounding objects, structures, etc., and recognizes the recognized objects around the vehicle 100. The position of the surrounding objects can be represented by representative points such as the center of gravity and corners of the object, or it can be represented by an area represented by the outline of the object. The objects recognized by the object recognition device 114 include, for example, structures, buildings, trees, guardrails, telephone poles, parked vehicles, pedestrians, and other objects in addition to surrounding vehicles. Such a function is used to recognize surrounding objects of the vehicle 100 during autonomous driving.

[0066] The object recognition device 114 collects the data detected by each sensor at a predetermined timing to generate detection data 115. The object recognition device 114 generates the detection data 115 by sampling at a predetermined sampling interval. Figure 4 114 is a diagram showing an example of the content of the detection data 115 generated by the object recognition device 114. The detection data 115 includes, for example, the vehicle position, the travel direction, the detection results of each sensor, the date and time, etc. The detection data 115 is an example of sensor detection information showing the detection results of the external sensing unit 110.

[0067] The vehicle position is data indicating the position at which an image or the like is acquired. The object recognition device 114 acquires position data from the navigation device 120 at each sampling period and sets it as the vehicle position. The traveling direction data is data recording the traveling direction of the vehicle 100. The object recognition device 114 acquires the traveling direction data from changes in the position data or the like.

[0068] Camera 1, camera 2, ... include image data captured in multiple directions around the vehicle 100. Radar 1, ... include data on the results of detecting objects by the radar device 112 in multiple directions around the vehicle 100. Detector 1, ... include data on objects detected by the detector 113 in multiple directions around the vehicle 100. Date and time data is information on the date and time when images, detection results, etc. are acquired.

[0069] Figure 51 is a diagram showing an example of the structure of the navigation device 120. The navigation device 120 performs route guidance according to the route that the vehicle 100 travels to the destination. The navigation device 120 includes, for example, a GNSS (Global Navigation Satellite System) receiver 121, a navigation HMI 122, and a route determination unit 123, and stores map information 126 in a storage unit 125 such as a HDD (Hard Disk Drive) or a flash memory.

[0070] The GNSS receiver 121 determines the position (latitude, longitude, altitude) of the vehicle 100 based on signals received from GNSS satellites. The position of the vehicle 100 can be determined or supplemented by an INS (Inertial Navigation System) using the output of the vehicle sensor 60. The navigation device 120 generates position data and travel direction data of the vehicle 100 based on the received data of the GNSS receiver 121.

[0071] The navigation HMI 122 includes a display device, a speaker, a touch panel, buttons, etc. The navigation HMI 122 may be partially or entirely common with the aforementioned HMI 110. The route determination unit 123 refers to the map information 126 to determine, for example, a route from the position of the vehicle 100 determined by the GNSS receiver 121 (or an input arbitrary position) to the destination input by the passenger using the navigation HMI 122 (for example, including information related to the via points when traveling to the destination).

[0072] The map information 126 is information that represents the shape of a road by, for example, lines representing the road and nodes connected by the lines. The map information 126 may include the curvature of the road, POI (Point Of Interest) information, etc. The POI includes information on places with high beauty obtained from the information processing device 200 as described later.

[0073] The map information 126 can be updated at any time by accessing the information processing device 200 via the communication device 130 and the network NW. The map information 126 can be further supplemented with information related to POIs of network users obtained via the network NW. The POIs on the route being driven can be displayed on the navigation HMI 122 .

[0074] The navigation device 120 performs route guidance using the navigation HMI 122 based on the route determined by the route determination unit 123. In addition, the navigation device 120 can be implemented by the function of a terminal device such as a smartphone or a tablet terminal owned by the user. In addition, the navigation device 120 can also send the current position and destination to the information processing device 200 and other navigation servers (not shown) via the communication device 130, and obtain the route sent back from them.

[0075] The communication device 130 performs wireless communication using, for example, a cellular network, a Wi-Fi network, Bluetooth (registered trademark), DSRC (Dedicated Short Range Communication), etc., and communicates with the information processing device 200 via the network NW.

[0076] The control unit 140 transmits the detection data 115 indicating the detection result by the external sensor unit 110 to the information processing device 200 via the communication device 130 and the network NW. In addition, the control unit 140 displays the information transmitted from the information processing device 200 on the navigation HMI 122 via the communication device 130 .

[0077] The control unit 140 is implemented by executing a program (software) through a processor such as a CPU (Central Processing Unit). In addition, part or all of the control unit 140, the external sensing unit 110, and the navigation device 120 can be implemented by hardware such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), or by the collaboration of software and hardware.

[0078] [Information processing device]

[0079] like Figure 1 As shown, the information processing device 200 includes, for example, an information acquisition unit 210 , an index derivation unit 220 , a landscape evaluation unit 230 , an information provision unit 240 , and a storage unit 260 .

[0080] The index derivation unit 220, the landscape evaluation unit 230, and the information provision unit 240 are implemented by executing programs (software) by processors such as a CPU (Central Processing Unit). In addition, some or all of these functional units can be implemented by hardware such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), or by the collaboration of software and hardware. The program can be pre-stored in a storage device such as an HDD (Hard Disk Drive) or a flash memory, or in a removable storage medium such as a DVD or CD-ROM, or installed in a storage device by equipping the storage medium with a drive device (not shown).

[0081] The information acquisition unit 210 includes, for example, a NIC (Network Interface Card) for connecting to the network NW. The information acquisition unit 210 acquires the detection data 115 from the external sensor unit 110 mounted on the vehicle via the network NW.

[0082] The index deriving unit 220 derives a plurality of indices representing the beauty of the scenery around the vehicle 100 based on the detection data 115 acquired by the information acquiring unit 210. The index deriving unit 220 derives the plurality of indices by performing image analysis, calculation, etc. based on the detection data 115. The method of deriving the plurality of indices will be described in detail later.

[0083] The landscape evaluation unit 230 evaluates the beauty level of the location where the detection data 115 is acquired based on the plurality of indices derived by the index derivation unit 220. The evaluation method of the beauty level will be described in detail later.

[0084] The information providing unit 240 also transmits the information of the beauty level evaluated by the landscape evaluating unit 230 to the vehicle 100 via the network NW. When the information processing device 200 is a navigation server, the information reflecting the beauty level evaluated by the landscape evaluating unit 230 may be reflected in the route search result.

[0085] The storage unit 260 is implemented by, for example, a RAM (Random Access Memory), a ROM (Read Only Memory), a HDD (Hard Disk Drive), a flash memory, or a hybrid storage device obtained by combining multiple of them. In addition, a part or all of the storage unit 260 may be an external device such as a NAS (Network Attached Storage) or an external storage server that can be accessed by the information processing device 200. The storage unit 260 stores, for example, map information 261 and landscape information 262.

[0086] The map information 261 is information that expresses the shape of a road by, for example, lines representing the road and nodes connected by the lines. The map information 261 includes the curvature of the road, POI information, and the like.

[0087] The landscape information 262 stores information related to the beauty level evaluated by the landscape evaluation unit 230. The information related to the beauty level is stored in association with, for example, a plurality of coordinates (positions) of nodes or links stored in the map information 261. As the coordinates, POIs may be associated.

[0088] [Information generation method]

[0089] Next, a method for generating information related to beauty by the information processing device 200 is described. The index deriving unit 220 refers to the detection data 115 acquired by the information acquiring unit 210 and derives a plurality of index values ​​X1 to Xn (n is an arbitrary natural number) related to beauty in association with positions.

[0090] The index deriving unit 220 derives the average, maximum, and other statistical values ​​of the indexes for each direction obtained based on the data of the plurality of sensors for the front, rear, left, and right directions around the vehicle 100 as the index value Xn. The index deriving unit 220 can derive the index value Xn for each direction. The so-called deriving the index value Xn for each direction means deriving the index value Xn for each direction such as the forward direction from the driver's seat and the front passenger seat, the direction from the side windows to the outside, and the direction from the rear window to the rear.

[0091] The index deriving unit 220 derives index values ​​related to, for example, distance, transmittance, color, contrast, moving object, height, etc., as an index indicating the beauty level.

[0092] Regarding the distance, the index deriving unit 220 derives the index using the distance data of the radar device 112 recorded in the detection data 115. The index deriving unit 220 can use the distance data obtained by the external sensing unit 110 to calculate the distance, and can also calculate the distance corresponding to the pixel based on the difference of the change of the image data based on the plurality of image data and the focal length.

[0093] When deriving the index of the distance, the index deriving unit 220 calculates the distribution of the distance detected in the front, rear, left, and right directions of the vehicle 100 based on the distance data. For example, when looking at the 360 ​​degrees around the vehicle, the distance is obtained in one degree of horizontal angle and the distribution of the distance is calculated. The distribution is represented in the form of a histogram. Figure 6 is a histogram showing the distribution of the distances. Then, the index deriving unit 220 derives an index value showing the spread of the distribution of the distances, for example, using the variance value of the distribution of the distances as a parameter.

[0094] When deriving an index of transmittance, the index deriving unit 220 calculates a contrast index (frequency component) of pixels at a distance greater than a given distance in the image. For example, the pixels at a distance greater than a given distance are extracted from an image (which may not be a stereo image) using a stereo camera. The index deriving unit 220 performs a two-dimensional FFT process in a direction perpendicular to the horizontal direction on the image of the extracted portion to calculate a spatial spectrum. The index deriving unit 220 uses the calculated spatial spectrum as a contrast index of transmittance. A high frequency component of a portion of the scenery at a distance greater than a given distance indicates a high transmittance.

[0095] When deriving the contrast index, the index deriving unit 220 performs two-dimensional FFT processing on the image in the same manner as described above, calculates the spatial spectrum, and uses the spatial spectrum as a contrast index of the texture in the image. When the contrast index of the texture in the image is high, it is highly likely that the place where the image was taken is a landscape with a high proportion of texture, such as a forest or a rock surface.

[0096] When deriving color indices of hue and chroma, the index deriving unit 220 calculates the hue and chroma of each pixel in the image. The index deriving unit 220 divides the image into a plurality of regions, for example, and calculates the distribution of hue and chroma for each region. The distribution is represented in the form of a histogram. Figure 7 The index deriving unit 220 uses the distribution of hue and saturation as a color index in the image.

[0097] The index deriving unit 220 compares the distribution of hue and chroma of each area, and extracts the place where the image is taken as a scenic spot with characteristic color when a characteristic distribution that is significantly different from the surrounding areas is obtained. The high or low color index value becomes an index of whether the color distribution is characteristic relative to the surrounding areas. The area with a characteristic distribution relative to the surrounding areas in the image is likely to be a landscape with characteristic color, such as flowers and the sea.

[0098] When deriving an index related to a moving object, the index deriving unit 220 derives the index using a plurality of position data and a plurality of image data. When the subject is moving, the index related to the moving object is derived based on the amount of movement of the subject in the image.

[0099] The index deriving unit 220 calculates the movement of the subject (moving object) from the difference in changes in the image data based on the plurality of images and the movement data of the vehicle 100 .

[0100] Figure 8 It is a graph showing the movement of the subject of the continuously captured images. For example, when an image is captured at (B) after a given time of the image captured at (A), in (B), the movement distance L1 of the large distant subject P is smaller than the movement distance L2 of the close subject Q. When the subject area is large and movement is detected with a small movement distance L relative to the vehicle 100, the index deriving unit 220 can derive the index value as an index of a moving object.

[0101] The index deriving unit 220 extracts a subject of a given size or larger based on the continuously captured images, for example. The index deriving unit 220 extracts the subject P based on the image (A) and the image (B), for example. The index deriving unit 220 recognizes that the subject P is a large subject when the ratio of the area S1 in the image of the subject P to the area S of the image: S1 / S exceeds a given threshold.

[0102] The index deriving unit 220 calculates the index of the movement of the subject P based on the image (A) and the image (B). The index of the movement of the subject P is calculated based on, for example, the correlation between the moving distance L1 of the subject P in the image (A) and the image (B) and the moving distance L of the vehicle during the period when the image (A) and the image (B) are captured. In the index value of the movement, the subject area is large and the moving distance L1 of the subject is small relative to the moving distance L of the vehicle 100, which is an index for capturing a distant landscape. The index value of the moving object is an index for a scene in which a large subject such as a mountain, a tree, or a building enters the field of view.

[0103] When deriving an index related to altitude, the index deriving unit 220 uses the three-dimensional position data of the navigation device 120 recorded in the detection data 115. The index deriving unit 220 extracts places at altitudes above a given threshold. The altitude index value is an index of scenery that the view that can be overlooked is expanding.

[0104] Fig. 9 2 is a diagram showing an example of the content of the derived index value data 222. The landscape evaluation unit 230 evaluates the beauty of the spot where the detection data 115 is acquired based on the plurality of index values ​​X1 to Xn derived by the index deriving unit 220.

[0105] The landscape evaluation unit 230 extracts distances greater than a given value based on the index value of the distance. A place where the index value of the distance is greater than a given value indicates that the space is expanding and is estimated to have a higher correlation with the beautiful scenery. The landscape evaluation unit 230 combines the derived multiple index values ​​X1 to Xn with the index value Xn having correlation at the extracted place to group them. The grouping is determined according to each landscape category such as "mountain" or "coast".

[0106] The landscape evaluation unit 230 multiplies each of the grouped index values ​​Xn by a weighted coefficient αn. The landscape evaluation unit 230 increases the coefficient αn of each index value Xn in the group according to the synergy of the groups and adds the index values ​​Xn of the groups, thereby calculating a score (evaluation value) for each group as shown in equation (1).

[0107] Group evaluation value (1) = (α1·X1+α2·X2+α3·X3)

[0108] Group evaluation value (2) = (α4·X4+α5·X5)

[0109] … (1)

[0111] The so-called synergy of grouping is to give weights to the coefficients αn of the index values ​​Xn associated with places with high beauty. For example, when the landscape is "sea", weights are given to the coefficients αn of index values ​​such as color and transmittance. When the landscape is "mountain", weights are given to the coefficients of index values ​​such as distance, color, moving objects, and height. The landscape evaluation unit 230 compares the scores of each group, sorts them in order, and selects the group with the highest score based on formula (2) as the final evaluation value. The group with the highest score represents the representative landscape of the place where the detection data 115 is obtained.

[0112] Final evaluation value = MAX{(α1·X1+α2·X2+α3·X3), (α4·X4+α5·X5), …}(2)

[0113] The landscape evaluation unit 230 may also assign weights to the coefficient αn in correspondence with the season in which the detection data 115 is obtained based on the detection data 115 obtained during a given period, and select scenery that is characteristic of the period. For example, the landscape evaluation unit 230 may increase the weights of the coefficients of distance, transmittance, color, and contrast in the spring and autumn seasons, and select places with a high degree of beauty that have characteristic colors of scenery such as red leaves and cherry blossoms.

[0114] The landscape evaluation unit 230 may also evaluate the beauty by assigning weights to the coefficient αn in correspondence with the location and time at which the detection data 115 is obtained. For example, even in the same place, the landscape may change according to time zones such as sunrise and sunset. Time zones such as sunrise and sunset present a characteristic landscape in which the red hue of the chromaticity in the image increases. The landscape evaluation unit 230 may, for example, assign weights to the coefficient αn of the color index value Xn in correspondence with the time. Thus, the landscape evaluation unit 230 may select a place with a high beauty at a characteristic time in color, such as sunset or sunrise at a lakeside or a coast.

[0115] When the index deriving unit 220 derives the index value Xn for each detection direction of each sensor, the landscape evaluation unit 230 may evaluate the beauty by assigning a weight to the coefficient αn in correspondence with the position and the moving direction where the detection data 115 is acquired. For example, when the landscape evaluation unit 230 evaluates that the scenery with a high beauty is expanding in the direction in which the vehicle 100 is moving, the coefficient αn of the index value Xn in the moving direction may be assigned a larger weight than that in other directions.

[0116] The landscape evaluation unit 230 can rank the beauty degree based on the grouped scores, and store information related to the beauty degree in each category in the storage unit 260 as landscape information 262. The categories are classified into, for example, "beautiful scenery ranking from high to low", "colorful attractions", "landmark sightseeing and learning attractions", "mountains", "sea", "night view" and the like.

[0117] The landscape evaluation unit 230 may also assign a scenic spot name to the extracted places with high scenic beauty by referring to the map information 261. The landscape evaluation unit 230 may also assign a scenic spot name to the places with high scenic beauty based on the information retrieved via the network NW. The landscape evaluation unit 230 generates information related to the scenic beauty and adds it to the map information 261 as POI data. The network NW includes, for example, a WAN (Wide Area Network), a LAN (Local Area Network), the Internet, a dedicated line, a wireless base station, a carrier, and some or all of the like.

[0118] Fig.10 This is a diagram showing an example of the content of the POI data 231 of a place with a high degree of beautiful scenery.

[0119] Furthermore, the landscape evaluation unit 230 may generate data by associating the driving data of the vehicle 100 with the final evaluation value. Fig.11 2 is a graph showing an example of the relationship between the driving data of the vehicle 100 and the final evaluation value. The landscape evaluation unit 230 establishes a correspondence between the driving data and the final evaluation value, and extracts the places whose final evaluation values ​​exceed a given threshold as places with high beauty. The landscape evaluation unit 230 can add the data of the places with high beauty on the extracted driving route to the map information 261.

[0120] In the information processing system 1, the evaluation of the beauty can be performed based on the information sensed by the plurality of vehicles 100. In this case, the landscape information 262 and the POI of the map information 261 stored in the storage unit 260 can be updated in real time by statistically processing the information sensed by the plurality of vehicles 100.

[0121] The information providing unit 240 provides the information of the POI with a high degree of beauty stored in the map information 261 to the vehicle 100. For example, when the information providing unit 240 provides information to the vehicle 100, when the passenger performs an operation of setting a route to the destination on the navigation device 120, information of a place with a high degree of beauty is provided corresponding to the route to the destination. As a result, the navigation HMI 122 displays the scenic spots with a high degree of beauty on the route to the destination.

[0122] The information providing unit 240 may also select a place with high beauty degree according to the category of beauty degree selected by the passenger of the vehicle 100, and provide it to the vehicle 100. In addition, the navigation device 120 may also set a route according to the category of beauty degree selected by the passenger. For example, when operating on the navigation HMI 122, the passenger may select a category such as "beautiful scenery ranking from high to low", "colorful attractions", "landmark sightseeing and learning attractions", etc. If the passenger selects a category such as "beautiful scenery ranking from high to low", the navigation device 120 sets a route according to "beautiful scenery ranking from high to low", and the passenger can drive on a route with high beauty degree.

[0123] Next, the processing executed in the information processing system 1 will be described. Fig.12 1 is a flowchart showing an example of the flow of processing executed in the information processing system 1. First, the information acquisition unit 210 acquires the detection data 115 from the vehicle 100 via the network NW (step S100). The index derivation unit 220 derives a plurality of index values ​​Xn indicating the beauty level based on the detection data 115 (step S110).

[0124] The landscape evaluation unit 230 evaluates the beauty of the location where the sensor detection information is acquired based on the plurality of index values ​​Xn (step S120 ). The information providing unit provides information related to the beauty to the vehicle 100 (step S130 ).

[0125] As described above, according to the information processing system 1 , the degree of beautiful scenery can be automatically indexed based on the detection data 115 of the surroundings of the vehicle 100 detected by the vehicle 100 .

[0126] Thus, according to the information processing system 1 , the beauty level at the location where the detection data 115 was acquired can be evaluated according to the location and time at which the detection data 115 was acquired.

[0127] In addition, according to the information processing system 1, the passenger can set a route in the navigation device 120 according to the category of the beauty level, and the passenger can drive on a route with high beauty level.

[0128] The above-mentioned information processing system 1 can also be used in an autonomous driving vehicle 300 . Fig.13 3 is a block diagram of a case where an automatic driving vehicle 300 is used in the information processing system 2. The navigation device 320 outputs a route to a destination to the recommended lane determination device 360. The recommended lane determination device 360 ​​refers to a map more detailed than the map data possessed by the navigation device 320 to determine a recommended lane for the vehicle to travel, and outputs the recommended lane to the automatic driving control device 350.

[0129] The automatic driving control device 350 controls part or all of the driving force output device 370 including the engine and the electric motor, the braking device 380 , and the steering device 390 based on the information input from the external sensing unit 310 , so as to travel along the recommended lane input from the recommended lane determination device 360 ​​.

[0130] Here, the autonomous driving vehicle 300 may also change the recommended lane based on the evaluation of the landscape evaluation unit 230. The autonomous driving vehicle 300 may change at least one of the lane change and the distance relationship with other vehicles so as to improve the evaluation of the beauty of the landscape evaluation unit 230. For example, the recommended lane determination device 360 ​​may determine the recommended lane, change lanes, or return to the original lane after changing lanes, so that the index of the beauty of the landscape evaluation unit 230 becomes higher. In addition, the recommended lane determination device 360 ​​may determine the recommended lane in the presence of parallel vehicles or approaching vehicles, increase the distance between vehicles, change lanes, overtake, etc., so that the index of the beauty of the landscape evaluation unit 230 becomes higher.

[0131] In such an automatic driving vehicle 300, it is assumed that the passengers have more opportunities to see the scenery compared to a manually driven vehicle. Therefore, by using the information processing system 2 of the embodiment, the passengers can select a route with good scenery to the destination during the automatic driving.

[0132] [Modifications]

[0133] In the above embodiment, an example is given of a case where the autonomous driving vehicle 300 communicates with the information processing device 200 to derive an index value such as the beauty along the vehicle's driving path. The index value for the surrounding environment of the autonomous driving vehicle 300 may be derived on the side of the autonomous driving vehicle 300 instead of on the information processing device 200 located outside the autonomous driving vehicle 300. Furthermore, the index value for the surrounding environment of the autonomous driving vehicle 300 may also be derived corresponding to the position of the vehicle. In the following description, the same names and figure marks are used for the same structures as in the above embodiment (the configuration and number of the devices may be different), and repeated descriptions are omitted as appropriate.

[0134] Fig.14 : is a diagram showing an example of the structure of an autonomous driving vehicle 300 involved in a modified example. In the modified example, the information processing device 200 is mounted on the autonomous driving vehicle 300. The information processing device 200 is, for example, built into the navigation device 320 or connected to the outside thereof. The so-called external connection means that the information processing device 200 connected to the navigation device 320 by wire or wirelessly operates as a part of the function of the navigation device 320. In addition, the autonomous driving control device 350 of the autonomous driving vehicle 300 (hereinafter also referred to as the self-vehicle) is an example of the first evaluation unit, and the information processing device 200 is an example of the second evaluation unit.

[0135] The automatic driving control device 350 evaluates the properties of the environment around the vehicle based on the detection data 315 detected by the external sensing unit 310, and performs driving support for the vehicle such as avoidance control based on automatic driving, collision avoidance and accident avoidance based on ADAS (Advanced driver-assistance systems) based on the evaluation result. The detection data 315 includes at least a part or all of the contents of the detection data 115 (see Figure 4 ).

[0136] The information processing device 200 includes an information acquisition unit 210, a landscape evaluation unit 230, an information provision unit 240, and a communication unit 250. Different from the evaluation of the attributes of the driving support for the driving of the vehicle, the information processing device 200 evaluates the attributes of the environment around the vehicle based on the detection data 315 detected by the external sensing unit 310, and supports the determination of the vehicle path of the navigation device 320 based on the evaluation result. In addition, in the modified example, the automatic driving control device 350 and the information processing device 200 are examples of multiple evaluation units. The automatic driving control device 350 is an example of the first evaluation unit, and the information processing device 200 is an example of the second evaluation unit.

[0137] First, the evaluation of the properties of the automatic driving control device 350 will be described. The automatic driving control device 350 obtains detection data 315 (sensor detection information) from the external sensing unit 310. The detection data 315 includes, for example, position data obtained from the GNSS and data obtained from other vehicle sensors in addition to data obtained from the external sensing unit 310.

[0138] The automatic driving control device 350 stores data based on sensor values ​​such as the vehicle position, moving speed, and posture state quantity among the detection data 315 in the vehicle sensor data storage unit 351, and stores data such as pixel information of the captured image data calculated based on the detection data 315, differential information of the difference of a plurality of detection data, and basic index values ​​required for deriving attribute information in the temporary memory 352. The so-called basic index value is a parameter determined mathematically in image processing.

[0139] The basic index value in this case is not limited to a value such as "beautiful scenery" as in the above-mentioned embodiment, and can be any parameter determined mathematically in image processing. The automatic driving control device 350 uses the detection data 315 to determine the control attribute. The so-called control attribute is data including a combination of an attribute value indicating whether the environment of each typed vehicle is in compliance with the attribute index indicating the degree of compliance.

[0140] Fig.15 35 is a diagram showing an example of the content of data included in the control attribute 355. The control attribute 355 is obtained by associating attribute values ​​and attribute indicators with "attributes" that categorize the environment around the vehicle, such as "collision, congestion, curve, object," for example.

[0141] The attribute index is, for example, a set of parameters associated with the attribute among the multiple parameters obtained based on the detection data 315. For example, for an attribute such as "curve", parameters associated with the curve such as "lane, lane curvature, steering angle, acceleration" are extracted from the data such as "position, vehicle speed, acceleration, yaw rate, steering angle, other vehicles, people, objects, relative distance, relative speed, relative angle, lane, lane curvature" obtained through the detection data 315 to establish a correspondence as the attribute index of the curve.

[0142] The plurality of parameters are acquired, for example, based on the detection data 315. The detection data includes data recognized within a given distance around the vehicle and data detected in the vehicle.

[0143] The attribute index indicates the degree of the attribute, and is, for example, the magnitude of an attribute index value calculated based on the magnitude of a vector composed of a plurality of parameters extracted in association with the attribute.

[0144] The attribute value is, for example, an evaluation value for determining whether the attribute is met. The attribute value is represented by, for example, a binary value of 0 or 1 based on the comparison result between the attribute index value and the threshold. The attribute value is associated with 1 when the attribute is met, and is associated with 0 when the attribute is not met.

[0145] The automatic driving control device 350 inputs the data stored in the vehicle sensor data storage unit 351 and the data stored in the temporary memory 352 to the evaluation function 354, and determines whether the attribute is met based on the calculation result. The evaluation function 354 is composed of a multi-layer neural network with an intermediate layer.

[0146] The multilayer neural network is a multilayered neural network used in deep learning. The evaluation function 354 is set by, for example, performing deep learning using learning data. The evaluation function 354 is set by previously performed learning.

[0147] For example, in deriving the control properties of the vehicle's surroundings, the automatic driving control device 350 inputs multiple images contained in the data stored in the temporary memory 352 into the evaluation function 354, and compares the result of the feature quantity when each pixel of the image is used as an input vector through a multi-layer neural network operation.

[0148] At this time, the evaluation function 354 is constructed as follows: a large number of input vectors and the correct attributes at that time are used as the result of teacher data learning in advance, and the weights of each layer are optimized to maintain the feature quantity corresponding to the correct attribute, thereby outputting the correlation between the new input vector and each attribute. For example, the evaluation function 354 is constructed as follows: the attributes include the boundaries of the shapes contained in the image, the attributes of the objects, and the attribute information of the driving environment. When multiple images of a time series are input, the transition from the previous time and the state that will occur next are also included. The automatic driving control device 350 uses the evaluation function 354 to continuously obtain the evaluation values ​​of the vehicle, road, pedestrian, object, and scene attributes of the input image to recognize the external state.

[0149] The automatic driving control device 350 inputs the vehicle-related data stored in the vehicle sensor data storage unit 351 into the evaluation function 354, calculates a plurality of parameters including the recognized object, and extracts a plurality of parameters for each attribute. The automatic driving control device 350 uses the evaluation function 354 to calculate a score (attribute index value) for each group.

[0150] The automatic driving control device 350 compares the calculated attribute index value with the benchmark. The automatic driving control device 350 determines the benchmark adaptability of the attribute index value by comparing whether the attribute index value is above the threshold, below the threshold, or within a given range. When the attribute index value satisfies the benchmark, the automatic driving control device 350 determines that the attribute is in compliance with the attribute and assigns 1 to the attribute value. When the attribute index value does not satisfy the benchmark, the automatic driving control device 350 determines that the attribute is not in compliance with the attribute and assigns 0 to the attribute value.

[0151] The automatic driving control device 350 generates a control attribute based on the content of the determined attribute value (refer to Fig.15 ). For example, when the attribute value is 1 and the attribute is satisfied, the automatic driving control device 350 calculates additional information for the attribute. The additional information will be described later.

[0152] The automatic driving control device 350 adds additional information to the attribute index value of the attribute whose attribute value is 1, and generates a control attribute value 356. The control attribute value 356 is used for the control of the driving support of the own vehicle as described later.

[0153] The following describes a specific attribute calculation method. For example, when evaluating an attribute such as "collision", the automatic driving control device 350 calculates a collision probability as an attribute index, determines an attribute value based on the calculated collision probability, and determines whether the attribute such as collision is satisfied.

[0154] For example, in calculating the collision probability, the automatic driving control device 350 inputs a plurality of image data into the evaluation function 354 to recognize moving objects around the vehicle and calculate the current positions of the objects and the vehicle. When recognizing a plurality of objects, the automatic driving control device 350 determines the type of each object, assigns an ID to each object, and calculates the collision probability of each object with the vehicle.

[0155] The automatic driving control device 350 inputs the detected and recognized data of "vehicle speed, acceleration, yaw rate, position" etc. into the evaluation function 354 to make a future prediction of the positional relationship between the moving object and the vehicle at a time ahead of a predetermined given time.

[0156] The automatic driving control device 350 performs deep learning based on a multi-layer neural network through the evaluation function 354, executes future predictions of the positional relationship between the moving object and the self-vehicle corresponding to the set prediction time, and calculates the probability of collision between the self-vehicle and the surrounding objects based on the overlap ratio between the predicted area where the self-vehicle may be located, i.e., the position distribution, and the area where the surrounding objects may be located, i.e., the position distribution.

[0157] At this time, the automatic driving control device 350 multiplies each coupling of the multilayer neural network of the evaluation function 354 by a weight coefficient pre-stored in the storage unit to calculate the collision probability between the vehicle and the surrounding objects.

[0158] The automatic driving control device 350 compares the calculated collision probability with the threshold value, and when the collision probability does not meet the reference, the automatic driving control device 350 assigns 0 to the attribute value and determines that there will be no collision. When the collision probability meets the reference, the automatic driving control device 350 assigns 1 to the attribute value and determines that there is a possibility of collision.

[0159] For example, when determining that there is a possibility of collision, the automatic driving control device 350 generates a control attribute value 356 including additional information such as "attribute: collision, possibility of collision: yes, object: other vehicle (ID=*), position: (x1, y1)". The automatic driving control device 350 outputs the control attribute value 356 to the control units such as the driving force output device 370, the braking device 380, and the steering device 390 to perform driving support control such as avoiding the object.

[0160] The above-mentioned automatic driving control device can also be configured to estimate attribute determination and additional information for multiple attributes such as "the vehicle ahead changes its direction", "a pedestrian crosses", and "a signal state change". For example, the above-mentioned automatic driving control device 350 determines the control attribute at a higher frequency than the information processing device 200 described later. This is because the surrounding environment of the automatic driving vehicle 300 needs to be frequently sensed in order to support the automatic driving control of the automatic driving control device 350.

[0161] Next, the evaluation of the attribute of the information processing device 200 will be described. Here, the processing of the index derivation unit 220 in the above embodiment can also be regarded as the processing performed by the landscape evaluation unit 230. The information processing device 200 evaluates the attribute of the environment around the vehicle, for example, while the vehicle is traveling. In the information processing device 200, the information acquisition unit 210 further includes an information selection unit 211.

[0162] The information selection unit 211 includes two information paths, for example, a high-speed unit 212 which is a high-speed side bus for inputting and outputting data at high speed, and a low-speed unit 213 which is a low-speed side bus used for random reading.

[0163] The high-speed section 212 is used, for example, for communication with a large data volume between the automatic driving control device 350 and the landscape evaluation unit 230. The high-speed section 212 communicates data acquired by the external sensing unit 310 of the automatic driving vehicle 300, GNSS, the vehicle sensor 60, and the like.

[0164] For example, when 60 images are captured by the external sensing unit 310 during a given time unit (AT), data reduced to about 10 images is sent to the landscape evaluation unit 230 via the high-speed unit 212. The given time unit is a given time length set when generating a data set, as described later.

[0165] The data set may be generated not in a given time unit, but by dividing the image data into given image regions. The data divided into image regions may be used in the calculation of a convolutional neural network, for example.

[0166] The low-speed portion 213 is, for example, an information path when the landscape evaluation unit 230 reads the attribute value generated by the automatic driving control device 350 at any time. The amount of information sent and received in the low-speed portion 213 is smaller than the amount of information sent and received in the high-speed portion 212.

[0167] The low speed unit 213 is also used in the communication between the automatic driving vehicle 300 and other devices provided outside. The automatic driving control device 350 outputs the control attribute value 356 generated at a given timing as transmission data to the communication data generation / storage unit 233 (data generation unit) via the low speed unit 213. The communication data generation / storage unit 233 transmits the generated transmission data to the external server 400 via the communication unit 250 connected to the low speed unit 213, for example.

[0168] The landscape evaluation unit 230 includes, for example, a definition update unit 232 and a communication data generation / storage unit 233. The landscape evaluation unit 230 determines an environmental attribute representing an attribute of an environment of a location to be evaluated using the detection data 315. The environmental attribute is data including an attribute of an environment surrounding each typed vehicle and an attribute index representing the degree of the attribute.

[0169] Fig.16 This is a diagram showing an example of the content of data included in the environmental attribute 255. Here, the environmental attribute 255 is mainly generated as a sensory index related to the sensory perception of the passenger. The so-called sensory index is a mathematically determined parameter that represents the sensory perception of the passenger to the environment of the evaluation object location, such as "beautiful scenery". The sensory index is calculated for attributes such as beautiful scenery, riding experience, and congestion.

[0170] The landscape evaluation unit 230 obtains information stored in the temporary memory 352 and information stored in the vehicle sensor data storage unit 351, for example, via the high-speed unit 212 of the information acquisition unit 210, inputs the obtained information into the evaluation function 234, and determines a plurality of environmental attributes of the surrounding environment at the location where the autonomous driving vehicle 300 is traveling while driving. The evaluation function 234 is composed of, for example, a multi-layer neural network having an intermediate layer. The landscape evaluation unit 230 uses the evaluation function 234 to calculate an attribute index for an attribute. The landscape evaluation unit 230 determines an attribute value based on the calculated attribute index. The landscape evaluation unit 230 evaluates the landscape of the surrounding environment based on the determined attribute value and the magnitude of the attribute index.

[0171] For example, when evaluating the "comfort of the road", the landscape evaluation unit 230 inputs data of a plurality of parameters related to the comfort of the road, such as acceleration, vibration, sound, congestion, and the number of stops, from the detection data 315 into the evaluation function 234. For example, the landscape evaluation unit 230 calculates an attribute index of the comfort based on the change pattern of the characteristic parameters of the "comfort of the road" learned in the evaluation function 234. The landscape evaluation unit 230 determines whether the location of the evaluation object meets the attribute of the "comfort of the road" based on the calculation result of the attribute index.

[0172] The landscape evaluation unit 230 determines whether the attribute index value calculated based on the magnitude of the attribute index satisfies the benchmark and determines the attribute value of "road comfort". The landscape evaluation unit 230 determines the benchmark adaptability of the attribute index value by comparing whether the attribute index value is above a threshold, below a threshold, or within a given range.

[0173] Specifically, the landscape evaluation unit 230 compares the magnitude of the vector of the derived multiple attribute index values ​​with the threshold used in each attribute, calculates the attribute value, and calculates the degree to which the location of the evaluation object meets the given attribute. The attribute value can be represented by a binary value of 0 or 1 indicating whether the attribute is met, or can be represented by a step-by-step numerical value of more than 2 values.

[0174] The landscape evaluation unit 230 generates an environmental attribute 255 (see Fig.16 ). For example, when the attribute value satisfies the benchmark and meets the attribute, the landscape evaluation unit 230 adds additional information to the attribute index value of the attribute to generate an environmental attribute value 236. The landscape evaluation unit 230 generates an environmental attribute value 236 including additional information such as "attribute: road comfort, comfort: relatively comfortable, position: (x1, y1)". The landscape evaluation unit 230 derives the evaluation value of the perceptual index at the location of the evaluation object in correspondence with the location where the detection data 315 was obtained based on multiple indicators and vehicle information including position information. The landscape evaluation unit 230 outputs the generated environmental attribute value 236 to the communication data generation / storage unit 233.

[0175] The definition updating unit 232 is suitable for adding or changing the attribute information of the defined attribute. The definition updating unit 232 adds or changes the attribute information based on the input information received by the user's input operation. For example, the user inputs the category of any attribute that he or she wishes to investigate the environment around the vehicle. The definition updating unit 232 updates the definition of the attribute used in the evaluation function 234.

[0176] The landscape evaluation unit 230 can set the attribute that the user wishes to investigate based on information input by the user to the definition update unit 232, and derive the evaluation result.

[0177] The communication data generation / storage unit 233 selectively generates transmission data to the outside. The transmission data is transmitted to, for example, an external server 400 described later. The communication data generation / storage unit 233 generates one communication data set (transmission data) in a given time unit (AT).

[0178] The communication data generation / storage unit 233 associates, for example, the environmental attribute value 236 in a given time unit, the position information, the time information, the value of the representative detection data 315, and the timestamp containing the representative information of the time, and generates a data set of transmission data, and stores it in the storage unit. In this case, the position information may be information on the section of the road specified by the nodes and links in the map information.

[0179] The communication data generation / storage unit 233 transmits the transmission data to the automatic driving control device 350, the external server 400 provided outside and described later, etc. in units of data sets. However, the communication data generation / storage unit 233 does not generate transmission data when there is no attribute value or when the attribute value is less than the threshold value, for example. The communication data generation / storage unit 233 selectively generates transmission data to be sent to the outside based on the content of the calculated attribute value.

[0180] The communication data generation / storage unit 233 sequentially transmits data related to the attributes of the environment around the autonomous driving vehicle 300 to the autonomous driving control device 350 via the low speed unit 213, for example, at given time intervals. According to the above configuration, the autonomous driving vehicle 300 can evaluate a plurality of indicators of the surrounding environment at the coordinates (position) of the vehicle while moving.

[0181] The autonomous driving vehicle 300 equipped with the above-mentioned information processing device 200 can be further connected to an external server 400 having the function of the information processing device 200 via the network NW. The communication data generation / storage unit 233 transmits the transmission data to the external server 400 via the communication unit 250 connected to the low-speed unit 213 and capable of communicating with the outside. Through such processing, the autonomous driving vehicle 300 can evaluate the environmental attributes even in an area with poor communication conditions.

[0182] Fig.17 : is a diagram showing an example of the structure of the external server 400. The external server 400 obtains information related to environmental attribute values ​​from the autonomous driving vehicle 300, re-evaluates the environmental attributes by taking into account the changed information, and provides the autonomous driving vehicle 300 with the evaluation results.

[0183] For example, environmental attribute values ​​such as "beautiful scenery" vary depending on the construction of buildings, weather, temperature, and the shape of the vehicle. The external server 400 obtains information related to the environment from vehicles actually traveling in the area, and adds information such as weather and temperature to re-evaluate the environmental attributes, and also provides information to the autonomous driving vehicle 300 and other vehicles.

[0184] The external server 400 includes an information acquisition unit 410, an environment evaluation unit 420, and an information provision unit 430. The environment evaluation unit 420 includes an evaluation function 422, an environment information storage unit 423, and an update unit 424 for adding derived attribute information.

[0185] The environmental information storage unit 423 stores three-dimensional terrain information of the evaluation target area, weather and temperature information, and vehicle geometry information including the size of multiple vehicles, etc. This information is time information that can change over time. The environmental evaluation unit 420 obtains the time information from the network NW at a given timing and changes the time information if there is a change.

[0186] The environment evaluation unit 420 obtains the transmission data transmitted by the communication data generation / storage unit 233 via the information acquisition unit 410 as the received data 421. In addition to the received data 421, the environment evaluation unit 420 inputs the time information stored in the environment information storage unit 423 into the evaluation function 422 to derive the environmental attribute value of the environment around the vehicle.

[0187] For example, a multilayer neural network is used as the evaluation function 422. The environment evaluation unit 420 calculates the environment attribute according to differences in terrain, weather / temperature, vehicle size, and the like, for example.

[0188] For example, in the evaluation of the attribute of "beautiful scenery", if the weather at the location with high beauty is rainy or cloudy, or the beauty is deteriorated due to the seat position of the vehicle, and the environmental attribute value obtained from the autonomous driving vehicle 300 indicates "poor beauty", the environmental evaluation unit 420 changes the environmental attribute value of "beautiful scenery" of the location currently being evaluated using the evaluation function 422. The evaluation function 422 increases the weight coefficient of the parameter in the calculation of the multilayer neural network in accordance with the change of the environment such as the change of the weather and the change of the seat position of the vehicle or the change of the viewpoint position.

[0189] When the environmental attribute value of the location of the derived evaluation object is significantly different from the environmental attribute value of the received data, the updating unit 424 updates the definition of the evaluation function 422 and adds the derived environmental attribute value as a new environmental attribute value. The updating unit 424 generates update information 405 including the updated environmental attribute value. The updating unit 424 sends the update information 405 to the autonomous driving vehicle 300.

[0190] The updating unit 424 provides information to other autonomous driving vehicles. The updating unit 424 transmits a response signal 406 via the information providing unit 430. The response signal 406 includes, for example, the update information 405 derived from the external server 400 based on the transmitted data.

[0191] In this way, the autonomous driving vehicle 300 can receive, via the information acquisition unit 210 , a response signal 406 sent back from the external server 400 in response to the transmission data as a result of the transmission data being transmitted via the communication unit 250 .

[0192] In the autonomous driving vehicle 300, when a response signal 406 is received, the information providing unit 240 provides landscape information representing information related to the landscape around the autonomous driving vehicle 300 to the autonomous driving vehicle 300 based on either the attribute information determined on the autonomous driving vehicle 300 side or the additional attribute information contained in the response signal 406 obtained from the external server.

[0193] Thus, the autonomous driving vehicle 300 can use attribute information updated in the external server 400 in addition to attribute information determined in the autonomous driving vehicle 300. Furthermore, the external server 400 can provide environmental attributes corresponding to changes in the environment over time, such as the shape of the vehicle and future weather changes. According to the information processing device 200, the support of the autonomous driving driving control of the autonomous driving vehicle 300 can be made more accurate.

[0194] The above describes the embodiment for implementing the present invention, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be added within the scope of the present invention. For example, the index derivation unit 220 and the landscape evaluation unit 230 of the information processing device 200 can also be provided on the vehicle 100 side.

Claims

1. An information processing device, characterized in that: have: an acquisition unit that acquires sensor detection information indicating a detection result from a sensor mounted on the vehicle; and The sensor detection information acquired by the acquisition unit is input into a plurality of evaluation units defined in advance corresponding to the purpose used in the control, so that the evaluation results of the plurality of evaluation units for the surrounding environment, i.e., a plurality of indicators, are derived at different frequencies on the vehicle. The plurality of evaluation units include: a first evaluation unit that uses the sensor detection information to determine a control attribute, wherein the control attribute is data including a combination of an attribute value and an attribute index, the attribute value indicating whether the control attribute complies with each of the pre-typed environments of the vehicle, and the attribute index indicating the degree of compliance; and a second evaluation unit that uses the sensor detection information to determine an environmental attribute, wherein the environmental attribute represents a sensory index associated with the sensory perception of the passenger of the vehicle with respect to the surrounding environment at the location corresponding to the detection result, i.e., an attribute associated with the beauty of the landscape; The attribute index is a set of parameters associated with the attribute among a plurality of parameters obtained based on the sensor detection information, and indicates the degree of the attribute. The plurality of indices, which are evaluation results of the first evaluation unit and the second evaluation unit, are used to control one or both of driving support for avoidance control of the vehicle and support for path determination.

2. The information processing device according to claim 1, characterized in that The second evaluation unit derives an evaluation value of a sensitivity index at the location in association with a location where the sensor detection information is acquired, based on the plurality of indices and vehicle information including position information.

3. The information processing device according to claim 2, characterized in that: The second evaluation unit derives the evaluation value of the sensitivity index in association with a time point included in a unit of a given time length.

4. The information processing device according to claim 2 or 3, characterized in that: The information processing device further includes an information providing unit that provides information related to scenery to the vehicle based on the evaluation value.

5. The information processing device according to claim 4, characterized in that The information providing unit provides information related to the scenery to the vehicle in accordance with the association between the category of the beauty level selected by the passenger of the vehicle and the environmental attribute.

6. The information processing device according to any one of claims 1 to 3, characterized in that: The vehicle is an autonomous vehicle, The vehicle performs at least one of a lane change and a change in distance relationship with other vehicles so as to improve the evaluation of the beautiful scenery by the evaluation unit.

7. The information processing device according to claim 1, characterized in that: The first evaluation unit determines the control attribute at a higher frequency than the second evaluation unit.

8. The information processing device according to claim 1, characterized in that The information processing device comprises: an updating unit that updates the definitions of the plurality of evaluation units; The information processing device can add the attribute information outputted from the plurality of evaluation units.

9. The information processing device according to claim 4, characterized in that: The information processing device includes a data generating unit that associates the sensor detection information and the plurality of indices with the position information and selectively generates transmission data to be transmitted to the outside according to the contents of the plurality of indices.

10. The information processing device according to claim 9, characterized in that: The information processing device includes: a communication unit that receives a response signal sent back to the transmission data transmitted to the outside; The information providing unit provides landscape information indicating information on landscape around the vehicle based on either the attribute information determined by the vehicle or the updated attribute information included in the response signal.

11. An information processing method, characterized in that: Executed by computer: Acquire sensor detection information indicating detection results from sensors mounted on the vehicle, By inputting the acquired sensor detection information into a plurality of evaluation units defined in advance corresponding to the purpose used in the control, the evaluation results of the plurality of evaluation units for the surrounding environment, i.e., a plurality of indicators, are derived at different frequencies on the vehicle. The plurality of evaluation units include: a first evaluation unit that uses the sensor detection information to determine a control attribute, wherein the control attribute is data including a combination of an attribute value and an attribute index, the attribute value indicating whether the control attribute complies with each of the pre-typed environments of the vehicle, and the attribute index indicating the degree of compliance; and a second evaluation unit that uses the sensor detection information to determine an environmental attribute, wherein the environmental attribute represents a sensory index associated with the sensory perception of the passenger of the vehicle with respect to the surrounding environment at the location corresponding to the detection result, i.e., an attribute associated with the beauty of the landscape; The attribute index is a set of parameters associated with the attribute among a plurality of parameters obtained based on the sensor detection information, and indicates the degree of the attribute. Furthermore, the computer uses the plurality of indicators, which are evaluation results of the first evaluation unit and the second evaluation unit, to control one or both of driving support for avoidance control of the vehicle and support for path determination.

12. A program product comprising a program, characterized in that The program causes the computer to execute: Acquire sensor detection information indicating detection results from sensors mounted on the vehicle, By inputting the acquired sensor detection information into a plurality of evaluation units defined in advance corresponding to the purpose used in the control, the evaluation results of the plurality of evaluation units for the surrounding environment, i.e., a plurality of indicators, are derived at different frequencies on the vehicle. The plurality of evaluation units include: a first evaluation unit that uses the sensor detection information to determine a control attribute, wherein the control attribute is data including a combination of an attribute value and an attribute index, the attribute value indicating whether the control attribute complies with each of the pre-typed environments of the vehicle, and the attribute index indicating the degree of compliance; and a second evaluation unit that uses the sensor detection information to determine an environmental attribute, wherein the environmental attribute represents a sensory index associated with the sensory perception of the passenger of the vehicle with respect to the surrounding environment at the location corresponding to the detection result, i.e., an attribute associated with the beauty of the landscape; The attribute index is a set of parameters associated with the attribute among a plurality of parameters obtained based on the sensor detection information, and indicates the degree of the attribute. Furthermore, the computer is caused to use the plurality of indicators, which are evaluation results of the first evaluation unit and the second evaluation unit, for one or both of driving support for avoidance control of the vehicle and support for path determination.

Images