Information processing apparatus, information processing method, and program

Abstract

To appropriately generate driving assistance information including information on an object located outside a vehicle.SOLUTION: An information processing device 20 for generating driving assistance information for a vehicle 4, comprises: a detection data acquisition unit 30 that acquires detection data D1 which is three-dimensional data obtained by detecting an object Oj located outside the vehicle 4; a data complementary unit 41 that extracts object data Do which is three-dimensional data of an object corresponding to the detection data D1 from three-dimensional data of a plurality of objects stored in advance, and generates complementary data D2 which is three-dimensional data by complementing the detection data D1 with the object data Do; and a data synthesis unit 42 that synthesizes the complementary data D2 and vehicle data Dv which is three-dimensional data of the vehicle 4 to generate synthetic data D3 which is three-dimensional data including the vehicle 4 and an outside region of the vehicle 4.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and a program for generating driving support information for a vehicle.

Background Art

[0002] Devices for providing driving support information for a vehicle are known for improving vehicle safety and preventing accidents. Patent Document 1 discloses a driving support information providing device that provides information until a state is reached where it is not possible to secure the minimum space required for opening and closing a flip-up tailgate of a vehicle, regardless of the shape of an obstacle existing behind the vehicle.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, with the device described in Patent Document 1, only local information about an object located outside the vehicle can be obtained, and sufficient driving support information may not be generated.

[0005] The present disclosure provides an information processing apparatus and the like that can generate driving support information including information about an object located outside the vehicle.

Means for Solving the Problems

[0006] An information processing apparatus according to an aspect of the present disclosure is an information processing apparatus that generates driving support information for a vehicle, and includes a detection data acquisition unit that acquires detection data, which is three-dimensional data obtained by detecting an object located outside the vehicle, and an object data extraction unit that extracts object data, which is three-dimensional data of an object corresponding to the detection data, from among three-dimensional data of a plurality of objects stored in advance, and generates complementary data, which is three-dimensional data, by complementing the detection data with the object data. Receive an input from the user to select a specific door among a plurality of doors provided in the vehicle and an input to select the door opening / closing angle state of the specific door, read vehicle data which is three-dimensional data of the vehicle in the door opening / closing angle state of the specific door selected by the user, The complementary data and the vehicle A data synthesis unit that generates synthesis data, which is three-dimensional data including the vehicle and an outer region of the vehicle, by synthesizing both data.

[0007] An information processing apparatus according to an aspect of the present disclosure is an information processing apparatus that generates driving support information for a vehicle, and includes a detection data acquisition unit that acquires detection data, which is three-dimensional data obtained by detecting an object located outside the vehicle, and an object data extraction unit that extracts object data, which is three-dimensional data of an object corresponding to the detection data, from among three-dimensional data of a plurality of objects stored in advance, and a synthesis unit that generates synthesis data, which is three-dimensional data, by synthesizing the detection data, the object data, and Receive an input from the user to select a specific door among a plurality of doors provided in the vehicle and an input to select the door opening / closing angle state of the specific door, of the The three-dimensional data of the vehicle.

[0008] An information processing method according to an aspect of the present disclosure is an information processing method that generates driving support information for a vehicle, and includes acquiring detection data, which is three-dimensional data obtained by detecting an object located outside the vehicle, extracting object data, which is three-dimensional data of an object corresponding to the detection data, from among three-dimensional data of a plurality of objects stored in advance, generating complementary data, which is three-dimensional data, by complementing the detection data with the object data, Receive an input from the user to select a specific door among a plurality of doors provided in the vehicle and an input to select the door opening / closing angle state of the specific door, read vehicle data which is three-dimensional data of the vehicle in the door opening / closing angle state of the specific door selected by the user, The complementary data and the vehicle And generating synthesis data, which is three-dimensional data including the vehicle and an outer region of the vehicle, by synthesizing both data.

[0009] A program according to an aspect of the present disclosure is a program for causing a computer to execute the above information processing method.

Advantages of the Invention

[0010] According to the information processing apparatus and the like of the present disclosure, driving support information including information on an object located outside the vehicle can be generated.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Mode for Carrying Out the Invention

[0012] (Background Leading to the Present Disclosure) The background leading to the present disclosure will be described with reference to FIGS. 1 and 2.

[0013] FIG. 1 is a diagram showing an example of an image provided by the information providing system of the comparative example. In the figure, an image displayed on the monitor in the vehicle cabin when the host vehicle backs and parks backward is shown.

[0014] FIG. 1(a) shows an image when looking at the rear of the host vehicle. In this image, other vehicles 6b, 6s, a pole 7, etc. located behind are shown, but if the driver is not accustomed to viewing this image, the actual distance from the host vehicle to other vehicles 6b, 6s, and pole 7 may not be grasped.

[0015] FIG. 1(b) shows an around view image when looking at the vehicle 4, which is the host vehicle, from above. In this image, other vehicles 6s, a pole 7, etc. located on the side are shown, but the farther away from the screen, the more other vehicles 6s and pole 7 are represented as spreading obliquely, and the driver may feel uncomfortable. Thus, in the comparative example, there is a problem that information on objects located around the vehicle 4 cannot be appropriately generated and sufficient driving support information cannot be provided.

[0016] FIG. 2 is a diagram showing an example of an image provided by the information providing system of the present disclosure.

[0017] Fig. 2(a) shows a side view of the vehicle 4, (b) shows a top view of the vehicle 4, and (c) shows an oblique top view of the vehicle 4. As shown in Fig. 2, by providing the vehicle 4 and the objects located around the vehicle 4 as image information that can be objectively or bird's-eye viewed from outside the vehicle 4, sufficient driving support information can be provided to the driver. The information processing apparatus and the like of the present disclosure have the following configuration in order to appropriately generate the driving support information as described above.

[0018] Hereinafter, embodiments will be specifically described with reference to the drawings. Each of the embodiments described below shows a comprehensive or specific example. The numerical values, shapes, materials, components, arrangement positions and connection forms of the components, steps, order of steps, etc. shown in the following embodiments are merely examples and are not intended to limit the present disclosure. In addition, among the components in the following embodiments, the components not described in the independent claims indicating the most general concept are described as optional components. Also, each figure is a schematic diagram and is not necessarily drawn precisely. In each figure, the same constituent members are denoted by the same reference numerals. In addition, the forms realized by arbitrarily combining two or more of the plurality of embodiments are also included in the present disclosure.

[0019] (Embodiment) [Configuration of Information Processing Apparatus and Information Providing System] The configuration of the information processing apparatus and the information providing system according to the embodiment will be described with reference to Figs. 3 to 8.

[0020] Fig. 3 is a block diagram showing an information processing apparatus 20 and an information providing system 1 according to the embodiment.

[0021] The information processing apparatus 20 is an apparatus that generates driving support information for the vehicle 4. For example, when the vehicle 4 parks, the information processing apparatus 20 generates driving support information for the vehicle 4. The information providing system 1 is a system that provides the driving support information generated by the information processing apparatus 20 and is mounted inside the vehicle 4 (see Fig. 4).

[0022] As shown in FIG. 3, the information providing system 1 includes a detection device 10 that detects an object and an information processing device 20 that generates driving support information. The information providing system 1 also includes a vehicle information acquisition unit 91, an operation switching unit 92, a monitor control unit 95, and a monitor 96.

[0023] The detection device 10 is a device that detects an object Oj (see FIGS. 5 and 6) located outside the vehicle 4, and is, for example, a stereo camera having a plurality of cameras. The detection device 10 may be a monocular camera using SLAM (Simultaneous Localization and Mapping) technology, or may be a lidar (LiDAR: Light Detection and Ranging) that performs light detection and distance measurement.

[0024] FIG. 4 is a diagram showing the installation position of the detection device 10 included in the information providing system 1.

[0025] The detection device 10 is constituted by, for example, a plurality of detection devices 11, 12, 13, and 14. The detection device 11 is provided on the rear body of the vehicle 4 to detect an object Oj located behind the vehicle 4. The detection device 12 is provided on the vehicle interior side of the front glass (windshield) to detect an object Oj located in front of the vehicle 4. The detection device 13 is provided on the right body or side mirror of the vehicle 4 to detect an object Oj on the right side of the vehicle 4. The detection device 14 is provided on the left body or side mirror of the vehicle 4 to detect an object Oj on the left side of the vehicle 4. For example, when the detection device 11 is a stereo camera, two cameras are arranged at the same height position in the vertical direction with a predetermined interval in the horizontal direction. The same applies to the other detection devices 12 to 14.

[0026] The object Oj detected by the detection device 10 is, for example, another vehicle 6b, 6s, a pole 7, a wall, a guardrail, etc. located behind and to the side of the vehicle 4. The object Oj may be a vehicle stopper 8 (see FIG. 2) located behind the vehicle 4, or may be a white line indicating the parking area of the vehicle 4. The object Oj may be an object having a form different from that of the driving road surface, the parking road surface, and the parking road surface, such as a curb or a gutter.

[0027] The detection device 10 obtains the three-dimensional distance between the vehicle 4 and the object Oj by detecting the object Oj, and generates detection data D1 which is the three-dimensional data of the object Oj. The detection data D1 includes data indicating the distance between the vehicle 4 and the three-dimensional point cloud of the object Oj. The detection data D1 generated by the detection device 10 is output to the information processing device 20.

[0028] As shown in FIG. 3, the information processing device 20 includes a detection data acquisition unit 30, a synthesis unit 40, a storage unit 70, a vehicle data selection unit 71, and an image generation unit 80. The storage unit 70 stores a computer program for operating the information processing device 20. The functions of each component included in the information processing device 20 are realized by executing the computer program stored in the storage unit 70.

[0029] The detection data acquisition unit 30 acquires the detection data D1 output from the detection device 10. Hereinafter, the detection data D1 obtained by detecting the rear and the right side of the vehicle 4 when the vehicle 4 parks backward will be described. The same applies to the left side and the front.

[0030] FIG. 5 is a diagram showing an example of detecting an object Oj located behind the vehicle 4 when the vehicle 4 parks backward.

[0031] In Fig. 5(a), an example of detecting another vehicle 6b located behind by a stereo camera is shown, and in Fig. 5(b), an example of detecting another vehicle 6b located behind by a LiDAR (Light Detection and Ranging) is shown. When the vehicle 4 performs reverse parking, the detection data acquisition unit 30 acquires detection data D1 of an object Oj located behind the vehicle 4. As shown in Fig. 5, the range that can be detected by the detection device 10 is the solid line portion shown in the figure, and the broken line portion cannot be detected. Therefore, the detection data D1 acquired by the detection data acquisition unit 30 is only the three-dimensional data of the rear part of the other vehicle 6b.

[0032] Fig. 6 is a diagram showing an example of detecting an object Oj located on the side of the vehicle 4 when the vehicle 4 performs reverse parking.

[0033] In Fig. 6(a), an example of detecting another vehicle 6s located on the side by a stereo camera is shown, and in Fig. 6(b), an example of detecting another vehicle 6s located on the side by a monocular camera using SLAM (Simultaneous Localization and Mapping) technology is shown. When the vehicle 4 performs reverse parking, the detection data acquisition unit 30 acquires detection data D1 of an object Oj located on the side of the vehicle 4. As shown in Fig. 6, the range that can be detected by the detection device 10 is the solid line portion shown in the figure, and the broken line portion cannot be detected. Therefore, the detection data D1 acquired by the detection data acquisition unit 30 is only the three-dimensional data of the side part of the other vehicle 6s.

[0034] Thus, when the range that can be detected by the detection device 10 is a part of the outer shape of the object Oj, the information that can be acquired by the detection data acquisition unit 30 is only a part of the information of the entire outer shape of the object Oj, and it is not possible to generate the driving support information required for the driver only with this information. Therefore, in the present embodiment, in addition to the above detection data D1, the driving support information is generated using the three-dimensional data of the object and the three-dimensional data of the vehicle 4 that are stored in advance in the storage unit 70.

[0035] The storage unit 70 stores 3D data of a plurality of objects. The 3D data of the objects is 3D data based on a predetermined origin and includes data of the entire outer shape of existing objects. The 3D data of the objects may be data indicating the outer shape of a standardized product or data indicating the outer shape of a product disclosed in an external information network. These 3D data of the objects may be updated and added in a timely manner whenever a new product is created. The 3D data of the objects stored in the storage unit 70 can be read from the data complementing unit 41 of the synthesizing unit 40.

[0036] Also, the storage unit 70 stores vehicle data Dv, which is 3D data of the vehicle 4. The vehicle data Dv is 3D data based on a predetermined origin and includes data of the entire outer shape of the vehicle 4. The vehicle data Dv is stored in the storage unit 70 when the information processing device 20 is mounted on the vehicle 4 or when an option is added to the vehicle 4. The vehicle data Dv stored in the storage unit 70 can be read from the data synthesizing unit 42 via a vehicle data selecting unit 71, which will be described later.

[0037] Note that the vehicle data Dv includes both data in a state where the doors of the vehicle 4 are closed and data in a state where the doors of the vehicle 4 are open. In this case, the doors of the vehicle 4 may be side doors 4s or rear doors 4b.

[0038] The vehicle data selection unit 71 selects at least one of the data of the vehicle data Dv in the state where the doors of the vehicle 4 are closed and the data in the state where the doors of the vehicle 4 are open. The vehicle data selection unit 71 selects data regarding the opening and closing state of the doors, for example, based on a selection switch input by the user. The selected data may be, for example, data in a state where three side doors 4s are closed and one side door 4s is open, or data in a state where four side doors 4s are closed and one back door 4b is open. The vehicle data selection unit 71 may select vehicle data Dv including information regarding the opening and closing angles of each door based on information regarding the opening and closing angles of each door set in advance. The data regarding the vehicle data Dv selected by the vehicle data selection unit 71 is output to the composition unit 40.

[0039] The composition unit 40 includes a data complement unit 41 and a data composition unit 42.

[0040] The data complement unit 41 extracts object data Do, which is the three-dimensional data of the object corresponding to the detection data D1, from the three-dimensional data of a plurality of objects stored in the storage unit 70. For example, the data complement unit 41 extracts, as the object data Do, the data having the highest correlation with the detection data D1 from the three-dimensional data of a plurality of objects. The degree of correlation may be determined by a correlation coefficient or a cosine similarity. Further, the data complement unit 41 may extract the object data Do corresponding to the detection data D1 by performing AI (Artificial Intelligence) analysis on the three-dimensional image.

[0041] FIG. 7 is a diagram schematically showing the three-dimensional data processed by the information processing apparatus 20. In this figure, the three-dimensional data is shown in a planar shape. In this example, an example in which the objects Oj located behind and on the right side of the vehicle 4 are other vehicles 6b and 6s will be described.

[0042] Figure 7 shows the rear detection data D1 and the side detection data D1. Figure 7 also shows the three-dimensional data of a plurality of objects, the extracted object data Do, and the vehicle data Dv. Figure 7 also shows the rear complement data D2, the side complement data D2, and the composite data D3.

[0043] The data complementing unit 41 extracts the object data Do corresponding to the detection data D1 of another vehicle 6b located behind the vehicle 4 from the three-dimensional data of the plurality of objects. For example, the data complementing unit 41 extracts the object data Do having the part closest to the detection data D1 at the rear part of another vehicle 6b from the three-dimensional data of the plurality of objects.

[0044] Then, the data complementing unit 41 generates the complement data D2, which is the rear three-dimensional data, by complementing the rear detection data D1 with the extracted object data Do. The rear complement data D2 is generated by adding the data (dashed line) other than the part overlapping with the rear detection data D1 among the extracted object data Do to the detection data D1 (solid line).

[0045] The data complementing unit 41 also extracts the object data Do corresponding to the detection data D1 of another vehicle 6s located on the side of the vehicle 4 from the three-dimensional data of the plurality of objects. For example, the data complementing unit 41 extracts the object data Do having the part closest to the detection data D1 at the side part of another vehicle 6s from the three-dimensional data of the plurality of objects.

[0046] Then, the data complementing unit 41 generates the complement data D2, which is the side three-dimensional data, by complementing the side detection data D1 with the extracted object data Do. The side complement data D2 is generated by adding the data (dashed line) other than the part overlapping with the side detection data D1 among the extracted object data Do to the detection data D1 (solid line).

[0047] In this way, the data completion unit 41 extracts a plurality of object data Do corresponding to a plurality of detection data D1 from among the three-dimensional data of a plurality of objects, and generates a plurality of completion data D2 by complementing each of the plurality of detection data D1 with the plurality of object data Do. The plurality of completion data D2 generated by the data completion unit 41 is output to the data synthesis unit 42.

[0048] The data synthesis unit 42 generates synthesis data D3 by synthesizing the rear completion data D2, the side completion data D2, and the vehicle data Dv.

[0049] Note that the information processing device 20 performs similar processing also in the front and the left side. That is, the data synthesis unit 42 synthesizes the rear, front, right side, and left side completion data D2 and the vehicle data Dv to generate synthesis data D3 including the vehicle 4 and the peripheral area of the vehicle 4. In this way, the synthesis unit 40 extracts object data Do corresponding to the detection data D1 from among the three-dimensional data of a plurality of objects, and generates synthesis data D3 by synthesizing the detection data D1, the object data Do, and the vehicle data Dv. The synthesis data D3 generated by the data synthesis unit 42 is output to the image generation unit 80.

[0050] The image generation unit 80 generates a predetermined image including the vehicle 4 and the outer area of the vehicle 4 based on the synthesis data D3 output from the data synthesis unit 42. The image generation unit 80 generates at least one of an image of the vehicle 4 seen from the side (see (a) of FIG. 2), an image of the vehicle 4 seen from above (see (b) of FIG. 2), and an image of the vehicle 4 seen obliquely (see (c) of FIG. 2) as the predetermined image. In this way, the image generation unit 80 generates an image with a changed viewing angle with respect to the vehicle 4, for example, an image obtained by overlooking the vehicle 4 and its surroundings. The image generated by the image generation unit 80 is not limited to a moving image, and may be a frame-by-frame still image.

[0051] The predetermined image generated by the image generation unit 80 is output to the monitor control unit 95. Note that the predetermined image output to the monitor control unit 95 is preset by operation keys on the monitor 96.

[0052] The vehicle information acquisition unit 91 is communicatively connected to the ECU (Electronic Control Unit) of the vehicle 4 and acquires the vehicle information output from the vehicle 4. The vehicle information is, for example, information regarding the presence or absence of parking start, and includes the ignition information and the shift lever information of the vehicle 4. The vehicle information acquisition unit 91 acquires the vehicle information output from the ECU and outputs the vehicle information to the operation switching unit 92.

[0053] The operation switching unit 92 switches the operations of the information processing device 20 and the monitor control unit 95 based on the vehicle information output from the vehicle information acquisition unit 91. For example, when the ignition is off, the operation switching unit 92 outputs a signal for stopping the operations of the information processing device 20 and the monitor control unit 95 to the information processing device 20 and the monitor control unit 95. Also, when the ignition is on, the operation switching unit 92 outputs a signal that enables the information processing device 20 to operate.

[0054] In addition, the operation switching unit 92 switches the driving support information by the information processing device 20 and the monitor control unit 95 according to the shift lever of the vehicle 4, that is, whether the vehicle 4 moves forward or backward. For example, when the ignition is on and the shift lever indicates reverse, the operation switching unit 92 operates the information processing device 20 to generate driving support information processing corresponding to reverse, and outputs an instruction signal for displaying an image corresponding to reverse to the monitor control unit 95.

[0055] The monitor control unit 95 generates an image for projection on the monitor 96 based on the command from the operation switching unit 92 and the output from the image generation unit 80. The monitor 96 provided in the vehicle interior displays an image based on the command of the monitor control unit 95.

[0056] [Image provided by the information providing system] FIG. 8 is a diagram showing an example of an image provided by the information providing systems of the comparative example and the embodiment.

[0057] The figure shows an image when viewed from the side of the vehicle 4 during reverse parking. Figure 8(a) is an image provided in the comparative example. In this image, only a part of the outer shape of another vehicle 6b is shown, and the state behind the vehicle 4 cannot be fully grasped.

[0058] Figure 8(b) is an image provided in the embodiment. This image is generated, for example, by changing the viewing point for the three-dimensional composite data D3. The image shown in Figure 8(b) is generated by viewing the three-dimensional composite data D3 from the side. According to this image, parts that could not be detected by the detection device 10 are also shown, and the state behind the vehicle 4 can be easily grasped.

[0059] Figure 9 is a diagram showing another example of an image provided by the information providing system 1.

[0060] Figure 9(a) shows an image of the state where the back door 4b is closed, (b) shows an image of the state where the back door 4b is open, and (c) shows an image indicating the movable area of the back door 4b. According to this image, it is possible to confirm in advance whether the vehicle will hit the rear object Oj when the back door 4b is opened after the vehicle 4 has finished parking. Note that the display of the opening and closing of the back door 4b can be selected in advance by the user.

[0061] Figure 10 is a diagram showing another example of an image provided by the information providing system 1.

[0062] The figure shows a plurality of images when viewed from the side of the vehicle 4 during reverse parking. Another vehicle 6b and a parking block 8 located behind the vehicle 4 are fixed, and the state of the vehicle 4 moving relative to them is shown. Also, the figure virtually shows the state where the back door 4b is open. According to this image, the vehicle 4 and the object Oj located behind the vehicle 4 can be provided as information objectively visible from outside the vehicle 4.

[0063] Figure 11 is a diagram showing another example of an image provided by the information providing systems of the comparative example and the embodiment.

[0064] shows an image of the vehicle 4 when parking backward as seen from above. (a) of FIG. 11 is an image provided in the comparative example. In this image, only a part of the outer shape of the other vehicle 6s is shown, and the state on the side of the vehicle 4 cannot be fully grasped.

[0065] (b) of FIG. 11 is an image provided in the embodiment. This image is generated, for example, by changing the viewing point with respect to the three-dimensional composite data D3. The image shown in (b) of FIG. 11 is generated by viewing the three-dimensional composite data D3 from above. According to this image, parts that could not be detected by the detection device 10 are also shown, and the state on the side of the vehicle 4 can be easily grasped.

[0066] FIG. 12 is a diagram showing another example of the image provided by the information providing system 1.

[0067] (a) of FIG. 12 shows an image of the side door 4s in a closed state, and (b) shows an image of the two side doors 4s in an open state. According to this image, it is possible to confirm in advance whether the vehicle will collide with the side object Oj when the side door 4s is opened after the vehicle 4 has finished parking. Note that the presence or absence of the display of the opening and closing of the side door 4s can be selected in advance by the user. Also, the opening and closing angle of the side door 4s can be set in advance by the user.

[0068] FIG. 13 is a diagram showing another example of the image provided by the information providing system 1.

[0069] The figure shows a plurality of images of the vehicle 4 when parking backward as seen from above. The other vehicle 6s, the pole 7 located on the side of the vehicle 4, and the car stopper 8 located behind are fixed, and the state in which the vehicle 4 moves relative to them is shown. Also, the figure virtually shows the side door 4s in an open state. According to this image, the vehicle 4 and the objects Oj located on the side and rear of the vehicle 4 can be provided as information that can be seen from above outside the vehicle 4.

[0070] [Information Processing Method] The information processing method according to the embodiment will be described with reference to FIG. 14. The information processing method shown in this example is a method for generating driving support information for the vehicle 4 when the vehicle 4 parks.

[0071] FIG. 14 is a flowchart showing the information processing method according to the embodiment. Note that FIG. 14 describes not only the information processing device 20 but also the processes executed in the information providing system 1.

[0072] First, the information providing system 1 determines whether there is a request for driving support information (step S11). Specifically, the vehicle information acquisition unit 91 checks whether it has acquired a request signal for driving support information from the ECU. If there is no request for driving support information (No in S11), the process returns to step S11 and waits for a request for driving support information. If there is a request for driving support information (Yes in S11), the information processing device 20 performs various processes for generating driving support information.

[0073] The detection data acquisition unit 30 acquires detection data D1, which is three-dimensional data obtained by detecting an object Oj located outside the vehicle 4 (step S12). The detection data D1 is data obtained by the detection device 10 installed in the vehicle 4. Note that the detection data D1 is three-dimensional data of a part of the outer shape of the object Oj located outside the vehicle 4. The detection data acquisition unit 30 acquires the detection data D1 for the rear and side of the vehicle 4 respectively.

[0074] The data completion unit 41 extracts object data Do, which is the 3D data of the object corresponding to the detection data D1, from among the 3D data of a plurality of objects stored in advance (step S13). The 3D data of the plurality of objects is the data of the objects stored in the storage unit 70 and includes the 3D data of the entire outer shape of the objects. The data completion unit 41 extracts, as the object data Do, data with a high correlation to the detection data D1 from among the 3D data of the plurality of objects stored in the storage unit 70. The data completion unit 41 extracts the respective object data Do corresponding to the detection data D1 at the rear and sides of the vehicle 4.

[0075] Also, the data completion unit 41 generates complementary data D2, which is 3D data, by complementing the detection data D1 with the object data Do (step S14). For example, the data completion unit 41 generates the complementary data D2 by adding to the detection data D1 the data other than the part overlapping with the detection data D1 in the object data Do. The data completion unit 41 generates the respective complementary data D2 at the rear and sides of the vehicle 4.

[0076] Next, the data synthesis unit 42 generates synthesized data D3, which is 3D data including the vehicle 4 and the outer region of the vehicle 4, by synthesizing the complementary data D2 and the vehicle data Dv (step S15). For example, the data synthesis unit 42 generates the synthesized data D3 by synthesizing the complementary data D2 at the rear of the vehicle 4, the complementary data D2 at the side of the vehicle 4, and the vehicle data Dv. The vehicle data Dv is the 3D data of the vehicle 4 stored in the storage unit 70 and includes the data of the entire outer shape of the vehicle 4. Note that the data synthesis unit 42 may read from the storage unit 70 the vehicle data Dv in which data related to the opening and closing of the doors is reflected.

[0077] Next, the image generation unit 80 generates a predetermined image including the vehicle 4 and the outer region of the vehicle 4 based on the composite data D3 (step S16). The predetermined image is, for example, at least one of an image of the vehicle 4 viewed from the side, an image viewed from above, and an image viewed obliquely. The monitor control unit 95 controls the monitor based on the predetermined image generated by the image generation unit 80 and causes the monitor 96 to display the image (step S17).

[0078] Next, the information providing system 1 determines again whether there is a request for driving support information (step S18). Specifically, the vehicle information acquisition unit 91 checks whether it is continuously acquiring a driving support information request signal from the ECU. If there is a request for driving support information (Yes in S18), the process returns to step S12 and the provision of driving support information continues. If there is no request for driving support information (No in S18), the information processing device 20 ends various processes for generating driving support information. By executing the above steps S11 to S18, the driving support information for the vehicle 4 can be appropriately generated and provided.

[0079] [Modification Example 1 of the Embodiment] The information processing device 20 of the present embodiment can be applied not only when the vehicle 4 is parking but also when the vehicle 4 and another vehicle 6s pass by each other on a narrow road. The information providing system 1 of the modification example 1 provides driving support information, for example, by receiving a predetermined switch input when the vehicle 4 is stopped or traveling slowly.

[0080] FIG. 15 is a diagram showing an image provided by the information providing system 1 of the modification example 1 of the embodiment.

[0081] Fig. 15(a) shows a rear view of vehicle 4 and other vehicles 6s, (b) shows a top view of vehicle 4 and other vehicles 6s, and (c) shows an obliquely upward view of vehicle 4 and other vehicles 6s. As shown in Fig. 15, by providing image information that allows vehicle 4 and the objects Oj located around vehicle 4 to be viewed from above or objectively outside vehicle 4, it becomes possible to provide sufficient driving support information to the driver even when driving on a narrow road.

[0082] [Modification Example 2 of the Embodiment] The information processing device 20 of this embodiment can be applied not only when vehicle 4 parks but also when vehicle 4 stops on the road shoulder. The information providing system 1 of Modification Example 2 provides driving support information by receiving a predetermined switch input, for example, when vehicle 4 is stopped or traveling slowly.

[0083] Fig. 16 is a diagram showing an image provided by the information providing system 1 of Modification Example 2 of the embodiment.

[0084] Fig. 16(a) shows a rear view of vehicle 4, (b) shows a top view of vehicle 4, and (c) shows an obliquely upward view of vehicle 4. As shown in Fig. 16, by providing image information that allows vehicle 4 and the objects Oj located around vehicle 4 to be viewed from above or objectively outside vehicle 4, it becomes possible to provide sufficient driving support information to the driver even when vehicle 4 stops on the road shoulder.

[0085] (Summary) The information processing apparatus 20 according to this embodiment is an information processing apparatus that generates driving support information for the vehicle 4. The information processing apparatus 20 includes a detection data acquisition unit 30 that acquires detection data D1, which is three-dimensional data obtained by detecting an object Oj located outside the vehicle 4, a data complementation unit 41 that extracts object data Do, which is three-dimensional data of an object corresponding to the detection data D1, from among the three-dimensional data of a plurality of objects stored in advance, and generates complementation data D2, which is three-dimensional data, by complementing the detection data D1 with the object data Do, and a data synthesis unit 42 that generates synthesis data D3, which is three-dimensional data including the vehicle 4 and the outer region of the vehicle 4, by synthesizing the complementation data D2 and vehicle data Dv, which is three-dimensional data of the vehicle 4.

[0086] In this way, by complementing the three-dimensional detection data D1 with the three-dimensional object data Do to generate three-dimensional complementation data D2, and synthesizing the complementation data D2 and the three-dimensional vehicle data Dv to generate three-dimensional synthesis data D3, it is possible to generate driving support information including information on the object Oj located outside the vehicle 4.

[0087] Further, the detection data D1 may be three-dimensional data of a part of the outer shape of the object Oj located outside the vehicle 4, and the object data Do may include three-dimensional data of the entire outer shape of the object stored in advance.

[0088] In this way, since the object data Do includes data of the entire outer shape of the object, even if the detection data D1 is data of a part of the outer shape of the object Oj, it is possible to appropriately generate three-dimensional complementation data D2. Thereby, by synthesizing the complementation data D2 and the three-dimensional vehicle data Dv to generate three-dimensional synthesis data D3, it is possible to generate driving support information including information on the object Oj located outside the vehicle 4.

[0089] Further, the data complementation unit 41 may generate the complementation data D2 by adding data other than the part overlapping with the detection data D1 in the object data Do to the detection data D1.

[0090] According to this, based on the detection data D1 of the actually detected object Oj, complementary data D2 that conforms to reality can be generated. Thereby, driving support information including information on the object Oj located outside the vehicle 4 can be generated.

[0091] Further, the information processing device 20 further includes a storage unit 70 that stores three-dimensional data of a plurality of objects. The data complementing unit 41 may extract, as object data Do, data having a high correlation with the detection data D1 from among the three-dimensional data of the plurality of objects stored in the storage unit 70.

[0092] By extracting, as object data Do, data having a high correlation with the detection data D1 in this way, the extraction accuracy of the object data Do can be improved. Thereby, the complementary data D2 can be appropriately generated, and driving support information including information on the object Oj located outside the vehicle 4 can be generated.

[0093] Further, the information processing device 20 further includes a storage unit 70 in which vehicle data Dv is stored, and the data synthesizing unit 42 may generate synthesized data D3 based on the vehicle data Dv stored in the storage unit 70.

[0094] According to this, driving support information including information on the vehicle 4 and the object Oj located outside the vehicle 4 can be generated. Thereby, it becomes possible to provide information that looks down on or objectively appears from outside the vehicle 4.

[0095] Further, the information processing device 20 may generate driving support information when the vehicle 4 parks.

[0096] Thereby, when the vehicle 4 parks, driving support information including information on the object Oj located outside the vehicle 4 can be generated.

[0097] Further, the detection data acquisition unit 30 acquires a plurality of detection data D1, which are detection data of objects Oj located at least behind and to the sides of the vehicle 4. The data complementation unit 41 extracts a plurality of object data Do based on the plurality of detection data D1, and generates a plurality of complemented data D2 by complementing the plurality of detection data D1 with the plurality of object data Do. The data synthesis unit 42 may generate synthesized data D3 based on the plurality of complemented data D2 and vehicle data Dv.

[0098] In this way, by generating the synthesized data D3 using the rear and side detection data D1, object data Do, and complemented data D2, it is possible to generate driving support information including information on the vehicle 4 and the objects Oj located around the vehicle 4. As a result, it becomes possible to provide information that appears to be viewed from above or objectively from outside the vehicle 4.

[0099] Further, the information processing device 20 further includes an image generation unit 80 that generates a predetermined image including the vehicle 4 and an outer region of the vehicle 4 based on the synthesized data D3 generated by the data synthesis unit 42. The image generation unit 80 may generate at least one of an image viewed from the side of the vehicle 4, an image viewed from above, and an image viewed obliquely as the predetermined image.

[0100] As a result, it becomes possible to provide information that appears to be viewed from above or objectively from outside the vehicle 4.

[0101] Further, the vehicle data Dv may include at least one of data in a state where the doors of the vehicle 4 are closed and data in a state where the doors of the vehicle 4 are open.

[0102] According to this, it becomes possible to generate and provide driving support information including information regarding the opening and closing of the doors of the vehicle 4.

[0103] The information processing apparatus 20 according to this embodiment is an information processing apparatus that generates driving support information for the vehicle 4. The information processing apparatus 20 includes a detection data acquisition unit 30 that acquires detection data D1, which is three-dimensional data obtained by detecting an object Oj located outside the vehicle 4, and an object data extraction unit that extracts object data Do, which is three-dimensional data of an object corresponding to the detection data D1 from among the three-dimensional data of a plurality of objects stored in advance, and generates composite data D3, which is three-dimensional data, by synthesizing the detection data D1, the object data Do, and the three-dimensional data (vehicle data Dv) of the vehicle 4.

[0104] In this way, by synthesizing the three-dimensional detection data D1, the three-dimensional object data Do, and the three-dimensional vehicle data Dv to generate the three-dimensional composite data D3, it is possible to generate driving support information including information on the object Oj located outside the vehicle 4.

[0105] The information processing method according to this embodiment is an information processing method for generating driving support information for the vehicle 4. The method includes acquiring detection data D1, which is three-dimensional data obtained by detecting an object Oj located outside the vehicle 4, extracting object data Do, which is three-dimensional data of an object corresponding to the detection data D1 from among the three-dimensional data of a plurality of objects stored in advance, generating complementary data D2, which is three-dimensional data, by complementing the detection data D1 with the object data Do, and generating composite data D3, which is three-dimensional data including the vehicle 4 and the outer region of the vehicle 4, by synthesizing the complementary data D2 and the vehicle data Dv, which is the three-dimensional data of the vehicle 4.

[0106] In this way, by generating the three-dimensional complementary data D2 by complementing the three-dimensional detection data D1 with the three-dimensional object data Do, and generating the three-dimensional composite data D3 by synthesizing the complementary data D2 and the three-dimensional vehicle data Dv, it is possible to generate driving support information including information on the object Oj located outside the vehicle 4.

[0107] The program according to this embodiment is a program for causing a computer to execute the above information processing method.

[0108] This enables the computer to execute the above-described information processing method according to a program.

[0109] Note that these general or specific aspects may be implemented by a system, a method, an integrated circuit, a computer program, or a non-transitory recording medium such as a CD-ROM readable by a computer, or may be implemented by any combination of a system, a method, an integrated circuit, a computer program, or a recording medium. The program may be pre-stored in the recording medium or may be supplied to the recording medium via a wide area communication network including the Internet or the like.

[0110] (Other Embodiments) The embodiments etc. have been described above. However, the above embodiments etc. are essentially preferred examples and are not intended to limit the scope of the present invention, its applications, or its uses.

[0111] In the above embodiment, an example in which the detection device 10 outputs the detection data D1 to the detection data acquisition unit 30 has been shown, but it is not limited thereto. For example, when the raw data detected by the detection device 10 is output to the detection data acquisition unit 30, the detection data acquisition unit 30 may generate the detection data D1 including three-dimensional distance information based on this raw data to acquire the detection data D1.

[0112] In the above embodiment, an example in which the data complementation unit 41 extracts the predetermined object data Do from the three-dimensions of a plurality of objects stored in the storage unit 70 has been shown, but it is not limited thereto. For example, when the vehicle 4 is provided with a wireless module, the data complementation unit 41 may access an external server via the wireless module and extract the predetermined object data Do from the three-dimensional data of a plurality of objects pre-stored in the server.

[0113] In the above-described embodiment, the case where the vehicle 4 parks backward has been described, but the present invention is not limited thereto. For example, the information providing system 1 can generate driving support information for the vehicle 4 and provide it to the driver even when the vehicle 4 parks forward or parks in a row.

[0114] In the above-described embodiment, an example in which one image is projected on the monitor 96 has been shown, but the present invention is not limited thereto. For example, when the monitor 96 is split-displayed, the three images of FIGS. 2(a), 2(b), and 2(c) may be simultaneously displayed on the monitor 96. Further, in addition to FIGS. 2(a) to 2(c), the image shown in FIG. 1(a) may be simultaneously displayed.

[0115] As described above, the present disclosure has been described based on the embodiments and the modified examples of the embodiments. However, the present disclosure is not limited to these embodiments and the modified examples of the embodiments.

[0116] Further, the information processing apparatus according to each of the above-described embodiments and the modified examples of the embodiments is realized by a program using a computer, and such a program may be stored in a storage device.

[0117] Further, each processing unit included in the information processing apparatus according to each of the above-described embodiments and the modified examples of the embodiments is typically realized as an LSI which is an integrated circuit. These may be individually formed into one chip, or may be formed into one chip so as to include some or all of them.

[0118] Further, the integration into an integrated circuit is not limited to an LSI, and it may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) which can be programmed after manufacturing the LSI, or a reconfigurable processor which can reconfigure the connection and setting of circuit cells inside the LSI may be used.

[0119] In each of the above-described embodiments and modified examples of the embodiments, each component may be configured by dedicated hardware or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory.

[0120] Also, all the numbers used above are for illustrative purposes to specifically explain the present disclosure, and the embodiments and modified examples of the embodiments of the present disclosure are not limited to the illustrated numbers.

[0121] Also, the division of functional blocks in the block diagram is an example, and a plurality of functional blocks may be realized as one functional block, one functional block may be divided into a plurality, or some functions may be transferred to other functional blocks. Also, the functions of a plurality of functional blocks having similar functions may be processed by a single piece of hardware or software in parallel or time-division.

[0122] Also, the order in which each step in the flowchart is executed is for illustrative purposes to specifically explain the present disclosure, and other orders may be used. Also, some of the above steps may be executed simultaneously (in parallel) with other steps.

[0123] In addition, forms obtained by applying various modifications that can be conceived by those skilled in the art to the embodiments and modified examples of the embodiments, and forms realized by arbitrarily combining the components and functions in the embodiments and modified examples of the embodiments without departing from the spirit of the present disclosure are also included in the present disclosure.

Industrial Applicability

[0124] The present disclosure can be widely used in fields such as automotive technology as an information processing device that generates driving support information for vehicles.

Explanation of Signs

[0125] 1 Information providing system 4 Vehicle 4b Back door 4s Side door 6b, 6s Other vehicles 7 Pole 8 Vehicle stopper 10, 11, 12, 13, 14 Detection device 20 Information processing device 30 Detection data acquisition unit 40 Synthesis unit 41 Data complementation unit 42 Data synthesis unit 70 Storage unit 71 Vehicle data selection unit 80 Image generation unit 91 Vehicle information acquisition unit 92 Operation switching unit 95 Monitor control unit 96 Monitor D1 Detection data D2 Complemented data D3 Synthesized data Do Object data Dv Vehicle data Oj Object

Claims

1. An information processing apparatus for generating driving support information for a vehicle, a detection data acquisition unit that acquires detection data, which is three-dimensional data obtained by detecting an object located outside the vehicle; a data complementation unit that extracts object data, which is three-dimensional data of an object corresponding to the detection data, from among the three-dimensional data of a plurality of objects stored in advance, and generates complementation data, which is three-dimensional data, by complementing the detection data with the object data; an input for selecting a specific door among a plurality of doors provided in the vehicle and an input for selecting a door opening / closing angle state of the specific door are received from a user, vehicle data, which is three-dimensional data of the vehicle in the door opening / closing angle state of the specific door selected by the user, is read, and composite data, which is three-dimensional data including the vehicle and an outer region of the vehicle, is generated by synthesizing the complementation data and the vehicle data; An information processing apparatus comprising the above.

2. The detection data is three-dimensional data of a part of the outer shape of an object located outside the vehicle, The object data includes three-dimensional data of the entire outer shape of the object stored in advance The information processing apparatus according to claim 1.

3. The data complementation unit generates the complementation data by adding data other than the part overlapping with the detection data in the object data to the detection data The information processing apparatus according to claim 1.

4. Further comprising a storage unit that stores three-dimensional data of the plurality of objects, The data complementation unit extracts, as the object data, data having a high correlation with the detection data from among the three-dimensional data of the plurality of objects stored in the storage unit The information processing apparatus according to claim 1.

5. Further comprising a storage unit in which the vehicle data is stored, The data synthesis unit generates the synthesized data based on the vehicle data stored in the storage unit. The information processing apparatus according to claim 1.

6. The information processing apparatus generates the driving support information when the vehicle parks. The information processing apparatus according to any one of claims 1 to 5.

7. The detection data acquisition unit acquires a plurality of the detection data which are detection data of objects located at least behind and on the sides of the vehicle. The data complementation unit extracts a plurality of the object data based on the plurality of the detection data, and generates a plurality of the complemented data by complementing the plurality of the detection data with the plurality of the object data. The data synthesis unit generates the synthesized data based on the plurality of the complemented data and the vehicle data. The information processing apparatus according to claim 1.

8. Further, an image generation unit is provided which generates a predetermined image including the vehicle and an outer region of the vehicle based on the synthesized data generated by the data synthesis unit. The image generation unit generates at least one of an image viewed from the side, an image viewed from above, and an image viewed obliquely with respect to the vehicle as the predetermined image. The information processing apparatus according to claim 1.

9. The vehicle data includes at least one of data in a state where the door of the vehicle is closed and data in a state where the door of the vehicle is open. The information processing apparatus according to claim 1.

10. An information processing apparatus for generating driving support information for a vehicle, comprising: a detection data acquisition unit that acquires detection data which is three-dimensional data obtained by detecting an object located outside the vehicle; Extract object data, which is 3D data of an object corresponding to the detection data, from the 3D data of a plurality of pre-stored objects, receive from the user an input for selecting the detection data, the object data, and a specific door among a plurality of doors provided in the vehicle, and an input for selecting the door opening / closing angle state of the specific door, and generate composite data, which is 3D data, by synthesizing the 3D data of the vehicle in the door opening / closing angle state of the specific door selected by the user. A synthesizing unit for An information processing apparatus comprising

11. An information processing method for generating driving support information for a vehicle, comprising Obtain detection data, which is 3D data obtained by detecting an object located outside the vehicle, Extract object data, which is 3D data of an object corresponding to the detection data, from the 3D data of a plurality of pre-stored objects, Generate complementary data, which is 3D data, by complementing the detection data with the object data, Receive from the user an input for selecting a specific door among a plurality of doors provided in the vehicle and an input for selecting the door opening / closing angle state of the specific door, read vehicle data, which is 3D data of the vehicle in the door opening / closing angle state of the specific door selected by the user, and generate composite data, which is 3D data including the vehicle and an outer region of the vehicle, by synthesizing the complementary data and the vehicle data. Information processing method.

12. A program for causing a computer to execute the information processing method according to Claim 11.

Images