Information processing device, information processing method, and program

Abstract

To smoothly perform guidance to a suitable parking space even with respect to a vehicle of a type (e.g., a vehicle with a load carrying platform) where the size of the vehicle is dynamically changed in response to a use state.SOLUTION: A control section acquires, from a parking lot database, information on a parking space suitable for the size of a vehicle at present in a predetermined range from a reference position which includes at least one of the present position of the vehicle and a destination of the vehicle. The control section supports driving to the parking space by expansion-displaying a real physical space by integrating the real physical space around the vehicle with a virtual space based on the acquired information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] Conventionally, vehicle navigation systems with various functions have been proposed. For example, Patent Document 1 below proposes a navigation system that guides a vehicle to a destination, and that acquires parking lot map data including at least information about parking spaces provided in parking lots around the destination and condition information indicating selection conditions for selecting a parking space determined by a vehicle occupant, and guides the vehicle from the current position of the vehicle to a destination parking space. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-144075 [Patent Document 2] Japanese Patent Publication No. 2021-140479 [Patent Document 3] Japanese Patent Publication No. 2022-156099 [Patent Document 4] Japanese Patent Application Publication No. 2023-117752 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the conventional navigation system may not always be able to guide a vehicle of a variable size to an appropriate parking space. An aspect of the embodiment of the present disclosure is to guide a vehicle of a type whose size dynamically changes depending on the usage situation (e.g., a vehicle with a cargo bed) to a parking space that is suitable for that vehicle. [Means for solving the problem]

[0005] In one aspect, an embodiment of the disclosure is exemplified by an information processing device. The device includes a control unit that acquires, from a parking space database, information on parking spaces that fit the vehicle's current size within a predetermined range from a reference position that includes at least one of the vehicle's current location and the vehicle's destination. Based on the acquired information, the control unit then performs driving assistance to the parking space by displaying an expanded real physical space that combines information on the real physical space around the vehicle with information on the virtual space. [Effects of the Invention]

[0006] The information processing device can guide a type of vehicle whose size changes dynamically depending on the usage situation to a parking space that is suitable for that vehicle. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram illustrating an information system according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of the structure of a vehicle that is the target of processing by the information system of the embodiment. [Figure 3] FIG. 3 is a hardware configuration diagram of a computer that can be used as the server, the parking space DB, or the vehicle-mounted device. [Figure 4] Figure 4 shows an example of a display when driving assistance is provided using XR display. [Figure 5] FIG. 5 is a diagram illustrating an example of data stored in the parking space DB. [Figure 6] FIG. 6 is a diagram illustrating a correspondence table that defines user attributes and priorities when selecting a parking space. [Figure 7] FIG. 7 is a flowchart illustrating the processing of the server or the vehicle-mounted device. DETAILED DESCRIPTION OF THE INVENTION

[0008] An information processing device, an information processing method, and a program according to an embodiment will be described below with reference to the drawings. The information processing device is exemplified by a server 1 or an in-vehicle device 3. In this embodiment, the vehicle 50 can dynamically change its size (at least one of length, width, and height) depending on the current situation. Therefore, the information processing device determines the current size of the vehicle 50 in order to obtain information on parking spaces that fit the current size of the vehicle 50. Meanwhile, a parking space database, exemplified by parking space DB2, contains the geographical locations of parking spaces and the maximum vehicle size that can be parked therein. Therefore, the information processing device obtains information on parking spaces that can accommodate the vehicle 50 of its current size from the parking space database and displays an augmented version of the actual physical space, thereby providing driving assistance to the parking space in a manner that is easy for the user to understand.

[0009] (System Configuration) 1 is a diagram illustrating an information system 100 according to this embodiment. The information system 100 includes a server 1, a parking space database (hereinafter referred to as a parking space DB2), an on-board device 3 mounted on a vehicle 50, and a camera 4 connectable to the on-board device 3. In the information system 100, the server 1, the parking space DB2, and the on-board device 3 can communicate with each other via a network N1.

[0010] The network N1 includes a wireless network and a wired network. The network N1 is, for example, a Long Term Evolution (LTE) or a fifth generation mobile communication system (5G ), and mobile communication systems such as the sixth generation mobile communication system (6G), wireless LANs (Local Area Networks), etc. The network N1 also includes public networks such as the Internet.

[0011] The server 1 is a normal computer, for example, as shown in Fig. 3. However, the server 1 is not limited to the single computer shown in Fig. 3. The server 1 may be a system in a cloud environment, for example, in which multiple computers cooperate via a network N1 or the like to execute processing using virtualized resources.

[0012] The server 1 cooperates with, for example, the in-vehicle device 3 to assist driving of the vehicle 50. The server 1 receives, for example, from the in-vehicle device 3, information such as the current position of the vehicle 50, the destination of the vehicle 50, and the current size of the vehicle 50, as well as a request to search for a parking space and a request for driving assistance to the searched parking space.

[0013] The current position of the vehicle 50 is position information of the point where the vehicle 50 is currently located, such as latitude and longitude. The destination of the vehicle 50 is a destination set in the navigation system of the vehicle-mounted device 3. The current size of the vehicle 50 is, for example, the length of the vehicle 50 in the direction of travel, the horizontal width perpendicular to the direction of travel of the vehicle 50 (also referred to as vehicle width), and the height of the vehicle 50 from the ground (also referred to as vehicle height). In this embodiment, the information system 100 assumes that the size of the vehicle 50, for example, the height, changes dynamically depending on the situation. The server 1 or the vehicle-mounted device 3 searches the parking space DB2 for a parking space that is suitable for the current size of the vehicle 50 and can accommodate the vehicle 50.

[0014] The driving assistance request may include information on the attributes of the user of the vehicle 50 (such as the driver, passengers, or owner of the vehicle 50). Examples of the user attributes include age, gender, preferences, and the presence of children or infants as passengers. When the driving assistance request includes information on the user attributes, the server 1 or the like may search for parking spaces that have desirable characteristics for the user attributes. Just search for parking spaces in DB2.

[0015] Then, for example, the server 1 transmits information for providing driving assistance such as route guidance (car navigation) on a route from the current position of the vehicle 50 to the searched parking space to the in-vehicle device 3. The in-vehicle device 3 provides driving assistance to the driver of the vehicle 50 in accordance with the information transmitted from the server 1. In this embodiment, the server 1 causes the in-vehicle device 3 to provide driving assistance using augmented reality (AR), mixed reality (MR), virtual reality (VR), and other cross reality (Extended Reality / Cross Reality (XR)).

[0016] The parking space DB2 is an example of a parking database that stores information about local parking spaces and provides the stored information in response to external search requests. The hardware configuration of the parking space DB2 itself is the same as that of the server 1, and is illustrated in, for example, FIG. 3. However, like the server 1, the parking space DB2 may also be a system in a cloud environment.

[0017] Parking Space DB2, for example, is a service that uses multiple Multi-access Edge Computing (MEC) servers. The parking space DB2 may be provided for each area in which the vehicle 50 travels, and may store information about parking spaces in that area. When the server 1 receives a parking space search request from the in-vehicle device 3, the server 1 may identify the location area of ​​the vehicle 50 from the current position of the vehicle 50 and access the parking space DB2 that stores information about that location area. However, the in-vehicle device 3 may identify the location area of ​​the vehicle 50 from the current position of the vehicle 50 without going through the server 1, and access the parking space DB2 that stores information about that location area.

[0018] The information held by the parking space DB2 includes, for example, identification information SID that identifies each parking space and information about the parking space specified by the identification information SID. One item of information about the parking space specified by the identification information SID is the location of the parking space. The parking space information also includes the size (length, width, height) of the largest vehicle 50 that can be parked in the parking space, characteristic information, and whether the parking space is currently vacant or not (vacant / occupied) (see FIG. 5). Details of the information held by the parking space DB2 will be explained separately with reference to FIG. 5.

[0019] The vehicle 50 is a vehicle whose size, for example, the height of the vehicle 50, can be changed dynamically or freely depending on the usage situation. The height of the vehicle 50 can be changed, for example, by changing the vertical dimensions (length, height) of the wall sections erected on the bottom surface of the box-shaped loading platform 51. In a simple configuration, the height of the vehicle 50, for example, the vertical dimensions (length, height) of the wall sections of the box-shaped loading platform 51, may be fixed. In this case, the height of the vehicle 50 is variable depending on the amount of luggage loaded on the loading platform 51. However, the width or length of the loading platform 51 of the vehicle 50 may also be extendable or retractable, for example, within the limits of the Road Traffic Act.

[0020] The vehicle 50 includes an in-vehicle device 3 and a camera 4. The in-vehicle device 3 can provide multimedia information including video, sound, and the like to a user inside the vehicle. The hardware configuration of the in-vehicle device 3 is similar to that of the computer illustrated in FIG. 3, for example. The in-vehicle device 3 cooperates with the server 1 to provide driving assistance using XR such as AR, MR, and VR. However, the division of processing between the server 1 and the in-vehicle device 3 can vary and is not limited. For example, the in-vehicle device 3 may simply operate as an information output device, and the server 1 may actually perform driving assistance processing using XR.

[0021] On the other hand, the server 1 simply acts as an interface with the parking space DB 2, and the in-vehicle device 3 essentially performs XR-based parking space information acquisition based on the information about the parking space acquired through the server 1. In this case, the vehicle-mounted device 3 may store the information about the parking space acquired from the server 1 in the main storage device 12, the external storage device 13, etc. (see FIG. 3). The vehicle-mounted device 3 may then periodically access the server 1 and update the information about the parking space as needed. The vehicle-mounted device 3 may also access the parking space DB2 directly without going through the server 1.

[0022] The camera 4 is connected to the in-vehicle device 3 and communicates with the vehicle through a Controller Area Network (CAN), FLEXR The camera 4 provides the captured image of the loading platform 51 to the vehicle-mounted device 3 via an in-vehicle Local Area Network (LAN) such as AY (registered trademark). However, the camera 4 may not be mounted on the vehicle, but may be carried by the user and connectable to the vehicle-mounted device 3. For example, the camera 4 may be mounted on the user's smartphone and capable of communicating with the vehicle-mounted device 3 via a wireless access network such as LTE, 5G, or 6G or a wireless LAN.

[0023] FIG. 2 is a diagram illustrating the structure of vehicles 50A and 50B that are targets of processing by the information system 100 of this embodiment. Vehicles 50A and 50B are specific examples of vehicle 50 and are collectively referred to as vehicle 50. FIG. 2(A) illustrates the structure of vehicle 50A. Vehicle 50A is a truck. A box-shaped loading platform 51 of vehicle 50A has a bottom and a wall portion erected on the bottom. This wall portion has a fixed portion 51A and a variable portion 51B. A user can change the load capacity of vehicle 50A by adjusting the height of variable portion 51B between 0 and an upper limit value (LM). However, variable portion 51B does not have to be a wall portion of loading platform 51, and may simply be the cargo itself, secured by a sheet, rope, or the like.

[0024] The width of the loading platform 51 may be adjustable by moving the left and right walls of the vehicle 50A toward the outside or inside of the vehicle 50A in the traveling direction of the vehicle 50A. The loading platform 51 may also be extendable to the rear of the vehicle 50A. The vehicle 50 includes an in-vehicle device 3, cameras 4, 4F, and 4R, and obstacle sensors 6 (6U, 6FL, and 6BL in FIG. 2). Two cameras (referred to as side cameras) are provided in the mirrors on the left and right sides of the vehicle 50. The camera 4 is installed, for example, on the ceiling outer wall of the passenger compartment 52, and captures an image of the rear of the vehicle 50A on the opposite side from the traveling direction of the vehicle 50A, providing the captured image to the in-vehicle device 3. The in-vehicle device 3 measures the height (Lx) of the adjustable section 51B from the provided image. The in-vehicle device 3 calculates the height (Lx) of the adjustable section 51B in the following manner. Here, the height (Lx) of the variable portion 51B refers to the dimension of the portion of the variable portion 51B on the upper side in the vertical direction, with the outer surface of the ceiling of the passenger compartment 52 as the reference (height 0).

[0025] When the height (Lx) of the variable portion 51B is measured, for example, the visual axis of the camera 4 is parallel to the horizontal direction, and the angle of view is fixed. The vehicle-mounted device 3 stores an image when the height of the variable portion 51B reaches the upper limit value (LM) within the angle of view of the camera 4. The vehicle-mounted device 3 also stores the actual measured value of the upper limit value (LM). The vehicle-mounted device 3 calculates the measured value of the height (Lx) of the variable portion 51B from the ratio (hx / h0) of the height (hx) of the portion of the variable portion 51B corresponding to Lx in the image provided when the height (Lx) of the variable portion 51B reaches the upper limit value (LM) to the height (h0) of the portion of the variable portion 51B corresponding to LM in the image provided when the height (Lx) of the variable portion 51B reaches the upper limit value (LM). The height Lx can be calculated by LM × (hx / h0), where "×" indicates multiplication. In this case, the height of the entire vehicle 50A can be obtained by adding the height H1 of the ceiling outer wall of the passenger compartment 52 from the ground to the height Lx of the variable portion 51B.

[0026] However, the camera 4 does not have to be installed on the ceiling exterior wall of the vehicle interior 52, and may be an image capturing device carried by the user, such as a smartphone. The user transmits an image captured by the user, for example, from the side of the vehicle 50A, to the vehicle-mounted device 3. From the transmitted image, the vehicle-mounted device 3 determines the height (h1) of the ceiling exterior wall of the vehicle 50A from the ground in the image, and the height (hx) of the variable part 51B from the ground. The vehicle-mounted device 3 also determines the actual measured value of the height H1 of the ceiling exterior wall of the vehicle interior 52 from the ground. The vehicle-mounted device 3 can obtain the measured value of the height (Hx) of the variable part 51B from the ground surface by H1×(hx / h1).

[0027] A camera 4F is provided on the front of the vehicle 50A. The camera 4F takes pictures of the area in front of the vehicle 50A and provides the captured images to the vehicle-mounted device 3 via the in-vehicle LAN. A camera 4R is provided on the rear of the vehicle 50A. The camera 4R takes pictures of the area behind the vehicle 50A and provides the images to the vehicle-mounted device 3 via the in-vehicle LAN. Side cameras that capture images to the left and right of the vehicle are mounted on the mirrors on both sides of the vehicle 50A.

[0028] Camera 4F, camera 4R, and the two side cameras are connected to a surround view system (also called a view-around system). The surround view system provides a surround view image of the surroundings of vehicle 50A viewed from above in a plan view to vehicle-mounted device 3. Vehicle-mounted device 3 measures the vehicle width and the length of vehicle 50A in the front-to-rear direction from the surround view image of vehicle 50A.

[0029] FIG. 2(A) illustrates, of the five obstacle sensors 6, an obstacle sensor 6U provided on the ceiling outer wall of the passenger compartment 52, an obstacle sensor 6FL on the front left side of the vehicle (e.g., the left end of the bumper), and an obstacle sensor 6BL on the rear left side of the vehicle (e.g., the left end of the bumper). FIG. 2(A) omits the remaining obstacle sensor 6 on the front right side of the vehicle (e.g., the right end of the bumper) and the obstacle sensor 6 on the rear right side of the vehicle (e.g., the right end of the bumper). The five obstacle sensors 6 detect the distance to obstacles above, in front of, on the left front side, on the right front side, on the left rear side, and on the right rear side of the vehicle 50A. The obstacle sensors 6 may be ultrasonic sensors known as clearance sonar, or optical sensors such as millimeter-wave radar.

[0030] FIG. 2(B) illustrates the structure of vehicle 50B. Vehicle 50B is a passenger car equipped with a cargo bed 51 on the outer ceiling wall. Like vehicle 50A, vehicle 50B is equipped with cameras 4, 4F, and 4R, two side cameras, and obstacle sensors 6 (6U, 6FL, and 6BL in FIG. 2). The wall of box-shaped cargo bed 51 of vehicle 50B also has fixed portion 51A and variable portion 51B. The user can change the load capacity of vehicle 50A by adjusting the height of variable portion 51B between 0 and an upper limit (LM). However, variable portion 51B does not have to be a wall of cargo bed 51, and may simply be the cargo itself, secured with a sheet, rope, or the like.

[0031] The width of the loading platform 51 may be adjustable by moving the left and right walls of the platform 51 toward the outside or inside of the vehicle 50B relative to the traveling direction of the vehicle 50B. The platform 51 may also be extendable toward the rear of the vehicle 50B. The camera 4 is installed, for example, on the outer wall of the ceiling of the hood, captures images of the rear of the vehicle 50B in the opposite direction to the traveling direction of the vehicle 50B, and provides the captured images to the vehicle-mounted device 3. The positions of the cameras 4F and 4R in the vehicle 50B are the same as those in the vehicle 50A, and capture the front and rear of the vehicle 50B, respectively. The vehicle 50B also has side cameras similar to those in the vehicle 50A. The vehicle-mounted device 3 measures the height (Lx) of the variable section 51B from the provided images. The vehicle-mounted device 3 calculates the height (Lx) of the variable section 51B as follows. Here, the height (Lx) of the variable section 51B refers to the portion vertically above the hood of the vehicle 50B (height 0).

[0032] When the height (Lx) of the variable portion 51B is measured, for example, the visual axis of the camera 4 is parallel to the horizontal direction, and the angle of view is fixed. The vehicle-mounted device 3 stores an image when the height of the variable portion 51B reaches the upper limit value (LM) within the angle of view of the camera 4. The vehicle-mounted device 3 also stores the actual measured value of the upper limit value (LM). The vehicle-mounted device 3 calculates the measured value of the height (Lx) of the variable portion 51B from the height (hx) of the variable portion 51B in the image provided when the height of the variable portion 51B is measured, and the height (h0) of the variable portion 51B in the image when the height Lx of the variable portion 51B reaches the upper limit value (LM). The height Lx can be calculated by LM × (hx / h0). In this case, the overall height of the vehicle 50A can be calculated by adding the height H2 of the hood to the height (Lx) of the variable portion 51B.

[0033] However, camera 4 does not have to be installed on the outer wall of the ceiling of the hood, and may be a smartphone carried by the user. From the image transmitted from the smartphone, vehicle-mounted device 3 determines the height of vehicle 50B from the ground in the same manner as vehicle 50A. Also, the processing of the surround view image based on images captured by cameras 4F and 4R, the two side cameras, in FIG. 5(B) is the same as in FIG. 5(A). Also, obstacle sensor 6 in FIG. 5(B) is the same as in FIG. 2(A).

[0034] 3 is a hardware configuration diagram of a computer applicable to the server 1, the parking space DB 2, or the in-vehicle device 3. As already mentioned, the server 1, the parking space DB 2, etc. may be in a cloud environment in which computers such as those shown in FIG. 3 cooperate on a network N1 to virtually provide resources. The smartphone carried by the user also has a similar configuration to that shown in FIG. 3. However, the smartphone of this embodiment is equipped with a photographing device equivalent to the camera 4.

[0035] The computer of this embodiment has a CPU 11, a main memory device 12, and external devices, and executes information processing and communication processing using a computer program. The CPU 11 is also called a processor. The CPU 11 is not limited to a single processor, and may have a multi-processor configuration. The CPU 11 may also include a graphics processing unit (GPU), a digital signal processor (DSP), etc.

[0036] The CPU 11 executes a computer program that has been deployed in an executable manner in the main memory device 12, and provides computer processing. The main memory device 12 stores the computer program executed by the CPU 11, data processed by the CPU 11, etc. The CPU 11 and the main memory device 12 are collectively referred to as the control unit 10.

[0037] Examples of external devices include an external storage device 13, an output device 14, an operation device 15, and a communication device 16. The external storage device 13 is used, for example, as a storage area that supplements the main storage device 12, and stores computer programs executed by the CPU 11, data processed by the CPU 11, etc.

[0038] The output device 14 is, for example, a display device such as a liquid crystal display or an electroluminescence panel. However, the output device 14 may also include a speaker or other device for outputting sound. The operation device 15 is, for example, a touch panel with a touch sensor superimposed on a display. The communication device 16 communicates with a computer such as a network N1 (see FIG. 1).

[0039] (Example of output using XR) 4 shows a display example when the server 1 and the in-vehicle device 3 cooperate with each other, or when the in-vehicle device 3 independently performs driving assistance by XR display on the output device 14 of the in-vehicle device 3. In the display example of FIG. 4, a graphics object marked with a "P" indicating a parking space and a summary explanation are displayed on the image ahead of the vehicle 50 captured by the camera 4F on the front of the vehicle 50.

[0040] The summary description includes information such as parking size OK, length: S1, width: W1, height: H1, and characteristics: compatible. Here, parking size OK exemplifies that the displayed parking space has a compartment size that allows a vehicle 50 of the current size to be parked. The length, width, and height exemplify the size of the largest vehicle 50 that can be parked in the parking space. Compatibility with characteristics means that the characteristics of the displayed parking space match the attributes of the user riding in the vehicle 50. This example shows that it complies with the following.

[0041] That is, the server 1 and the in-vehicle device 3 cooperate with each other, or the in-vehicle device 3 independently combines information such as images of real physical space with information such as virtual graphic objects and character strings in virtual space, and assists the user in driving to a parking space through their vision. Note that during driving assistance, the server 1 or the in-vehicle device 3 may output sound from the speaker of the output device 14.

[0042] (Example data) Fig. 5 shows an example of data stored in the parking space DB2. In the example of Fig. 5, the parking space DB2 is, for example, in a table format. However, the parking space DB2 is not limited to a table format. For example, the parking space DB2 may store data in a keyword=parameter sequence, with keywords and parameters (values) associated with each other.

[0043] In the table of FIG. 5, each row is one record. Also, each column in the table of FIG. 5 is an element in one record. However, in the example of FIG. 5, the record in the first row illustrates the title of each element. Each record has the following elements: identification information SID, position, length, width, height, characteristic information, and vacant / occupied.

[0044] The identification information SID is information that uniquely identifies each parking space in the parking space DB2. The identification information SID is assigned by the parking space DB2, for example, when the parking space information is registered in the parking space DB2. The location is information about the geographical location of each parking space, such as latitude and longitude. The length, width, and height are the size (three-dimensional dimensions) of the largest vehicle 50 that can be parked in the parking space. The information on the location, length, width, and height is set in the parking space DB2 by the manager, owner, etc. of the parking space when the parking space information is registered in the parking space DB2. However, among these pieces of information, the information on the length, width, and height may be updated by information from the in-vehicle device 3 of the vehicle 50 that uses the parking space (see FIG. 7).

[0045] The characteristic information is information that exemplifies the characteristics of a parking space. For example, the distance (M1m) from each parking space to the exit of the parking lot that includes that parking space is stored. The surrounding environment (high, medium, low, etc.) exemplifies the quality of the environmental conditions around the parking lot that includes the parking space. The brightness of the passage (high, medium, low, etc.) exemplifies the brightness of the lighting in the passage to the exit of the parking lot that includes the parking space. The fee exemplifies the parking fee per unit time (Y yen / H), for example. The characteristic information is set in the parking space DB2 by the manager, owner, etc. of the parking space when the parking space information is registered in the parking space DB2.

[0046] FIG. 6 is a diagram illustrating a correspondence table that defines user attributes and priorities when selecting a parking space, which are stored in the main storage device 12 or the external storage device 13 by the server 1 or the in-vehicle device 3. The correspondence table is data in tabular format. Each row in the table is one record. The correspondence table stores two elements (each column) in correspondence with each other within one record. The information in FIG. 6 is created, for example, based on a questionnaire from users who have used a parking space.

[0047] The elements in the first column store the user's attributes. The elements in the second column store the characteristics of parking spaces that are prioritized when selecting a parking space according to the user's attributes. For example, if the user's attributes include "infant," "child," and "elderly," priority is given to parking spaces that are close to the exit of the parking lot that contains that parking space, i.e., parking spaces that have the characteristic of requiring a short walking distance within the parking lot after the user gets out of the car. For "elderly," priority is also given to the brightness of the walking path. Note that "infant," "child," "elderly," etc. may also be defined and categorized by age.

[0048] If the user's attribute is "luxury-oriented," the luxury of the facility and area including the parking space is prioritized when selecting a parking space. If the user's attribute is "price-first," a parking space with a low parking fee is prioritized. If the user's attribute is "not good at driving," the relative size of the parking space to the vehicle 50 and the relative size of the access road from the current position of the vehicle 50 to the parking space to the vehicle 50 are prioritized. The relative size to the vehicle 50 is the difference between the size of the vehicle 50 and the size of the largest vehicle 50 that can be parked. For example, if the vehicle width is 1.8 m and the narrowest road width of the access road is 2.5 m, the relative size is determined to be 0.7. The relative size of the access road to the vehicle 50 is the difference between the vehicle width and the width of the access road.

[0049] (Processing example) FIG. 7 is a flowchart illustrating the processing of the server 1 or the vehicle-mounted device 3. The server 1 may instruct the vehicle-mounted device 3 to execute each step (S1 to S10) of FIG. 7 one by one to execute the processing of FIG. 7. In addition, the server 1 may execute the processing of FIG. 7 according to the Hypertext Transfer Protocol (HTTP). 7 may be executed via a browser program on the vehicle-mounted device 3 through communication via the network N1. Alternatively, the vehicle-mounted device 3 may execute the process of FIG. 7 independently. Furthermore, the vehicle-mounted device 3 may communicate with the server 1 via the network N1, receive instructions for at least a part of the process of FIG. 7 from the server 1, and execute the process of FIG. 7. However, in the following description, the vehicle-mounted device 3 will be described as executing the process of FIG. 7.

[0050] In the process of FIG. 7, the in-vehicle device 3 first accepts a destination setting from the user of the vehicle 50 via the user interface of the in-vehicle device 3 (S1). Next, the in-vehicle device 3 photographs the vehicle 50 with the camera 4 and measures the height of the vehicle 50 based on the photographed image. The in-vehicle device 3 also measures the length and width of the vehicle 50 from the surround view image (S2). The method for measuring the height, length, and width of the vehicle 50 is as described with reference to FIG. 2. The process of S2 is an example of a process for acquiring the current size of the vehicle 50. The process of S2 is also an example of acquiring images photographed by the two side cameras, cameras 4, 4F, and 4R, mounted on the vehicle 50 or images photographed by an imaging device equivalent to camera 4 carried by an occupant of the vehicle 50, and measuring the size of the vehicle based on the acquired images.

[0051] Next, the vehicle-mounted device 3 acquires user attributes, for example, from the main storage device 12 (S3). The user attributes are, for example, attributes of the owner of the vehicle 50, the driver, and their family members. The user attributes may be registered in advance by the user, for example, via a user interface of the vehicle-mounted device 3. However, if the vehicle occupants include unregistered users, the vehicle-mounted device 3 may accept attribute settings for the unregistered users in the processing of S3.

[0052] Next, the vehicle-mounted device 3 searches the parking space DB2 for an appropriate vacant parking space that is located within a predetermined range from the reference position, can accommodate the vehicle 50, and matches the user's attributes (S4). Here, the reference position includes, for example, the current position of the vehicle 50 and the destination set in S1. However, the reference position can be set or changed by, for example, the user. The current position of the vehicle 50 is determined by the Global Positioning System (GPS) provided in the vehicle-mounted device 3. The predetermined range is determined by a GPS (Global Positioning System). GPS is also called a Global Navigation Satellite System (GNSS). Here, the predetermined range may be specified by the user, for example.

[0053] That is, the vehicle-mounted device 3 accesses the parking space DB2 via the server 1 and searches for currently available parking spaces that are located within a predetermined range from the reference position and have a size that can accommodate the vehicle 50. The process of S4 searches for parking spaces that are suitable for the current size of the vehicle 50 within a predetermined range from the reference position that includes at least one of the current position of the vehicle 50 and the destination of the vehicle 50. The process in S4 is also an example of searching for data in the parking space DB2 based on the measured size of the vehicle.

[0054] When multiple parking spaces are searched, the vehicle-mounted device 3 prioritizes and selects a parking space with characteristics that match the user's attributes. The parking space with characteristics that match the user's attributes may be selected from the correspondence table illustrated in FIG. 6, for example. If a parking space has characteristics that match the current user's attributes in the correspondence table illustrated in FIG. 6, for example, the vehicle-mounted device 3 may add 1 point to the matching characteristics. Furthermore, with regard to the priority criteria when selecting a parking space, such as "proximity to the exit," "brightness of the aisle," and "luxury," the vehicle-mounted device 3 may, for example, add 1 point to "top" in a ranking such as top, middle, and bottom, add 0 point to "middle," and add -1 point to "bottom." Furthermore, for "price priority," the in-vehicle device 3 may classify the searched parking spaces into, for example, three groups based on the fee per unit time, and assign a bonus of 1 to the group with the lowest fee, 0 to the middle group, and -1 to the group with the highest fee. For "relative size of the parking space to the vehicle," the in-vehicle device 3 may classify the searched parking spaces into, for example, three groups based on the difference between the width and the vehicle width of the parking space, and assign bonus points in the same way as for "price priority." However, the in-vehicle device 3 may have a threshold for the difference and exclude parking spaces where the difference between the width and the vehicle width of the parking space does not reach the threshold. The in-vehicle device 3 may then select the parking space with the highest total score. The server 1 may perform all of the processing in S4, or the in-vehicle device 3 may directly access the parking space DB2 and perform the processing in S4 without going through the server 1.

[0055] In the process of S4, the vehicle-mounted device 3 (or the server 1) may determine the parking space to which the user should actually be guided by comparing the information on multiple parking spaces (sizes of the vehicle 50 that can be parked) acquired from the parking space DB2 with the size of the vehicle itself. For example, the vehicle-mounted device 3 etc. may first acquire information on multiple parking spaces located within a predetermined range from the reference position. Then, from the acquired information, the vehicle-mounted device 3 etc. may determine a parking space that can accommodate the vehicle 50 and has characteristics that match the attributes of the user.

[0056] Next, the vehicle-mounted device 3 performs driving assistance using XR display (or augmented reality display) based on the search results (S5). The vehicle-mounted device 3 displays an image, such as that shown in FIG. 4, or outputs sound or audio on the output device 14. This XR display (or augmented reality display) can be said to be an example of a display that extends the real physical space by combining information about the real physical space around the vehicle 50 with information about the virtual space.

[0057] The vehicle-mounted device 3 then determines whether the vehicle 50 has arrived at the destination (S6). If the vehicle 50 has not arrived at the destination, the vehicle-mounted device 3 repeats the process of S5. On the other hand, when the vehicle 50 arrives at the destination, it checks whether parking is possible (S7). For example, the vehicle 50 may measure the size of the parking space in front of the vehicle using the front camera 4F or a surround view image. The vehicle-mounted device 3 then determines whether the vehicle 50 can be parked in the parking space (S8).

[0058] If the vehicle 50 can be parked in the parking space, the in-vehicle device 3 continues driving assistance, completes parking, and measures the size of the parking space (S9). For example, the in-vehicle device 3 detects lines or the like that separate the parking space from the surround view image. Then, the in-vehicle device 3 measures the distance (Dx) from the vehicle to the line or the like by multiplying the actual vehicle width, length, or length (D0) of the vehicle in the surround view image by the ratio (dx / d0) of the distance (dx) from the vehicle to the line or the like to the length (d0) of the vehicle's width, length, or other specific part of the vehicle. However, if the parking space is surrounded by walls, such as a multi-story parking garage, the in-vehicle device 3 may measure the distance (Dx) from the vehicle to the line or the like by multiplying the actual vehicle width, length, or other specific part of the vehicle (D0) by the ratio (Dx=D0×(dx / d0)). All you have to do is measure the distance from the image to the surrounding walls.

[0059] The vehicle-mounted device 3 may also measure the distance to surrounding walls, surrounding vehicles, etc. using four obstacle sensors 6. The vehicle-mounted device 3 also measures the distance to the ceiling of the space above the parking space using the obstacle sensor 6U on the exterior ceiling wall. If the parking space is flat and there are no walls separating the parking space, the vehicle-mounted device 3 determines that an adjacent vehicle is parked in an adjacent parking space if the distance (DDx) to the surrounding vehicle, etc. is shorter than the width or length (W0) of the vehicle, and determines half of that distance as the margin on one side of the parking space. In this case, the width (W0) of the vehicle + the distance (DDx) is the width or length of the parking space in that direction. On the other hand, if the distance to the surrounding vehicle, etc. is longer than the width or length (W0) of the vehicle, the vehicle-mounted device 3 may assume that another vehicle is parked in a parking space with, for example, n spaces available, and temporarily place a vehicle of the same size as the vehicle in the available parking space to calculate the distance. For example, the measured distance to surrounding vehicles, etc., can be calculated by finding an integer n such that the width (W0) of the vehicle itself x n + the inter-vehicle distance (DDx) x (n+1), and the inter-vehicle distance (DDx) that is shorter than the width or length (W0) of the vehicle itself. The in-vehicle device 3 can then determine that half of the inter-vehicle distance (DDx) is the margin on one side of the parking space. The in-vehicle device 3 then reports the measured parking space data (actually measured three-dimensional dimensions) together with the SID to the parking space DB2 via the server 1 (S10).

[0060] On the other hand, if the determination in S8 indicates that parking is not possible, the in-vehicle device 3 may report to the parking space DB2 that the vehicle 50 of its current size cannot be parked in the parking space (identification information SID). The server 1 may execute the processes of S10 and S11 based on the measurement results or determination results of the in-vehicle device 3. The in-vehicle device 3 may also execute the processes of S10 and S11 without going through the server 1. The process of S10 is an example of a process in which the three-dimensional dimensions of the parking space in which the vehicle 50 has parked are actually measured and the measured three-dimensional dimensions are reported to the parking space DB2. The parking space DB2 updates the size of the vehicle 50 that can be parked in that parking space based on the reported three-dimensional dimensions of the parking space with the identification information SID.

[0061] (Effects of the embodiment) As described above, the control unit 10 of the in-vehicle device 3 or the server 1 acquires, from the parking space DB2, information on parking spaces that fit the current size of the vehicle 50 within a predetermined range from a reference position that includes at least one of the current position of the vehicle 50 and the destination of the vehicle 50. Then, based on the acquired information, the control unit 10 performs XR display (or augmented reality display) that expands the real physical space by combining information on the real physical space around the vehicle 50 with information on the virtual space. Then, the control unit 10 performs driving assistance to the parking space using the XR display or the like. Therefore, the in-vehicle device 3 or the server 1 can guide the user to a suitable parking space even for a type of vehicle 50 whose size, such as height, dynamically changes depending on the usage situation (e.g., a vehicle with a cargo bed).

[0062] The parking space DB2 also stores the geographical location of the parking space and the maximum vehicle size that can be parked for each parking space. The control unit 10 of the in-vehicle device 3 or the server 1 acquires images taken by a camera 4 or the like mounted on the vehicle 50 or by an image capturing device equivalent to the camera 4 carried by an occupant of the vehicle 50, and measures the size of the vehicle based on the acquired images. The control unit 10 then searches for data in the parking space DB2 based on the measured size of the vehicle. Therefore, the in-vehicle device 3 or the server 1 can acquire information on available parking spaces in real time for a vehicle 50 whose size dynamically changes depending on usage. The in-vehicle device 3 or the server 1 can then assist the user's driving and guide the user to a parking space based on the acquired information.

[0063] The parking space DB2 can be accessed from the vehicle-mounted device 3 or the server 1 through the network N1. The control unit 10 then generates a three-dimensional map of the parking space where the vehicle 50 is parked. The dimensions are actually measured, and the measured three-dimensional dimensions are reported to the parking space DB 2. Therefore, the vehicle-mounted device 3 or the server 1 can update the information in the parking space DB 2 to the latest state based on the actually measured data.

[0064] <Other embodiments> The above-described embodiment is merely an example, and the present disclosure may be modified and implemented as appropriate without departing from the spirit thereof. Furthermore, the processes and means described in the present disclosure may be freely combined and implemented as long as no technical contradictions arise. Furthermore, a process described as being performed by one device may be shared and executed by multiple devices. Alternatively, a process described as being performed by different devices may be executed by a single device. In a computer system, the hardware configuration (server configuration) by which each function is realized can be flexibly changed.

[0065] The present disclosure can also be realized by supplying a computer program implementing the functions described in the above embodiments to a computer, and having one or more processors in the computer read and execute the program. Such a computer program may be provided to the computer via a non-transitory computer-readable storage medium connectable to the computer's system bus, or via a network N1. Examples of non-transitory computer-readable storage media include any type of disk, such as a magnetic disk, a hard disk drive (HDD), an optical disk (CD-ROM, DVD, Blu-ray, etc.), a read-only memory (ROM), a random access memory (RAM), an EPROM, an EEPROM, a magnetic card, a flash memory, or an optical card. [Explanation of symbols]

[0066] 1 server 2 Parking Space DB 3 Onboard equipment 4, 4F, 4R Camera

Claims

1. Obtaining information on parking spaces that fit the current size of the vehicle within a predetermined range from a reference position that includes at least one of the current position of the vehicle and the destination of the vehicle from a parking lot database; An information processing device having a control unit that, based on the acquired information, performs driving assistance to the parking space by displaying an expanded real physical space that combines information about the real physical space around the vehicle with information about the virtual space.

2. The parking lot database stores the geographical location of the parking space and the size of the vehicle that can be parked for each parking space; the control unit acquires an image taken by a camera mounted on the vehicle or an image taken by a camera carried by an occupant of the vehicle, and measures the size of the vehicle based on the acquired image; The information processing device according to claim 1, wherein data in the parking lot database is searched for based on the measured size of the vehicle.

3. the parking lot database is accessible from the information processing device via a network; The information processing device according to claim 1 , wherein the control unit actually measures three-dimensional dimensions of the parking space in which the vehicle is parked, and reports the measured three-dimensional dimensions to the parking space database.

4. The computer acquires, from a parking lot database, information on parking spaces that fit the current size of the vehicle within a predetermined range from a reference position that includes at least one of the current position of the vehicle and the destination of the vehicle; An information processing method that provides driving assistance to the parking space by displaying an expanded real physical space that combines information about the real physical space around the vehicle with information about the virtual space based on the acquired information.

5. The computer acquires, from a parking space database, information on parking spaces that are suitable for the current size of the vehicle within a predetermined range from a reference position that includes at least one of the current position of the vehicle and the destination of the vehicle; Based on the acquired information, the program assists driving to the parking space by displaying an expanded version of the real physical space around the vehicle that combines information on the real physical space around the vehicle with information on the virtual space.

Images