Carriage simulation method

Abstract

The method performed by the terminal device includes: accepting, via an input portion, an input by a user of an environmental condition in which the user studies a purchased vehicle and places the vehicle; transmitting, via the communication unit, first information indicating the vehicle and environmental conditions; transmitting, via the communication unit, second information indicating an object placed by the user in the compartment of the vehicle; acquiring, via the communication unit, an image indicating an object placed in the vehicle under the environmental conditions, the image being generated on the basis of the first information and the second information; and outputting the image via an output section.

Description

Technical Field

[0001] The present disclosure relates to a method. Background Art

[0002] Conventionally, there has been a technology for providing a simulation of a vehicle cabin to users researching the purchase of a vehicle. For example, Japanese Unexamined Patent Application Publication No. 2017-182560 discloses a vehicle interior simulation device that can display a three-dimensional image of the entire interior by sequentially combining selection images of the vehicle interior and confirm it step by step to obtain a final interior schematic diagram. Summary of the Invention

[0003] Sometimes, although a user wants to know the situation of personal belongings placed in the vehicle they intend to purchase, it is difficult to confirm this in an actual vehicle. As such, there is room for improvement in the technology for providing a simulation of a vehicle cabin to users researching the purchase of a vehicle.

[0004] The method according to the present disclosure is a method executed by a terminal device including an input unit, an output unit, a communication unit, and a control unit, wherein the processing performed by the control unit includes: receiving, via the input unit, an input by the user of the vehicle the user is researching the purchase of and the environmental conditions in which the vehicle is placed; sending, via the communication unit, first information indicating the vehicle and the environmental conditions; sending, via the communication unit, second information indicating an object the user has placed in the vehicle cabin; acquiring, via the communication unit, an image generated based on the first information and the second information and indicating the object placed in the vehicle under the environmental conditions; and outputting, via the output unit, the image.

[0005] According to the present disclosure, it is possible to provide a technology for improving the simulation of a vehicle cabin. Brief Description of the Drawings

[0006] The features, advantages, and technical and industrial significance of exemplary embodiments of the present invention will be described below with reference to the accompanying drawings, in which like reference numerals denote like elements, and in the drawings: Figure 1 is a diagram showing a schematic configuration of a system according to a first embodiment; Figure 2 is a flowchart showing the operation of a system according to a first embodiment; Figure 3 is a diagram for explaining an image output by a terminal device according to a first embodiment; Figure 4Ais a flowchart showing the operation of the system according to the second embodiment; Figure 4B is a flowchart showing the operation of the system according to the second embodiment. Detailed implementation mode First embodiment

[0007] Hereinafter, the first embodiment of the present disclosure will be described with reference to the drawings. In each drawing, the same or corresponding parts are denoted by the same reference numerals. In the description of the present embodiment, for the same or corresponding parts, the description will be appropriately omitted or simplified.

[0008] Refer to Figure 1 , the configuration of the system 1 according to the present embodiment will be described. The system 1 includes an information processing device 10 and a terminal device 20. The information processing device 10 can communicate with the terminal device 20 via a network 30.

[0009] The information processing device 10 is a computer installed in a facility such as a data center. The information processing device 10 is, for example, a server belonging to a cloud computing system or other computing systems. The information processing device 10 is operated by a vehicle dealer.

[0010] The terminal device 20 is held by a user. The terminal device 20 is, for example, a mobile device such as a mobile phone, a smartphone, or a tablet computer, or a PC. "PC" is an abbreviation for personal computer.

[0011] The network 30 includes the Internet, at least one WAN, at least one MAN, or any combination thereof. "WAN" is an abbreviation for wide area network. "MAN" is an abbreviation for metropolitan area network. The network 30 may also include at least one wireless network, at least one optical network, or any combination thereof. The wireless network is, for example, an ad hoc network, a cellular network, a wireless LAN, a satellite communication network, or a terrestrial microwave network. "LAN" is an abbreviation for local area network.

[0012] First, an overview of the present embodiment will be described, and the details will be described later. The terminal device 20 receives the input of the vehicle that the user intends to purchase and the environmental conditions for placing the vehicle by the user, sends the first information indicating the vehicle and the environmental conditions, and sends the second information indicating the object placed by the user in the vehicle compartment. The terminal device 20 obtains an image representing the object placed in the vehicle under the environmental conditions generated based on the first information and the second information, and outputs the image.

[0013] The environmental conditions represent the environment in which the vehicle compartment is placed, such as weather, time period, and the degree of brightness brought by the lighting fixtures in the vehicle compartment. As described below, it can be input by the user. According to this embodiment, it is possible to easily confirm together the vehicle compartment of the vehicle that the user is researching and purchasing, and the object placed in the compartment, under the specified environmental conditions. Thus, it is easy for the user to imagine the vehicle compartment when they are riding. Therefore, the technology of providing a simulation of the vehicle compartment can be improved.

[0014] Refer to Figure 1 , and describe the configuration of the information processing device 10 according to this embodiment. The information processing device 10 includes a control unit 11, a storage unit 12, and a communication unit 13.

[0015] The control unit 11 includes at least one processor, at least one programmable circuit, at least one dedicated circuit, or any combination thereof. The processor is a general-purpose processor such as a CPU or GPU, or a dedicated processor dedicated to specific processing. "CPU" is an abbreviation for central processing unit. "GPU" is an abbreviation for graphics processing unit. The programmable circuit is, for example, an FPGA. "FPGA" is an abbreviation for field-programmable gate array. The dedicated circuit is, for example, an ASIC. "ASIC" is an abbreviation for application specific integrated circuit. The control unit 11 controls each part of the information processing device 10 while performing processing related to the operation of the information processing device 10.

[0016] The storage unit 12 includes at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or any combination thereof. The semiconductor memory is, for example, a RAM, a ROM, or a flash memory. "RAM" is an abbreviation for random access memory. "ROM" is an abbreviation for read only memory. The RAM is, for example, an SRAM or a DRAM. "SRAM" is an abbreviation for static random access memory. "DRAM" is an abbreviation for dynamic random access memory. The ROM is, for example, an EEPROM. "EEPROM" is an abbreviation for electrically erasable programmable read only memory. The flash memory is, for example, an SSD. "SSD" is an abbreviation for solid-state drive. The magnetic memory is, for example, an HDD. "HDD" is an abbreviation for hard disk drive. The storage unit 12 functions as, for example, a main storage device, an auxiliary storage device, or a cache memory. In the storage unit 12, information used in the operation of the information processing device 10 and information obtained through the operation of the information processing device 10 are stored.

[0017] The communication unit 13 includes at least one communication module. The communication module is, for example, a module that supports a wired LAN communication standard such as Ethernet (registered trademark) or a wireless LAN communication standard such as IEEE 802.11. "IEEE" is the abbreviation of Institute of Electrical and Electronics Engineers. The communication unit 13 communicates with devices other than the information processing device 10. The communication unit 13 receives information used in the operation of the information processing device 10 and also transmits information obtained through the operation of the information processing device 10.

[0018] Refer to Figure 1 , the configuration of the terminal device 20 according to the present embodiment will be described. The terminal device 20 includes a control unit 21, a storage unit 22, a communication unit 23, an input unit 24, an output unit 25, and a photographing unit 26. The hardware configurations of the control unit 21, the storage unit 22, and the communication unit 23 of the terminal device 20 are the same as the hardware configurations of the control unit 11, the storage unit 12, and the communication unit 13 of the information processing device 10, respectively. The description thereof will be omitted here.

[0019] The input unit 24 includes at least one input interface. The input interface is, for example, a physical key, a capacitive key, a pointing device, a touch screen provided integrally with the display, or a microphone. The input unit 24 accepts operations for inputting data used in the operation of the terminal device 20. The input unit 24 may also be connected to the terminal device 20 as an external input device instead of being provided in the terminal device 20. As a connection method, for example, any method such as USB, HDMI (registered trademark), or Bluetooth (registered trademark) can be used. "USB" is an abbreviation for Universal Serial Bus. "HDMI (registered trademark)" is an abbreviation for High-Definition Multimedia Interface.

[0020] The output unit 25 includes at least one output interface. The output interface is, for example, a display or a speaker. The display is, for example, an LCD or an organic EL display. "LCD" is an abbreviation for liquid crystal display. "EL" is an abbreviation for electro luminescence. The output unit 25 outputs data obtained through the operation of the terminal device 20. The output unit 25 may also be connected to the terminal device 20 as an external output device instead of being provided in the terminal device 20. As a connection method, for example, any method such as USB, HDMI (registered trademark), or Bluetooth (registered trademark) can be used.

[0021] The imaging unit 26 is a camera that images the surroundings of the terminal device 20. The imaging unit 26 may be a TOF (Time Of Flight) camera, a stereo camera, or the like that can image the subject three-dimensionally. The imaging unit 26 can output the acquired image to the control unit 21. The imaging unit 26 is provided, for example, on the surface opposite to the output unit 25.

[0022] The functions of the information processing device 10 or the terminal device 20 are implemented by executing the program according to this embodiment in a processor serving as the control unit 11 or the control unit 21. That is, the functions of the information processing device 10 or the terminal device 20 are implemented by software. The program causes the computer to execute the operations of the information processing device 10 or the terminal device 20, thereby causing the computer to function as the information processing device 10 or the terminal device 20. That is, the computer functions as the information processing device 10 or the terminal device 20 by executing the operations of the information processing device 10 or the terminal device 20 according to the program.

[0023] The program can be pre-stored in a non-transitory computer-readable medium. The non-transitory computer-readable medium is, for example, a flash memory, a magnetic recording device, an optical disc, a magneto-optical recording medium, or a ROM. The distribution of the program is carried out, for example, by selling, transferring, or leasing a removable medium such as an SD card, a DVD, or a CD-ROM storing the program. "SD" is an abbreviation for Secure Digital. "DVD" is an abbreviation for digital versatile disc. "CD-ROM" is an abbreviation for compact disc read only memory. The program can also be distributed by saving the program in a register of a server and transferring the program from the server to other computers. The program can also be provided as a program product.

[0024] The computer, for example, temporarily saves the program stored in the removable medium or the program transferred from the server in the main storage device. Then, the computer reads the program saved in the main storage device through the processor and executes processing according to the read program through the processor. The computer can also directly read the program from the removable medium and execute processing according to the program. The computer can execute processing successively according to the received program each time the program is transferred from the server to the computer. It is also possible to execute processing by using a so-called ASP-type service that realizes functions only through execution instructions and result acquisition without transferring the program from the server to the computer. "ASP" is an abbreviation for application service provider. The program includes a quasi-program that is information used for processing by an electronic computer. For example, data that is not a direct instruction for a computer but has the nature of specifying the processing of the computer belongs to the "quasi-program".

[0025] Part or all of the functions of the information processing device 10 or the terminal device 20 can also be realized by a programmable circuit or a dedicated circuit as the control unit 11 or the control unit 21. That is, part or all of the functions of the information processing device 10 or the terminal device 20 can also be realized by hardware.

[0026] Refer to Figure 2 and Figure 3 , and describe the operation of the system 1 according to this embodiment. Figure 2 Among the operations shown, the operation of the terminal device 20 corresponds to the method according to this embodiment. Hereinafter, it is assumed that the communication between the information processing device 10 and an external device is carried out via the communication unit 13 and the network 30, and the communication between the terminal device 20 and an external device is carried out via the communication unit 23 and the network 30.

[0027] In Figure 2In S101, the control unit 11 of the information processing device 10 sends setting information representing at least one sales vehicle and environmental options for placing the vehicle to the terminal device 20. The setting information can be pre-stored in the storage unit 12 of the information processing device 10. The environmental options include weather, time period, and the degree of brightness brought by the lighting fixtures in the vehicle compartment. In this example, the environmental options for weather include "rain", "sunny", "cloudy", etc. The environmental options for the time period include "morning", "noon", "evening", "night", etc. The environmental options for the degree of brightness brought by the lighting fixtures in the vehicle compartment include "high", "medium", "low", etc. Not limited thereto, the environmental options can also include temperature, humidity, etc. The setting information can include interior options for multiple compartments of the same vehicle. The interior options can include, for example, the materials, colors, etc. of seats, instrument panels, etc.

[0028] In S102, the control unit 21 of the terminal device 20 obtains the setting information by receiving it.

[0029] In S103, the control unit 21 outputs at least one vehicle and environmental options represented by the setting information via the output unit 25. The output can be performed by any method. For example, the control unit 21 can output at least one vehicle and environmental options by displaying them as icons, etc. on the output unit 25. In this example, the control unit 21 outputs vehicles CA, CB, and CC, and the three weather conditions of "rain", "sunny", and "cloudy" as environmental options via the output unit 15. In addition, the control unit 21 outputs the three time periods within a day of "morning", "noon", and "night", and the three degrees of brightness brought by the lighting fixtures in the vehicle compartment of "low", "medium", and "high" via the output unit 15. When the setting information includes interior options for the vehicle compartment, the control unit 21 outputs the interior options.

[0030] In S104, the control unit 21 of the terminal device 20 receives the input of the vehicle that the user is considering purchasing and the environmental conditions for placing the vehicle via the input unit 24. The input can be received by any method. For example, the control unit 11 can receive the operation of the user selecting the icons output in S103 as the input via the input unit 24. In this example, it is assumed that the control unit 21 receives the input of vehicle CA as the vehicle that the user is considering purchasing. In this example, it is assumed that the control unit 11 receives the input of the weather of "sunny", the time period of "night", and the degree of brightness brought by the lighting fixtures in the vehicle compartment of "medium" as the environmental conditions. When the setting information includes interior options for the vehicle compartment, the control unit 21 can also receive the input of the user for the interior options.

[0031] In S105, the control unit 21 generates first information indicating the vehicle and environmental conditions for which input was received in S104, and sends the first information to the information processing device 10. The first information may also include the interior decoration options of the vehicle cabin input by the user.

[0032] In S106, the control unit 11 of the information processing device 10 obtains the first information by receiving it.

[0033] In S107, the control unit 21 of the terminal device 20 generates and sends second information indicating the object placed by the user in the vehicle cabin. The second information can be generated by any method. For example, the control unit 21 can obtain a two-dimensional captured image of the object captured by the user using the imaging unit 26 from the imaging unit 26, and generate second information including the captured image. In this case, the control unit 21 can also obtain a plurality of two-dimensional captured images of the object captured from multiple directions to generate the second information. Without being limited thereto, the control unit 21 can obtain a three-dimensional captured image from the imaging unit 26 and generate second information including the three-dimensional captured image. In this example, it is assumed that the control unit 11 generates second information including a three-dimensional captured image of the bag B and the tissue box K as the object.

[0034] In S108, the control unit 11 of the information processing device 10 obtains the second information by receiving it.

[0035] In S109, the control unit 11 obtains first three-dimensional data of the vehicle cabin under the environmental conditions indicated by the first information. The three-dimensional data includes, for example, a three-dimensional model, point cloud data, mesh data, etc. The first three-dimensional data can be obtained by any method. For example, the control unit 11 refers to the information indicating each vehicle cabin stored in advance in the storage unit 12, and obtains the three-dimensional data of the vehicle indicated by the first information. The control unit 11 can also use any image processing technique to apply the environmental conditions indicated by the first information to the three-dimensional data, and obtain the first three-dimensional data with adjusted light irradiation mode, etc. generated. When the first information includes the interior decoration options of the vehicle cabin, the control unit 21 obtains the first three-dimensional data of the cabin provided with the interior decoration options under the environmental conditions.

[0036] In S110, the control unit 11 acquires second three-dimensional data representing the object indicated by the second information. The acquisition of the second three-dimensional data can be performed by any method. For example, the control unit 11 can also generate, based on a plurality of two-dimensional captured images representing the object captured from a plurality of directions, second three-dimensional data representing the object in a three-dimensional shape using any image processing technique and acquire it. The control unit 11 can also extract three-dimensional data from the three-dimensional captured image indicated by the second information and acquire it. Without being limited thereto, the control unit 11 can also read out the three-dimensional data of the object from a database of three-dimensional data stored in advance in the storage unit 12 and acquire it. In this example, it is assumed that the control unit 11 extracts three-dimensional data from the three-dimensional captured images of the package B and the tissue box K indicated by the second information and acquires it.

[0037] In S111, the control unit 11 generates an image obtained by synthesizing the first three-dimensional data and the second three-dimensional data. In this example, the synthesized image is a three-dimensional image of the vehicle cabin under the environmental conditions indicated by the first information in which the object indicated by the second information is placed. Without being limited thereto, as the synthesized image, the control unit 11 can also generate a two-dimensional image. The generation of the synthesized image can be performed by any method. For example, the control unit 11 can generate an image of the object placed at a preset position such as on the seat in the vehicle cabin. This position can also be preset by the user. The control unit 11 can apply the environmental conditions indicated by the first information to the object using any image processing technique and generate an image of the object in which the light irradiation mode and the like are adjusted.

[0038] In S112, the control unit 11 sends the generated image to the terminal device 20.

[0039] In S113, the control unit 21 of the terminal device 20 receives the image from the information processing device 10.

[0040] In S114, the control unit 21 outputs the received image via the output unit 25. Figure 3 FIG. is an example of an image output by the output unit 25 of the terminal device 20. In Figure 3 , the cabin R of the vehicle CA is shown under the environmental conditions indicated by the first information where the weather is "sunny", the time period is "night", and the degree of cabin lighting is "medium". In the cabin R, the package B and the tissue box K as the objects indicated by the second information are placed. By viewing the three-dimensional image output by the output unit 25, the user can imaginarily experience the vehicle cabin with a sense of presence. After that, the operation of the system 1 ends. Second Embodiment

[0041] Next, a second embodiment of the present disclosure will be described. The configuration of the system 1 according to the second embodiment is the same as that of the above-described first embodiment, so the description thereof will be omitted. The configurations of the information processing apparatus 10 and the terminal apparatus 20 according to the second embodiment are the same as those of the above-described embodiments, so the description thereof will be omitted.

[0042] Hereinafter, with reference to Figure 4A and Figure 4B the operation of the system 1 according to this embodiment will be described.

[0043] Figure 4A S201 to S206 of Figure 2 are the same as S101 to S106 of the above-described first embodiment, so the description thereof will be omitted.

[0044] In S207, the control unit 21 receives an input of the usage of the user's vehicle via the input unit 24. The usage may include, for example, "commuting", "going to school", "fishing", "golf", "daily use", etc. For example, the control unit 21 may also display a plurality of icons indicating the usage on the output unit 25 and receive an operation by which the user designates at least one of the icons via the input unit 24. In this example, it is assumed that the control unit 21 has received an input designating "golf" as the usage of the user's vehicle.

[0045] In S208, the control unit 21 generates information indicating the usage and sends it to the information processing apparatus 10.

[0046] In S209, the control unit 11 of the information processing apparatus 10 receives the information indicating the usage.

[0047] In S210, the control unit 11 generates information indicating at least one item corresponding to the usage represented by the information received in S209, and sends it to the terminal device 20. The at least one item corresponding to the usage means an item that is highly likely to be brought in by the user and placed in the vehicle compartment when the vehicle is used for that usage. The at least one item corresponding to the usage can be stored in advance in the storage unit 12 of the information processing device 10 in association with the usage. For example, when the usage is "daily use", the items corresponding to the usage may include "tissue box", "seat cushion", and "shopping bag", etc. When the usage is "commuting", the items corresponding to the usage may include "business bag", "PC bag", etc. When the usage is "fishing", the items corresponding to the usage may include "fishing tools", "cooler box", etc. When the usage is "golf", the items corresponding to the usage may include "golf bag", "course bag", etc. In this example, as the at least one item corresponding to the usage, the control unit 11 generates information indicating "golf bag" and "course bag", and sends it to the terminal device 20.

[0048] In S211, the terminal device 20 outputs, via the output unit 25, at least one item corresponding to the usage represented by the information received in S210. In this example, the control unit 11 outputs "golf bag" and "course bag" via the output unit 25. The items can also be displayed by icons or the like.

[0049] In S212, the control unit 21 receives, via the input unit 24, the input of one or more items selected by the user. In this example, it is assumed that the control unit 21 receives the input of "golf bag" selected by the user from "golf bag" and "course bag".

[0050] In Figure 4B S213, the control unit 21 generates second information indicating the item input in S212 as the object, and sends it to the information processing device 10. In this example, it is assumed that the control unit 21 generates second information indicating the item "golf bag" as the object, and sends it to the information processing device 10.

[0051] In S214, the control unit 11 of the information processing device 10 obtains the second information by receiving it.

[0052] S215 is the same as S109 of the above-mentioned first embodiment Figure 2 so the description is omitted.

[0053] In S216, the control unit 11 acquires second three-dimensional data representing the object indicated by the second information. The acquisition of the second three-dimensional data can be performed by any method. For example, the control unit 11 may also acquire the three-dimensional data of the object by reading out the three-dimensional data of the object from the database of the three-dimensional data stored in advance in the storage unit 12. In this example, it is assumed that the control unit 11 acquires the three-dimensional data of the golf bag indicated by the second information.

[0054] S217 to S220 are the same as S111 to S114 according to the above-described first embodiment, and thus the description thereof is omitted. Figure 2

[0055] The present disclosure is not limited to the above-described embodiments. For example, two or more blocks described in the block diagram may be integrated, or one block may be divided. For two or more steps described in the flowchart, instead of being executed in time series according to the description, they may be executed in parallel or in a different order according to the processing capabilities of the devices that execute each step or as needed. In addition, changes can be made without departing from the gist of the present disclosure.

[0056] For example, in S114 of the above-described first embodiment or S220 of the second embodiment, the control unit 21 of the terminal device 20 may also arrange a plurality of images and output them to the output unit 25 at the same time. The plurality of images may be images generated by the information processing device 10 in S111 according to the above-described Figure 2 or Figure 4B S217 of the above-described, in which any one of the vehicle cabin, environmental conditions, object, and articles corresponding to the usage is different. Thereby, the user can comprehensively compare the images of the vehicle cabin under different conditions.

Claims

1. A method, which is performed by a terminal device having an input unit, an output unit, a communication unit, and a control unit, wherein: The processing performed by the control unit includes: receiving, via the input unit, an input from a user of a vehicle that the user is researching to purchase and an environmental condition in which the vehicle is placed; transmitting, via the communication unit, first information representing the vehicle and the environmental condition; transmitting, via the communication unit, second information indicating an object placed by the user in the compartment of the vehicle; acquiring, via the communication unit, an image generated based on the first information and the second information and showing the object placed on the vehicle under the environmental condition; and The image is output via the output unit.

2. The method according to claim 1, wherein: The image is a three-dimensional image obtained by synthesizing first three-dimensional data representing the cabin of the vehicle under the environmental condition and second three-dimensional data representing the object.

3. The method according to claim 1 or 2, wherein: The environmental conditions also include the degree of brightness provided by lighting fixtures in the vehicle cabin.

4. The method according to claim 1 or 2, wherein: The processing performed by the control unit includes: receiving, via the input unit, an input of a usage purpose of the vehicle by the user; Outputting at least one item corresponding to the usage purpose via the output unit; receiving, via the input unit, input of one or more items selected by the user; and The second information indicating that the item selected by the user is the target object is transmitted via the communication unit.

5. The method according to claim 1 or 2, wherein: The terminal device includes a photographing unit. The processing performed by the control unit includes: obtaining a captured image captured by the user through the imaging unit and showing the object; The second information includes the captured image.

Images