Support device and support method

Abstract

To improve a technique for storing baggage which a user brings into a vehicle.SOLUTION: A support device 20 includes a control section 25 for distinguishing the type of baggage which a user brings into a vehicle 10 from a taken camera image and measuring the dimension of the distinguished baggage, and presenting a predetermined baggage storage to the user when the distinguished type of the baggage is predetermined or the measured dimension of the baggage is more than a threshold.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an assistance device and an assistance method. [Background technology]

[0002] Conventionally, there are known techniques for storing luggage that a user brings into a vehicle. For example, Patent Document 1 discloses an information providing device that prevents a user and the user's luggage from forgetting to drop off at a predetermined location. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-229053 Summary of the Invention [Problem to be solved by the invention]

[0004] However, when passengers have a lot of luggage to bring onto the vehicle, they have difficulty finding a place to store it. Therefore, there is room for improvement in the technology for storing luggage that passengers bring onto the vehicle.

[0005] In view of the above circumstances, an object of the present disclosure is to improve technology relating to the storage of luggage brought into a vehicle by a user. [Means for solving the problem]

[0006] An assistance device according to one embodiment of the present disclosure is an assistance device having an input unit equipped with a camera and a control unit that acquires camera images captured by the camera, wherein the control unit determines the type of luggage that a user brings into a vehicle from the acquired camera images, measures the dimensions of the determined luggage, and if the determined type of luggage is a specified luggage or if the measured dimensions of the luggage are larger than a threshold value, presents the user with a specified luggage storage area.

[0007] An assistance method according to one embodiment of the present disclosure includes using an assistance device to determine the type of luggage a user brings into a vehicle from an acquired camera image, measure the dimensions of the determined luggage, and, if the determined type of luggage is a specified luggage or if the measured dimensions of the luggage are larger than a threshold, present the user with a specified luggage storage area. [Effects of the Invention]

[0008] According to one embodiment of the present disclosure, technology for storing luggage brought into a vehicle by a user is improved. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a block diagram illustrating a schematic configuration example of a system according to an embodiment of the present disclosure. [Figure 2] 10 is a flowchart illustrating an example of the operation of the support device. [Figure 3] FIG. 10 is a schematic diagram illustrating an example of the operation of the support device. [Figure 4] FIG. 1 is a schematic diagram showing an example of a luggage storage area in a vehicle. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present disclosure will be described.

[0011] (Outline of the embodiment) An overview of a system 1 according to an embodiment of the present disclosure will be described with reference to Fig. 1. The system 1 includes a vehicle 10, an assistance device 20, and a server 30. The vehicle 10, the assistance device 20, and the server 30 are communicably connected to a network 2 including, for example, the Internet and a mobile communication network.

[0012] The vehicle 10 is, for example, an automobile, but is not limited to this and may be any vehicle. The automobile may be, but is not limited to, a gasoline-powered vehicle, a BEV (Battery Electric Vehicle), a HEV (Hybrid Electric Vehicle), a PHEV (Plug-in Hybrid Electric Vehicle), or a FCEV (Fuel Cell Electric Vehicle). In the present disclosure, the vehicle 10 is assumed to be, but is not limited to, a public bus (on-demand bus) that operates on a predetermined route. The number of vehicles 10 included in the system 1 may be determined arbitrarily. The vehicle 10 is capable of communicating with the assistance device 20 and the server 30 via the network 2.

[0013] The support device 20 is a terminal device installed at a bus stop where the vehicle 10 stops. The support device 20 presents a storage location (luggage storage area) for luggage to be brought into the vehicle 10 to a user who boards the vehicle 10. The support device 20 is connected to the vehicle 10 and the server 30 via the network 2 so as to be able to communicate with them.

[0014] The server 30 is, for example, a computer owned by a business operator that operates the vehicle 10. The server 30 is capable of communicating with the vehicle 10 and the assistance device 20 via the network 2.

[0015] First, an outline of this embodiment will be described, and details will be described later. The assistance device 20 determines the type of luggage that the user 3 brings into the vehicle 10 from the acquired camera image, measures the dimensions of the determined luggage, and if the determined type of luggage is a predetermined luggage or if the measured dimensions of the luggage are larger than a threshold, presents a predetermined luggage storage area to the user 3.

[0016] As described above, according to this embodiment, if the type of luggage is a predetermined luggage, or if the measured dimensions of the luggage are larger than a threshold value, a predetermined luggage storage area is presented to the user 3 of the vehicle 10. This allows the user 3 to place bulky luggage in the predetermined luggage storage area, so that the luggage does not get in the way of other users, improving the comfort of the user 3 and improving the technology for storing luggage brought into the vehicle by users.

[0017] Next, each component of the system 1 will be described in detail.

[0018] (Vehicle configuration) As shown in FIG. 1, a vehicle 10 includes a communication unit 11, a storage unit 12, and a control unit 13.

[0019] The communication unit 11 includes a communication interface for wirelessly connecting to the network 2. The communication interface for connecting to the network 2 is compatible with mobile communication standards such as, but not limited to, 4G (4th Generation) or 5G (5th Generation).

[0020] The storage unit 12 includes one or more memories. The memories may be, for example, semiconductor memories, magnetic memories, or optical memories, but are not limited to these. Each memory included in the storage unit 12 may function as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 12 stores any information used in the operation of the vehicle 10. For example, the storage unit 12 may store a system program, an application program including a car navigation application, embedded software, vehicle operation information, map information, and the like. The information stored in the storage unit 12 may be updatable with information obtained from the network 2 via the communication unit 11, for example.

[0021] The control unit 13 includes one or more processors, one or more programmable circuits, one or more dedicated circuits, or a combination thereof. The processor may be, for example, a general-purpose processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), or a dedicated processor specialized for a specific process, but is not limited to these. The programmable circuit may be, for example, but is not limited to, an FPGA (Field-Programmable Gate Array). The dedicated circuit may be, for example, but is not limited to, an ASIC (Application Specific Integrated Circuit). The control unit 13 controls the overall operation of the vehicle 10.

[0022] (Configuration of support device) As shown in FIG. 1, the support device 20 includes a communication unit 21, an input unit 22, an output unit 23, a storage unit 24, and a control unit 25.

[0023] The communication unit 21 includes a communication interface for wirelessly connecting to the network 2. The communication interface for connecting to the network 2 corresponds to, for example, a mobile communication standard, a wired LAN standard, or a wireless LAN standard, but is not limited to these and may correspond to any communication standard.

[0024] The input unit 22 is configured to include an input interface that can input camera images and audio. The input interface is, for example, a camera 22A that captures camera images of the user 3 and the luggage that the user 3 brings into the vehicle 10, and a microphone 22B that collects the audio of the user 3. However, the input interface is not limited to these. The input unit 22 outputs the camera images captured by the camera 22A and the audio collected by the microphone 22B to the control unit 25.

[0025] The camera 22A is, for example, a video camera installed on a pole at a bus stop where the vehicle 10 stops, and captures moving or still images. The camera 22A captures images of the face and body of the user 3, as well as luggage that the user 3 brings into the vehicle 10.

[0026] The output unit 23 includes at least one audio output interface capable of outputting audio, and at least one display interface capable of displaying text or video. The audio output interface is, for example, a speaker 23A. The display interface is, for example, a display 23B such as an LCD or an organic EL display.

[0027] The storage unit 24 includes one or more memories. Each memory included in the storage unit 24 may function as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 24 stores any information used in the operation of the assistance device 20. For example, the storage unit 24 may store a system program and an application program including a measurement app that measures the dimensions of luggage. The storage unit 24 registers user information of the user 3. The information stored in the storage unit 24 may be updatable with information obtained from the network 2 via the communication unit 21, for example.

[0028] The control unit 25 includes one or more processors, one or more programmable circuits, one or more dedicated circuits, or a combination thereof. The control unit 25 controls the overall operation of the assistance device 20.

[0029] (Server configuration) As shown in FIG. 1, the server 30 includes a communication unit 31, a storage unit 32, and a control unit 33.

[0030] The communication unit 31 includes one or more communication interfaces connected to the network 2. The communication interfaces may correspond to, for example, a mobile communication standard, a wired LAN standard, or a wireless LAN standard, but are not limited to these and may correspond to any communication standard. In this embodiment, the server 30 communicates with the vehicle 10 and the assistance device 20 via the communication unit 31 and the network 2.

[0031] The storage unit 32 includes one or more memories. Each memory included in the storage unit 32 may function, for example, as a main storage device, an auxiliary storage device, or a cache memory. The storage unit 32 stores any information used in the operation of the server 30. For example, the storage unit 32 may store a system program, a database, an application program, etc. Furthermore, the storage unit 32 includes a database 32A that stores user information of the user 3. The information stored in the storage unit 32 may be updatable with information obtained from the network 2 via the communication unit 31, for example.

[0032] The control unit 33 includes one or more processors, one or more programmable circuits, one or more dedicated circuits, or a combination thereof. The control unit 33 controls the overall operation of the server 30.

[0033] (Operation flow of the support device) The operation of the assistance device 20 according to this embodiment will be described with reference to Fig. 2. This operation relates to storing luggage brought into the vehicle 10 by the user 3.

[0034] S101: The control unit 25 receives a notification from the server 30 that the user 3 has purchased a ticket for the vehicle 10.

[0035] In the present disclosure, vehicle 10 is assumed to be an on-demand bus (public bus), but is not limited to this. User 3 purchases a boarding ticket online on a website published by server 30 owned by a business operator that operates vehicle 10. When the purchase of the boarding ticket is completed, control unit 25 receives a notification from server 30 that the purchase of the boarding ticket for vehicle 10 by user 3 who plans to board vehicle 10 has been completed.

[0036] S102: The control unit 25 registers the user information of the user 3.

[0037] When purchasing a ticket online, user 3 inputs user information on a website published by server 30. Control unit 33 of server 30 accepts the user information entered by user 3 and registers the accepted user information in database 32A provided in memory unit 32. Control unit 33 transmits the registered user information of user 3 to the vehicle 10 for which user 3 reserves a ride and to the assistance device 20 installed at the bus stop where vehicle 10 stops. Control unit 25 registers the received user information of user 3 in memory unit 24 as user information of the person who made the reservation.

[0038] The user information includes basic information such as the address, name, and contact information (telephone number, email address) of the user 3. The user information also includes a facial image for identifying the user 3. Furthermore, the user information may include a voice model of the user 3 who uttered a predetermined phrase.

[0039] S103: The control unit 25 acquires the camera image taken by the camera 22A.

[0040] FIG. 3 is a schematic diagram illustrating an example of the operation of the assistance device 20. In the example shown in FIG. 3, four users 3 (e.g., 3A, 3B, 3C, and 3D) board a vehicle 10 at a bus stop where the assistance device 20 is installed. In the present disclosure, the assistance device 20 is assumed to be installed on a pole at the bus stop, but the installation location of the assistance device 20 is not limited thereto. The assistance device 20 is equipped with a camera 22A that captures moving or still images. The users 3 (3A, 3B, 3C, and 3D) carry luggage of various types and sizes, and bring these items into the vehicle 10. The input unit 22 uses the camera 22A to capture facial images of the users 3 and camera images of the luggage that the users 3 bring into the vehicle 10, and outputs the captured camera images to the control unit 25.

[0041] The control unit 25 verifies the identity of the user 3 by comparing the captured facial image of the user 3 with the facial image of the user 3 included in the user information registered in the storage unit 24. The control unit 25 approves the user 3, whose identity has been verified, to board the vehicle 10. The input unit 22 may use the microphone 22B to collect the voice of the user 3 uttering a predetermined phrase, such as "name of user 3" or "getting in, getting out." The control unit 25 may verify the identity of the user 3 by comparing the voice of the user 3 uttering the predetermined phrase with a voice model of the user 3 extracted by performing a voiceprint analysis of the predetermined phrase uttered by the user 3 included in the user information.

[0042] S104: The control unit 25 determines the type of baggage b that the user 3 brings into the vehicle 10 from the acquired camera image.

[0043] In this disclosure, the type of luggage b refers to a classification based on the type of luggage b, such as a bag, handbag, backpack, travel bag, or suitcase. However, the type of luggage b is not limited to these. The type of luggage b may also be classified based on regulations for transporting luggage b, such as carry-on baggage b1, checked baggage b2, and baggage b3 that cannot be carried on board the vehicle 10. Carry-on baggage b1 is baggage that can be carried on board by each user 3 and brought into the passenger compartment, and includes, for example, a bag, handbag, backpack, or travel bag. Checked baggage b2 is baggage that is accepted by the vehicle 10 (transportation company) for transport by the user 3, and includes, for example, a backpack, travel bag, or suitcase. Backpacks and travel bags are classified as either carry-on baggage b1 or checked baggage b2 depending on their dimensions. Baggage b3 that cannot be carried on board the vehicle 10 is baggage that cannot be transported due to laws and regulations or transportation company regulations, such as animals, plants, or hazardous materials.

[0044] S105: The control unit 25 measures the dimensions of the identified package b.

[0045] The control unit 25 starts a measurement application stored in the storage unit 24 for measuring the dimensions of the package b, and measures the dimensions of the package b. The measurement application uses AR (Augmented Reality) technology to measure the dimensions (height x width x depth (cm)) of the object captured by the camera, the total dimensions of the three sides (height + width + depth (cm)), and the area of ​​the object (cm 2 In the present disclosure, the dimensions of package b are the sum of the three sides of the subject, but are not limited to this and may be any dimensions. Note that the method for measuring the dimensions of package b is not limited to the method using a measurement app, and any method may be used.

[0046] S106: The control unit 25 checks whether the type of the identified package b is a predetermined package bp. If it is the predetermined package bp, proceed to S108, and if it is not the predetermined package bp, proceed to S107.

[0047] The predetermined luggage bp is, for example, a suitcase. Since the predetermined luggage bp is relatively bulky, placing the luggage bp in a predetermined luggage storage area prevents the luggage b from getting in the way of other users, improving the comfort of user 3. However, the predetermined luggage bp is not limited to a suitcase and may be set arbitrarily.

[0048] S107: The control unit 25 determines whether the measured dimension of the package b is greater than the threshold value α. If it is greater than or equal to the threshold value α, the process proceeds to S108, and if it is not greater than or equal to the threshold value α, the process proceeds to S109.

[0049] The control unit 25 measures the dimensions of the baggage b included in the acquired camera image using the measurement app. For example, if the height x width x depth (cm) dimensions of a travel bag are 30 x 60 x 26 (cm), the control unit 25 measures the total dimensions of the three sides of the travel bag to be 116 (cm).

[0050] In the present disclosure, the threshold value α is defined as, for example, 100 (cm). The control unit 25 determines that the dimensions of the travel bag whose total dimension of the above-mentioned three sides is 116 (cm) are larger than the threshold value α.

[0051] Furthermore, the threshold value α may include multiple threshold values ​​αi. In the present disclosure, the threshold value α includes a first threshold value α1 and a second threshold value α2 that is greater than the first threshold value α1, but the number of threshold values ​​is not limited to two. Specifically, the first threshold value α1 may be set to 50 cm, and the second threshold value α2 may be set to 100 cm. In this case, since the total dimension of the three sides of the travel bag described above is 116 cm, the dimensions of the travel bag are determined to be greater than the second threshold value α2.

[0052] S108: The control unit 25 presents the user 3 with a predetermined luggage storage area s.

[0053] FIG. 4 is a schematic diagram showing an example of a luggage storage area in a vehicle. In the example shown in FIG. 4, the predetermined luggage storage area s is a trunk room s1 under the passenger compartment and a luggage shelf s2 installed above the passenger compartment. For the user 3, the control unit 25 (i) suggests storing luggage b in the trunk room s1 under the passenger compartment if it determines that luggage b is checked baggage b2; (ii) suggests storing luggage b in the trunk room s1 if the measured dimensions of luggage b are larger than the second threshold value α2; (iii) suggests storing luggage b on the luggage shelf s2 if the measured dimensions of luggage b are larger than the first threshold value α1 but equal to or smaller than the second threshold value α2; and (iv) does not suggest the predetermined luggage storage area s if the measured dimensions of luggage b are equal to or smaller than the first threshold value α1. In case (iv), user 3 may carry baggage b into the passenger cabin as carry-on baggage b1, sit in a seat, and hold baggage b, or store baggage b in luggage rack s2, or, if there are many empty seats in the passenger cabin, temporarily place baggage b on an empty seat s3 until another user is seated. However, the designated baggage storage area s is not limited to trunk room s1 and luggage rack s2, and any location within vehicle 10 may be indicated. Note that baggage b whose total dimensions of its three sides are larger than second threshold value α2 and therefore larger than third threshold value α3 may be included in baggage b3 that cannot be brought onto vehicle 10.

[0054] The assistance device 20 includes an output unit 23 having a speaker 23A and a display 23B. The control unit 25 may present the predetermined luggage storage area s to the user 3 by voice output from the speaker 23A, text and images displayed on the display 23B, or both voice and text and images. Furthermore, when baggage b3 that cannot be brought onto the vehicle 10 is detected, the control unit 25 may use the speaker 23A and the display 23B to notify the user 3 that the baggage b3 cannot be brought onto the vehicle 10.

[0055] S109: The control unit 25 determines whether the vehicle 10 has passed all the stops on the route. If all the stops have been passed, the information processing ends. If all the stops have not been passed, the control unit 25 returns to S103 and continues the information processing.

[0056] As described above, the assistance device 20 of this embodiment determines the type of luggage that the user 3 brings into the vehicle 10 from the acquired camera image, measures the dimensions of the determined luggage, and if the determined type of luggage is a specified luggage or if the measured dimensions of the luggage are equal to or larger than a threshold value, presents the user 3 with a specified luggage storage area.

[0057] According to this configuration, if the type of baggage b is a predetermined baggage bp, or if the measured dimensions of baggage b are larger than the threshold value α, a predetermined baggage storage area is presented to the user 3 of the vehicle 10. This allows the user 3 to place bulky baggage in the predetermined baggage storage area, so that the baggage does not get in the way of other users, improving the comfort of the user 3 and improving the technology for storing baggage brought into the vehicle by users.

[0058] Although the present disclosure has been described based on the drawings and examples, it should be noted that those skilled in the art may make various modifications and alterations based on the present disclosure. Therefore, it should be noted that these modifications and alterations are included in the scope of the present disclosure. For example, the functions included in each component or step can be rearranged so as not to be logically inconsistent, and multiple components or steps can be combined or divided into one.

[0059] For example, in the above-described embodiment, the configuration and operation of the assistance device 20 may be distributed among multiple computers that can communicate with each other. Also, for example, an embodiment in which some or all of the components of the assistance device 20 are provided in the vehicle 10 may be possible. For example, an in-vehicle device installed in the vehicle 10 may include some or all of the components of the assistance device 20.

[0060] Also, an embodiment is possible in which, for example, a general-purpose computer functions as the assistance device 20 according to the above-described embodiment. Specifically, a program describing the processing content for realizing each function of the assistance device 20 according to the above-described embodiment is stored in the memory of the general-purpose computer, and the program is read and executed by a processor. Therefore, the present disclosure can also be realized as a program executable by a processor, or a non-transitory computer-readable medium storing the program.

[0061] In one example, the assistance device 20 may be used to provide MaaS (Mobility as a Service), which is a service that utilizes mobility. [Explanation of symbols]

[0062] 1 System 2 Network 3,3A,3B,3C,3D User 10 vehicles 11 Communications Department 12 Storage section 13 Control Unit 20 Support equipment 21 Communications Department 22 Input section 22A Camera 22B Microphone 23 Output section 23A Speaker 23B Display 24 Memory section 25 Control Unit 30 servers 31 Communications Department 32 Storage section 32A Database 33 Control Unit

Claims

1. an input unit having a camera; a control unit that acquires a camera image taken by the camera, The control unit determines the type of luggage that the user is bringing into the vehicle from the acquired camera image, measures the dimensions of the determined luggage, and if the determined type of luggage is a specified luggage or if the measured dimensions of the luggage are larger than a threshold value, presents the user with a specified luggage storage area.

2. The support device according to claim 1, The predetermined luggage storage area is a trunk room under the passenger compartment and a luggage rack installed above the passenger compartment, the threshold values ​​include a first threshold value and a second threshold value that is greater than the first threshold value; The control unit presents the user with the luggage shelf as the specified luggage storage area when the measured luggage dimensions are greater than the first threshold value and less than or equal to the second threshold value, and presents the trunk room as the specified luggage storage area when the measured luggage dimensions are greater than the second threshold value.

3. The support device according to claim 1, further comprising an output unit having a speaker and a display; The control unit presents the specified luggage storage area to the user by audio output from the speaker, text and images displayed on the display, or both the audio and the text and images.

4. The support device according to claim 1, The control unit measures the dimensions of the luggage included in the acquired camera image using a measurement app.

5. With the assistance device, Identifying the type of luggage brought into the vehicle by the user from the acquired camera image; Measuring the dimensions of the identified luggage; If the determined type of the luggage is a predetermined luggage, or if the measured dimensions of the luggage are larger than a threshold value, presenting a predetermined luggage storage area to the user; How to support this.

6. A MaaS (Mobility as a Service) providing method using the assistance device according to claim 1.

Images