Information processing system

Abstract

To provide an information processing system capable of specifying a highly accurate position of a user.SOLUTION: An information processing system comprises a plurality of moving bodies and an information processing device, in which the plurality of moving bodies each include a communication device capable of communication using terahertz waves, the information processing device comprises a control unit that executes transmitting a sensing request to a first moving body; receiving a sensing result including a detection position of a detected object from the first moving object; and notifying a second moving body of information related to a position of a predetermined user when the sensing result is a result obtained by detecting the predetermined user. Sensing by the first moving body is sensing by a transmitting / receiving device using terahertz waves.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to an information processing system, and more particularly to an information processing system capable of identifying the highly accurate position of a user.

Background Art

[0002] Patent Document 1 discloses determining the position of a pedestrian using a camera and a radar mounted on a vehicle.

Prior Art Document

Patent Document

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] One aspect of the present disclosure aims to provide an information processing system capable of identifying the highly accurate position of a user.

Means for Solving the Problems

[0005] One aspect of the present disclosure is <了 an information processing system including a plurality of moving bodies and an information processing device, wherein the plurality of moving bodies are provided with communication devices capable of communicating using terahertz waves, and the information processing device transmits a sensing request to a first moving body, receives a sensing result including the detection position of a detected object from the first moving body, and when the sensing result is a result of detecting a predetermined user, notifies the second moving body of information related to the position of the predetermined user, and includes a control unit that executes wherein the sensing by the first moving body is sensing using terahertz waves by the communication device. This is an information processing system characterized by the following:

[0006] Another aspect of this disclosure is, An information processing method performed by an information processing system including multiple mobile bodies and information processing devices, The information processing device transmits a sensing request to the first mobile object, The first mobile device performs sensing using a communication device capable of communication using terahertz waves and transmits the sensing results to the information processing device. The information processing device receives sensing results from the first moving object, including the detection position of the detected object. The information processing device, when the sensing result is the result of detecting a predetermined user, notifies the second mobile body of information related to the location of the predetermined user. This is an information processing method that includes [the following]. [Effects of the Invention]

[0007] According to the aspects of this disclosure, it becomes possible to identify the user's location with high precision and to process information based on high-precision location information. [Brief explanation of the drawing]

[0008] [Figure 1] A diagram illustrating the overview of the information processing system according to the embodiment. [Figure 2] A diagram showing the configuration of the information processing device and vehicle in the embodiment. [Figure 3] A flowchart showing the processing flow performed by the information processing device in the embodiment. [Figure 4] This diagram illustrates an example of the process for determining whether the sensing target is the target user. [Figure 5] A diagram illustrating another example of the process for determining whether the sensing target is the target user. [Modes for carrying out the invention]

[0009] (overview) If a user's location can be accurately determined, location information can be used more effectively. For example, ride-hailing services may need to consider which side of the road a user is on when moving their vehicle. In such cases, location detection with a precision finer than the road width is required, but GPS devices may not be able to provide sufficient accuracy.

[0010] Incidentally, recent 5G communications utilize terahertz waves. Furthermore, object detection using terahertz waves with 5G communication equipment has been proposed. In this disclosure, this type of sensing is also referred to as 5G sensing. By using 5G sensing, highly accurate position detection is possible.

[0011] One aspect of this disclosure is an information processing system including a plurality of mobile bodies and an information processing device. The mobile bodies are typically vehicles (including electric vehicles, hybrid vehicles, and gasoline vehicles), but may also be flying objects or other mobile bodies. The mobile bodies are equipped with communication devices capable of communication using terahertz waves. This communication device is a device capable of sensing using terahertz waves, and may be, for example, a 5G or next-generation mobile communication device, but may also be a terahertz wave radar device.

[0012] The information processing device transmits a sensing request to the first mobile body, receives sensing results from the first mobile body including the detection location of the detected object, and, if the sensing results indicate the detection of a predetermined user, notifies the second mobile body of information related to the location of the predetermined user.

[0013] The sensing results include information on at least one of the location, shape, or type of the sensed object, or sensing data that can be used to calculate such information. In other words, the determination of the object's location, shape, type, etc., based on the sensing data may be performed by a moving object, an information processing device, or other device.

[0014] Information related to the user's position may be the user's position itself or any information obtained based on the user's position. Examples of information obtained based on the user's position include the route to the user's position.

[0015] The present disclosure can be used in a service that dispatches a moving body to the user's position. For example, the control unit of the information processing device acquires first position information representing the position of the predetermined user, selects a moving body that moves to the position of the predetermined user as the second moving body, and based on the sensing result of the first moving body, acquires second position information, which is the position information of the predetermined user with higher accuracy than the first position information. The route from the current position of the second moving body to the position indicated by the second position information may be searched considering on which side of the road the second position information faces, and the searched route may be notified to the second moving body. In this way, the second moving body can be directed to the location where the user is located. At that time, since it is considered on which side of the road it is located, access (boarding or item delivery, etc.) to the second moving body by the user becomes easy. Note that the sensing request for the first moving body from the information processing device may be made when the second moving body is located within a predetermined range from the first position information. This is because when the second moving body reaches close to the user, it is sufficient to know the detailed position of the user. In the present disclosure, the determination as to whether the sensing result is a result of detecting a predetermined user may be made by the moving body, the information processing device, or another device. As an example, the information processing device may make the determination as follows.

[0016] In the present disclosure, the determination as to whether the sensing result is a result of detecting a predetermined user may be made by the moving body, the information processing device, or another device. As an example, the information processing device may make the determination as follows.

[0017] In this disclosure, the control unit of the information processing device may perform the following actions: obtain a User Equipment (UE) ID associated with a predetermined user; obtain the location of the UE having the UE ID from a location information server; and determine that the sensing result is the result of detecting a predetermined user if the sensing result is the result of detecting a person and the detected location is within the range of the UE's location. Since a UE associated with a user can be considered to be carried by that user, the location of the UE can be considered to be the location of the user. The location of the UE is managed by a mobile communication system (e.g., a 5G core system). Therefore, if the detected location is within the range of the UE's location, the sensing result can be considered to be the result of detecting a user, and the location of the user can be identified with a higher accuracy than that managed by the mobile communication system.

[0018] In this disclosure, the sensing by the first mobile body includes sensing by a camera, and the control unit of the information processing device may determine that the sensing result is a result of detecting the predetermined user if the feature quantities of the detected object obtained from the sensing result match the feature quantities of the predetermined user that have been stored in advance. The user feature quantities can be, for example, feature quantities representing facial features, or feature quantities representing body or movement features. In this way, it is possible to determine whether or not the detected object is the target user from the features obtained from the camera image.

[0019] According to this disclosure, the user's location can be accurately determined, enabling the realization of effective services such as dispatching a vehicle to the user's location.

[0020] This disclosure further includes computer programs for causing a computer to perform each step of the above method, and computer programs for implementing the above network node or information processing system using a computer. This disclosure further includes computer-readable media on which the above computer programs are recorded.

[0021] (Embodiment 1) Embodiments of this disclosure will be described below with reference to the drawings. The following embodiments are for illustrative purposes only, and this disclosure is not limited to the configuration of these embodiments. For example, an information processing system for a ride-hailing service is described below, but it is applicable to any information processing system that performs information processing using user location information.

[0022] <System Overview> Figure 1 is a diagram illustrating the outline of the information processing system (dispatch service system) according to this embodiment. The information processing system 100 includes a server device 110 and vehicles 120A and 120B, and dispatches one of the vehicles (vehicle 120B in Figure 1) to user 130 in response to a dispatch request from user 130. Here, it is assumed that the server device 110 knows the location of each vehicle 120A and 120B. In the following, when it is not necessary to distinguish between vehicle 120A and vehicle 120B, they will be referred to as vehicle 120.

[0023] Refer to Figure 1 for a brief explanation of how the system works.

[0024] User 130 uses their mobile device (UE; User Equipment) 130A to request a ride from the server device 110 (Step 1). The ride request includes at least the user ID and the user's current location (or pickup location), and may also include information such as the pickup time and destination.

[0025] The server device 110 selects an assigned vehicle to dispatch to the user 130 and sends a dispatch instruction to the assigned vehicle (step 2). This dispatch instruction includes at least the pickup location.

[0026] The server device 110 transmits a sensing request to vehicle 120B located near the pickup location (step 3). Upon receiving the sensing request, vehicle 120A performs sensing of surrounding objects (step 4) and transmits the sensing results to the server device 110 (step 5). This sensing is performed using terahertz waves transmitted and received by the mobile communication radio wave transceiver equipped in vehicle 120A. Sensing by vehicle 120A may also be performed by other sensor devices such as cameras or LiDAR.

[0027] The server device 110 verifies whether the sensing result obtained from vehicle 120B is a result of detecting user 130 (step 6). If the sensing result is the location of user 130, the server device 110 has obtained the user's location with high accuracy. Then, the server device 110 searches for a route from the vehicle 120B's current location to the user's location (step 7). In this route search, the server device 110 considers which side of the road the user's location faces and seeks a route that will guide vehicle 120B to vehicles adjacent to the sidewalk where the user is located. The server device 110 transmits an updated dispatch instruction to vehicle 120B, including the determined route and the user's location with high accuracy.

[0028] By transmitting a route based on the user's highly accurate location to the vehicle 120B in this way, the vehicle 120B can easily reach the user.

[0029] <Structure> Figure 2A shows an example configuration of an information processing device (computer) 200 that can operate as a server device 110. The information processing device 200 can be configured using a dedicated or general-purpose information processing device (computer) such as a personal computer (PC), workstation (WS), or server machine. However, the information processing device 200 may also be a collection of one or more computers (cloud).

[0030] The information processing device 200 includes a processor 201 acting as a processing unit or control unit (controller), a main memory 202, an auxiliary memory 203, an input / output device 204, and a communication device 205, all interconnected via a bus. The processor 201 loads a program stored in the auxiliary memory 203 into the main memory 202 and executes it, thereby performing the processing described later.

[0031] The main memory 202 is used as at least one of the following: a program and data storage area, a program deployment area, a program work area, and a communication data buffer area. The main memory consists of RAM (Random Access Memory), or a combination of RAM and ROM (Read Only Memory).

[0032] The auxiliary storage device 203 is used as a storage area for data and programs. A non-volatile storage medium is used for the auxiliary storage device. The non-volatile storage medium is, for example, a hard disk. These include solid-state drives (SSDs), flash memory, or EEPROMs (electrically erasable programmable read-only memory). The auxiliary storage device 203 may also include a drive device for a disk recording medium.

[0033] The input / output device 204 includes input devices such as keys, buttons, pointing devices, and touch panels, and output devices such as liquid crystal displays, and performs data input from the operator and data output to the operator.

[0034] The communication device 205 communicates with the vehicle 120, the mobile terminal 130A, and other devices. The communication method is wireless communication (5G, wireless LAN (Wi-Fi®), BLE, etc.). It can be either a wireless connection or a wired connection.

[0035] Figure 2B shows an example of a vehicle configuration. The vehicle 210 includes a control device 211, a vehicle drive unit 212, a communication device 213, and a group of sensors. The control device 211 is a computer including a processor and memory. The vehicle drive unit 212 controls the vehicle's movement. The communication device 213 is a device that communicates with the server device 110, base stations, and other devices, and in this embodiment, it is a wireless communication device that performs wireless communication based on the 5G standard. In addition to frequencies below 6 GHz (sub-6), the communication device 213 also supports communication using terahertz waves in the 28 GHz / 39 GHz band.

[0036] The sensor group 214 includes, as an example, a GPS device 214A, a millimeter-wave radar 214B, a Lidar 214C, a camera 214D, and a 5G wireless device 214E. The GPS device 214A acquires positional information based on GPS signals. The millimeter-wave radar 214B senses surrounding objects using millimeter waves. The Lidar 214C senses surrounding objects using near-infrared laser pulses. The camera 214D acquires images of surrounding objects using visible light or near-infrared light. The 5G wireless device 214E senses surrounding objects using terahertz waves used for 5G mobile communication. The 5G wireless device 214E is the same as the communication device 213, but since it can be used for both communication and sensing purposes, it is shown as both a communication device 213 and a 5G wireless device 214E in Figure 2B. The sensor group 214 may also include sensors that acquire information about objects around the vehicle other than those mentioned above, sensors that acquire information about objects inside the vehicle, sensors that acquire information about the vehicle's status, etc.

[0037] Since the mobile terminal 130A is similar to the information processing device 200 in that it includes a processor (control unit), main memory, auxiliary memory, input device, and communication device, a detailed explanation will be omitted. The communication device is compatible with the 5G mobile communication standard.

[0038] <Processing> The operation of the information processing system according to this embodiment will be described below, mainly focusing on the processing performed by the server device 110. Figure 3 is a flowchart showing the processing flow performed by the server device 110. In the following description, the fact that the processor (control unit) 201 of the server device 110 performs the processing will also be expressed simply as the server device 110 performing the processing.

[0039] In step S301, the server device 110 receives a dispatch request from the mobile terminal 130A. The dispatch request is a request from user 130 to request any vehicle, and includes user 130's user ID and dispatch destination. The dispatch destination of the mobile terminal 130A may be obtained in any way. For example, the dispatch destination may be the location obtained by the GPS device installed in the mobile terminal 130A, or the dispatch destination may be the location specified by the user using the mobile terminal 130A. The information representing the dispatch destination corresponds to the first location information in this disclosure.

[0040] In step S302, the server device 110 selects a vehicle to assign to the dispatch request from among several vehicles and sends a dispatch instruction to the selected assigned vehicle. The method of selecting the assigned vehicle is not particularly limited, but a typical method is to select a vehicle that is close to the dispatch destination from among the available vehicles. Here, it is assumed that vehicle 120B shown in Figure 1 is selected as the assigned vehicle. The selected assigned vehicle corresponds to the second mobile vehicle in this disclosure. The dispatch instruction includes at least the dispatch destination, and may also include information about the user 130 and information about the destination. Typically, there is one assigned vehicle, but there may be multiple vehicles.

[0041] In step S302, the server device 110 determines whether the assigned vehicle 120B has arrived near the dispatch destination. Since each vehicle 120 periodically transmits its current location, acquired by the GPS device 214A, to the server device 110, the server device 110 can determine the current location of each vehicle 120. The degree of proximity for "near the dispatch destination" can be determined as appropriate according to the system requirements, but for example, it may be set as being within 1 km of the dispatch destination. If the assigned vehicle 120B has arrived near the dispatch destination, the system proceeds to step S304; otherwise, it waits.

[0042] In step S304, the server device 110 selects a vehicle to request sensing from among vehicles located near the dispatch destination. The criteria for "near the dispatch destination" here may be the same as or different from those in step S302, and in this embodiment, it is set to be within 300m, which is closer than the criteria in step S302. Here, it is assumed that vehicle 120A shown in Figure 1 is selected as the vehicle to perform sensing. The selected sensing vehicle corresponds to the first mobile body in this disclosure. Note that there is not limited to one sensing vehicle; there may be multiple vehicles.

[0043] In step S305, the server device 110 sends a sensing request to the sensing vehicle 120A. The sensing request can take any form as long as it is clear that it is requesting the sensing of objects around the vehicle, especially people. The sensing request may or may not include information such as the user ID and physical characteristics of user 130 and the UE ID of the mobile terminal 130A.

[0044] Upon receiving a sensing request, vehicle 120A performs sensing of surrounding objects using the 5G wireless device 214E. Specifically, vehicle 120A obtains information about surrounding objects by transmitting terahertz waves used for mobile communication and acquiring the reflected waves. This sensing is also called 5G sensing. In addition to sensing using the 5G wireless device 214E, vehicle 120A may also perform sensing using at least one of the following: millimeter-wave radar 214B, Lidar 214C, or camera 214D. From the obtained sensing data, the shape and type of the detected object can be determined. Furthermore, from the obtained sensing data, the position of the detected object, particularly its position relative to vehicle 120A, can be determined. When vehicle 120A determines that a person or user 130 has been detected, it transmits the sensing results, including the detection location, to the server device 110. Vehicle 120A may also transmit sensor data to the server device 110, which may then determine whether the detected object is a person and calculate the detection location.

[0045] In step S306, the server device 110 receives sensing results from the sensing vehicle 120A.

[0046] In step S307, the server device 110 determines whether the sensing result received from the sensing vehicle 120A is a result of detecting user 130. Details of this determination will be described later. If the sensing result is not related to user 130, Return to step S306 and receive another sensing result. If the sensing result relates to user 130, proceed to step S308.

[0047] In step S308, the server device 110 considers the detected position included in the sensing result to be the position of user 130 and searches for a route from the current position of vehicle 120B to the sensing result position. At this time, the detected position included in the sensing result is more accurate than the initial dispatch destination (first position information) and its relative relationship with the position of vehicle 120A can be determined, so it is possible to determine which side of the road the sensing vehicle 120A is traveling on is located on. Therefore, the server device 110 performs a route search specifying which side to approach from to the position of user 130, making it possible to guide the vehicle to the sidewalk facing the position of user 130. The information representing the detected position included in the sensing result corresponds to the second position information in this disclosure.

[0048] In step S309, the server device 110 transmits an updated dispatch instruction to the assigned vehicle 120B, which includes the highly accurate location of user 130 obtained as a sensing result and the route determined in step S308. As a result, the assigned vehicle 120B can determine the highly accurate location of user 130 and understand the route to get there, and can easily move to where user 130 is.

[0049] Referring to Figure 4, an example of determining whether the detected object in the sensing in step S307 is user 130 will be explained. Figure 4(A) is a flowchart detailing the process in step S307.

[0050] In step S401, the server device 110 obtains the UE ID of UE 130A associated with user 130. In step S402, the server device 110 queries the 5G core (5GC) system for the location of the UE. The 5G system manages the locations of UEs and can provide this location to external parties as a location service. The server device 110 can obtain the location of the UE by querying for a location that includes the UE ID. In step S403, it is determined whether the sensing location is within the UE location obtained from the 5G core system. If the sensing location is within the UE location, the server device 110 determines that the sensing target is user 130 (step S404); otherwise, it determines that the sensing target is not a user (step S405). The UE location provided by the 5G core system includes an error larger than the sensing accuracy and is shown within a certain range including the UE location 411 and the location error 412, for example, as shown in Figure 4B. In contrast, the sensing results are more accurate and are shown, for example, as positions 413 and 414 in Figure 4B. Position 413 is within the range of the UE position, and therefore, when the sensing position is position 413, it is determined that the object being sensed is the user 130. On the other hand, position 414 is outside the range of the UE position, and therefore, when the sensing position is position 414, it is determined that the object being sensed is not the user 130.

[0051] Referring to Figure 5, another example of determining whether the detected object in the sensing in step S307 is user 130 is illustrated. Figure 5 is a flowchart detailing the process in step S307.

[0052] In step S501, the server device 110 calculates feature quantities of the object to be sensed from the sensing data. For example, feature quantities related to the shape of the object may be obtained from the results of 5G sensing, or feature quantities representing the facial and body features of the detected object may be obtained from images obtained by the camera. The feature quantities representing the features of the object and the encoder for calculating the feature quantities can be implemented using known methods.

[0053] In step S502, the server device 110 acquires the features of user 130. The server device 110 has previously acquired the user's features, associated them with the user ID, and stored them in a memory device, and can then retrieve the features using the user ID as a key as needed.

[0054] In step S503, the server device 110 determines whether the feature quantities match or not. If they match, it determines that the object to be sensed is the user 130 (step S504). If they do not match, it determines that the object to be sensed is not the user 130 (step S505).

[0055] Although the flowcharts shown in Figures 4A and 5 were explained assuming execution by the server device 110, they may also be executed by the vehicle 120A or other devices.

[0056] <Effects of the Embodiment> According to this embodiment, the location of user 130 can be determined with high accuracy. Therefore, a route can be presented to vehicle 120B to guide it to the user's location. Furthermore, since the vehicles to which sensing is requested are limited to those around the user's location, processing and communication loads can be suppressed. In addition, by making the sensing request when the assigned vehicle 120B approaches the vicinity of the user's location, it is possible to prevent the user from moving after sensing and rendering the sensing useless.

[0057] (Other variations) The embodiments described above are merely examples, and this disclosure may be modified as appropriate without departing from its essence.

[0058] For example, in the above embodiment, the sensing request is made when the assigned vehicle 120B approaches the vicinity of the user's location, but the sensing request may be made without adding such a condition. Also, in the above embodiment, a dispatch instruction is sent in step S302, and then an updated dispatch instruction is sent in step S309. However, the dispatch instruction in step S302 may be omitted, and only a dispatch instruction including the user's highly accurate location and the route to it may be sent. In this way, various modifications are possible.

[0059] Furthermore, the technology disclosed herein is applicable to services other than ride-hailing services. For example, it is applicable to services that dispatch a mobile vehicle to a user 130, such as delivery services. It is also applicable to any service that acquires and utilizes the high-precision location of user 130 without dispatching a mobile vehicle.

[0060] The present disclosure can also be realized by supplying a computer program implementing the functions described in the embodiments above to a computer, and having one or more processors in the computer read and execute the program. Such a computer program may be provided to the computer by a non-temporary computer-readable storage medium that can be connected to the computer's system bus, or it may be provided to the computer via a network. Non-temporary computer-readable storage mediums include, for example, any type of disk such as magnetic disks (floppy disks, hard disk drives (HDDs), etc.), optical disks (CD-ROMs, DVDs, Blu-ray discs, etc.), read-only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic cards, flash memory, optical cards, and any type of medium suitable for storing electronic instructions. [Explanation of symbols]

[0061] 100: Ride-hailing service system (information processing system) 110: Server device (information processing device) 120A, 120B: Vehicle 130: User 130A: Mobile device

Claims

1. An information processing system including multiple mobile bodies and information processing devices, The aforementioned multiple mobile bodies are equipped with communication devices capable of communication using terahertz waves. The aforementioned information processing device is Sending a sensing request to the first mobile object, The first mobile body receives sensing results including the detection position of the detected object, If the sensing result indicates the detection of a predetermined user, information related to the location of the predetermined user is notified to the second mobile unit. It includes a control unit that performs the following: The sensing by the first mobile body is sensing using terahertz waves by the communication device. An information processing system characterized by the following features.

2. The aforementioned communication device is a portable communication device used for mobile communication. The information processing system according to feature 1.

3. The control unit, To acquire first location information representing the location of the predetermined user, Selecting the moving body to be moved to the predetermined user's position as the second moving body, When the second mobile body is located within a predetermined range from the first position information, a sensing request is transmitted to the first mobile body. Based on the sensing results, a second location information, which is the location information of a predetermined user with higher accuracy than the first location information, is obtained. The system searches for a path from the current position of the second mobile body to the position indicated by the second position information, taking into consideration which side of the road the second position information faces, and notifies the second mobile body of the searched path. The information processing system according to claim 1, characterized in that it further performs the following.

4. The control unit, Obtaining the UE ID associated with the aforementioned specified user, Obtaining the location of the UE having the aforementioned UE ID from the location information server, If the sensing result is the result of detecting a person and the detection location is within the range of the location of the UE, then it is determined that the sensing result is the result of detecting the predetermined user. The information processing system according to claim 1, characterized in that it further performs the following.

5. The sensing by the first mobile body includes sensing by a camera, The control unit determines that the sensing result is a result of detecting the predetermined user when the characteristic quantity of the detected object obtained from the sensing result matches the characteristic quantity of the predetermined user that has been stored in advance. The information processing system according to feature 1.

Images