Information processing system

Abstract

To provide an information processing system capable of specifying a highly accurate position of a user.SOLUTION: An information processing system comprises a plurality of moving bodies and an information processing device, in which the plurality of moving bodies each include a communication device capable of communication using terahertz waves, the information processing device comprises a control unit that executes transmitting a sensing request to a first moving body; receiving a sensing result including a detection position of a detected object from the first moving object; and notifying a second moving body of information related to a position of a predetermined user when the sensing result is a result obtained by detecting the predetermined user. Sensing by the first moving body is sensing by a transmitting / receiving device using terahertz waves.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to an information processing system, and more particularly to an information processing system capable of identifying the highly accurate position of a user.

Background Art

[0002] Patent Document 1 discloses determining the position of a pedestrian using a camera and a radar mounted on a vehicle.

Prior Art Document

Patent Document

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] One aspect of the present disclosure aims to provide an information processing system capable of identifying the highly accurate position of a user.

Means for Solving the Problems

[0005] One aspect of the present disclosure is an information processing system including a plurality of moving bodies and an information processing device, wherein the plurality of moving bodies are provided with communication devices capable of communicating using terahertz waves, and the information processing device transmits a sensing request to a first moving body, receives a sensing result including the detection position of a detected object from the first moving body, and when the sensing result is a result of detecting a predetermined user, notifies information related to the position of the predetermined user to a second moving body, and includes a control unit that executes wherein the sensing by the first moving body is sensing using terahertz waves by the communication device. An information processing system, characterized by the above.

[0006] Another aspect of the embodiments of the present disclosure is An information processing method executed by an information processing system including a plurality of mobile bodies and an information processing device, wherein the information processing device transmits a sensing request to a first mobile body, the first mobile body performs sensing by a communication device capable of communicating using terahertz waves and transmits a sensing result to the information processing device, the information processing device receives a sensing result including a detection position of a detected object from the first mobile body, when the sensing result is a result of detecting a predetermined user, the information processing device notifies the second mobile body of information related to the position of the predetermined user, and an information processing method including the above.

Advantages of the Invention

[0007] According to the embodiments of the present disclosure, it is possible to specify the accurate position of a user and perform information processing based on the accurate position information.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Modes for Carrying Out the Invention

[0009] (Summary) If the user's location information can be accurately grasped, the location information can be used more effectively. For example, there is a requirement to move a vehicle considering on which side of the sidewalk the user is located, such as in a carpooling service. In such a case, location detection with a finer accuracy than the road width is required, but sufficient accuracy may not be obtained with a GPS device.

[0010] By the way, in recent 5G communications, communications using terahertz waves have been carried out. Furthermore, object detection using terahertz waves by a communication device for 5G communication has been proposed. Such sensing is also referred to as 5G sensing in this disclosure. By using 5G sensing, highly accurate location detection is possible.

[0011] One aspect of this disclosure is an information processing system including a plurality of moving bodies and an information processing device. The moving body is typically a vehicle (including electric vehicles, hybrid vehicles, gasoline vehicles), but may also be an aircraft or other moving body. The moving body is provided with a communication device capable of communicating using terahertz waves. This communication device is a device capable of sensing using terahertz waves, and as an example, it is a 5G or later-generation mobile communication device, but may also be a terahertz radar device.

[0012] The information processing device transmits a sensing request to the first moving body, receives a sensing result including the detection position of the detected object from the first moving body, and when the sensing result is a result of detecting a predetermined user, notifies the second moving body of information related to the position of the predetermined user.

[0013] The sensing result includes at least any one of the position, shape, and type of the sensed object, or sensing data from which the information can be calculated. In other words, the determination of the position, shape, type, etc. of the object based on the sensing data may be performed by the moving body, the information processing device, or other devices.

[0014] Information related to the user's location may be the user's location itself or any information obtained based on the user's location. Examples of information obtained based on the user's location include the route to the user's location.

[0015] This disclosure can be used in a service that dispatches a moving body to the user's location. For example, the control unit of the information processing device acquires first position information representing the location of the predetermined user, selects a moving body that moves to the location of the predetermined user as the second moving body, and based on the sensing result by the first moving body, acquires second position information, which is the location information of the predetermined user with higher accuracy than the first position information. The route from the current position of the second moving body to the position indicated by the second position information may be searched considering on which side of the road the second position information is located, and the searched route may be notified to the second moving body. In this way, the second moving body can be directed to the location where the user is located. At that time, since it is considered on which side of the road it is located, access (boarding or item delivery, etc.) to the second moving body by the user becomes easy. Note that the sensing request for the first moving body from the information processing device may be made when the second moving body is located within a predetermined range from the first position information. This is because when the second moving body reaches close to the user to a certain extent, it is sufficient to know the detailed location of the user. In this disclosure, the determination as to whether the sensing result is a result of detecting a predetermined user may be made by the moving body, the information processing device, or other devices. As an example, the information processing device may make the determination as follows.

[0016] In this disclosure, the determination as to whether the sensing result is a result of detecting a predetermined user may be made by the moving body, the information processing device, or other devices. As an example, the information processing device may make the determination as follows.

[0017] In the present disclosure, the control unit of the information processing apparatus acquires a UE (User Equipment) ID associated with the predetermined user, acquires the position of the UE having the UE ID from a position information server, and determines that the sensing result is a result of detecting the predetermined user when the sensing result is a result of detecting a person and the detection position is included in the range of the position of the UE. Since the UE associated with the user can be regarded as being carried by the user, the position of the UE can be regarded as the position of the user. The position of the UE is managed by a mobile communication system (for example, a 5G core system). Therefore, when the detection position is included in the range of the position of the UE, the sensing result can be regarded as a result of detecting the user, and the position of the user can be specified with higher accuracy than the accuracy managed by the mobile communication system.

[0018] In the present disclosure, the sensing by the first moving body includes sensing by a camera, and the control unit of the information processing apparatus may determine that the sensing result is a result of detecting the predetermined user when a feature amount of a detected object obtained from the sensing result matches a feature amount of the predetermined user stored in advance. As the feature amount of the user, for example, a feature amount representing a facial feature or a feature amount representing a feature of the body or its movement can be adopted. In this way, it is possible to determine whether the detection target is the target user from the features obtained from the camera image.

[0019] According to the present disclosure, since the position of the user can be accurately grasped, it is possible to realize an effective service such as dispatching a vehicle to the user.

[0020] The present disclosure further includes a computer program for causing a computer to execute each step of the above method, and a computer program for realizing the above network node or information processing system using a computer. The present disclosure further includes a computer-readable medium having the above computer program recorded thereon.

[0021] (Embodiment 1) Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. The following embodiments are merely illustrative for explanation, and the present disclosure is not limited to the configurations of the embodiments. For example, hereinafter, an information processing system related to a car-sharing service will be exemplified, but it is applicable to an information processing system that performs any information processing using user location information.

[0022] <System Overview> FIG. 1 is a diagram for explaining the overview of an information processing system (car-sharing service system) according to the present embodiment. The information processing system 100 includes a server device 110 and vehicles 120A and 120B, and dispatches one of the vehicles (vehicle 120B in FIG. 1) to the user 130 in response to a car-sharing request from the user 130. Here, it is assumed that the server device 110 grasps the positions of each of the vehicles 120A and 120B. In the following, when it is not necessary to distinguish between vehicle 120A and vehicle 120B, they are collectively referred to as vehicle 120.

[0023] Referring to FIG. 1, the operation of the system will be briefly described.

[0024] The user 130 makes a car-sharing request to the server device 110 using the mobile terminal (UE; User Equipment) 130A that the user has (step 1). The car-sharing request includes at least a user ID and the current location (or pick-up location) of the user, and may further include information such as the pick-up time and the destination.

[0025] The server device 110 selects an assigned vehicle to dispatch to the user 130 and transmits a car-sharing instruction to the assigned vehicle (step 2). This car-sharing instruction includes at least the pick-up location.

[0026] The server device 110 transmits a sensing request to the vehicle 120B located near the pickup position (step 3). The vehicle 120A that has received the sensing request executes sensing of surrounding objects (step 4) and transmits the sensing result to the server device 110 (step 5). This sensing is performed using terahertz waves transmitted and received by the radio wave transmission and reception device for mobile communication provided in the vehicle 120A. The sensing by the vehicle 120A may be executed by other sensor devices such as a camera or Lidar.

[0027] The server device 110 checks whether the sensing result obtained from the vehicle 120B is a result of detecting the user 130 (step 6). If the sensing result is the position of the user 130, it means that the server device 110 has obtained the high-precision position of the user 130. Therefore, the server device 110 searches for a route from the current position of the vehicle 120B to the user position (step 7). In this route search, considering on which side of the road the user position faces, a route that can guide the vehicle 120B to the vehicles in contact with the sidewalk where the user is located is required. The server device 110 transmits an updated vehicle dispatch instruction including the obtained route and the high-precision position of the user to the vehicle 120B.

[0028] By transmitting the route based on the high-precision position of the user to the vehicle 120B in this way, the vehicle 120B can easily reach the user.

[0029] <Configuration> FIG. 2A is a diagram showing a configuration example of an information processing device (computer) 200 that can operate as the server device 110. The information processing device 200 can be configured using a dedicated or general-purpose information processing device (computer) such as a personal computer (PC), a workstation (WS), or a server machine. However, the information processing device 200 may be an aggregate (cloud) of one or two or more computers.

[0030] The information processing apparatus 200 includes a processor 201 as a processing unit or a control unit (controller) interconnected via a bus, a main memory device 202, an auxiliary storage device 203, an input / output device 204, and a communication device 205. By loading and executing the program stored in the auxiliary storage device 203 into the main memory device 202 by the processor 201, the processes described later are executed.

[0031] The main memory device 202 is used as at least one of a storage area for programs and data, a deployment area for programs, a working area for programs, and a buffer area for communication data. The main memory device is composed of a RAM (Random Access Memory), or a combination of a RAM and a ROM (Read Only Memory).

[0032] The auxiliary storage device 203 is used as a storage area for data and programs. A non-volatile storage medium is applied to the auxiliary storage device. The non-volatile storage medium is, for example, a hard disk drive, a Solid State Drive (SSD), a flash memory, or an EEPROM (Electrically Erasable Programmable Read-Only Memory), etc. Also, the auxiliary storage device 203 can include a drive device for a disk recording medium.

[0033] The input / output device 204 includes input devices such as keys, buttons, pointing devices, and touch panels, and output devices such as liquid crystal displays, and performs input of data from an operator and output of data to the operator.

[0034] The communication device 205 communicates with the vehicle 120, the mobile terminal 130A, and other devices. The form of communication may be wireless communication (such as 5G, wireless LAN (Wi-Fi (registered trademark)), BLE, etc. ) or may be wired communication.

[0035] Figure 2B is a diagram showing a configuration example of a vehicle. The vehicle 210 includes a control device 211, a vehicle drive device 212, a communication device 213, and a sensor group. The control device 211 is a computer including a processor and a storage device. The vehicle drive device 212 is in charge of the driving control of the vehicle. The communication device 213 is a device that communicates with the server device 110, a base station, and other devices, and in this embodiment, it is a wireless communication device that performs wireless communication based on the 5G standard. In addition to frequencies of 6 GHz or less (sub-6), the communication device 213 also supports communication using terahertz waves in the 28 GHz / 39 GHz band.

[0036] The sensor group 214 includes, as an example, a GPS device 214A, a millimeter-wave radar 214B, a Lidar 214C, a camera 214D, and a 5G wireless device 214E. The GPS device 214A acquires position information based on a GPS signal. The millimeter-wave radar 214B senses surrounding objects using millimeter waves. The Lidar 214C senses surrounding objects using near-infrared laser pulses. The camera 214D acquires an image of surrounding objects using visible light or near-infrared light. The 5G wireless device 214E senses surrounding objects using terahertz waves used for 5G mobile communication. Although the 5G wireless device 214E is the same as the communication device 213 as a device, since it can be used for both communication purposes and sensing purposes, in Figure 2B, it is shown as both the communication device 213 and the 5G wireless device 214E. Further, the sensor group 214 may include sensors that acquire information about objects around the vehicle other than those described above, sensors that acquire information about objects inside the vehicle, sensors that acquire information about the state of the vehicle, and the like.

[0037] The mobile terminal 130A is similar to the information processing device 200 in that it includes a processor (control unit), a main storage device, an auxiliary storage device, an input device, and a communication device, and thus detailed description thereof is omitted. Note that the communication device is compatible with the 5G mobile communication standard.

[0038] <Processing> Hereinafter, the operation of the information processing system according to the present embodiment will be mainly described centering on the processing performed by the server device 110. FIG. 3 is a flowchart showing the flow of processing performed by the server device 110. In the following description, the fact that the processor (control unit) 201 of the server device 110 performs processing is also simply expressed as the server device 110 performs processing.

[0039] In step S301, the server device 110 receives a vehicle allocation request from the mobile terminal 130A. The vehicle allocation request is a request for the user 130 to request any vehicle, and includes the user ID of the user 130 and the vehicle allocation destination. The vehicle allocation destination in the vehicle allocation request may be obtained in any manner. For example, the position obtained by the GPS device provided in the mobile terminal 130A may be used as the vehicle allocation destination, or the position specified by the user using the mobile terminal 130A may be used as the vehicle allocation destination. The information representing the vehicle allocation destination corresponds to the first position information in the present disclosure.

[0040] In step S302, the server device 110 selects a vehicle to be allocated to the vehicle allocation request from among a plurality of vehicles, and transmits a vehicle allocation instruction to the selected allocated vehicle. The method for selecting the allocated vehicle is not particularly limited, but a typical method is to select a vehicle close to the vehicle allocation destination from among the available vehicles. Here, it is assumed that the vehicle 120B shown in FIG. 1 is selected as the allocated vehicle. The selected allocated vehicle corresponds to the second moving body in the present disclosure. The vehicle allocation instruction includes at least the vehicle allocation destination, and may also include information about the user 130 and information about the destination, etc. The allocated vehicle is typically one, but may be a plurality.

[0041] In step S302, the server device 110 determines whether the assigned vehicle 120B has arrived near the vehicle dispatching destination. Since each vehicle 120 periodically transmits its current position acquired by the GPS device 214A to the server device 110, the server device 110 can grasp the current position of each vehicle 120. The degree of "near the vehicle dispatching destination" can be appropriately determined according to system requirements. For example, it may be conditioned that the vehicle is located within 1 km from the vehicle dispatching destination. If the assigned vehicle 120B has arrived near the vehicle dispatching destination, the process proceeds to step S304; otherwise, it waits.

[0042] In step S304, the server device 110 selects a vehicle for which to request sensing from among the vehicles near the vehicle dispatching destination. The criterion for "near the vehicle dispatching destination" here may be the same as or different from that in step S302. In this embodiment, it is conditioned that the vehicle is within 300 m, which is closer than the criterion in step S302. Here, it is assumed that the vehicle 120A shown in FIG. 1 is selected as the vehicle for which to perform sensing. The selected sensing vehicle corresponds to the first moving body in the present disclosure. Note that the number of sensing vehicles does not have to be one, and a plurality of vehicles may be used.

[0043] In step S305, the server device 110 transmits a sensing request to the sensing vehicle 120A. The sensing request may be in any form as long as it is understood that the request is for sensing objects around the vehicle, particularly people. The sensing request may or may not include information such as the user ID and external characteristics of the user 130, and the UE ID of the mobile terminal 130A.

[0044] Upon receiving a sensing request, vehicle 120A performs sensing of surrounding objects using 5G wireless device 214E. Specifically, vehicle 120A transmits terahertz waves used for mobile communication and obtains reflected waves, thereby acquiring information about surrounding objects. This sensing is also referred to as 5G sensing. In addition to sensing using 5G wireless device 214E, vehicle 120A may perform sensing using at least one of millimeter-wave radar 214B, Lidar 214C, and camera 214D. From the obtained sensing data, the shape and type of the detected object can be grasped. Also, the position of the detected object, particularly the relative position with respect to vehicle 120A, can be grasped from the obtained sensing data. When vehicle 120A determines that a person or user 130 has been detected, it transmits the sensing result including the detection position to server device 110. Note that vehicle 120A may transmit sensor data to server device 110, and server device 110 may determine whether the detection target is a person and calculate the detection position.

[0045] In step S306, server device 110 receives the sensing result from sensing vehicle 120A.

[0046] In step S307, server device 110 determines whether the sensing result received from sensing vehicle 120A is a result of detecting user 130. Details of this determination will be described later. If the sensing result is not related to user 130, the process returns to step S306 to receive another sensing result. If the sensing result is related to user 130, the process proceeds to step S308.

[0047] In step S308, the server device 110 regards the detection position included in the sensing result as the position of the user 130, and searches for a route from the current position of the vehicle 120B to the sensing result position. At this time, since the detection position included in the sensing result is more accurate than the initial vehicle allocation destination (first position information) or the relative relationship with the position of the vehicle 120A is known, it is possible to grasp on which side of the road the sensing vehicle 120A is traveling. Therefore, the server device 110 performs route search specifying from which side to approach the position of the user 130, so that it is possible to provide route guidance to the vehicle facing the sidewalk where the user 130 is located. Information representing the detection position included in the sensing result corresponds to the second position information in the present disclosure.

[0048] In step S309, the server device 110 transmits an updated vehicle allocation instruction including the highly accurate position of the user 130 obtained as the sensing result and the route obtained in step S308 to the assigned vehicle 120B. As a result, the assigned vehicle 120B can know the highly accurate position of the user 130 and can grasp the route thereto, and can easily move to where the user 130 is.

[0049] With reference to FIG. 4, an example of determining whether the detected object in the sensing in step S307 is the user 130 will be described. FIG. 4(A) is a flowchart showing details of the process of step S307.

[0050] In step S401, the server device 110 acquires the UE ID of the UE 130A associated with the user 130. In step S402, the server device 110 inquires about the location of the UE to the 5G Core (5GC) system. In the 5G system, the location of the UE is managed and can be provided to the outside as a location service. The server device 110 can acquire the location of the UE by making a location inquiry including the UE ID. In step S403, it is determined whether the sensing location is within the UE location obtained from the 5G core system. If the sensing location is within the UE location, the server device 110 determines that the sensing target is the user 130 (step S404), and otherwise determines that the sensing target is not the user (step S405). The UE location provided by the 5G core system includes an error larger than the accuracy of sensing. For example, as shown in FIG. 4B, it is shown within a certain range including the UE location 411 and the location error 412. On the other hand, the sensing result is more accurate and is shown as, for example, the location 413 or the location 414 in FIG. 4B. The location 413 is within the range of the UE location. Therefore, when the sensing location is the location 413, it is determined that the sensing object is the user 130. On the other hand, the location 414 is outside the range of the UE location. Therefore, when the sensing location is the location 414, it is determined that the sensing object is not the user 130.

[0051] Referring to FIG. 5, another example of determining whether the detected object in the sensing in step S307 is the user 130 will be described. FIG. 5 is a flowchart showing the details of the process of step S307.

[0052] In step S501, the server device 110 calculates the feature amount of the sensing object from the sensing data. For example, a feature amount related to the shape of the object may be obtained from the result of 5G sensing, or a feature amount representing the face or body features of the detection target may be obtained from the image obtained by the camera. The feature amount representing the features of the object and the encoder for calculating the feature amount may be realized using a known method.

[0053] In step S502, the server device 110 acquires the feature amount of the user 130. The server device 110 may acquire the feature amount of the user in advance, associate it with the user ID, store it in the storage device, and acquire the feature amount using the user ID as a key as needed.

[0054] In step S503, the server device 110 determines whether the feature amounts match. If they match, it is determined that the sensing target is the user 130 (step S504). If they do not match, it is determined that the sensing target is not the user 130 (step S505).

[0055] Although the flowcharts shown in FIGS. 4A and 5 have been described as being executed by the server device 110, they may be executed by the vehicle 120A or other devices.

[0056] <Effect of the Embodiment> According to the present embodiment, the position of the user 130 can be accurately grasped. Therefore, a route for guiding the vehicle 120B to the position where the user is located can be presented. In addition, since the vehicle that requests sensing is limited to the vicinity of the existence position of the user 130, it is possible to suppress the impossibility of processing and communication. Further, by setting the timing when the assigned vehicle 120B approaches the vicinity of the user position for the sensing request, it is possible to suppress the situation where the user moves after sensing and the sensing becomes wasted.

[0057] (Other Modification Examples) The above embodiment is merely an example, and the present disclosure can be appropriately modified and implemented without departing from the gist thereof.

[0058] For example, in the above-described embodiment, the sensing request is made when the assigned vehicle 120B approaches near the user's position. However, the sensing request may be made without adding such a condition. Also, in the above-described embodiment, after sending the vehicle allocation instruction in step S302, an updated version of the vehicle allocation instruction is sent in step S309. However, it is also possible to omit the vehicle allocation instruction in step S302 and only send a vehicle allocation instruction including the user's high-precision position and the route thereto. Thus, various modifications are possible.

[0059] In addition, the technology of the present disclosure is also applicable to services other than the vehicle allocation service. For example, it is applicable to a service that dispatches a moving body to the user 130 such as a delivery service. Also, it is applicable to any service that acquires and uses the high-precision position of the user 130 without involving the dispatch of a moving body.

[0060] The present disclosure can also be realized by supplying a computer program that implements the functions described in the above embodiment to a computer and causing one or more processors included in the computer to read and execute the program. Such a computer program may be provided to the computer by a non-transitory computer-readable storage medium connectable to the system bus of the computer, or may be provided to the computer via a network. The non-transitory computer-readable storage medium includes, for example, any type of disk such as a magnetic disk (floppy (registered trademark) disk, hard disk drive (HDD), etc.), an optical disk (CD-ROM, DVD disk, Blu-ray disk, etc.), a read-only memory (ROM), a random access memory (RAM), an EPROM, an EEPROM, a magnetic card, a flash memory, an optical card, and any type of medium suitable for storing electronic instructions.

Explanation of Reference Numerals

[0061] 100: Vehicle Allocation Service System (Information Processing System) 110: Server Device (Information Processing Device) 120A, 120B: Vehicle 130: User 130A: Mobile terminal

Claims

1. An information processing system including a plurality of moving bodies and an information processing device, wherein the plurality of moving bodies are provided with communication devices capable of communicating using terahertz waves, and the information processing device transmits a sensing request to a first moving body, receives a sensing result including a detection position of a detected object from the first moving body, and when the sensing result is a result of detecting a predetermined user, notifies information related to the position of the predetermined user to a second moving body, and includes a control unit that executes the above, wherein the sensing by the first moving body is sensing using terahertz waves by the communication device, An information processing system, characterized by the above.

2. The communication device is a mobile communication device used for mobile communication, The information processing system according to claim 1, characterized by the above.

3. The control unit acquires first position information representing the position of the predetermined user, selects, as the second moving body, a moving body that moves to the position of the predetermined user, transmits a sensing request to the first moving body when the second moving body is located within a predetermined range from the first position information, acquires second position information, which is position information of the predetermined user with higher accuracy than the first position information, based on the sensing result, searches for a path from the current position of the second moving body to the position indicated by the second position information, taking into account on which side of the road the position indicated by the second position information faces, and notifies the searched path to the second moving body, The information processing system according to claim 1, further characterized by the above.

4. The control unit acquires a UE ID related to the predetermined user, acquires the position of a UE having the UE ID from a position information server, and determines that the sensing result is a result of detecting the predetermined user when the sensing result is a result of detecting a person and the detection position is included in the range of the position of the UE, The information processing system according to claim 1, further characterized by the above.

5. The sensing by the first moving body includes sensing by a camera, When the feature amount of the detected object obtained from the sensing result matches the feature amount of the predetermined user stored in advance, the control unit determines that the sensing result is a result of detecting the predetermined user. The information processing system according to claim 1, characterized in that.

Images