Processing system, processing method, and processing program

Abstract

A processor of the processing system is configured to execute the following contents: acquiring desired content (Cd) of a user service desired by a user; acquiring service capability information indicating a provision capability (Pc) of a user service in association with an autonomous traveling device standby within a field of view (Av) visually confirmed by the user by a wearable terminal worn by the user; an XR-emphasized image (Ipe) superimposed on the field of view (Av), the XR-emphasized image being an image obtained by searching for and emphasizing autonomous travel devices within the field of view (Av) for the autonomous travel devices having the provision capability (Pc) matching the desired content (Cd) from the provision capabilities (Pc) indicated by the service capability information; and providing a user service by driving an autonomous travel device selected by the user in response to the overlapping display of the XR emphasized image (Ipe) within the field of view area (Av).

Description

[0001] Cross-reference of related applications

[0002] This application is based on Japanese Patent Application No. 2023-87917 filed on May 29, 2023, and incorporates the contents of the base application in its entirety by reference. Technical Field

[0003] This disclosure relates to a processing technique for performing service association processing associated with user services provided to users. Background Technology

[0004] The technology disclosed in Patent Document 1 attempts to appropriately guide users by reserving autonomous driving devices corresponding to the service ranges that divide the path from which the autonomous driving device provides guidance services to the user.

[0005] Existing technical documents

[0006] Patent documents

[0007] Patent Document 1: Japanese Patent No. 6956422.

[0008] In the technology disclosed in Patent Document 1, since the autonomous driving device corresponding to the service area is automatically booked within the segmented service area, it is difficult to reflect the user's preferences in the booking. However, the reflection of user preferences is not limited to guided services. Although it becomes a particularly important factor when user services with diverse content are provided by autonomous driving devices, the user's own search for autonomous driving devices becomes complicated. Summary of the Invention

[0009] The technical problem of this invention is to improve the search efficiency of autonomous driving devices that can provide user services that meet user preferences.

[0010] The technical means of this disclosure used to solve the technical problem are described below.

[0011] The first method of this disclosure is,

[0012] A processing system having a processor that performs service association processing associated with user services provided to a user.

[0013] The processor is configured to execute the following:

[0014] To obtain the desired content of user services that users want;

[0015] Service capability information representing the ability to provide user services is acquired in association with an autonomous driving device (Ma) that is standing still within a field of view confirmed by the user's vision through a wearable terminal worn by the user.

[0016] The XR-emphasized image is overlaid relative to the field of view. This XR-emphasized image is an image obtained by searching for autonomous driving devices within the field of view whose capabilities match the desired content from the capabilities provided by service capability information, and then emphasizing them.

[0017] User services are provided by driving an autonomous driving device selected by the user within the field of view, which responds to the overlay display of an XR-emphasized image.

[0018] The second method of this disclosure is,

[0019] A processing method, executed by a processor, for performing service association processing associated with user services provided to a user, the processing method comprising the following:

[0020] To obtain the desired content of user services that users want;

[0021] Service capability information representing the ability to provide user services is acquired in association with an autonomous driving device that is in standby within a field of view confirmed by the user's vision through a wearable terminal worn by the user.

[0022] The XR-emphasized image is overlaid relative to the field of view. This XR-emphasized image is an image obtained by searching for autonomous driving devices within the field of view whose capabilities match the desired content from the capabilities provided by service capability information, and then emphasizing them.

[0023] User services are provided by driving an autonomous driving device selected by the user within the field of view, which responds to the overlay display of an XR-emphasized image.

[0024] The third method disclosed herein is,

[0025] A processing program includes instructions stored in a storage medium for performing service association processing associated with providing user services to a user, and the service association processing is executed by a processor.

[0026] The command is used to execute the following:

[0027] To obtain the desired content of user services that users want;

[0028] Service capability information representing the ability to provide user services is acquired in association with an autonomous driving device that is in standby within a field of view confirmed by the user's vision through a wearable terminal worn by the user.

[0029] The XR-emphasized image is overlaid relative to the field of view. This XR-emphasized image is an image obtained by searching for autonomous driving devices within the field of view whose capabilities match the desired content from the capabilities provided by service capability information, and then emphasizing them.

[0030] User services are provided by driving an autonomous driving device selected by the user within the field of view, which responds to the overlay display of an XR-emphasized image.

[0031] Thus, according to the first to third methods, service capability information representing the ability to provide user services is acquired in association with autonomous driving devices that are idle within the field of vision confirmed by the user through a wearable terminal. Therefore, by acquiring the desired content of the user service desired by the user, and focusing on the autonomous driving devices within the field of vision, the service capability information indicates which autonomous driving device's provision capabilities match the desired content. An XR-emphasized image, obtained by searching for and emphasizing autonomous driving devices whose capabilities match the desired content, is overlaid on the field of vision to represent the user's selection of that device. Therefore, the search efficiency for autonomous driving devices capable of providing user services that meet the user's preferences within the field of vision can be improved.

[0032] The fourth method disclosed herein is,

[0033] A processing system having a processor that performs service association processing associated with user services provided to a user.

[0034] The processor is configured to execute the following:

[0035] To obtain the desired content of user services that users want;

[0036] Service capability information representing the ability to provide services to the user is acquired in association with an autonomous driving device that is standing still in a background area of ​​a display background shown to the user as a mobile terminal carried by the user.

[0037] The XR-emphasized image is overlaid on the background video projected onto the background area. This XR-emphasized image is obtained by searching for autonomous driving devices within the background area whose capabilities match the desired content from the capabilities provided by the service capability information, and then emphasizing them.

[0038] User services are provided by driving an autonomous driving device selected by the user, which is activated by overlaying an XR-emphasized image in a background area.

[0039] The fifth method disclosed herein is,

[0040] A processing method, executed by a processor, for performing service association processing associated with user services provided to a user, the processing method comprising the following:

[0041] To obtain the desired content of user services that users want;

[0042] Service capability information representing the ability to provide services to the user is acquired in association with an autonomous driving device that is standing still in a background area of ​​a display background shown to the user as a mobile terminal carried by the user.

[0043] The XR-emphasized image is overlaid on the background video projected onto the background area. This XR-emphasized image is obtained by searching for autonomous driving devices within the background area whose capabilities match the desired content from the capabilities provided by the service capability information, and then emphasizing them.

[0044] User services are provided by driving an autonomous driving device selected by the user, which is activated by overlaying an XR-emphasized image in a background area.

[0045] The sixth method disclosed herein is,

[0046] A processing program includes instructions stored in a storage medium for performing service association processing associated with providing user services to a user, and the service association processing is executed by a processor.

[0047] The command is used to execute the following:

[0048] To obtain the desired content of user services that users want;

[0049] Service capability information representing the ability to provide services to the user is acquired in association with an autonomous driving device that is standing still in a background area of ​​a display background shown to the user as a mobile terminal carried by the user.

[0050] The XR-emphasized image is overlaid on the background video projected onto the background area. This XR-emphasized image is obtained by searching for autonomous driving devices within the background area whose capabilities match the desired content from the capabilities provided by the service capability information, and then emphasizing them.

[0051] User services are provided by driving an autonomous driving device selected by the user, which is activated by overlaying an XR-emphasized image in a background area.

[0052] Thus, according to methods four through six, service capability information representing the ability to provide user services is acquired in association with an autonomous driving device that is standing still in a background area displayed to the user as a background from a mobile terminal carried by the user. Therefore, by acquiring the desired content of the user service desired by the user, for the autonomous driving devices in the background area, the service capability information indicates which autonomous driving device's provision capabilities match the desired content. Accordingly, an XR-emphasized image is obtained by searching for an autonomous driving device whose capabilities match the desired content and highlighting it. This selection of the device is then overlaid on a background video projected in the background area. Therefore, the search efficiency for autonomous driving devices capable of providing user services that meet user preferences through operation in the background area can be improved. Attached Figure Description

[0053] Figure 1 This is an overall structural diagram showing the network connection environment of the processing system in the first embodiment.

[0054] Figure 2 This is a schematic diagram illustrating a wearable terminal according to the first embodiment.

[0055] Figure 3 This is a schematic diagram used to illustrate a hypothetical three-dimensional voxel in the infrastructure system of the first embodiment.

[0056] Figure 4 This is a block diagram illustrating the processing system of the first embodiment.

[0057] Figure 5 This is a flowchart illustrating the processing flow of the first embodiment.

[0058] Figure 6 This is a flowchart illustrating the processing flow of the first embodiment.

[0059] Figure 7 This is a schematic diagram showing the display state of the wearable terminal according to the first embodiment.

[0060] Figure 8 This is a schematic diagram showing the display state of the wearable terminal according to the first embodiment.

[0061] Figure 9 This is a schematic diagram showing the display state of the wearable terminal according to the first embodiment.

[0062] Figure 10 This is an overall structural diagram showing the network connection environment of the processing system in the second embodiment.

[0063] Figure 11 This is a schematic diagram illustrating a wearable terminal according to the second embodiment.

[0064] Figure 12 This is a block diagram illustrating the processing system of the second embodiment.

[0065] Figure 13 This is a flowchart illustrating the processing flow of the second embodiment.

[0066] Figure 14 This is a flowchart illustrating the processing flow of the second embodiment.

[0067] Figure 15 This is a schematic diagram showing the display state of the mobile terminal according to the second embodiment.

[0068] Figure 16 This is a schematic diagram showing the display state of the mobile terminal according to the second embodiment.

[0069] Figure 17 This is a schematic diagram showing the display state of a wearable terminal in a modified example of the first embodiment.

[0070] Figure 18 This is a schematic diagram showing the display state of a wearable terminal in a modified example of the first embodiment. Detailed Implementation

[0071] Hereinafter, several embodiments of the present disclosure will be described with reference to the accompanying drawings. Furthermore, in each embodiment, corresponding constituent elements are sometimes labeled with the same symbol, and repeated descriptions are omitted. Additionally, where only a portion of the structure is described in each embodiment, the structures of other previously described embodiments can be applied to the other parts of that structure. Moreover, not only can combinations of structures specified in the descriptions of each embodiment be made, but structures of multiple embodiments can also be partially combined with each other even without specification, provided that such combinations do not particularly hinder them.

[0072] (First Implementation)

[0073] Figure 1 The processing system 1 of the first embodiment shown performs service association processing related to the user services provided to the user Us. Therefore, the processing system 1 is connected via the communication network Nc to the wearable terminal Wt worn by the pre-registered user Us, the autonomous driving device Ma, and the infrastructure system 6. Here, it is assumed that there are multiple wearable terminals Wt and autonomous driving devices Ma connected to the processing system 1. It is assumed that at least one infrastructure system 6 is connected to the processing system 1.

[0074] The wearable terminal Wt is configured such that, at least when connected to the processing system 1, it is worn on the face by the user Us of the system 1 and is capable of hands-free operation. For example... Figure 2As shown, the wearable terminal Wt is an optically transparent electronic device equipped with a display unit 2 and a sensor unit 3, such as an HMD (Head Mounted Display) or smart glass. Furthermore, regarding the wearable terminal Wt, Figure 2 The schematic image shows the appearance of the visual field area Av from the user Us side, corresponding to the portion of one eye in the user Us's eyes.

[0075] like Figure 1 As shown, the wearable terminal Wt controls the transmission and reception of data via the communication network Nc through the communication unit. Based on control commands from system 1 received via the communication network Nc, the wearable terminal Wt is controlled by the control unit to perform service association processing in conjunction with the processing system 1.

[0076] Under the action control of the control unit of the wearable terminal Wt, such as virtual image projection type or retinal projection type, etc. Figure 2 The display unit 2 shown above displays a real image of the field of view Av that can be visually confirmed by the user Us through glass or a lens, overlaying it with the image required for service association processing (see below). Figures 7-9 This allows the user Us to visually confirm the action. Simultaneously, under the motion control of the wearable terminal Wt, Figure 2 The sensor unit 3 shown acquires sensing information through sensing processing corresponding to the service association processing. As the sensor unit 3, at least one of the following can be used: a camera, an inertial sensor, a GNSS (Global Navigation Satellite System) sensor, an eye potential sensor, a gaze sensor, an infrared sensor, a geomagnetic sensor, a motion sensor, a touch sensor, and a microphone. With this structure, the sensor unit 3 can acquire input representing the user Us's intention as sensing information.

[0077] Figure 1The autonomous driving device Ma shown is an automated vehicle or robot capable of autonomously moving in any direction—forward, backward, left, or right—by electrically driving its wheels 5 based on sensing information from sensor unit 4. The autonomous driving device Ma controls the transmission and reception of data via communication network Nc through a communication unit. Based on control commands from system 1 received via communication network Nc, the autonomous driving device Ma is driven and controlled by the control unit to jointly perform service association processing with processing system 1. Under this drive control, sensor unit 4 acquires sensing information through sensing processing corresponding to the service association processing. Sensor unit 4 may be, for example, at least one of the following: a camera, an inertial sensor, a GNSS sensor, LiDAR (Light Detection and Ranging / Laser Imaging Detection and Ranging), and sonar.

[0078] Infrastructure system 6, serving as the common foundation of the distributed architecture, is the basic system that shares the 3D spatial information Ti accumulated in the infrastructure database Di. Infrastructure system 6 controls the transmission and reception of data via the communication network Nc through communication units. Infrastructure system 6 continuously collects the 3D spatial information Ti provided to individual distributed systems such as processing system 1 that constitute the distributed architecture through control units, and updates the accumulated information in infrastructure database Di with the latest information.

[0079] Here, in particular, the infrastructure system 6 will be used as an object for information accumulation in three-dimensional space, such as Figure 3 As shown, the system is virtually divided into multiple imaginary three-dimensional voxels Vi (i.e., a three-dimensional mesh) arranged in a three-dimensional pattern and its information is managed. Infrastructure system 6 assigns each spatial ID to each voxel Vi as three-dimensional spatial information Ti through a dataset associated with metadata, such as... Figure 1 As shown, this information is accumulated in the infrastructure database Di. Here, the three-dimensional spatial information Ti can also include, for example, two-dimensional raster information associated only with the lower surface, of the voxels Vi that constitute the two-dimensional arrangement along the ground in the three-dimensional arrangement.

[0080] The data used to construct the three-dimensional spatial information Ti can also be collected from at least one of the wearable terminal Wt shared with the processing system 1, other wearable terminals, and mobile terminals such as smartphones or tablets. The data used to construct the three-dimensional spatial information Ti can also be collected from at least one of the autonomous driving device Ma shared with the processing system 1 and other mobile bodies.

[0081] The data used to construct the 3D spatial information Ti can also be collected from at least one of the following: infrastructure sensors equipped with cameras and / or LiDAR, such as communication base stations, smart poles, and smart streetlights. The data used to construct the 3D spatial information Ti can also be collected from servers that provide at least one of the following: map services, weather services, communication services, traffic management services, feature management services, and aviation management services, as well as user services (described later) processed by processing system 1.

[0082] The construction data for the three-dimensional spatial information Ti collected in this way can be, for example, at least one type of image data such as video, still images, and point group images. The construction data for the three-dimensional spatial information Ti can also be at least one type of secondary data generated by image processing of such image data within the information accumulation object space for a person including the user Us of the processing system 1, such as location data, motion data, posture data, gaze data, action data, and crowd flow data, as well as intent data representing the intent of the user Us (described later), while ensuring information security.

[0083] The data used to construct the three-dimensional spatial information Ti can also be at least one type of secondary data, such as position data, motion data, and posture data, generated by image processing of image data of a wearable terminal Wt containing a user Us within the information accumulation object space. Alternatively, the data used to construct the three-dimensional spatial information Ti can also be at least one type of secondary data, such as position data, motion data, and posture data, generated by image processing of image data of a moving body containing an autonomous driving device Ma within the information accumulation object space.

[0084] The data used to construct the three-dimensional spatial information Ti can also be sound data containing the voices of users Us in the processing system 1, collected within the information accumulation object space. The data used to construct the three-dimensional spatial information Ti can also be at least one type of secondary data that ensures information security, such as location data, motion data, action data, crowd flow data, conversation data, and intent data representing the intentions of users Us (described later), which has been analyzed through sound recognition processing of such sound data for users Us.

[0085] The construction data for the three-dimensional spatial information Ti can also be at least one of the following: two-dimensional and / or three-dimensional map data, GIS (Geographic Information System) data, road network data, meteorological data, communication data, route data, traffic data, feature management data, BIM (Building Information Modelling) data, POI (Point of Interest) data, aviation management data, and time data. The construction data for the three-dimensional spatial information Ti can also be service data associated with at least one of the user services processed by processing system 1, such as guidance services, transport services, and photography services.

[0086] like Figure 1 As shown, the processing system 1 is a distributed computer system comprising a communication system 10 and a control system 20, and for example, includes at least one of cloud servers and edge servers. At least a portion of each system 10, 20 in the processing system 1 may also be composed of a communication unit and a control unit of a wearable terminal Wt. At least a portion of each system 10, 20 in the processing system 1 may also be composed of a communication unit and a control unit of an autonomous driving device Ma. At least a portion of each system 10, 20 in the processing system 1 may also be composed of a communication unit and a control unit of the infrastructure system 6.

[0087] The communication system 10 is primarily composed of communication equipment used to construct the communication network Nc. The control system 20 is connected to the communication system 10 via at least one wired communication line and one wireless communication line. The control system 20 is constructed by including at least one dedicated computer. The dedicated computer constructing the control system 20 has at least one memory 22 and one processor 24.

[0088] In such a processing system 1, the control system 20 executes multiple instructions of a processing program stored in the memory 22 via the processor 24. Thus, the control system 20 constructs multiple functional blocks for performing service-related processing associated with user services provided to the user Us. For example... Figure 4 As shown, the constructed functional blocks include an identification block 200, a search block 210, and a drive control block 220.

[0089] Processing system 1 uses these blocks 200, 210, and 220 together to execute the service association processing method, according to... Figure 5 , 6The processing flow shown is executed. This processing flow is executed in response to the sensor unit 3 of the wearable terminal Wt acquiring service request input representing the user Us's intention to request user services as sensing information. Furthermore, each "S" in this processing flow represents a multiple step executed by the multiple instructions included in the processing procedure of the first embodiment.

[0090] In processing step S100, the identification block 200 (refer to...) Figure 4 The visual field area Av is identified by the user Us through the wearable terminal Wt. At this time, the visual field area Av is defined as an object in the information accumulation space of the infrastructure system 6, such as in a station, terminal station, facility entrance, or landmark. Figure 2 As shown, the overlap between the standby area Aw and the standby area of ​​multiple autonomous driving devices Ma is identified.

[0091] Therefore, in S100, if the autonomous driving device Ma is not present in the standby area Aw that overlaps with the field of view area Av, the current execution of the processing flow ends. In this case, an XR notification image can be overlaid on the field of view area Av to notify of the unfavorable search result caused by the absence of an available autonomous driving device Ma.

[0092] The recognition processing of the field of view region Av in S100 is based on sensing information from at least one of the sensor units 3 of the wearable terminal Wt, such as camera information and inertial information. As a result, the field of view region Av is recognized as an area extending in the orientation of the user Us's face or gaze, or in the orientation of the wearable terminal Wt.

[0093] exist Figure 5 In S110 shown, the identification block 200 obtains the desired content Cd of the user service desired by user Us (see reference). Figure 7 At this time, the desired content Cd is identified by sensing information about the face of the wearable terminal Wt worn by the user Us. Therefore, when the sensing information used to identify the desired content Cd is input into the sensor unit 3 as the user Us's intention data, this information is acquired by the wearable terminal Wt.

[0094] As input to the desired content Cd in S110, gesture input, gaze input, facial expression input, or voice input from user Us can also be sensed by sensor unit 3. At this time, the display unit 2 of the wearable terminal Wt can also be an XR receiving image Ir used to receive the input of the desired content Cd from user Us via sensor unit 3, such as... Figure 7As shown, it is displayed overlapping with the field of view area Av. Furthermore, it is desired that the identification of content Cd can also be achieved based on the three-dimensional spatial information Ti obtained from the infrastructure database Di via the communication system 10.

[0095] In the projects identified as desired content Cd in S110, such as Figure 7 As shown, it includes at least a category of user services that can be provided to the user Us of the processing system 1. As a category of user services corresponding to the desired content Cd, at least one of the following is prepared: guidance service, transport service, and photography service.

[0096] In S110, if the category of the identified desired content Cd is any type of service, at least one voice-related item may be included in the content Cd, such as whether or not voice output is needed, whether or not voice recognition function is needed, and whether or not dialogue function is needed. When the category of the desired content Cd is either a guidance service or a transport service, at least one path-related item may be included in the content Cd, such as the route to the destination, movement speed, arrival time, and service provision time (i.e., guidance time or transport time).

[0097] When the desired category of content Cd is guided service, for destinations such as tourist attractions, at least one of the following tourism-related items may be included in the content Cd: permissible crowding level, availability of desired facilities, and availability of nature. When the desired category of content Cd is guided service, optional items representing other service categories and expressing the desired service may also be included in the content Cd as an optional feature.

[0098] Here, in the optional items in the desired content Cd, which represent the handling service as a handling function in the guidance service, at least one baggage-related item may be included, such as the number of bags, size, weight, type, and whether temperature management is required. The same applies to handling services that are independent of the guidance service.

[0099] The optional items in the desired content Cd, representing the photography service as a photography function within the guidance service, may include at least one photography-related item such as photography schedule, photography time, photography location, whether image or data transmission is required, and whether photo printing is required. The same applies to cases involving photography services independent of the guidance service.

[0100] exist Figure 5 In S120 shown, search block 210 (refer to...) Figure 4The autonomous driving device Ma, which is in standby mode in standby area Aw within the field of view Av identified by S100, acquires the capability to provide user services (see reference) in association with the capability to provide user services Pc. Figure 8 The service capability information can be the provision capability Pc required to provide user services of the category prepared in processing system 1, and the information on the holding capabilities of the autonomous driving devices Ma that are in standby areas Aw within the field of view Av.

[0101] Here, the Pc providing various services may also include at least one of the following driving-related capabilities: remaining battery capacity, remaining driving range, maximum driving speed, minimum driving road width, driving road surface condition, and obstacle avoidance level corresponding to congestion level. The Pc providing various services may also include at least one of the following voice-related capabilities: whether voice output is available, whether voice recognition function is available, and whether dialogue function is available.

[0102] The provision capability Pc for the handling service provided as a handling function in the guidance service may also include at least one of the following handling-related capabilities: the presence or absence of a loading compartment, the size of the loadable items, the total weight of the loadable luggage, the types of luggage that can be loaded, the presence or absence of refrigeration function, and the presence or absence of refrigeration function. The same applies to the provision capability Pc for the handling service independent of the guidance service.

[0103] The photography-oriented service provisioning capability PC provided as part of the bootstrapping service can also include at least one photography-related capability, such as the presence or absence of a camera function, the presence or absence of image or video data transmission function, and the presence or absence of photo printing function. The same applies to the photography-oriented service provisioning capability PC independent of the bootstrapping service.

[0104] In S120, service capability information associated with such provision capability Pc can also be obtained via communication system 10 from the communication unit of the autonomous driving device Ma, which is in standby area Aw within the field of view Av. Service capability information can also be obtained based on three-dimensional spatial information Ti obtained via communication system 10 from infrastructure database Di. Service capability information can also be obtained by capturing a QR code (registered trademark) displayed on the display unit of the autonomous driving device Ma, which is in standby area Aw within the field of view Av, using a sensor unit 3 such as a camera in the wearable terminal Wt.

[0105] exist Figure 5In S130, search block 210 searches for autonomous driving devices Ma whose provisioning capabilities Pc match the desired content Cd obtained in S110, based on the service capability information obtained by the autonomous driving device Ma within the field of view Av in S120. At this time, the matching degree R of the provisioning capability Pc relative to the desired content Cd is obtained (refer to...). Figure 8 ).

[0106] In S130, regarding the ability Pc to provide user services for the category represented by the desired content Cd, a matching degree R is obtained as an indicator related to the fit rate obtained by rating each item other than the category contained in the content Cd. For example, as shown in the following mathematical formula 1, by utilizing the maximum fit rate rm of each item... i The cumulative value of the index is used to score each item of index i to obtain the fitness rate r. i The matching degree R can be obtained by normalizing the cumulative value. Furthermore, the matching degree R can be calculated by adjusting the compliance rate r for each item. i The cumulative value can be obtained by standardizing the calculation, or by inputting the compliance rate r of each project into a machine learning model. i To obtain.

[0107] [Mathematical Expression 1]

[0108]

[0109] In S130, the autonomous driving device Ma, whose matching degree R obtained in this way falls within the recommendation range, is extracted as the search result of the matching device Mam, which holds the providing capability Pc that matches the desired content Cd. The recommendation range is defined as a range where the suitability rate is above or below a threshold but below the maximum value, so that the matching degree R relative to the desired content Cd becomes a range that can be recommended to the user Us. Here, the threshold for determining the lower limit of the recommendation range, for example, when the matching degree R is obtained through the above-described standardization, can be set to a specific value between 50% and 80% (when the maximum value is standardized to 100%), or a specific value between 0.5 and 0.8 (when the maximum value is standardized to 1).

[0110] exist Figure 5 In S140 shown, search block 210 makes S130 pass through as shown Figure 8 As shown, the XR-emphasized image Ipe of the matching device Ma, which emphasizes the search, is overlaid on the display unit 2 of the wearable terminal Wt relative to the field of view Av. Therefore, in S140, the user-oriented service provision capability Pc corresponding to the desired content Cd obtained in S110 is extracted from the provision capability Pc represented by the service capability information obtained by the autonomous driving device Ma in S120.

[0111] In S140, such as Figure 8 As shown, the XR capability image Ip, as a candidate for the XR emphasis image Ipe, is overlaid on the field of view Av by the display unit 2 to notify the autonomous driving device Ma of the provision capability Pc corresponding to the desired content Cd for user services. Specifically, in order to notify the matching degree R obtained through S130 along with the provision capability Pc, the XR capability image Ip is aligned and overlaid on the corresponding autonomous driving device Ma within the field of view Av. Therefore, the overlaid display position of the XR capability image Ip is adjusted, for example, based on sensing information from at least one of the sensor units 3 and 4 and / or three-dimensional spatial information Ti obtained from the infrastructure database Di, so that its position around the autonomous driving device Ma is visually confirmed. At this time, the field of view Av of the overlaid display of the XR capability image Ip can be fixed to the area identified through S100, or updated according to S100 based on the viewpoint movement of the user Us.

[0112] In S140, the XR capability image Ip that is overlaid on the XR capability image Ip in the field of view area Av, and which emphasizes the matching device Mam whose matching degree R between the provided capability Pc and the desired content Cd is within the recommended range, is set as an XR emphasis image Ipe. Therefore, for the XR capability image Ip set for the XR emphasis image Ipe, for XR capability images Ip that are not set for the XR emphasis image Ipe, they are emphasized by at least one difference related to the text display and / or the window display as its background, such as color, size, and line width.

[0113] In S140, if fewer matching devices Mam than the number of autonomous driving devices Ma present in the field of view Av are found in S130, the XR capability images Ip, both set and unset, are overlaid on the XR emphasis image Ipe. Conversely, in S140, if the same number of matching devices Mam as the number of autonomous driving devices Ma present in the field of view Av are found in S130, the XR emphasis image Ipe only overlays the set XR capability images Ip. Furthermore, in S140 where no matching device Mam is found in S130, the XR emphasis image Ipe only overlays the unset XR capability images Ip.

[0114] exist Figure 5In S150, as shown, the search block 210, in response to the overlapping display performed in S140, determines whether the user Us has selected any autonomous driving device Ma within the field of view Av. At this time, the presence or absence of the user Us's selection is identified by facial sensing information from the wearable terminal Wt. Therefore, the sensing information used to identify the presence or absence of the user Us's selection is input as the user Us's intention data to the sensor unit 3, and thus acquired by the wearable terminal Wt.

[0115] As the selection input for the autonomous driving device Ma in S150, the sensor unit 3 can also sense the user Us's gesture input, gaze input, facial expression input, or voice input. The selection input can also be an input selecting the location of the autonomous driving device Ma, or an input selecting the image Ip or Ipe corresponding to the autonomous driving device Ma. Any selection input here can be used to identify the selection location based on, for example, sensing information from at least one of the sensor units 3 and 4 and / or three-dimensional spatial information Ti obtained from the infrastructure database Di. Furthermore, the presence or absence of the user Us's selection and the identification of the selection location can also be achieved based on the three-dimensional spatial information Ti obtained from the infrastructure database Di through the communication system 10.

[0116] In S150, if the user Us's selection is recognized and affirmatively determined, such as Figure 5 As shown, S160 is executed. In S160, the drive control block 220 (refer to...) Figure 4 By controlling the drive within the field of vision Av of the autonomous driving device Ma selected via S150, user services of the category corresponding to the desired content Cd are provided. At this time, by reading the drive control mode of the autonomous driving device Ma that matches the desired content Cd from memory 22, user services are provided according to that drive control mode.

[0117] Here, the guidance service performed in S160 is provided through the following drive control mode: the autonomous driving device Ma is driven from the standby position to the current location of the guided user Us or a registered family member, and after executing path guidance to the destination, it returns to the standby area Aw. During this guidance service, the path for guiding the guided user can also be indicated by overlaying an XR prompt image in the field of view Av of the wearable terminal Wt, or by display output from the display unit held by the autonomous driving device Ma. The path for guiding the guided user can also be indicated by sound output from the sound unit of the wearable terminal Wt or the autonomous driving device Ma.

[0118] The handling service performed by S160 is provided through the following drive control mode: the autonomous driving device Ma, which holds the loading chamber, is driven from the standby position to the current position of the user Us. After accepting the luggage from the user Us to be loaded into the loading chamber, the luggage is transported to its destination and then returned to the standby area Aw. During this handling service, the user Us can also be prompted with the loading method of the luggage into the loading chamber by overlaying an XR prompt image onto the field of view area Av in the wearable terminal Wt, or by display output from the display unit held by the autonomous driving device Ma. During the handling service, the user Us can also be prompted with the loading of luggage into the loading chamber by sound output from the sound unit held by the wearable terminal Wt or the autonomous driving device Ma. Such a handling service can be provided as a handling function of a guided service when the desired content Cd is included.

[0119] The photography service provided by S160 is offered by the following drive control mode: the autonomous driving device Ma, carrying the camera unit, is driven from its standby position to the current location of the user Us or a landmark near the user Us, takes photos of the user Us from the surroundings, and then returns to the standby area Aw. During this time, the photography service can also alert the user Us to the photography schedule or time by overlaying XR cue images onto the field of view Av in the wearable terminal Wt, or by display output from the display unit held by the autonomous driving device Ma. The photography service can also alert the user Us to the photography schedule or time by sound output from the sound unit held by the wearable terminal Wt or the autonomous driving device Ma.

[0120] During the photography service conducted in S160, the autonomous driving device Ma, which drives around the user Us, can also capture images or videos containing the user Us in two-dimensional or three-dimensional form. During the photography service, the focus during photography can also be adjusted based on the distance from the user Us sensed by a sensor unit 4, such as LiDAR, within the autonomous driving device Ma.

[0121] In the photography service performed by S160, the direction of external light incidence during photography can be adjusted based on three-dimensional spatial information Ti obtained from the infrastructure database Di, or sensing information from sensor units 4 such as cameras in the autonomous driving device Ma. Image data captured by the user Us during the photography service can also be stored in the memory 22 of the control system 20 or in the storage medium within the wearable terminal Wt. The above photography service can also be provided as a photography function included in a guided service when desired content Cd is desired.

[0122] As user services are provided based on the drive control of such an autonomous driving device Ma, the viewpoint of user Us moves continuously. Therefore, in S160, by updating the field of view area Av according to S100, the drive of the autonomous driving device Ma, which provides user services, can be implemented at any time within the updated field of view area Av.

[0123] Therefore, in S150 above, for example, if the selection input by user Us is not executed during the elapsed period from the start of execution to the set time, or if user Us indicates that they do not wish to select the autonomous driving device Ma, a negative determination is made. In this case, it is determined that the selection made by user Us is rejected by the autonomous driving device Ma within the field of view Av, and thus... Figure 5 , 6 Perform S170 as shown.

[0124] exist Figure 6 In S170 shown, search block 210 determines whether an autonomous driving device Ma is waiting outside the identified field of view Av at a distance set by the user Us. At this time, the presence or absence of the autonomous driving device Ma at the set distance outside the field of view Av can be identified by using the three-dimensional spatial information Ti obtained from the infrastructure database Di through the communication system 10.

[0125] If a positive determination is made in S170, such as Figure 6 As shown, S180 is executed. In S180, search block 210 associates the service capability information representing the user service provision capability Pc with the autonomous driving device Ma identified in S170 that is idle outside the field of view Av, and obtains it as additional capability information. At this time, the acquisition of additional capability information is realized according to S120.

[0126] exist Figure 6 In S190 shown, search block 210 searches for a matching device Mam that matches the additional capability information provided by the autonomous driving device Ma outside the field of view Av obtained in S180 with the desired content Cd obtained in S110. The search at this time is also based on the matching degree R obtained according to S130.

[0127] exist Figure 6 In S200, search block 210 notifies the provision of capability Pc represented by the additional capability information obtained by the autonomous driving device Ma outside the field of view Av via S180. In S200, in order to implement such notification based on the XR capability image Ip, Figure 9 The additional capability image Ia shown is overlaid on the display unit 2 of the wearable terminal Wt relative to the field of view Av.

[0128] In S200, in particular, in order to notify the matching degree R obtained through S190 along with the provided capability Pc, the additional capability image Ia is displayed overlaid with, for example, an air region or a ground region in the field of view Av. Here, the additional capability image Ia may also indicate at least one of the following regarding the location of the corresponding autonomous driving device Ma: the direction from user Us, the distance from user Us, and the arrival time to user Us by calling. For the matching device Mam searched through S190 (in... Figure 9 In the example above, the autonomous driving device Ma, whose location is displayed as "robot 3m to the right," can have its additional capability image Ia set as an additional emphasis image Iae emphasizing device Ma, based on the XR emphasis image Ipe. Such an additional capability image Ia can be overlaid on a fixed relative position within the field of view Av relative to the glass or lens of display unit 2, or it can be overlaid on an XR capability image Ip (including the XR emphasis image Ipe relative to the matching device Ma) at a fixed relative position within the field of view Av relative to the autonomous driving device Ma. In either case, the field of view Av overlaid with the additional capability image Ia can be fixed to the area identified by S100, or updated according to S100 based on the viewpoint movement of user Us.

[0129] exist Figure 6 In S210, the search block 210, in response to the overlapping display performed in S200, determines in S150 whether the user Us has selected any autonomous driving device Ma outside the field of view Av. As a result, if a positive determination is made by recognizing the user Us's selection, S220 is executed. In S220, the drive control block 220 controls the autonomous driving device Ma selected in S210 to drive into the field of view Av, so as to provide user services corresponding to the desired content Cd according to S160. In other words, in S220, by controlling the drive of the selected autonomous driving device Ma into the field of view Av, the service of the desired content Cd can be provided.

[0130] The above Figure 5 , 6 The processing flow shown completes when the provision of the user service corresponding to the desired content Cd ends in either S160 or S220. Conversely, if a negative decision is made in either S170 or S210, the current execution of the processing flow completes after an XR notification image of the negative search result is overlaid on the visual field Av. In either of these cases, after the current execution is completed, the next service request input is obtained through the sensor unit 3 of the wearable terminal Wt, and the next execution of the processing flow begins.

[0131] (Effects)

[0132] The effects of the first embodiment described above will be explained below.

[0133] According to the first embodiment, service capability information representing the user service provision capability Pc is acquired in association with an autonomous driving device Ma that is waiting in the field of vision Av, which is visually confirmed by the user Us via a wearable terminal Wt worn by the user Us. Therefore, by acquiring the desired content Cd of the user service desired by the user Us, for the autonomous driving device Ma within the field of vision Av, the system focuses on autonomous driving devices Ma whose provision capabilities Pc match the desired content Cd from the service capability information. Thus, an XR emphasis image Ipe obtained by searching for and emphasizing the autonomous driving device Ma whose capability Pc matches the desired content Cd is overlaid on the field of vision Av, facing the user Us's selection of that device Ma. Therefore, the search efficiency for autonomous driving devices Ma capable of providing user services that meet user preferences through driving within the field of vision Av can be improved.

[0134] Furthermore, according to the first embodiment, the XR capability image Ip is displayed in alignment with the autonomous driving device Ma relative to the field of view Av, so that the autonomous driving device Ma within the field of view Av is notified of the provision capability Pc corresponding to the desired content Cd. Therefore, the XR capability image Ip that emphasizes the autonomous driving device Ma (specifically the matching device Ma) whose matching degree R with the provision capability Pc of the desired content Cd is within the recommended range in the XR capability image Ip of each autonomous driving device Ma is set as the XR emphasis image Ipe. As a result, the user Us can appropriately select the autonomous driving device Ma automatically searched based on the matching degree R between the desired content Cd and the provision capability Pc among the autonomous driving devices Ma within the field of view Av that has been notified of the provision capability Pc, even without prior reservation, for example. Therefore, not only can the search efficiency of autonomous driving devices Ma that can provide user services that meet user preferences be improved, but the search accuracy can also be improved by suppressing the gap between the desired content Cd and the provision capability Pc.

[0135] Here, according to the first embodiment, in order to notify the matching degree R along with the provision capability Pc, the XR capability image Ip is displayed overlaid relative to the field of view Av. Therefore, when the user Us notifies the autonomous driving device Ma providing the provision capability Pc within the field of view Av, they can intuitively select the autonomous driving device Ma that is emphasized along with the matching degree notification of the provision capability Pc relative to the desired content Cd. Thus, reliable and efficient searching for autonomous driving devices Ma that can provide user services that meet the user's preferences is achieved.

[0136] Furthermore, according to the first embodiment, when it is determined that the user Us's selection has been rejected by the autonomous driving device Ma within the field of view Av, the service capability information of the autonomous driving device Ma waiting outside the field of view Av is obtained as additional capability information. Therefore, the additional capability image Ia representing the provision capability Pc indicated by the additional capability information is overlaid on the field of view Av. Thus, even if no autonomous driving device Ma providing user services that meet the user's preferences is found within the field of view Av, the search results for that device Ma, which extends beyond the field of view Av, can be notified to the user Us. Therefore, fault safety during the efficient search of the autonomous driving device Ma can be ensured, giving the user Us peace of mind.

[0137] Furthermore, according to the first embodiment, the autonomous driving device Ma, selected by the user Us in response to the overlay display of the additional capability image Ia, is driven into the field of vision Av to provide user services. Thus, by selecting the autonomous driving device Ma, which is notified through efficient searching outside the field of vision Av, the user Us can receive user services tailored to their preferences from the device Ma, which is automatically invoked into the field of vision Av.

[0138] (Second Implementation)

[0139] The second embodiment is a variation of the first embodiment.

[0140] like Figure 10 As shown, the processing system 2001 of the second embodiment is connected via a communication network Nc to a pre-registered user Us-carried mobile terminal Mt, an autonomous driving device Ma, and an infrastructure system 6 that provides infrastructure information. Here, multiple terminals are assumed to be the mobile terminal Mt connected to the processing system 1.

[0141] The mobile terminal Mt is configured such that, at least when connected to the processing system 2001, the user Us of the system 2001 can hold and operate it with their fingers. For example... Figure 11 As shown, the mobile terminal Mt is a small electronic device equipped with a display unit 2002 and a sensor unit 2003, such as a smartphone or tablet terminal. The mobile terminal Mt is as follows... Figure 10 As shown, the transmission and reception of data via the communication network Nc are controlled by the communication unit. The mobile terminal Mt, based on control commands from the system 2001 via the communication network Nc, is controlled by the control unit to jointly perform service association processing with the processing system 2001.

[0142] Under the action control of the control unit of the mobile terminal Mt, such as the LCD panel or OLED panel, etc. Figure 11The display unit 2002 shown implements the screen display required for service association processing (see below). Figure 15 , 16 Meanwhile, under the motion control of the mobile terminal Mt, Figure 11 The sensor unit 2003 shown acquires sensing information through sensing processing corresponding to service association processing. As sensor unit 3, at least one of the following is used: a camera, an inertial sensor, a GNSS sensor, a touch sensor, and a microphone. With this structure, sensor unit 2003 can acquire input representing the user Us's intention as sensing information.

[0143] Figure 10 The autonomous driving device Ma shown is configured similarly to the processing system 1 of the first embodiment, except that it is driven and controlled by the control unit according to control commands from the system 2001 via the communication network Nc to jointly perform service association processing with the processing system 2001. The control system 2020 of the processing system 2001 is configured similarly to the control system 20 of the first embodiment, except that at least a portion of it can be composed of the control unit of the mobile terminal Mt, and that it executes the processing flow using the mobile terminal Mt described later.

[0144] In order to perform service-related processing in Control System 2020, according to Figure 13 , 14 The processing flow shown is as follows: Figure 12 The blocks 200, 210, and 220 constructed as shown jointly execute the processing method of the service-related processing system 2001.

[0145] In processing step S2100, identification block 200 (refer to...) Figure 12 The background region Ab, which is the display background displayed from the display unit 2002 to the user Us, is identified via the mobile terminal Mt carried by the user Us. At this time, the background region Ab is identified as a region Ab extending along the orientation of the mobile terminal Mt based on sensing information from at least one of the sensor units 2003 of the mobile terminal Mt, such as camera information and inertial information. In particular, the background region Ab can be identified based on the field of view Av identified via S100 of the first embodiment, such as... Figure 11 As shown, it is identified as overlapping with the standby area Aw where multiple autonomous driving devices Ma can standby. The processing in the case where no autonomous driving device Ma is present in the standby area Aw that overlaps with the background area Ab is based on S100 of the first embodiment.

[0146] exist Figure 13In S2110 shown, the identification block 200 obtains the content Cd desired by the user Us based on the sensing information of the mobile terminal Mt carried by the user Us. This sensing information is obtained through the mobile terminal Mt at the moment the user Us's intention data is input into the sensor unit 2003, and is thus used for the identification of the desired content Cd. The items of the desired content Cd and their input are based on S110 of the first embodiment.

[0147] exist Figure 13 In S2120 shown, search block 210 (refer to...) Figure 12 The autonomous driving device Ma, which is in standby mode in the standby area Aw within the background area Ab identified by S2100, acquires the capability to provide user services Pc (see reference). Figure 15 The service capability information of the Pc is provided. Details of the capability provided are based on S120 of the first embodiment.

[0148] exist Figure 13 In S2130 shown, search block 210 searches for autonomous driving devices Ma whose provisioning capabilities Pc match the desired content Cd obtained in S2110 from the service capability information obtained by the autonomous driving device Ma in the background region Ab via S2120. Matching degree R (see...) Figure 15 The acquisition of ) and the extraction of the matching device Mam based thereon are based on S130 of the first embodiment.

[0149] exist Figure 13 In the S2140 shown, search block 210 causes the display unit 2002 of the mobile terminal Mt to display an XR emphasized image Ipe, which is as follows: Figure 15 As shown, the matching device Mam searched via S2130 is emphasized. At this time, the XR-emphasized image Ipe is overlaid with the background video Ib projected onto the background region Ab. Here, the background video Ib is obtained, for example, based on sensing information from sensor unit 4 such as a camera, or three-dimensional spatial information Ti obtained from the infrastructure database Di. The background video Ib can also use primary video data captured from the background region Ab, or it can use secondary video data processed from that primary data.

[0150] In S2140, such as Figure 15 As shown, the XR capability image Ip, which is a candidate for the XR emphasis image Ipe, is overlaid on the background video Ib by the display unit 2002 to notify the user-oriented service provision capability Pc corresponding to the desired content Cd by the autonomous driving device Ma. In particular, in order to notify the matching degree R obtained by S2130 together with the provision capability Pc, the XR capability image Ip is aligned and overlaid in the background video Ib according to the corresponding autonomous driving device Ma.

[0151] The overlapping display position of the XR capability image Ip in S2140 is adjusted, for example, based on sensing information from at least one of the sensor units 2003 and 4 and / or three-dimensional spatial information Ti obtained from the infrastructure database Di, to a position surrounding the location of the autonomous driving device Ma reflected in the background video Ib. At this time, the background area Ab reflected in the background video Ib of the overlapping display of the XR capability image Ip can be fixed to the area identified by S2100, or updated according to S2100 based on the viewpoint movement of the user Us. In the above S2140, the extraction of the provision capability Pc corresponding to the desired content Cd, and the setting and display of images Ip and Ipe, are based on S140 of the first embodiment.

[0152] exist Figure 13 In S2150, as shown, search block 210, in response to the overlapping display performed in S2140, determines whether user Us has selected any autonomous driving device Ma within the background area Ab. At this time, the presence or absence of user Us's selection is identified based on sensor information from mobile terminal Mt. Therefore, the sensor information used to identify the presence or absence of user Us's selection is input as user Us's intention data to sensor unit 2003, and thus acquired through mobile terminal Mt. The details of the selection input for autonomous driving device Ma are based on S150 of the first embodiment.

[0153] In S2150, if the user Us's selection is recognized and affirmatively determined, such as Figure 13 As shown, S2160 is executed. In S2160, the drive control block 220 (refer to...) Figure 12 By controlling the drive within the background area Ab of the autonomous driving device Ma selected via S2150, user services of the category corresponding to the desired content Cd are provided. Details of the provided user services and drives are based on S160 of the first embodiment.

[0154] In S2150 described above, for example, if the selection input made by the user Us is not executed during the elapsed period from the start of execution to the set time, or if the user Us inputs an intention not to select the autonomous driving device Ma, a rejection determination is made. In this case, it is determined that the selection made by the user Us is rejected by the autonomous driving device Ma in the background area Ab, and thus... Figure 13 , 14 Perform S2170 as shown.

[0155] exist Figure 14 In S2170 shown, search block 210 determines whether an autonomous driving device Ma is waiting outside the identified background area Ab at a distance set by the user Us. The determination of presence or absence is based on S170 of the first embodiment.

[0156] If a positive determination is made in S2170, such as Figure 14 As shown, S2180 is executed. In S2180, the search block 210 associates the autonomous driving device Ma, which was identified in S2170 and is waiting outside the background area Ab, and obtains additional capability information representing the user service provision capability Pc according to S120 and S180 of the first embodiment.

[0157] exist Figure 14 In S2190 shown, search block 210 searches for a matching device Mam that matches the provision capability Pc of the additional capability information obtained by the autonomous driving device Ma outside the background region Ab in S2180 with the desired content Cd obtained in S2110. The search at this time is based on S190 of the first embodiment, which is based on the matching degree R.

[0158] exist Figure 14 In S2200, search block 210 notifies the provision of capability Pc represented by the additional capability information obtained by the autonomous driving device Ma outside the background region Ab via S2180. In S2200, in order to implement such notification based on the XR capability image Ip, Figure 16 The additional capability image Ia shown is overlaid on the background video Ib of the background area Ab by the display unit 2002 of the mobile terminal Mt.

[0159] In S2200, in particular, in order to notify the matching degree R obtained through S2190 along with the provided capability Pc, the additional capability image Ia is overlaid on a background video Ib, such as an air area or a ground area. Here, the additional capability image Ia may also indicate at least one of the following regarding the location of the corresponding autonomous driving device Ma: the direction from user Us, the distance from user Us, and the arrival time to user Us via a call. For the matching device Mam found through S2190 (in... Figure 16In the example above, the autonomous driving device Ma, whose location is displayed as "robot 3m to the right," can have its additional capability image Ia set as an additional emphasis image Iae emphasizing device Ma, based on the XR emphasis image Ipe. Such an additional capability image Ia can be overlaid on the screen of the display unit 2002 at a fixed relative position relative to the background video Ib reflecting the background area Ab, or it can be overlaid on the XR capability image Ip (including the XR emphasis image Ipe relative to the matching device Ma) at a fixed relative position relative to the autonomous driving device Ma reflected in the background video Ib reflecting the background area Ab. In either case, the background area Ab reflected in the background video Ib overlaid with the additional capability image Ia can be fixed to the area identified by S2100, or updated according to S2100 based on the viewpoint movement of the user Us.

[0160] exist Figure 14 In S2210, the search block 210, in response to the overlapping display performed in S2200, determines in S2150 whether the user Us has selected any autonomous driving device Ma outside the background area Ab. As a result, if a positive determination is made by recognizing the user Us's selection, S2220 is executed. In S2220, the drive control block 220 controls the autonomous driving device Ma selected in S2210 to drive into the background area Ab, so as to provide user services corresponding to the desired content Cd according to S2160. In other words, in S2220, by controlling the drive of the selected autonomous driving device Ma into the background area Ab, the service of the desired content Cd can be provided.

[0161] The above Figure 13 , 14 The processing flow shown ends when the provision of the user service corresponding to the desired content Cd is completed in either S2160 or S2220. Conversely, if a negative decision is made in either S2170 or S2210, the current execution of the processing flow is complete after an XR notification image of the negative search result is overlaid on the background video Ib of the background area Ab. In either of these cases, after the current execution ends, the next service request input is obtained through the sensor unit 2003 of the mobile terminal Mt, and the next execution of the processing flow begins.

[0162] (Effects)

[0163] The effects of the second embodiment described above will be explained below.

[0164] According to the second embodiment, service capability information representing the user service provision capability Pc is obtained in association with an autonomous driving device Ma that is idle in a background area Ab, which is the display background shown to the user Us by a mobile terminal Mt carried by the user Us. Therefore, by obtaining the desired content Cd of the user service desired by the user Us, for the autonomous driving device Ma in the background area Ab, the autonomous driving device Ma whose provision capability Pc matches the desired content Cd is selected from the provision capabilities Pc represented by the service capability information. Accordingly, an XR emphasized image Ipe obtained by searching for an autonomous driving device Ma (specifically a matching device Mam) whose capability Pc matches the desired content Cd and emphasizing it is overlaid on the background video Ib reflected in the background area Ab and displayed to the user Us's selection of the device Ma. Therefore, the search efficiency of the autonomous driving device Ma that can provide user services that meet the user's preferences by driving in the background area Ab can be improved.

[0165] Furthermore, according to the second embodiment, the XR capability image Ip is displayed in alignment with the background video Ib of the region Ab, overlaid with respect to the autonomous driving device Ma, so that the autonomous driving device Ma within the background region Ab is notified of the provision capability Pc corresponding to the desired content Cd. Therefore, the XR capability image Ip of the autonomous driving device Ma (specifically the matching device Ma) whose matching degree R with the provision capability Pc of the desired content Cd is within the recommended range in the XR capability image Ip emphasizing each autonomous driving device Ma is set as the XR emphasis image Ipe. As a result, the user Us can appropriately select the autonomous driving device Ma automatically searched based on the matching degree R between the desired content Cd and the provision capability Pc among the autonomous driving devices Ma within the background region Ab that has been notified of the provision capability Pc, even without prior reservation, for example. Therefore, not only can the search efficiency of autonomous driving devices Ma that can meet user preferences be improved, but the search accuracy can also be improved by suppressing the gap between the desired content Cd and the provision capability Pc.

[0166] Here, according to the second embodiment, in order to notify the matching degree R along with the provided capability Pc, the XR capability image Ip is overlaid relative to the background video Ib of the background region Ab. Thus, when the user Us notifies the autonomous driving device Ma providing the provided capability Pc within the background region Ab, they can intuitively select the autonomous driving device Ma that is emphasized along with the matching degree notification of the provided capability Pc relative to the desired content Cd. Therefore, reliable and efficient searching for autonomous driving devices Ma that can provide user services that meet user preferences is possible.

[0167] Furthermore, according to the second embodiment, when it is determined that the user Us's selection was rejected from the autonomous driving device Ma within the background region Ab, the service capability information of the autonomous driving device Ma waiting outside the background region Ab is obtained as additional capability information. Therefore, the additional capability image Ia representing the provision capability Pc indicated by the additional capability information is overlaid on the background video Ib of the background region Ab. Thus, even if no autonomous driving device Ma capable of providing user services that meet the user's preferences is found within the background region Ab, the search results of the device Ma extending outside the background region Ab can be notified to the user Us. Therefore, fault safety during the efficient search of the autonomous driving device Ma can be ensured, giving the user Us peace of mind.

[0168] Furthermore, according to the second embodiment, the autonomous driving device Ma selected by the user Us in response to the overlay display of the additional capability image Ia is driven into the background area Ab to provide user services. Thus, by selecting the autonomous driving device Ma that is notified through efficient searching outside the background area Ab, the user Us can receive user services tailored to their preferences from the device Ma that is automatically called into the background area Ab.

[0169] (Other implementation methods)

[0170] The above describes several embodiments, but this disclosure is not limited to these embodiments. It can be applied to various embodiments and combinations without departing from the spirit of this disclosure.

[0171] In variations of the first and second embodiments, the dedicated computer constituting the control system 20 and 2020 of the processing system 1 and 2001 may also have at least one of digital circuits and analog circuits as a processor. Here, the digital circuit is at least one of, for example, ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array), SOC (System on a Chip), PGA (Programmable Gate Array), and CPLD (Complex Programmable Logic Device). Furthermore, such a digital circuit may also have a memory for storing programs.

[0172] In variations of the first and second embodiments, the processing systems 1 and 2001 may also be non-connected to the infrastructure system 6. In this variation, the three-dimensional spatial information Ti obtained from the infrastructure database Di can be replaced by sensing information from the connection elements Wt, Ma, and Mt of the processing systems 1 and 2001.

[0173] In variations of the first and second embodiments, processing systems 1 and 2001 may also obtain three-dimensional spatial information Ti, which is not associated with voxel Vi, from the infrastructure database Di. In variations of the first and second embodiments, processing systems 1 and 2001 may also obtain two-dimensional mesh information from the infrastructure database Di instead of the three-dimensional spatial information Ti.

[0174] In a variation of the first embodiment, where the mobile terminal Mt is connected to the processing system 1 and functions as part of the wearable terminal Wt, as in the second embodiment, the processing flow may be executed by acquiring a service request input as sensing information from the sensor unit 2003 constituting that part. In another variation of the first embodiment, where the mobile terminal Mt is connected to the processing system 1 and functions as part of the wearable terminal Wt, as in the second embodiment, the desired content Cd may be acquired in step S110 of the processing flow based on the sensing information from the sensor unit 2003 constituting that part.

[0175] In a variation of the second embodiment, a smartwatch, a video-transparent or non-transparent wearable terminal can also be used as a mobile terminal Mt. In another variation of the second embodiment, where the wearable terminal Wt is connected to the processing system 2001 and functions as part of the mobile terminal Mt as in the first embodiment, the processing flow can be executed by acquiring service request input as sensing information from the sensor unit 3 constituting that part. In another variation of the second embodiment, where the wearable terminal Wt is connected to the processing system 2001 and functions as part of the mobile terminal Mt as in the first embodiment, the desired content Cd can be acquired based on the sensing information from the sensor unit 3 constituting that part in step S2110 of the processing flow.

[0176] In variations of the first and second embodiments, by skipping S170-S220 and S2170-2220, the current execution of the processing flow can also end in the case of a negative determination in S150 and S2150. In S140 and S2140 of the variations of the first and second embodiments, the matching degree R can also be removed from the notification object based on the XR capability image Ip.

[0177] In S140 and S2140 of the variations according to the first embodiment and the second embodiment, such as Figure 17 (This figure is a variation of the first embodiment.) As shown, it is also possible to display only the XR capability image Ip of the providing capability Pc of the matching device Mam, that is, the XR capability image Ip set in the XR enhancement image Ipe. However, for such a variation of the display, if the matching device Mam is not found through S130 and S2130, S140 to S160 and S2140 to S2160 can be skipped, and S170 and S2170 can be executed.

[0178] In S200 and S2200 of the variations of the first and second embodiments, such as Figure 18 (This figure is a variation of the first embodiment.) As shown, only the additional capability image Ia of the providing capability Pc of the matching device Mam can be displayed, that is, only the additional capability image Ia set in the additional emphasis image Iae can be displayed. However, for such a variation of the display, if the matching device Mam is not found through S190 and 2190, S200 to S220 and S2200 to S2220 can be skipped, and the current execution of the processing flow can be terminated.

[0179] In addition to the above description, the above-described embodiments and variations can also be implemented as processing systems 1 and 2001, each having at least one processor 24 and one memory 22 in control systems 20 and 2020, in the form of semiconductor devices (e.g., semiconductor chips).

[0180] (Postscript)

[0181] This specification discloses several technical ideas and combinations thereof, as listed below.

[0182] (Technical Idea 1)

[0183] A processing system having a processor (24) that performs service association processing associated with user services provided to users.

[0184] The processor is configured to execute the following:

[0185] Obtain the desired content (Cd) of the user service that the user desires;

[0186] Service capability information representing the user's ability to provide services (Pc) is acquired in association with an autonomous driving device (Ma) that is in standby within the field of view (Av) confirmed by the user's vision through the wearable terminal (Wt) worn by the user.

[0187] An XR-emphasized image (Ipe) is overlaid relative to the field of view. This XR-emphasized image is an image obtained by searching for autonomous driving devices within the field of view that match the desired content from the capabilities provided by the service capability information, and then emphasizing the corresponding autonomous driving devices.

[0188] The user service is provided by driving the autonomous driving device selected by the user in response to the overlapping display of the XR-emphasized image within the field of view.

[0189] (Technical Idea 2)

[0190] In the processing system described in technical concept 1

[0191] The overlaid XR emphasized image includes the following:

[0192] In order to notify the autonomous driving device within the field of view of the provision capabilities corresponding to the desired content, the XR capability image of the autonomous driving device that emphasizes the matching degree (R) of the provision capabilities with respect to the desired content within the recommended range is set as the XR emphasis image in the XR capability image (Ip) that is aligned and overlaid by the autonomous driving device in the field of view.

[0193] (Technical Idea 3)

[0194] In the processing system described in technical concept 2

[0195] The overlaid XR emphasized image includes the following:

[0196] In order to notify the matching degree along with the provided capability, the XR capability image is displayed overlapping with the field of view region.

[0197] (Technical Idea 4)

[0198] In the processing system described in technical concept 2 or 3

[0199] Obtaining the service capability information includes the following:

[0200] When it is determined that the user's selection of the autonomous driving device within the field of view is rejected, the service capability information of the autonomous driving device waiting outside the field of view is obtained as additional capability information.

[0201] The overlaid XR emphasized image includes the following:

[0202] The additional capability image (Ia) representing the provided capability, as indicated by the additional capability information, will be displayed overlapping with the field of view; and

[0203] The autonomous driving device, selected by the user in response to the overlay display of the additional capability images, will drive into the field of view to provide the user service.

[0204] (Technical Idea 5)

[0205] A processing system having a processor (24) that performs service association processing associated with user services provided to users.

[0206] The processor is configured to execute the following:

[0207] Obtain the desired content (Cd) of the user service that the user desires;

[0208] Service capability information representing the ability to provide services to the user is acquired in association with an autonomous driving device (Ma) that is standing still in a background area (Ab) of a display background shown to the user from a mobile terminal (Mt) carried by the user.

[0209] The XR emphasized image (Ipe) is overlaid with the background video (Ib) projected onto the background area. This XR emphasized image is obtained by searching for autonomous driving devices within the background area that match the desired content from the capabilities provided by the service capability information, and then emphasizing the corresponding autonomous driving device.

[0210] The user service is provided by driving the autonomous driving device selected by the user in response to the overlapping display of the XR-emphasized image within the background area.

[0211] (Technical Idea 6)

[0212] In the processing system described in technical concept 5

[0213] The overlaid XR emphasized image includes the following:

[0214] In order to notify the autonomous driving device in the background area of ​​the provision capability corresponding to the desired content, the XR capability image of the autonomous driving device that emphasizes the matching degree (R) of the provision capability with respect to the desired content within the recommended range in the XR capability image (Ip) that is aligned and overlaid by the autonomous driving device with the background video is set as the XR emphasis image.

[0215] (Technical Idea 7)

[0216] In the processing system described in technical concept 6

[0217] The overlaid XR emphasized image includes the following:

[0218] In order to notify the matching degree along with the provided capabilities, the XR capability image is overlaid relative to the background video.

[0219] (Technical Idea 8)

[0220] In the processing system described in technical concept 6 or 7

[0221] Obtaining the service capability information includes the following:

[0222] When it is determined that the user's selection of the autonomous driving device within the background area is rejected, the service capability information of the autonomous driving device waiting outside the background area is obtained as additional capability information.

[0223] The overlaid XR emphasized image includes the following:

[0224] The additional capability image (Ia) representing the provided capability, as indicated by the additional capability information, will be overlaid relative to the background video; and

[0225] The autonomous driving device, selected by the user in response to the overlay display of the additional capability image, will drive into the background area to provide the user service.

[0226] (Technical Idea 9)

[0227] In the processing system described in any one of technical concepts 2-4 and 6-8,

[0228] The overlaid XR emphasized image includes the following:

[0229] Regarding the ability to provide user services in the category represented by the desired content, the matching degree is obtained in relation to the suitability obtained by rating each item outside the category contained in the desired content.

[0230] Furthermore, the aforementioned technical ideas 1 to 9 can also be grasped through the various technical ideas of methods and procedures.

Claims

1. A processing system having a processor (24) for performing service association processing associated with user services provided to a user, characterized in that, The processor is configured to execute the following: Obtain the desired content (Cd) of the user service that the user desires; Service capability information representing the user's ability to provide services (Pc) is acquired in association with an autonomous driving device (Ma) that is in standby within the field of view (Av) confirmed by the user's vision through the wearable terminal (Wt) worn by the user. An XR-emphasized image (Ipe) is overlaid relative to the field of view. This XR-emphasized image is an image obtained by searching for autonomous driving devices within the field of view that match the provided capabilities from the provided capabilities represented by the service capability information, and emphasizing them. as well as The user service is provided by driving the autonomous driving device selected by the user in response to the overlapping display of the XR-emphasized image within the field of view.

2. The processing system according to claim 1, characterized in that, The overlaid XR emphasized image includes the following: In order to notify the autonomous driving device within the field of view of the provision capabilities corresponding to the desired content, the XR capability image of the autonomous driving device that emphasizes the matching degree (R) of the provision capabilities with respect to the desired content within the recommended range is set as the XR emphasis image in the XR capability image (Ip) that is aligned and overlaid by the autonomous driving device in the field of view.

3. The processing system according to claim 2, characterized in that, The overlaid XR emphasized image includes the following: In order to notify the matching degree along with the provided capability, the XR capability image is displayed overlapping with the field of view region.

4. The processing system according to claim 2 or 3, characterized in that, Obtaining the service capability information includes the following: When it is determined that the user's selection of the autonomous driving device within the field of view is rejected, the service capability information of the autonomous driving device waiting outside the field of view is obtained as additional capability information. The overlaid XR emphasized image includes the following: The additional capability image (Ia) representing the provided capability, as indicated by the additional capability information, will be displayed overlapping with the field of view; and The autonomous driving device, selected by the user in response to the overlay display of the additional capability images, will drive into the field of view to provide the user service.

5. A processing system having a processor (24) for performing service association processing associated with user services provided to a user, characterized in that, The processor is configured to execute the following: Obtain the desired content (Cd) of the user service that the user desires; Service capability information representing the ability to provide services to the user is acquired in association with an autonomous driving device (Ma) that is standing still in a background area (Ab) of a display background shown to the user from a mobile terminal (Mt) carried by the user. The XR emphasized image (Ipe) is overlaid with the background video (Ib) projected onto the background area. The XR emphasized image is an image obtained by searching for autonomous driving devices in the background area that match the provided capabilities from the provided capabilities represented by the service capability information and emphasizing them. as well as The user service is provided by driving the autonomous driving device selected by the user in response to the overlapping display of the XR-emphasized image within the background area.

6. The processing system according to claim 5, characterized in that, The overlaid XR emphasized image includes the following: In order to notify the autonomous driving device in the background area of ​​the provision capability corresponding to the desired content, the XR capability image of the autonomous driving device that emphasizes the matching degree (R) of the provision capability with respect to the desired content within the recommended range in the XR capability image (Ip) that is aligned and overlaid by the autonomous driving device with the background video is set as the XR emphasis image.

7. The processing system according to claim 6, characterized in that, The overlaid XR emphasized image includes the following: In order to notify the matching degree along with the provided capabilities, the XR capability image is overlaid relative to the background video.

8. The processing system according to claim 6 or 7, characterized in that, Obtaining the service capability information includes the following: When it is determined that the user's selection of the autonomous driving device within the background area is rejected, the service capability information of the autonomous driving device waiting outside the background area is obtained as additional capability information. The overlaid XR emphasized image includes the following: The additional capability image (Ia) representing the provided capability, as indicated by the additional capability information, will be overlaid relative to the background video; and The autonomous driving device, selected by the user in response to the overlay display of the additional capability image, will drive into the background area to provide the user service.

9. The processing system according to any one of claims 2, 3, 6, and 7, characterized in that, The overlaid XR emphasized image includes the following: Regarding the ability to provide user services in the category represented by the desired content, the matching degree is obtained in relation to the suitability obtained by rating each item outside the category contained in the desired content.

10. A processing method, executed by a processor (24), to perform service association processing associated with user services provided to a user, characterized in that, Includes the following content: Obtain the desired content (Cd) of the user service that the user desires; Service capability information representing the user's ability to provide services (Pc) is acquired in association with an autonomous driving device (Ma) that is in standby within the field of view (Av) confirmed by the user's vision through the wearable terminal (Wt) worn by the user. An XR-emphasized image (Ipe) is overlaid relative to the field of view. This XR-emphasized image is an image obtained by searching for autonomous driving devices within the field of view that match the provided capabilities from the provided capabilities represented by the service capability information, and emphasizing them. as well as The user service is provided by driving the autonomous driving device selected by the user in response to the overlapping display of the XR-emphasized image within the field of view.

11. A processing program comprising instructions stored in a storage medium (22) for performing service association processing associated with a user service provided to a user, and the service association processing being performed by a processor (24), characterized in that, The instruction is used to execute the following: Obtain the desired content (Cd) of the user service that the user desires; Service capability information representing the user's ability to provide services (Pc) is acquired in association with an autonomous driving device (Ma) that is in standby within the field of view (Av) confirmed by the user's vision through the wearable terminal (Wt) worn by the user. An XR-emphasized image (Ipe) is overlaid relative to the field of view. This XR-emphasized image is an image obtained by searching for autonomous driving devices within the field of view that match the provided capabilities from the provided capabilities represented by the service capability information, and emphasizing them. as well as The user service is provided by driving the autonomous driving device selected by the user in response to the overlapping display of the XR-emphasized image within the field of view.

12. A processing method, executed by a processor (24), to perform service association processing associated with user services provided to a user, characterized in that, Includes the following content: Obtain the desired content (Cd) of the user service that the user desires; Service capability information representing the ability to provide services to the user is acquired in association with an autonomous driving device (Ma) that is standing still in a background area (Ab) of a display background shown to the user from a mobile terminal (Mt) carried by the user. The XR emphasized image (Ipe) is overlaid with the background video (Ib) projected onto the background area. The XR emphasized image is an image obtained by searching for autonomous driving devices in the background area that match the provided capabilities from the provided capabilities represented by the service capability information and emphasizing them. as well as The user service is provided by driving the autonomous driving device selected by the user in response to the overlapping display of the XR-emphasized image within the background area.

13. A processing program comprising instructions stored in a storage medium (22) for performing service association processing associated with a user service provided to a user, and the service association processing being performed by a processor (24), characterized in that, The instruction is used to execute the following: Obtain the desired content (Cd) of the user service that the user desires; Service capability information representing the ability to provide services to the user is acquired in association with an autonomous driving device (Ma) that is standing still in a background area (Ab) of a display background shown to the user from a mobile terminal (Mt) carried by the user. The XR emphasized image (Ipe) is overlaid with the background video (Ib) projected onto the background area. The XR emphasized image is an image obtained by searching for autonomous driving devices in the background area that match the provided capabilities from the provided capabilities represented by the service capability information and emphasizing them. as well as The user service is provided by driving the autonomous driving device selected by the user in response to the overlapping display of the XR-emphasized image within the background area.

Images