Processing system, processing method, and processing program

Abstract

To improve search efficiency of an autonomous traveling device that provides user service satisfying user preference.SOLUTION: A processor of a processing system is configured to execute the steps of: acquiring a desired content Cd of user service desired by a user; acquiring service capability information indicating providing capability Pc of the user service in association with an autonomous traveling device that stands by in a visual field area Av visually recognized by the user through a wearable terminal worn by the user; searching for the autonomous traveling device having the providing capability Pc matching the desired content Cd from among the providing capability Pc indicated by the service capability information for each autonomous traveling device in the visual field area Av, and displaying an XR-emphasized image Ipe in a superimposed manner relative to the visual field area Av; and providing the user service by driving the autonomous traveling device selected by the user in the visual field area Av in response to a superimposed display of the XR emphasized image Ipe.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present disclosure relates to processing techniques for performing service-related processing related to user services provided to users. [Background technology]

[0002] The technology disclosed in Patent Document 1 aims to provide appropriate guidance to users by reserving autonomous mobile devices that fall within the service coverage ranges that are created by dividing a route where guidance services are provided to users by autonomous mobile devices. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6956422 Summary of the Invention [Problem to be solved by the invention]

[0004] In the technology disclosed in Patent Document 1, autonomous mobile devices whose service provision range falls within the divided service ranges are automatically reserved, making it difficult to reflect user preferences in the reservation. However, while reflecting user preferences is an especially important factor when providing user services of diversified content, not limited to guidance services, using autonomous mobile devices, it makes it cumbersome for users to search for autonomous mobile devices themselves.

[0005] An object of the present disclosure is to improve the search efficiency of an autonomous mobile device that can provide user services that satisfy user preferences. [Means for solving the problem]

[0006] The technical means of the present disclosure for solving the problems will be described below. Note that the claims and the reference symbols in parentheses in this section indicate the correspondence with the specific means described in the embodiments described later in detail, and do not limit the technical scope of the present disclosure.

[0007] A first aspect of the present disclosure is A processing system having a processor (24) for performing service-related processing associated with a user service provided to a user, the processing system comprising: The processor Acquiring desired content (Cd) of a user service desired by the user; Acquiring service capability information representing a capability (Pc) of providing a user service, by associating it with an autonomous mobile device (Ma) waiting within a visual field (Av) that is visually recognized by the user through a wearable device (Wt) worn by the user; Searching for and highlighting an autonomous driving device with a provision capability matching the desired content from among the provision capabilities indicated by the service capability information for each autonomous driving device within the field of view, and superimposing the highlighted XR enhanced image (Ipe) on the field of view; The system is configured to provide a user service by driving an autonomous driving device selected by a user within the field of view area in response to the superimposed display of an XR-enhanced image.

[0008] A second aspect of the present disclosure is A processing method executed by a processor (24) to perform service-related processing related to a user service provided to a user, comprising: Acquiring desired content (Cd) of a user service desired by the user; Acquiring service capability information representing a capability (Pc) of providing a user service, by associating it with an autonomous mobile device (Ma) waiting within a visual field (Av) that is visually recognized by the user through a wearable device (Wt) worn by the user; Searching for and highlighting an autonomous driving device with a provision capability matching the desired content from among the provision capabilities indicated by the service capability information for each autonomous driving device within the field of view, and superimposing the highlighted XR enhanced image (Ipe) on the field of view; Providing a user service by driving an autonomous driving device selected by a user within the field of view area in response to the superimposed display of an XR-enhanced image.

[0009] A third aspect of the present disclosure is A processing program including instructions stored in a storage medium (22) and executed by a processor (24) to perform service-related processing related to a user service provided to a user, A processing program including instructions stored in a storage medium (22) and executed by a processor (24) to perform service-related processing related to a user service provided to a user, Acquiring desired content (Cd) of a user service desired by the user; Acquiring service capability information representing a capability (Pc) of providing a user service, by associating it with an autonomous mobile device (Ma) waiting within a visual field (Av) that is visually recognized by the user through a wearable device (Wt) worn by the user; Searching for and highlighting an autonomous driving device with a provision capability matching the desired content from among the provision capabilities indicated by the service capability information for each autonomous driving device within the field of view, and superimposing the highlighted XR enhanced image (Ipe) on the field of view; The instructions include instructions to execute the following: providing a user service by driving an autonomous driving device selected by a user within the field of view area in response to the superimposed display of an XR-enhanced image.

[0010] According to the first to third aspects, service capability information indicating the capability of providing a user service is acquired by linking it to a standby autonomous mobile device within a field of view visible to the user through a wearable device worn by the user. Then, by acquiring the desired content of a user service desired by the user, attention is focused on an autonomous mobile device with a capability of providing a service that matches the desired content from the provision capabilities indicated by the service capability information for each autonomous mobile device within the field of view. This allows an XR-enhanced image that highlights and searches for autonomous mobile devices with a capability matching the desired content, and is superimposed on the field of view to guide the user in selecting the device. This improves the efficiency of searching for autonomous mobile devices that can provide a user service that satisfies the user's preferences by operating within the field of view.

[0011] A fourth aspect of the present disclosure is A processing system having a processor (24) for performing service-related processing associated with a user service provided to a user, the processing system comprising: The processor Acquiring desired content (Cd) of a user service desired by the user; Acquiring service capability information representing a capability (Pc) of providing a user service from a mobile terminal (Mt) carried by a user, linked to an autonomous mobile device (Ma) waiting in a background area (Ab) that serves as a display background for the user; Searching for and highlighting an autonomous driving device with a provision capability matching the desired content from among the provision capabilities indicated by the service capability information for each autonomous driving device in the background area, and superimposing the highlighted XR enhanced image (Ipe) on a background video (Ib) showing the background area; and providing a user service by driving an autonomous driving device selected by a user within a background area in response to the superimposed display of an XR-enhanced image.

[0012] A fifth aspect of the present disclosure is A processing method executed by a processor (24) to perform service-related processing related to a user service provided to a user, comprising: Acquiring desired content (Cd) of a user service desired by the user; Acquiring service capability information representing a capability (Pc) of providing a user service from a mobile terminal (Mt) carried by a user, linked to an autonomous mobile device (Ma) waiting in a background area (Ab) that serves as a display background for the user; Searching for and highlighting an autonomous driving device with a provision capability matching the desired content from among the provision capabilities indicated by the service capability information for each autonomous driving device in the background area, and superimposing the highlighted XR enhanced image (Ipe) on a background video (Ib) showing the background area; Providing a user service by driving an autonomous driving device selected by a user within a background area in response to the superimposed display of an XR-enhanced image.

[0013] A sixth aspect of the present disclosure is A processing program including instructions stored in a storage medium (22) and executed by a processor (24) to perform service-related processing related to a user service provided to a user, Acquiring desired content (Cd) of a user service desired by the user; Acquiring service capability information representing a capability (Pc) of providing a user service from a mobile terminal (Mt) carried by a user, linked to an autonomous mobile device (Ma) waiting in a background area (Ab) that serves as a display background for the user; Searching for and highlighting an autonomous driving device with a provision capability matching the desired content from among the provision capabilities indicated by the service capability information for each autonomous driving device in the background area, and superimposing the highlighted XR enhanced image (Ipe) on a background video (Ib) showing the background area; and providing a user service by driving an autonomous driving device selected by a user within a background area in response to the superimposed display of an XR-enhanced image.

[0014] According to the fourth to sixth aspects, service capability information indicating the capability to provide a user service is acquired from a mobile terminal carried by a user in a background area that serves as a display background for the user, and is associated with a waiting autonomous mobile device. Then, by acquiring the desired content of a user service desired by the user, attention is focused on an autonomous mobile device with a capability to provide the desired content from the provision capabilities indicated by the service capability information for each autonomous mobile device in the background area. This allows an XR-enhanced image that highlights an autonomous mobile device with a capability matching the desired content to be searched for and displayed superimposed on a background video showing the background area, allowing the user to select the device. This improves the efficiency of searching for an autonomous mobile device that can provide a user service that satisfies the user's preferences by operating within the background area. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is an overall configuration diagram showing a network connection environment of a processing system according to a first embodiment. [Figure 2] FIG. 1 is a schematic diagram illustrating a wearable terminal according to a first embodiment. [Figure 3] FIG. 2 is a schematic diagram for explaining three-dimensional voxels assumed in the infrastructure system according to the first embodiment. [Figure 4] 1 is a block diagram showing a processing system according to a first embodiment; [Figure 5] 3 is a flowchart showing a processing flow according to the first embodiment. [Figure 6] 3 is a flowchart showing a processing flow according to the first embodiment. [Figure 7] FIG. 2 is a schematic diagram showing a display state of the wearable terminal according to the first embodiment. [Figure 8] FIG. 2 is a schematic diagram showing a display state of the wearable terminal according to the first embodiment. [Figure 9] FIG. 2 is a schematic diagram showing a display state of the wearable terminal according to the first embodiment. [Figure 10]FIG. 10 is an overall configuration diagram showing a network connection environment of a processing system according to a second embodiment. [Figure 11] FIG. 10 is a schematic diagram showing a mobile terminal according to a second embodiment. [Figure 12] FIG. 10 is a block diagram showing a processing system according to a second embodiment. [Figure 13] 10 is a flowchart showing a processing flow according to a second embodiment. [Figure 14] 10 is a flowchart showing a processing flow according to a second embodiment. [Figure 15] FIG. 10 is a schematic diagram showing a display state of a mobile terminal according to a second embodiment. [Figure 16] FIG. 10 is a schematic diagram showing a display state of a mobile terminal according to a second embodiment. [Figure 17] FIG. 10 is a schematic diagram showing a display state of a wearable terminal according to a modified example of the first embodiment. [Figure 18] FIG. 10 is a schematic diagram showing a display state of a wearable terminal according to a modified example of the first embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, multiple embodiments of the present disclosure will be described with reference to the drawings. Note that corresponding components in each embodiment are designated by the same reference numerals, and redundant description may be omitted. Furthermore, when only a portion of the configuration is described in each embodiment, the configuration of another previously described embodiment may be applied to the remaining portions of the configuration. Furthermore, in addition to the combinations of configurations explicitly stated in the description of each embodiment, configurations of multiple embodiments may be partially combined together even if not explicitly stated, provided that there is no particular problem with the combination.

[0017] (First embodiment) 1 performs service-related processing related to user services provided to a user Us. To this end, the processing system 1 is connected to a wearable device Wt worn by a pre-registered user Us, an autonomous mobile device Ma, and an infrastructure system 6 via a communication network Nc. It is assumed that multiple wearable devices Wt and multiple autonomous mobile devices Ma are connected to the processing system 1. It is assumed that at least one infrastructure system 6 is connected to the processing system 1.

[0018] The wearable terminal Wt is configured so that a user Us of the processing system 1 can wear it on their face and operate it hands-free, at least when connected to the processing system 1. As shown in Fig. 2, the wearable terminal Wt is an optical see-through electronic device equipped with a display unit 2 and a sensor unit 3, such as an HMD or smart glasses. Regarding the wearable terminal Wt, Fig. 2 shows a schematic image of how a visual field area Av appears from the user Us's side in a portion corresponding to one of the user Us's eyes.

[0019] The wearable device Wt uses a communication unit to control the transmission and reception of data via a communication network Nc as shown in Fig. 1. The wearable device Wt is operated and controlled by the control unit in accordance with control commands from a processing system 1 via the communication network Nc so as to perform service-related processing in cooperation with the processing system 1.

[0020] Under operational control by the control unit of the wearable device Wt, the display unit 2 shown in FIG. 2, such as a virtual image projection type or retinal projection type, superimposes a display (see FIGS. 7 to 9 described below) required for service-related processing on a real image of the field of view Av viewed by the user Us through glasses or lenses, allowing the user Us to view the superimposed display. At the same time, under operational control of the wearable device Wt, the sensor unit 3 shown in FIG. 2 acquires sensing information through sensing processing in accordance with the service-related processing. The sensor unit 3 may be, for example, at least one of a camera, inertial sensor, GNSS sensor, electrooculography sensor, gaze sensor, infrared sensor, geomagnetic sensor, motion sensor, touch sensor, and microphone. This configuration enables the sensor unit 3 to acquire input representing the user Us's intention as sensing information.

[0021] The autonomous mobile device Ma shown in FIG. 1 is an autonomous mobile vehicle or an autonomous mobile robot that can autonomously travel in any direction, forward, backward, leftward, or rightward, by electrically driving wheels 5 based on sensing information from a sensor unit 4. The autonomous mobile device Ma uses a communication unit to control the transmission and reception of data via a communication network Nc. The autonomous mobile device Ma is driven and controlled by a control unit in accordance with control commands from a processing system 1 via the communication network Nc so as to perform service-related processing in cooperation with the processing system 1. Under this drive control, the sensor unit 4 acquires sensing information through sensing processing in accordance with the service-related processing. The sensor unit 4 may be at least one of a camera, an inertial sensor, a GNSS sensor, a LiDAR (Light Detection and Ranging / Laser Imaging Detection and Ranging), a sonar, or the like.

[0022] The infrastructure system 6 is a base system that shares three-dimensional spatial information Ti stored in an infrastructure database Di as a common foundation for the distributed architecture. The infrastructure system 6 controls the transmission and reception of data via a communication network Nc using a communication unit. The infrastructure system 6 collects the three-dimensional spatial information Ti, which is provided to individual distributed systems such as the processing system 1 that make up the distributed architecture, using a control unit as needed, and updates the information stored in the infrastructure database Di to the latest information.

[0023] In particular, the infrastructure system 6 manages information by virtually dividing the three-dimensional space to be stored into a plurality of three-dimensional voxels Vi (i.e., a three-dimensional grid) assumed to be arranged in a three-dimensional array as shown in Fig. 3. The infrastructure system 6 stores a data set linked by metadata for each space ID individually assigned to each voxel Vi as three-dimensional spatial information Ti in the infrastructure database Di as shown in Fig. 1. Of course, the three-dimensional spatial information Ti may include two-dimensional grid information linked, for example, only to the bottom surface of the voxels Vi that constitute the bottom layer of the two-dimensional array along the ground in the three-dimensional array.

[0024] The data constituting the three-dimensional spatial information Ti may be collected from at least one of a wearable device Wt that cooperates with the processing system 1, other wearable devices, and mobile devices such as smartphones or tablet devices. The data constituting the three-dimensional spatial information Ti may be collected from at least one of an autonomous mobile device Ma that cooperates with the processing system 1, and other moving bodies.

[0025] The data constituting the three-dimensional spatial information Ti may be collected from at least one of a communication base station, a smart pole, a smart street light, etc., equipped with infrastructure sensors such as a camera and / or LiDAR. The data constituting the three-dimensional spatial information Ti may be collected from a service provider that provides at least one of a map service, a weather service, a communication service, a traffic management service, a feature management service, an aviation management service, and a user service (described later) handled by the processing system 1.

[0026] The construction data collected in this manner to construct the three-dimensional space information Ti may be at least one type of image data, for example, videos, still images, images, etc. The construction data of the three-dimensional space information Ti may be at least one type of secondary data, for which information security is guaranteed, generated by image processing of such image data regarding people, including the user Us of the processing system 1, within the target space for information accumulation, such as position data, movement data, posture data, gaze data, behavior data, and people flow data, as well as intention data representing the intention of the user Us, which will be described later.

[0027] The construction data of the three-dimensional space information Ti may be at least one type of secondary data, such as position data, movement data, and posture data, generated by image processing of image data regarding human belongings, including the wearable terminal Wt of the user Us, within the information storage target space.The construction data of the three-dimensional space information Ti may be at least one type of secondary data, such as position data, movement data, and posture data, generated by image processing of image data regarding moving objects, including the autonomous mobile device Ma, within the information storage target space.

[0028] The construction data of the three-dimensional space information Ti may be voice data obtained by collecting voices of people, including the user Us of the processing system 1, in the information storage target space. The construction data of the three-dimensional space information Ti may be at least one type of secondary data, for which information security is guaranteed, analyzed regarding people, including the user Us, by voice recognition processing of such voice data, such as position data, movement data, behavior data, people flow data, and conversation data, as well as intention data expressing the intention of the user Us, which will be described later.

[0029] The construction data of the three-dimensional spatial information Ti may be, for example, at least one type of data selected from two-dimensional and / or three-dimensional map data, GIS (Geographic Information System) data, road network data, weather data, communication data, line data, traffic data, feature management data, BIM (Building Information Modeling) data, POI (Point of Interest) data, aviation management data, and time data. The construction data of the three-dimensional spatial information Ti may also be service data related to at least one type of user service handled by the processing system 1, such as a guidance service, a transportation service, and a photography service, which will be described later.

[0030] 1, the processing system 1 is a distributed computer system equipped with a communication system 10 and a control system 20, and constructed to include at least one of, for example, a cloud server and an edge server. At least a portion of each of the systems 10 and 20 in the processing system 1 may be configured by a communication unit and a control unit of a wearable device Wt. At least a portion of each of the systems 10 and 20 in the processing system 1 may be configured by a communication unit and a control unit of an autonomous mobile device Ma. At least a portion of each of the systems 10 and 20 in the processing system 1 may be configured by a communication unit and a control unit of an infrastructure system 6.

[0031] The communication system 10 is mainly composed of communication devices for constructing a communication network Nc. The control system 20 is connected to the communication system 10 via at least one of a wired communication line and a wireless communication line. The control system 20 is constructed to include at least one dedicated computer. The dedicated computer that constructs the control system 20 has at least one memory 22 and one processor 24.

[0032] In the processing system 1, the control system 20 executes a plurality of instructions of a processing program stored in the memory 22 using the processor 24. As a result, a plurality of functional blocks for performing service-related processing related to the user service provided to the user Us are constructed in the control system 20. The constructed functional blocks include a recognition block 200, a search block 210, and a drive control block 220, as shown in FIG.

[0033] The processing method in which the processing system 1 performs service-related processing through the cooperation of these blocks 200, 210, and 220 is executed according to the processing flow shown in Figures 5 and 6. This processing flow is executed in response to the sensor unit 3 of the wearable device Wt acquiring, as sensing information, a service request input indicating the user Us's intention to request the provision of a user service. Note that each "S" in this processing flow represents multiple steps executed by multiple commands included in the processing program of the first embodiment.

[0034] At S100 in the processing flow, the recognition block 200 (see FIG. 4) recognizes the field of view area Av visually recognized by the user Us through the wearable device Wt. At this time, the field of view area Av is recognized so as to overlap with a waiting area Aw where multiple autonomous mobile devices Ma can wait, as shown in FIG. 2, in an information storage target space of the infrastructure system 6, such as a station, terminal, facility entrance, or landmark.

[0035] Therefore, in S100, if the autonomous mobile device Ma is not present in the waiting area Aw that overlaps with the field of view Av, the current execution of the processing flow ends. In this case, an XR notification image may be superimposed on the field of view Av to notify the user that the search was unsuccessful due to the absence of an available autonomous mobile device Ma.

[0036] The recognition process of the visual field area Av in S100 is based on at least one of sensing information, such as camera information and inertial information, from the sensor unit 3 of the wearable device Wt. As a result, the visual field area Av is recognized as an area extending in the direction of the user Us's face or line of sight, or in the direction of the wearable device Wt.

[0037] 5, the recognition block 200 acquires desired content Cd (see FIG. 7) of a user service desired by the user Us. At this time, the desired content Cd is recognized based on sensing information about the face obtained by the wearable device Wt worn on the face of the user Us. Therefore, the sensing information for recognizing the desired content Cd is acquired through the wearable device Wt at the timing when the sensing information is input to the sensor unit 3 as intention data of the user Us.

[0038] The input of the desired content Cd in S110 may be a gesture input, a gaze input, a facial expression input, or a voice input by the user Us, which may be sensed by the sensor unit 3. At this time, on the display unit 2 of the wearable device Wt, an XR reception image Ir for receiving the input of the desired content Cd from the user Us by the sensor unit 3 may be superimposed on the field of view Av as illustrated in Fig. 7. Note that the desired content Cd may be recognized based on three-dimensional space information Ti obtained from the infrastructure database Di via the communication system 10.

[0039] 7, the items of the desired content Cd recognized in S110 include at least the types of user services that can be provided to the user Us of the processing system 1. As the types of user services corresponding to the desired content Cd, at least one of a guidance service, a transportation service, a photography service, etc. is prepared.

[0040] In S110, regardless of the type of service the desired content Cd recognized may include at least one voice-related item, such as whether or not voice output is required, whether or not a voice recognition function is required, whether or not a dialogue function is required, etc. If the type of desired content Cd is either a guidance service or a transportation service, at least one route-related item, such as a route including a destination, a moving pace, an arrival time, and a service provision time (i.e., guidance time or transportation time), etc. may be included in the desired content Cd.

[0041] If the type of desired content Cd is a guidance service, the content Cd may include at least one type of tourism-related item regarding a destination such as a tourist spot, such as an acceptable level of congestion, the presence or absence of desired facilities, the presence or absence of nature, etc. If the type of desired content Cd is a guidance service, the content Cd may include an optional item indicating a desire to use other types of services in combination as optional functions.

[0042] Here, among the desired contents Cd, the optional items indicating the desire to use a transportation service in combination with the guidance service as a transportation function may include at least one of the items related to luggage, such as the number of luggage, size, weight, type, whether temperature control is required, etc. This also applies to the case of a transportation service independent of the guidance service.

[0043] Among the desired content Cd, the optional items indicating the desire to use a photography service in combination with the guidance service as a photography function may include at least one photography-related item from among, for example, a photography schedule, photography timing, photography location, whether or not to send images or data, whether or not to print photos, etc. This also applies to photography services that are independent of the guidance service.

[0044] 5, the search block 210 (see FIG. 4) acquires service capability information indicating a user service provision capability Pc (see FIG. 8) associated with the autonomous mobile device Ma waiting in the waiting area Aw within the field of view Av recognized in S100. In this case, the service capability information may be information that can identify the capability possessed by each autonomous mobile device Ma waiting in the waiting area Aw within the field of view Av as the provision capability Pc required to provide user services of the types prepared in the processing system 1.

[0045] Here, the provision capability Pc for various services may include at least one of driving-related capabilities such as remaining battery power, cruising range, maximum possible driving speed, minimum possible driving road width, road surface condition, and obstacle avoidance level according to congestion level, etc. The provision capability Pc for various services may also include at least one of voice-related capabilities such as whether or not voice output is possible, whether or not a voice recognition function is available, and whether or not a dialogue function is available.

[0046] The capacity Pc for the transportation service provided as a transportation function in the guidance service may include at least one type of transportation-related capacity, for example, the presence or absence of a loading compartment, the loadable size, the total loadable luggage weight, the loadable luggage type, the presence or absence of a refrigeration function, the presence or absence of a freezing function, etc. This is also true in the case of the capacity Pc for the transportation service that is independent of the guidance service.

[0047] The provision capability Pc for the photography service provided as a photography function in the guidance service may include at least one of photography-related capabilities, such as the presence or absence of a camera function, the presence or absence of an image or video data transmission function, and the presence or absence of a photo printing function, etc. This also applies to the provision capability Pc for the replay service that is independent of the guidance service.

[0048] In S120, service capability information regarding such provision capability Pc may be acquired from the communication unit of the autonomous mobile device Ma waiting in the waiting area Aw within the field of view Av via the communication system 10. The service capability information may be acquired based on three-dimensional space information Ti obtained from the infrastructure database Di via the communication system 10. The service capability information may be acquired by having a sensor unit 3, such as a camera, in the wearable device Wt capture an image of a QR code (registered trademark) displayed on the display unit of the autonomous mobile device Ma waiting in the waiting area Aw within the field of view Av.

[0049] 5, the search block 210 searches for an autonomous mobile device Ma having a provision capability Pc that matches the desired content Cd obtained in S110 from the provision capabilities Pc represented by the service capability information obtained for each autonomous mobile device Ma within the field of view Av in S120. At this time, the matching degree R (see FIG. 8) of the provision capability Pc for the desired content Cd is obtained.

[0050] In S130, the matching degree R is obtained as an index correlated with the relevance rate, which is scored for each item other than the type included in the desired content Cd, with respect to the provision capability Pc of providing the user service of the type represented by the desired content Cd. For example, as shown in the following equation 1, the relevance rate r i The cumulative value of the maximum precision of each item is rm i The matching degree R can be obtained by normalizing the sum of the above values. i The integrated value of can be obtained by standardizing the accuracy of each item to the machine learning model, ri The information may be obtained by inputting:

number

[0051] In S130, autonomous mobile devices Ma whose acquired matching degree R falls within the recommended range are extracted as search results for matching devices Mam that have provision capabilities Pc that match the desired content Cd. The recommended range in this case is specified as a range in which the relevance rate is equal to or greater than a threshold and equal to or less than a maximum value, so that the matching degree R for the desired content Cd falls within a range that can be recommended to the user Us. Here, when the matching degree R is acquired by the normalization described above, the threshold that determines the lower limit of the recommended range may be set to a specific value between 50% and 80% (when the maximum value is normalized to 100%), or a specific value between 0.5 and 0.8 (when the maximum value is normalized to 1).

[0052] 5, the search block 210 causes the display unit 2 of the wearable device Wt to ​​superimpose on the field of view Av an XR-enhanced image Ipe that highlights the matching device Mam found in S130 as shown in Fig. 8. To achieve this, in S140, the provision capacity Pc for the user service corresponding to the desired content Cd obtained in S110 is extracted from the provision capacity Pc represented by the service capacity information obtained for each autonomous mobile device Ma in S120.

[0053] In S140, as shown in FIG. 8, the display unit 2 superimposes an XR capability image Ip, which is a candidate for the XR-enhanced image Ipe, on the field of view Av to notify the user service provision capability Pc corresponding to the desired content Cd for each autonomous mobile device Ma. In particular, the XR capability image Ip is superimposed and displayed in the field of view Av, aligned with the corresponding autonomous mobile device Ma, to notify the matching degree R obtained in S130 together with the provision capability Pc. The superimposed display position of the XR capability image Ip is adjusted so that it is visible around the autonomous mobile device Ma, for example, based on sensing information from at least one of the sensor units 3 and 4 and / or three-dimensional space information Ti obtained from the infrastructure database Di. In this case, the field of view Av on which the XR capability image Ip is superimposed may be fixed to the area recognized in S100 or updated in accordance with S100 as the user Us's viewpoint moves.

[0054] In S140, of the XR capability images Ip superimposed on the field of view Av, the image Ip highlighting the matching device Mam whose matching degree R of the provision capability Pc for the desired content Cd is within the recommended range is set as the XR-enhanced image Ipe. Therefore, the XR capability image Ip set as the XR-enhanced image Ipe is emphasized by making at least one difference, such as color, size, and line width, in the text display and / or the window display that serves as the background, compared to the XR capability images Ip that are not set as the XR-enhanced image Ipe.

[0055] In such S140, if S130 finds fewer matching devices Mam than the autonomous mobile devices Ma present in the field of view Av, the set and non-set XR capability images Ip will be superimposed on the XR-enhanced image Ipe. On the other hand, in S140, if S130 finds the same number of matching devices Mam as the autonomous mobile devices Ma present in the field of view Av, only the set XR capability images Ip will be superimposed on the XR-enhanced image Ipe. Furthermore, in S140, if no matching devices Mam are found by S130, only the non-set XR capability images Ip will be superimposed on the XR-enhanced image Ipe.

[0056] 5, in response to the superimposed display in S140, the search block 210 determines whether any of the autonomous mobile devices Ma in the field of view Av has been selected by the user Us. Whether or not a selection has been made by the user Us is determined based on facial sensing information acquired by the wearable device Wt. The sensing information for determining whether or not a selection has been made by the user Us is input to the sensor unit 3 as intention data of the user Us and is acquired through the wearable device Wt.

[0057] The selection input of the autonomous mobile device Ma in S150 may be a gesture input, a gaze input, a facial expression input, or a voice input by the user Us, which may be sensed by the sensor unit 3. The selection input in this case may be an input for selecting the location of the autonomous mobile device Ma, or an input for selecting images Ip, Ipe corresponding to the autonomous mobile device Ma. Here, for either selection input, the selection position may be recognized based on, for example, sensing information from at least one of the sensor units 3 and 4 and / or three-dimensional space information Ti obtained from the infrastructure database Di. Note that the recognition of whether or not the user Us has made a selection and the selection position may be realized based on three-dimensional space information Ti obtained from the infrastructure database Di via the communication system 10.

[0058] If a selection by the user Us is recognized in S150 and a positive determination is made, S160 is executed as shown in Fig. 5. In S160, the drive control block 220 (see Fig. 4) controls the drive of the autonomous mobile device Ma selected in S150 within the field of view area Av, thereby providing a user service of a type corresponding to the desired content Cd. At this time, a drive control pattern for the autonomous mobile device Ma that matches the desired content Cd is read from the memory 22, and the user service is provided in accordance with the drive control pattern.

[0059] Here, the guidance service according to S160 is provided by a drive control pattern in which the autonomous mobile device Ma is driven from a standby position to a position in front of a person to be guided, such as the user Us or a registered family member, and then performs route guidance to lead the person to the destination, and then returns to the standby area Aw. In this guidance service, the route along which the person to be guided may be presented by superimposing an XR presentation image on the field of view Av of the wearable device Wt or by display output from a display unit possessed by the autonomous mobile device Ma. In the guidance service, the route along which the person to be guided may be presented by audio output from an audio unit possessed by the wearable device Wt or the autonomous mobile device Ma.

[0060] The delivery service according to S160 is provided by a drive control pattern in which an autonomous mobile device Ma having a loading compartment is driven from a standby position to a position in front of a user Us, receives a request from the user Us to load luggage into the loading compartment, delivers the luggage to the destination, and then returns to the standby area Aw. In this case, the delivery service may present to the user Us a method for loading luggage into the loading compartment by superimposing an XR presentation image on the field of view Av of the wearable device Wt or by display output from a display unit possessed by the autonomous mobile device Ma. In the delivery service, the loading of luggage into the loading compartment may be presented to the user Us by audio output from an audio unit possessed by the wearable device Wt or the autonomous mobile device Ma. Such a delivery service may be provided as a delivery function of a guidance service when the desired content Cd is included.

[0061] The photography service by S160 is provided by a drive control pattern in which the autonomous mobile device Ma having an imaging unit is driven from a standby position to a position in front of the user Us or a landmark in the user Us's vicinity, photographs the user Us from the surroundings, and then returns to the standby area Aw. In this photography service, the photography schedule or photography timing may be presented to the user Us by superimposing an XR presentation image on the field of view Av of the wearable device Wt or by display output from a display unit possessed by the autonomous mobile device Ma. In the photography service, the photography schedule or photography timing may be presented to the user Us by audio output from an audio unit possessed by the wearable device Wt or the autonomous mobile device Ma.

[0062] In the photography service by S160, the autonomous mobile device Ma driven around the user Us may capture two-dimensional or three-dimensional still images or videos that include the user Us. In the photography service, the focus during photography may be adjusted based on the distance to the user Us sensed by a sensor unit 4 such as a LiDAR in the autonomous mobile device Ma.

[0063] In the photography service performed by S160, the incident direction of external light during photography may be adjusted based on three-dimensional spatial information Ti obtained from the infrastructure database Di or sensing information from a sensor unit 4, such as a camera, in the autonomous mobile device Ma. In the photography service, image data of the user Us may be stored in the memory 22 of the control system 20 or in a storage medium in the wearable device Wt. The photography service described above may be provided as a photography function for a guidance service when the desired content Cd is included in the desired content Cd.

[0064] The viewpoint of the user Us moves as the user service progresses under the drive control of the autonomous mobile device Ma. Therefore, in S160, the field of view Av is updated in accordance with S100, so that the autonomous mobile device Ma that provides the user service can be driven within the updated field of view Av as needed.

[0065] In the above-described S150, a negative determination is made, for example, if the user Us does not perform a selection input within the set time period from the start of execution, or if the user Us indicates that they do not want to select any autonomous mobile devices Ma, etc. In this case, it is determined that the selection by the user Us has been rejected for the autonomous mobile device Ma in the field of view Av, and S170 is executed as shown in FIGS.

[0066] 6, the search block 210 determines whether or not there is an autonomous mobile device Ma waiting within a set distance from the user Us outside the recognized field of view Av. At this time, the presence or absence of the autonomous mobile device Ma within the set distance outside the field of view Av may be recognized based on three-dimensional space information Ti obtained from the infrastructure database Di via the communication system 10.

[0067] If a positive determination is made in S170, S180 is executed as shown in Fig. 6. In S180, the search block 210 acquires service capability information that indicates the user service provision capability Pc as additional capability information, linking it to the autonomous mobile device Ma that is waiting outside the field of view Av and whose presence was recognized in S170. The acquisition of the additional capability information is realized in accordance with S120.

[0068] 6, the search block 210 searches for a matching device Mam whose provision capability Pc of the additional capability information acquired for each autonomous mobile device Ma outside the field of view Av in S180 matches the desired content Cd acquired in S110. This search is also realized based on the matching degree R acquired in accordance with S130.

[0069] 6, the search block 210 notifies the providing capability Pc represented by the additional capability information acquired for each autonomous mobile device Ma outside the field of view Av in S180. In S200, to realize such notification in accordance with the XR capability image Ip, the additional capability image Ia shown in FIG. 9 is superimposed on the field of view Av by the display unit 2 of the wearable device Wt.

[0070] In S200, the additional capability image Ia is superimposed on, for example, an air or ground area in the field of view Av to notify the matching degree R acquired in S190 together with the provision capability Pc. Here, the additional capability image Ia may notify at least one of the following regarding the location of the corresponding autonomous mobile device Ma: the direction from the user Us, the distance from the user Us, and the time it takes to reach the user Us by calling. For the matching device Mam found in S190 (in the example of FIG. 9, the location of the autonomous mobile device Ma displayed as "3m to the right, robot"), the additional capability image Ia may be set as an additional enhancement image Iae that emphasizes the device Mam in accordance with the XR enhancement image Ipe. Such an additional capability image Ia may be superimposed and displayed in the field of view Av at a fixed relative position to the glasses or lenses of the display unit 2, or may be superimposed and displayed as an XR capability image Ip (including the XR enhancement image Ipe for the matching device Mam) in the field of view Av at a fixed relative position to the autonomous mobile device Ma. In either case, the field of view area Av on which the additional capability image Ia is superimposed may be fixed to the area recognized by S100, or updated in accordance with S100 as the user Us moves their viewpoint.

[0071] In S210 shown in FIG. 6, the search block 210 determines, in response to the superimposed display in S200, whether any of the autonomous mobile devices Ma outside the field of view Av has been selected by the user Us, in accordance with S150. As a result, if a positive determination is made by recognizing the selection by the user Us, S220 is executed. In S220, the drive control block 220 controls the autonomous mobile device Ma selected in S210 to drive into the field of view Av in order to provide a user service corresponding to the desired content Cd in accordance with S160. In other words, in S220, the drive control of the selected autonomous mobile device Ma into the field of view Av enables the service of the desired content Cd to be provided.

[0072] 5 and 6, the current execution of the process flow is completed when the provision of the user service corresponding to the desired content Cd is completed in either S160 or S220. On the other hand, if a negative determination is made in either S170 or S210, the current execution of the process flow may be completed after an XR notification image notifying the user of the unsuccessful search is superimposed on the field of view Av. In either of these cases, after the current execution is completed, the next service request input is acquired via the sensor unit 3 of the wearable device Wt, and the next execution of the process flow will begin.

[0073] (Action and effect) The effects of the first embodiment described above will be explained below.

[0074] According to the first embodiment, service capability information indicating the user service provision capability Pc is acquired in association with a standby autonomous mobile device Ma within a field of view Av visually recognized by the user Us through a wearable device Wt worn by the user Us. Then, desired content Cd of a user service desired by the user Us is acquired, and the autonomous mobile device Ma with the provision capability Pc matching the desired content Cd is selected from the provision capabilities Pc indicated by the service capability information for each autonomous mobile device Ma within the field of view Av. This allows an XR-enhanced image Ipe, which highlights and searches for an autonomous mobile device Ma (specifically, a matching device Mam) with a capability Pc matching the desired content Cd, to be superimposed on the field of view Av to help the user Us select that device Ma. This improves the efficiency of searching for an autonomous mobile device Ma that can provide a user service that satisfies the user's preferences by operating within the field of view Av.

[0075] Furthermore, according to the first embodiment, the XR capability image Ip is aligned with and superimposed on the field of view Av for each autonomous mobile device Ma so as to notify each autonomous mobile device Ma within the field of view Av of the provision capability Pc corresponding to the desired content Cd. Therefore, among the XR capability images Ip for each autonomous mobile device Ma, the XR capability image Ip highlighting the autonomous mobile device Ma (specifically, the matching device Mam) whose matching degree R of the provision capability Pc for the desired content Cd falls within the recommended range is set as the XR-enhanced image Ipe. This allows the user Us to appropriately select an autonomous mobile device Ma automatically searched for based on the matching degree R between the desired content Cd and the provision capability Pc, without, for example, making a reservation in advance, from among the autonomous mobile devices Ma within the field of view Av notified of the provision capability Pc. Therefore, not only is the search efficiency for an autonomous mobile device Ma capable of providing a user service that satisfies the user's preferences improved, but the search accuracy can also be improved by minimizing the gap between the desired content Cd and the provision capability Pc.

[0076] According to the first embodiment, the XR capability image Ip is superimposed on the field of view Av to notify the matching degree R together with the provision capability Pc. This allows the user Us to intuitively select, from among the autonomous mobile devices Ma notified of their provision capabilities Pc within the field of view Av, the autonomous mobile device Ma that is highlighted along with the notification of the matching degree of the provision capability Pc for the desired content Cd. This can provide reliability to a highly efficient search for an autonomous mobile device Ma that can provide a user service that satisfies the user's preferences.

[0077] Furthermore, according to the first embodiment, if it is determined that the user Us's selection from among the autonomous mobile devices Ma within the field of view Av has been rejected, the service capability information of the autonomous mobile device Ma waiting outside the field of view Av is acquired as additional capability information. An additional capability image Ia notifying the user of the provision capability Pc represented by the additional capability information is then superimposed on the field of view Av. This allows the user Us to be notified of the search results for the device Ma expanded beyond the field of view Av, even if an autonomous mobile device Ma capable of providing a user service that satisfies the user's preferences is not found within the field of view Av. This ensures fail-safety in the highly efficient search for the autonomous mobile device Ma, giving the user Us a sense of security.

[0078] Furthermore, according to the first embodiment, the autonomous mobile device Ma selected by the user Us in response to the superimposed display of the additional capability image Ia is driven into the field of view Av to provide the user service. This allows the user Us to select the autonomous mobile device Ma notified by a highly efficient search outside the field of view Av, and the device Ma that is automatically called into the field of view Av can provide a user service that satisfies the user's preferences.

[0079] Second Embodiment The second embodiment is a modification of the first embodiment.

[0080] 10 , a processing system 2001 of the second embodiment is connected to a mobile terminal Mt carried by a pre-registered user Us, an autonomous mobile device Ma that performs autonomous driving, and an infrastructure system 6 that provides infrastructure information, via a communication network Nc. Here, it is assumed that a plurality of mobile terminals Mt are connected to the processing system 1.

[0081] The mobile terminal Mt is configured so that it can be held and operated by a user Us of the processing system 2001 with his / her fingers, at least when connected to the system 2001. As shown in Fig. 11 , the mobile terminal Mt is a small electronic device including a display unit 2002 and a sensor unit 2003, such as a smartphone or a tablet terminal. The mobile terminal Mt uses a communication unit to control transmission and reception of data via a communication network Nc, as shown in Fig. 10 . The mobile terminal Mt is operated and controlled by the control unit in accordance with control commands from the processing system 2001 via the communication network Nc, so as to perform service-related processing in cooperation with the processing system 2001.

[0082] Under the operational control of the control unit of the mobile terminal Mt, a display unit 2002 shown in Fig. 11, such as a liquid crystal panel or an organic EL panel, realizes a screen display (see Figs. 15 and 16 described below) required for service-related processing. At the same time, under the operational control of the mobile terminal Mt, a sensor unit 2003 shown in Fig. 11 acquires sensing information through sensing processing according to the service-related processing. The sensor unit 2003 may be, for example, at least one of a camera, an inertial sensor, a GNSS (Global Navigation Satellite System) sensor, a touch sensor, a microphone, or the like. This configuration enables the sensor unit 2003 to acquire input representing the user Us's intention as sensing information.

[0083] 10 is configured similarly to the processing system 1 of the first embodiment, except that the autonomous mobile device Ma is driven and controlled by a control unit in accordance with control commands from a processing system 2001 via a communication network Nc so as to cooperate with the processing system 2001 to perform service-related processing. A control system 2020 of the processing system 2001 is configured similarly to the control system 20 of the first embodiment, except that at least a part of the control system 2020 may be configured by a control unit of the mobile terminal Mt, and that the control system 2020 executes a processing flow using the mobile terminal Mt, which will be described later.

[0084] In order to perform service-related processing in the control system 2020, the processing system 2001 performs service-related processing in cooperation with the blocks 200, 210, and 220 constructed as shown in FIG. 12, and the processing method is executed according to the processing flows shown in FIGS.

[0085] In S2100 of the processing flow, the recognition block 200 (see FIG. 12 ) recognizes a background area Ab, which serves as a display background for the user Us from the display unit 2002, by pointing the mobile terminal Mt carried by the user Us. At this time, the background area Ab is recognized as an area Ab extending in the direction in which the mobile terminal Mt is facing, based on at least one of sensing information, such as camera information from the sensor unit 2003 of the mobile terminal Mt and inertial information. In particular, the background area Ab may be recognized so as to overlap with a waiting area Aw in which multiple autonomous mobile devices Ma can wait, as shown in FIG. 11 , in accordance with the field of view area Av recognized in S100 of the first embodiment. Processing in the case where no autonomous mobile device Ma is present in the waiting area Aw overlapping with the background area Ab is similar to S100 of the first embodiment.

[0086] 13, the recognition block 200 acquires desired content Cd desired by the user Us based on sensing information from the mobile terminal Mt carried by the user Us. The sensing information at this time is acquired through the mobile terminal Mt at the timing when it is input to the sensor unit 2003 as intention data of the user Us, and is used to recognize the desired content Cd. The items of the desired content Cd and their input are similar to S110 in the first embodiment.

[0087] 13, the search block 210 (see FIG. 12) acquires service capability information that indicates a user service provision capability Pc (see FIG. 15) associated with the autonomous mobile device Ma waiting in the waiting area Aw in the background area Ab recognized in S2100. Details of the provision capability Pc are similar to S120 in the first embodiment.

[0088] 13, the search block 210 searches for an autonomous mobile device Ma having a provision capability Pc that matches the desired content Cd acquired in S2110, from the provision capabilities Pc represented by the service capability information acquired for each autonomous mobile device Ma in the background area Ab in S2120. Acquisition of the matching degree R (see FIG. 15) and extraction of the matching device Mam based thereon are similar to S130 in the first embodiment.

[0089] 13, the search block 210 displays an XR-enhanced image Ipe, which enhances the matching device Mam found in S2130, on the display unit 2002 of the mobile terminal Mt, as shown in FIG. 15. At this time, the XR-enhanced image Ipe is displayed superimposed on a background video Ib showing a background area Ab. Here, the background video Ib is acquired based on sensing information obtained by a sensor unit 4 such as a camera, or three-dimensional space information Ti obtained from an infrastructure database Di. For the background video Ib, primary video data obtained by capturing the background area Ab may be used, or secondary video data obtained by processing the primary data may be used.

[0090] 15, the display unit 2002 superimposes an XR capability image Ip, which is a candidate for the XR-enhanced image Ipe, on the background video Ib to notify each autonomous mobile device Ma of the provision capability Pc for the user service corresponding to the desired content Cd. In particular, the XR capability image Ip is superimposed and aligned with the corresponding autonomous mobile device Ma on the background video Ib to notify the matching degree R obtained in S2130 together with the provision capability Pc.

[0091] The superimposed display position of the XR capability image Ip in S2140 is adjusted to a position surrounding the position where the autonomous mobile device Ma appears in the background video Ib, based on, for example, sensing information from at least one of the sensor units 2003, 2004 and / or three-dimensional space information Ti obtained from the infrastructure database Di. At this time, the background area Ab shown in the background video Ib superimposed with the XR capability image Ip may be fixed to the area recognized in S2100 or updated in accordance with S2100 as the user Us's viewpoint moves. In the above S2140, the extraction of the provision capability Pc corresponding to the desired content Cd and the setting and display of the images Ip and Ipe are similar to S140 in the first embodiment.

[0092] 13, in response to the superimposed display in S2140, the search block 210 determines whether any of the autonomous mobile devices Ma in the background area Ab has been selected by the user Us. At this time, whether or not a selection has been made by the user Us is recognized based on sensing information from the mobile terminal Mt. Therefore, the sensing information for recognizing whether or not a selection has been made by the user Us is input to the sensor unit 2003 as intention data of the user Us, and is thereby acquired through the mobile terminal Mt. Details of the selection input of the autonomous mobile device Ma are similar to S150 in the first embodiment.

[0093] If a selection by the user Us is recognized in S2150 and a positive determination is made, S2160 is executed as shown in Fig. 13. In S2160, the drive control block 220 (see Fig. 12) controls the drive of the autonomous mobile device Ma selected in S2150 within the background area Ab, thereby providing a user service of a type corresponding to the desired content Cd. The details of the user service provided and the drive are similar to those of S160 in the first embodiment.

[0094] In the above-described S2150, a negative determination is made, for example, if the user Us does not perform a selection input within the set time period from the start of execution, or if the user Us indicates that they do not want to select any autonomous mobile devices Ma, etc. In this case, it is determined that the selection by the user Us has been rejected for the autonomous mobile device Ma in the background area Ab, and S2170 is executed as shown in FIGS.

[0095] 14, the search block 210 determines whether or not an autonomous mobile device Ma is waiting outside the recognized background area Ab within a set distance from the user Us. The recognition of the presence or absence here is similar to S170 in the first embodiment.

[0096] If a positive determination is made in S2170, S2180 is executed as shown in Fig. 14. In S2180, the search block 210 associates additional capability information representing the user service provision capability Pc with the autonomous mobile device Ma whose presence was recognized in S2170 and which is waiting outside the background area Ab, in accordance with S120 and S180 of the first embodiment.

[0097] 14, the search block 210 searches for a matching device Mam whose provision capability Pc of the additional capability information acquired for each autonomous mobile device Ma outside the background area Ab in S2180 matches the desired content Cd acquired in S2110. This search conforms to S190 in the first embodiment, which is based on the matching degree R.

[0098] 14, the search block 210 notifies the providing capability Pc represented by the additional capability information acquired for each autonomous mobile device Ma outside the background area Ab in S2180. In S2200, to realize such notification in accordance with the XR capability image Ip, the additional capability image Ia shown in FIG. 16 is superimposed on the background video Ib of the background area Ab by the display unit 2002 of the mobile terminal Mt.

[0099] In particular, in S2200, the additional capability image Ia is superimposed on, for example, an air area or a ground area shown in the background video Ib so as to notify the matching degree R acquired in S2190 together with the provision capability Pc. Here, the additional capability image Ia may notify at least one of the following regarding the location of the corresponding autonomous mobile device Ma: the direction from the user Us, the distance from the user Us, and the time it takes to reach the user Us by calling. For the matching device Mam found in S2190 (in the example of FIG. 16, the autonomous mobile device Ma whose location is displayed as "robot, 3m to the right"), the additional capability image Ia may be set as an additional emphasis image Iae that emphasizes the device Mam in accordance with the XR emphasis image Ipe. Such additional capability image Ia may be superimposed and displayed at a fixed relative position on the screen of the display unit 2002 displaying the background video Ib of the background area Ab, or may be superimposed and displayed as an XR capability image Ip (including an XR-enhanced image Ipe for the matching device Mam) at a fixed relative position on the autonomous mobile device Ma displayed in the background video Ib of the background area Ab. In either case, the background area Ab displayed in the background video Ib superimposed with the additional capability image Ia may be fixed to the area recognized by S2100, or updated in accordance with S2100 as the user Us's viewpoint moves.

[0100] 14, in response to the superimposed display in S2200, the search block 210 determines, in accordance with S2150, whether any of the autonomous mobile devices Ma outside the background area Ab has been selected by the user Us. As a result, if a positive determination is made by recognizing the selection by the user Us, S2220 is executed. In S2220, the drive control block 220 controls the autonomous mobile device Ma selected in S2210 to drive into the background area Ab in order to provide a user service corresponding to the desired content Cd in accordance with S2160. In other words, in S2220, the drive control of the selected autonomous mobile device Ma into the background area Ab enables the service of the desired content Cd to be provided.

[0101] 13 and 14, the current execution of the process flow is completed when the provision of the user service corresponding to the desired content Cd is completed in either S2160 or S2220. On the other hand, if a negative determination is made in either S2170 or S2210, the current execution of the process flow may be completed after an XR notification image notifying the user of the search failure is superimposed on the background video Ib in the background area Ab. In either of these cases, after the current execution is completed, the next service request input is acquired via the sensor unit 2003 of the mobile terminal Mt, and the next execution of the process flow will be initiated.

[0102] (Action and effect) The effects of the second embodiment described above will be explained below.

[0103] According to the second embodiment, service capability information indicating a user service provision capability Pc is acquired from a mobile terminal Mt carried by a user Us, associated with a waiting autonomous mobile device Ma within a background area Ab that serves as a display background for the user Us. Accordingly, by acquiring the desired content Cd of a user service desired by the user Us, attention is focused on the provision capabilities Pc indicated by the service capability information for each autonomous mobile device Ma within the background area Ab, and an autonomous mobile device Ma having a provision capability Pc that matches the desired content Cd is selected. This results in an XR-enhanced image Ipe that highlights an autonomous mobile device Ma (specifically, a matching device Mam) with a capability Pc that matches the desired content Cd. This image is superimposed on a background video Ib showing the background area Ab, allowing the user Us to select the device Ma. This improves the efficiency of searching for an autonomous mobile device Ma that can provide a user service that satisfies the user's preferences by operating within the background area Ab.

[0104] Furthermore, according to the second embodiment, an XR capability image Ip is positioned and superimposed on the background video Ib of the background area Ab for each autonomous mobile device Ma so as to notify the provision capability Pc corresponding to the desired content Cd for each autonomous mobile device Ma in the background area Ab. Therefore, among the XR capability images Ip for each autonomous mobile device Ma, the XR capability image Ip highlighting the autonomous mobile device Ma (specifically, the matching device Mam) whose matching degree R of the provision capability Pc for the desired content Cd falls within the recommended range is set as the XR-enhanced image Ipe. This allows the user Us to appropriately select an autonomous mobile device Ma automatically searched for based on the matching degree R between the desired content Cd and the provision capability Pc, without, for example, making a reservation in advance, from among the autonomous mobile devices Ma in the background area Ab notified of their provision capabilities Pc. Therefore, not only is the search efficiency for an autonomous mobile device Ma capable of providing a user service that satisfies the user's preferences improved, but the search accuracy can also be improved by reducing the gap between the desired content Cd and the provision capability Pc.

[0105] According to the second embodiment, the XR capability image Ip is superimposed on the background video Ib of the background area Ab to notify the matching degree R together with the provision capability Pc. This allows the user Us to intuitively select, from among the autonomous mobile devices Ma notified of their provision capabilities Pc within the background area Ab, the autonomous mobile device Ma that is highlighted along with the notification of the matching degree of the provision capability Pc for the desired content Cd. This can provide reliability to a highly efficient search for an autonomous mobile device Ma that can provide a user service that satisfies the user's preferences.

[0106] Furthermore, according to the second embodiment, if it is determined that the user Us's selection from among the autonomous mobile devices Ma in the background area Ab has been rejected, the service capability information of the autonomous mobile device Ma waiting outside the background area Ab is acquired as additional capability information. An additional capability image Ia notifying the user of the provision capability Pc represented by the additional capability information is then superimposed on the background video Ib of the background area Ab. This allows the user Us to be notified of the search results for the device Ma expanded beyond the background area Ab, even if an autonomous mobile device Ma capable of providing a user service that satisfies the user's preferences is not found within the background area Ab. This ensures fail-safety in the highly efficient search for the autonomous mobile device Ma, giving the user Us a sense of security.

[0107] Moreover, according to the second embodiment, the autonomous mobile device Ma selected by the user Us in response to the superimposed display of the additional capability image Ia is driven into the background area Ab to provide the user service. This allows the user Us to select the autonomous mobile device Ma notified by a highly efficient search outside the background area Ab, and the user Us can receive a user service that satisfies their preferences from the device Ma that is automatically called into the background area Ab.

[0108] (Other embodiments) Although multiple embodiments have been described above, the present disclosure should not be construed as being limited to those embodiments, and can be applied to various embodiments and combinations within the scope that does not deviate from the gist of the present disclosure.

[0109] In the modified examples of the first and second embodiments, the dedicated computer constituting the control system 20, 2020 of the processing system 1, 2001 may have at least one of a digital circuit and an analog circuit as a processor. Here, the digital circuit is at least one of an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a system on a chip (SOC), a programmable gate array (PGA), and a complex programmable logic device (CPLD). Furthermore, such a digital circuit may have a memory that stores a program.

[0110] In a modification of the first and second embodiments, the processing system 1, 2001 may be disconnected from the infrastructure system 6. In this modification, sensing information from connection elements Wt, Ma, and Mt with the processing system 1, 2001 may be used instead of the three-dimensional space information Ti obtained from the infrastructure database Di.

[0111] In the modified examples of the first and second embodiments, three-dimensional spatial information Ti that is not linked to the voxel Vi may be acquired from the infrastructure database Di by the processing system 1, 2001. In the modified examples of the first and second embodiments, two-dimensional grid information may be acquired from the infrastructure database Di by the processing system 1, 2001 instead of the three-dimensional spatial information Ti.

[0112] In a modification of the first embodiment, when a mobile terminal Mt as in the second embodiment is connected to the processing system 1 and functions as part of a wearable terminal Wt, the processing flow may be executed in response to the sensor unit 2003 constituting the part acquiring a service request input as sensing information. In a modification of the first embodiment, when a mobile terminal Mt as in the second embodiment is connected to the processing system 1 and functions as part of a wearable terminal Wt, the desired content Cd in S110 of the processing flow may be acquired based on the sensing information of the sensor unit 2003 constituting the part.

[0113] In a modification of the second embodiment, a smart watch or a video see-through or non-transparent wearable terminal may be used as the mobile terminal Mt. In the modification of the second embodiment, when the wearable terminal Wt as in the first embodiment is connected to the processing system 2001 and functions as a part of the mobile terminal Mt, the processing flow may be executed in response to the sensor unit 3 constituting the part acquiring a service request input as sensing information. In the modification of the second embodiment, when the wearable terminal Wt as in the first embodiment is connected to the processing system 2001 and functions as a part of the mobile terminal Mt, the desired content Cd may be acquired in S2110 of the processing flow based on sensing information from the sensor unit 3 constituting the part.

[0114] In the modified examples of the first and second embodiments, S170 to S220 and S2170 to S2220 may be skipped, and in the case of a negative determination at S150 or S2150, the current execution of the processing flow may be terminated. At S140 and S2140 according to the modified examples of the first and second embodiments, the matching degree R may be excluded from the subject of notification by the XR performance image Ip.

[0115] In S140 and S2140 according to the modified examples of the first and second embodiments, only the XR capability image Ip notifying the provision capability Pc of the matching device Mam, i.e., only the XR capability image Ip set in the XR-enhanced image Ipe, may be displayed as shown in Figure 17 (the figure is a modified example of the first embodiment). However, in the display of such a modified example, if the matching device Mam is not found by S130 and S2130, S140 to S160 and S2140 to S2160 may be skipped, and S170 and S2170 may be executed.

[0116] In S200 and S2200 according to the modified examples of the first and second embodiments, as shown in Fig. 18 (the same figure is a modified example of the first embodiment), only the additional capability image Ia notifying the provision capability Pc of the matching device Mam, that is, only the additional capability image Ia set in the additional emphasis image Iae, may be displayed. However, in the display of such a modified example, if the matching device Mam is not found by S190 and S2190, S200 to S2200 and S2200 to S2220 may be skipped and the current execution of the processing flow may be completed.

[0117] In addition to the embodiments and modifications described above, the processing system 1, 2001 having at least one processor 24 and one memory 22 in the control system 20, 2020 may be implemented in the form of a semiconductor device (for example, a semiconductor chip).

[0118] (Additional remarks) This specification discloses the following technical ideas and combinations thereof.

[0119] (Technical thought 1) A processing system having a processor (24) for performing service-related processing associated with a user service provided to a user, the processing system comprising: The processor: Acquiring desired content (Cd) of the user service desired by the user; Acquiring service capability information indicating a capability (Pc) of providing the user service, by associating it with an autonomous mobile device (Ma) waiting within a visual field (Av) visible to the user through a wearable device (Wt) worn by the user; Searching for and highlighting an autonomous driving device having a provision capability that matches the desired content from the provision capabilities represented by the service capability information for each autonomous driving device within the field of view area, and superimposing the XR-enhanced image (Ipe) on the field of view area; and and providing the user service by driving the autonomous driving device selected by the user within the field of view in response to the superimposed display of the XR-enhanced image.

[0120] (Technical thought 2) The superimposed display of the XR-weighted image is A processing system as described in technical idea 1, which includes setting the XR capability image (Ip) that highlights an autonomous driving device whose matching degree (R) of the provision capability for the desired content is within a recommended range as the XR emphasized image, among the XR capability images (Ip) that are aligned and superimposed on the field of view for each autonomous driving device in the field of view, so as to notify each autonomous driving device of the provision capability corresponding to the desired content.

[0121] (Technical Thought 3) The superimposed display of the XR-weighted image is A processing system described in technical idea 2, which includes superimposing the XR capability image on the field of view area so as to notify the matching degree together with the provision capability.

[0122] (Technical Thought 4) obtaining the service capability information, When it is determined that the user's selection of the autonomous mobile device within the field of view has been rejected, acquiring the service capability information of the autonomous mobile device waiting outside the field of view as additional capability information; The superimposed display of the XR-weighted image is superimposing an additional capability image (Ia) notifying the provision capability represented by the additional capability information on the field of view; A processing system described in Technical Idea 2 or 3, which includes driving the autonomous driving device selected by the user in response to the superimposed display of the additional capability image into the field of view to provide the user service.

[0123] (Technical Thought 5) A processing system having a processor (24) for performing service-related processing associated with a user service provided to a user, the processing system comprising: The processor: Acquiring desired content (Cd) of the user service desired by the user; Acquiring service capability information indicating a capability (Pc) of providing the user service from a mobile terminal (Mt) carried by the user, linked to an autonomous mobile device (Ma) waiting in a background area (Ab) that serves as a display background for the user; Searching for and highlighting an autonomous driving device having a provision capability that matches the desired content from the provision capabilities represented by the service capability information for each autonomous driving device in the background area, and displaying the highlighted XR-enhanced image (Ipe) superimposed on a background video (Ib) showing the background area; and providing the user service by driving the autonomous driving device selected by the user within the background area in response to the superimposed display of the XR-enhanced image.

[0124] (Technical Thought 6) The superimposed display of the XR-weighted image is A processing system as described in technical idea 5, which includes setting the XR capability image (Ip) that highlights an autonomous driving device whose matching degree (R) of the provision capability for the desired content is within a recommended range as the XR emphasized image, among the XR capability images (Ip) that are aligned and superimposed on the background video for each autonomous driving device in the background area so as to notify the provision capability corresponding to the desired content for each autonomous driving device.

[0125] (Technical Thought 7) The superimposed display of the XR-weighted image is A processing system described in technical idea 6, which includes superimposing the XR capability image on the background video so as to notify the matching degree together with the provision capability.

[0126] (Technical Thought 8) obtaining the service capability information, When it is determined that the user's selection of the autonomous mobile device within the background area has been rejected, acquiring the service capability information of the autonomous mobile device waiting outside the background area as additional capability information; The superimposed display of the XR-weighted image is superimposing an additional capability image (Ia) notifying the providing capability represented by the additional capability information on the background video; The processing system described in Technical Idea 6 or 7 includes driving the autonomous driving device selected by the user in response to the superimposed display of the additional capability image into the background area to provide the user service.

[0127] (Technical Thought 9) The superimposed display of the XR-weighted image is A processing system described in any one of technical ideas 2 to 4, 6 to 8, which includes obtaining the matching degree correlated with the compatibility rate scored for each item other than the type contained in the desired content, regarding the ability to provide the user service for the type represented by the desired content.

[0128] The above-mentioned technical concepts 1 to 9 may be understood as the respective technical concepts of a method and a program. [Explanation of symbols]

[0129] 1, 2001: Processing system, 22: Memory, 24: Processor, Ab: Background area, Av: Field of view area, Cd: Desired content, Ia: Additional capability image, Ib: Background video, Ip: XR capability image, Ipe: XR enhanced image, Ma: Autonomous driving device, Mt: Mobile terminal, R: Matching degree, Wt: Wearable terminal

Claims

1. A processing system having a processor (24) for performing service-related processing related to a user service provided to a user, the processing system comprising: The processor: Obtaining desired content (Cd) of the user service desired by the user; Acquiring service capability information representing a capability (Pc) of providing the user service, by linking it to an autonomous mobile device (Ma) waiting within a visual field (Av) visible to the user through a wearable terminal (Wt) worn by the user; superimposing an XR-enhanced image (Ipe) on the field of view, the XR-enhanced image (Ipe) being obtained by searching for and highlighting an autonomous mobile device having a provision capability that matches the desired content from the provision capabilities represented by the service capability information for each autonomous mobile device within the field of view; and and providing the user service by driving the autonomous driving device selected by the user within the field of view in response to the superimposed display of the XR-enhanced image.

2. The superimposed display of the XR-weighted image is The processing system of claim 1 further comprises setting, as the XR-enhanced image, an XR capability image (Ip) that highlights an autonomous driving device whose matching degree (R) of the provision capability for the desired content is within a recommended range, among XR capability images (Ip) that are aligned and superimposed on the field of view for each autonomous driving device in the field of view, so as to notify the provision capability corresponding to the desired content for each autonomous driving device in the field of view.

3. The superimposed display of the XR-weighted image is The processing system according to claim 2 , further comprising: displaying the XR capability image superimposed on the field of view area so as to notify the matching degree together with the providing capability.

4. obtaining the service capability information, When it is determined that the user's selection of the autonomous mobile device within the field of view has been rejected, acquiring the service capability information of the autonomous mobile device waiting outside the field of view as additional capability information; The superimposed display of the XR-weighted image is superimposing an additional capability image (Ia) notifying the provision capability represented by the additional capability information on the field of view; The processing system of claim 2 or 3 further comprises driving the autonomous driving device selected by the user in response to the superimposed display of the additional capability image into the field of view to provide the user service.

5. A processing system having a processor (24) for performing service-related processing related to a user service provided to a user, the processing system comprising: The processor: Obtaining desired content (Cd) of the user service desired by the user; Acquiring service capability information representing a capability (Pc) of providing the user service from a mobile terminal (Mt) carried by the user, the information being linked to an autonomous mobile device (Ma) waiting in a background area (Ab) that serves as a display background for the user; Searching for and highlighting an autonomous mobile device having a provision capability that matches the desired content from the provision capabilities represented by the service capability information for each autonomous mobile device in the background area, and displaying the highlighted XR-enhanced image (Ipe) superimposed on a background video (Ib) showing the background area; and providing the user service by driving the autonomous driving device selected by the user within the background area in response to the superimposed display of the XR-enhanced image.

6. The superimposed display of the XR-weighted image is The processing system of claim 5 further comprises setting, as the XR-enhanced image, an XR capability image (Ip) that highlights an autonomous driving device whose matching degree (R) of the provision capability for the desired content is within a recommended range, among XR capability images (Ip) that are aligned and superimposed on the background video for each autonomous driving device in the background area so as to notify the provision capability corresponding to the desired content for each autonomous driving device.

7. The superimposed display of the XR-weighted image is The processing system according to claim 6 , further comprising: superimposing the XR capability image on the background video so as to notify the matching degree together with the provision capability.

8. obtaining the service capability information, When it is determined that the user's selection of the autonomous mobile device within the background area has been rejected, acquiring the service capability information of the autonomous mobile device waiting outside the background area as additional capability information; The superimposed display of the XR-weighted image is superimposing an additional capability image (Ia) notifying the providing capability represented by the additional capability information on the background video; The processing system of claim 6 or 7, further comprising driving the autonomous driving device selected by the user in response to the superimposed display of the additional capability image into the background area to provide the user service.

9. The superimposed display of the XR-weighted image is A processing system as described in any one of claims 2, 3, 6 and 7, which includes obtaining the matching degree correlated with the relevance rate scored for each item other than the type contained in the desired content, regarding the ability to provide the user service for the type represented by the desired content.

10. A processing method executed by a processor (24) to perform service-related processing related to a user service provided to a user, comprising: Obtaining desired content (Cd) of the user service desired by the user; Acquiring service capability information representing a capability (Pc) of providing the user service, by linking it to an autonomous mobile device (Ma) waiting within a visual field (Av) visible to the user through a wearable terminal (Wt) worn by the user; superimposing an XR-enhanced image (Ipe) on the field of view, the XR-enhanced image (Ipe) being obtained by searching for and highlighting an autonomous mobile device having a provision capability that matches the desired content from the provision capabilities represented by the service capability information for each autonomous mobile device within the field of view; and and providing the user service by driving the autonomous driving device selected by the user within the field of view in response to the superimposed display of the XR-enhanced image.

11. A processing program including instructions stored in a storage medium (22) and executed by a processor (24) to perform service-related processing related to a user service provided to a user, Obtaining desired content (Cd) of the user service desired by the user; Acquiring service capability information representing a capability (Pc) of providing the user service, by linking it to an autonomous mobile device (Ma) waiting within a visual field (Av) visible to the user through a wearable terminal (Wt) worn by the user; superimposing an XR-enhanced image (Ipe) on the field of view, the XR-enhanced image (Ipe) being obtained by searching for and highlighting an autonomous mobile device having a provision capability that matches the desired content from the provision capabilities represented by the service capability information for each autonomous mobile device within the field of view; and and providing the user service by driving the autonomous driving device selected by the user within the field of view in response to the superimposed display of the XR-enhanced image.

12. A processing method executed by a processor (24) to perform service-related processing related to a user service provided to a user, comprising: Obtaining desired content (Cd) of the user service desired by the user; Acquiring service capability information representing a capability (Pc) of providing the user service from a mobile terminal (Mt) carried by the user, the information being linked to an autonomous mobile device (Ma) waiting in a background area (Ab) that serves as a display background for the user; Searching for and highlighting an autonomous mobile device having a provision capability that matches the desired content from the provision capabilities represented by the service capability information for each autonomous mobile device in the background area, and displaying the highlighted XR-enhanced image (Ipe) superimposed on a background video (Ib) showing the background area; and providing the user service by driving the autonomous driving device selected by the user within the background area in response to the superimposed display of the XR-enhanced image.

13. A processing program including instructions stored in a storage medium (22) and executed by a processor (24) to perform service-related processing related to a user service provided to a user, Obtaining desired content (Cd) of the user service desired by the user; Acquiring service capability information representing a capability (Pc) of providing the user service from a mobile terminal (Mt) carried by the user, the information being linked to an autonomous mobile device (Ma) waiting in a background area (Ab) that serves as a display background for the user; Searching for and highlighting an autonomous mobile device having a provision capability that matches the desired content from the provision capabilities represented by the service capability information for each autonomous mobile device in the background area, and displaying the highlighted XR-enhanced image (Ipe) superimposed on a background video (Ib) showing the background area; and providing the user service by driving the autonomous driving device selected by the user within the background area in response to the superimposed display of the XR-enhanced image.

Images