Information processing device, information processing method, and program

Abstract

To flexibly and accurately respond to the diverse user demands for sensing processing. [Solution] The control unit receives a sensing intent in natural language from the sensing requester, specifying the event to be sensed. The control unit then transmits a sensing request corresponding to the sensing intent to a management device that distributes sensing data acquired by a wireless communication device. The control unit then receives sensing data in response to the sensing request from the management device. Furthermore, the control unit detects the event specified in the sensing intent from the received sensing data. Finally, the control unit transmits the detected event to the requester.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] In Non-Patent Document 1 below, 5G wireless sensing technology is described as a technology for obtaining data from wireless signals (signals such as reflection, refraction, and diffraction) affected by an object or environment. Also, 5G wireless sensing technology is described as obtaining information regarding the characteristics of an environment and / or an object within the environment (such as shape, size, orientation, speed, position, distance between objects, or relative movement).

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

[0004] However, conventional technologies do not anticipate scenarios such as multiple vehicles requesting sensing for a specific location after traveling at their own speed for a certain period of time to detect obstacles in the future, and then processing the obtained sensing results. Furthermore, they do not anticipate use cases where a vehicle warns the driver to take evasive action if there is an obstacle exhibiting dangerous behavior at that location. User requirements for sensing processing via wireless communication are diverse, and these are not the only examples. An aspect of the disclosed embodiment is its ability to flexibly and accurately respond to diverse user requirements for sensing processing. [Means for solving the problem]

[0005] In one aspect, the embodiments of the disclosure are exemplified by an information processing device including a control unit. The control unit receives a sensing intent in natural language from a sensing requester, specifying the event to be sensed. The control unit then transmits a sensing request corresponding to the sensing intent to a management device that distributes sensing data acquired by a wireless communication device. The control unit then receives sensing data in response to the sensing request from the management device. Furthermore, the control unit detects the event specified in the sensing intent from the received sensing data. The control unit then transmits the detected event to the requester. [Effects of the Invention]

[0006] This information processing device can flexibly and accurately respond to a wide range of user requests for sensing processing. [Brief explanation of the drawing]

[0007] [Figure 1] Figure 1 illustrates the configuration of a network to which the information processing device of the first embodiment is connected. [Figure 2] Figure 2 illustrates a vehicle that requests processing from the information processing device of this embodiment. [Figure 3] Figure 3 illustrates the components that make up the core network of a fifth-generation mobile communication system. [Figure 4] Figure 4 is a flowchart illustrating the event detection process by the information processing device of the first embodiment. [Figure 5] Figure 5 is an example of the data in the target location definition dictionary. [Figure 6] Figure 6 is an example of the data in the sensing target definition dictionary. [Figure 7] Figure 7 is an example of the data in the event type definition dictionary. [Figure 8] Figure 8 is a flowchart illustrating the event detection process of the second embodiment. [Figure 9] Figure 9 is a flowchart illustrating the event detection process of the third embodiment. [Figure 10] Figure 10 is a flowchart illustrating the event detection process of the third embodiment. [Figure 11] Figure 11 is a flowchart illustrating the sensing instruction aggregation process A of the fourth embodiment. [Figure 12] Figure 12 is a flowchart illustrating a modified example of sensing instruction aggregation process B. [Modes for carrying out the invention]

[0008] Hereinafter, this disclosure will describe the information processing device 6, the information processing method, and the program with reference to the drawings of the embodiment. In this embodiment, the information processing device 6 includes a control unit 60 (see Figure 3). The control unit 60 receives a sensing intent in natural language from a sensing requester, specifying the event to be sensed. The control unit 60 then transmits a sensing request corresponding to the sensing intent to a management device that distributes sensing data acquired by a base station 3A, UE2, etc. (see Figure 1), which is an example of a wireless communication device. The control unit 60 then receives sensing data in response to the sensing request from the management device. Furthermore, the control unit 60 detects the event specified in the sensing intent from the received sensing data. The control unit 60 then transmits the detected event to the requester.

[0009] Here, sensing is exemplified as a technique for obtaining data from radio signals (signals such as reflection, refraction, and diffraction) that have been affected by an object or environment, for example, in a mobile communication system. Furthermore, the radio signal is emitted by wireless communication equipment such as base stations 3A and UE2, and is received by the same or other wireless communication equipment as a signal that has been affected by reflection, refraction, diffraction, etc., from the object being sensed. The wireless communication equipment acquires sensing data from the received radio signal.

[0010] In this embodiment, the source of the sensing request is a computer connected to the mobile communication system, such as a UE2 mounted on a mobile device or carried by a person. The management device is, for example, a network function (NF) sensing in the core network (5GC) of a fifth-generation mobile communication system (5G network, also called 5GNW). This is known as 11n (see Figure 3). However, the configuration of this embodiment is not limited to fifth-generation mobile communication systems, but is also applicable to sixth-generation mobile communication systems and later.

[0011] Furthermore, in this embodiment, an event is, for example, that a sensing target is detected. Also, an event may be that information on the location where the sensing target is located is detected, the type of event including the attributes of the target, the situation of the target, for example, the size, material, presence or absence of movement, movement speed, etc. is detected. Based on such a sensing intention in natural language from the sensing requester, this information processing apparatus generates a corresponding sensing request, transmits the generated sensing request to the management apparatus, detects an event based on the sensing data received as a result, and transmits it to the requester.

[0012] <First Embodiment> Referring to FIGS. 1 to 4, an information processing apparatus 6, an information processing method, and a program according to the first embodiment will be described.

[0013] (Application Example) FIG. 1 is a diagram illustrating the configuration of a network N1 to which the information processing apparatus 6 of this embodiment is connected. As shown in FIG. 1, in this embodiment, the information processing apparatus 6 communicates with various devices connected to the network N1 and executes processing. In this embodiment, the information processing apparatus 6 and the devices connected to the network N1 are referred to as an information communication system 100. The network N1 is exemplified by a communication network including at least one of mobile communication systems such as Long Term Evolution (LTE), the fifth-generation mobile communication system (5G), and the sixth-generation mobile communication system (6G), and the Internet. is exemplified by a communication network including at least one of mobile communication systems such as Long Term Evolution (LTE), the fifth-generation mobile communication system (5G), and the sixth-generation mobile communication system (6G), and the Internet.

[0014] For example, the information processing apparatus 6 is connected to user devices (UE2-1 to UE2-3, etc.) mounted on vehicles C1 to C3, etc. via the network N1. The UE2-1 to UE2-3, etc. mounted on the vehicles C1 to C3 are devices that are requesters for requesting data on sensing results (hereinafter simply referred to as sensing data) by sensing processing. Hereinafter, when UE2-k (k is an integer) is collectively referred to, it is simply called UE2.

[0015] Further, the information processing apparatus 6 is connected to SENSING 11n, which is an example of a management apparatus, via the network N1. Further, SENSING 11n is connected to base stations 3A-1, 3A-2, UEs 2-4, UEs 2-5, etc. via the network N1. Hereinafter, when the base stations 3A-k (k is an integer) are collectively referred to, they are simply called base stations 3A. The base stations 3A and UEs 2 are examples of wireless communication apparatuses.

[0016] The base stations 3A, UEs 2, etc. have, for example, a transmitter and a receiver and operate as sensors. That is, these sensors execute radiation of electromagnetic waves (i.e., radio signals) by the transmitter and reception of reflected waves, refracted waves, diffracted waves, etc. from the sensing target by the receiver, and detect the sensing target. SENSING 11n instructs these sensors to detect (F4-1 to F4-4) and receives sensing data (from F5-1 to F5-4).

[0017] Note that the sensor may be, for example, a pair of a transmitter and a receiver of one base station 3. Further, the sensor may be, for example, a pair of a transmitter of the first base station 3A-1 and a receiver of a second base station 3A-2 other than the first base station 3A-1. Further, the sensor may be, for example, a pair of a transmitter and a receiver of one UE 2. Further, the sensor may be, for example, a pair of a transmitter of the first UE 2-1 and a receiver of a second UE 2-2 other than the first UE 2-1. Further, the sensor may be, for example, a pair of a transmitter of the base station 3A and a receiver of the UE 2. Further, the sensor may be, for example, a pair of a transmitter of the UE 2 and a receiver of the base station 3A. Note that, in the present embodiment, the signal transmitted by the transmitter to detect the sensing target is called a reference signal.

[0018] Furthermore, in this embodiment, the information processing device 6 may perform processing in cooperation with other computers on the network N1, such as a server 6A. The server 6A may, for example, manage various databases. Examples of databases include a map database, various dictionaries, a rule base, and a base station database that stores location information of base stations 3A. Also, the server 6A is not limited to a single computer, but may consist of multiple computers. In addition, the information processing device 6 and one or more servers 6A may be part of a virtual system called a cloud.

[0019] Figure 2 illustrates vehicles C1 to C3, which are requesting processing from the information processing device 6 of this embodiment. As shown in Figure 2, for example, vehicles C1 and C2 are traveling towards an area (geographical region) including point A1, and it is assumed that they wish to acquire information near point A1. Vehicle C3 is also traveling towards an area including point A2, and it is assumed that it wishes to acquire information near point A2. Here, the information desired by vehicles C1 to C3, etc., is, for example, the presence or absence of obstacles (the presence of bicycles, pedestrians, etc.).

[0020] Figure 2 shows three request sources. However, in reality, it is expected that sensing processing requests to the information processing device 6 will be issued from a very large number of devices. Furthermore, it is expected that the sensing processing requests to the information processing device 6 will vary widely in terms of the target of detection, the detection area (location), the detection conditions, the status of the target, the type of event, etc. In this embodiment, it is illustrated that the information processing device 6 efficiently processes a large number of diverse sensing processing requests.

[0021] In other words, as illustrated in Figure 1, UE2-1 to UE2-3 notify the information processing device 6 of the specifications for sensing processing, for example. In this embodiment, the specifications for sensing processing are called sensing intentions (F1-1 to F1-3). Sensing intentions include what to detect (target), where to detect (location, area), and how to detect (presence or absence, movement or absence, movement speed, detection accuracy, etc.). Presence or absence, movement or absence, movement speed, etc. can also be described as types of events.

[0022] When the information processing device 6 receives a sensing intent, it generates a sensing request (F2 in Figure 2). That is, the information processing device 6 analyzes the sensing intent and generates a sensing request. When analyzing the sensing intent, the information processing device 6 may also perform aggregation of sensing results. In other words, if there is already acquired sensing data that matches the sensing intent, the information processing device 6 can use the acquired sensing data without generating a new sensing request. Furthermore, if sensing processes corresponding to multiple sensing intents can be aggregated into a single sensing request, the information processing device 6 can perform aggregation.

[0023] The information processing device 6 then notifies SENSING 11n of the sensing request (F3). The sensing request can also be called the activation of a sensing task. The sensing request includes, for example, the sensing area (or location), notification conditions, detection accuracy, etc. Here, the sensing area (or location) is information that identifies the location where the sensing process is to be performed. The notification conditions are information that specifies the conditions under which the sensing results from the sensing process are to be reported. The notification conditions include the number of notifications and the timing of notifications. If there are multiple notifications, the notification cycle and the length of the notification period (or the start and end times of the notifications) may also be specified. The notifications may also continue until a notification termination message is sent. Furthermore, the notification conditions may be such that notifications are only generated when a specific event occurs. Thus, SENSING 11n can be considered an example of a management device that distributes sensing data acquired by a wireless communication device.

[0024] SENSING 11n instructs each sensor to perform sensing processing in the specified sensing area (or location) according to the sensing request (F4-1 to F4-4), and receives sensing data (F5-1 to F5-4). Then, SENSING 11n notifies the information processing device 6 of the sensing data according to the notification conditions (F6).

[0025] The information processing device 6 receives sensing data and performs filtering (F7). The filtering process selects sensing data that matches the sensing intent from the requester. This is a beneficial process. If sensing data that matches the sensing intent is obtained, it can be said that an event has been detected. Therefore, if the information processing device 6 obtains sensing data that matches the sensing intent from the requester, it notifies the requester of the obtained data (F8). The notification in F8 can be called a notification of a detected event.

[0026] In this embodiment, detecting an event includes, for example, detecting the presence or absence of a sensing object, detecting the location of the sensing object, and detecting the state and type of the sensing object. The location of the sensing object is a geographical location, such as latitude and longitude. The state of the sensing object includes the size, material, whether it is moving or not, and the speed of movement of the sensing object. The size, material, whether it is moving or not, and the speed of movement of the sensing object can also be referred to as the type of sensing or the type of event.

[0027] Here, SENSING 11n can obtain the position of a sensing target by instructing three or more sensing entities (base station 3A or other UE2) to measure the latitude and longitude of the sensing target using a triangulation method. Furthermore, SENSING 11n can obtain the material of a sensing target by instructing the sensing entities (base station 3A or other UE2) to measure the material based on the electromagnetic wave absorption rate of the sensing target, using the power of the electromagnetic waves radiated to the sensing entity (base station 3A or other UE2) and the power of the received reflected waves.

[0028] (Network example) Figure 3 illustrates the components (configurations) that make up the core network (5GC) of the fifth-generation mobile communication system (5G network, also called 5GNW) within the information and communication system 100. In this embodiment, the components of 5GC are collectively called Network Function (hereinafter NF11), and individually they are called, for example, Access and Mobility Management Function (hereinafter AMF 11b). Figure 3 shows each component Each term has a generic symbol along with an individual symbol in parentheses.

[0029] However, the types of NF11 are not limited to those shown in the examples in Figure 2. UPF (User Plane Function) 11a AMF (Access and Mobility Management Function)11b SMF (Session Management Function)11c PCF(Policy Control Function)11d NEF(Network Exposure Function)11e NRF(Network Repository Function)11g NSSF(Network Slice Selection Function)11h AUSF(Authentication Server Function)11i UDM(Unified Data Management)11j NWDAF(Network Data Analytics Function)11k SENSING (Sensing Function) 11n

[0030] UPF11a performs routing and forwarding of user packets (user plane packets sent and received by UE2), packet inspection, and QoS processing. UPF11a uses DN (Data It connects to Network 5. DN5 is an external data network (such as the Internet) outside of 5GC.

[0031] AMF11b is the location-based accommodation device for UE2 in the core network. AMF11b accommodates RAN3 (base stations) and performs subscriber authentication control, UE2 location (mobility), and registration area management. UDM11j is a database (storage device) that provides subscriber information and retrieves, registers, deletes, and modifies the status of UE2.

[0032] SMF11c manages PDU (Protocol Data Unit) sessions and controls UPF11a for QoS (Quality of Service) control and policy control. A PDU session is a virtual communication channel for data exchange between UE2 and DN5.

[0033] PCF11d performs QoS control, policy control, and billing control under the control of SMF11c. QoS control involves controlling the quality of communication, such as prioritizing packet forwarding. Policy control involves communication control, such as QoS, packet forwarding eligibility, and billing, based on network or subscriber information.

[0034] The NEF11e acts as an intermediary for communication between external nodes (external devices) such as the AF (Application Function) 12 and nodes (NFs) within the control plane. In other words, the NEF11e functions as a gateway (GW) between the core network and the external network. The AF 12 is, for example, an application server (external server) located outside the core network (e.g., connected to DN5).

[0035] NRF11g stores and manages information about the NFs that make up the core network. In response to an inquiry about an NF that the user wishes to use, NRF11g can return multiple candidate NFs to the inquirer.

[0036] NSSF11h has the function of selecting the network slice to be used by the subscriber from among the network slices generated by network slicing. A network slice is a virtual network with specifications tailored to its intended use.

[0037] AUSF11i is a subscriber authentication server that performs subscriber authentication under the control of AMF11b. NWDAF 11k has the function of collecting and analyzing data from each NF11, OAM (Operations, Administration, and Maintenance) terminal, AF12, etc. NWDAF 11k is an NF that provides analytical information related to 5GS.

[0038] UDM11j maintains subscriber-related information, provides subscriber information, and retrieves, registers, deletes, and modifies the status of UE2. SENSING 11n performs sensing services, including collecting sensing information from UE2, RAN3 (base station (gNB)), or other nodes, and providing the collected sensing information to UE2 or other external systems (AF12, DN5, etc.). Details of SENSING 11n will be described later.

[0039] AF12 is an NF that processes sensing data and provides application services using that sensing data to UE2 (terminal). For example, AF12 notifies UE2 (terminal) of the sensing results acquired by SENSING 11n within a specified spatial range. Alternatively, an application program executed on UE2 (terminal) may also operate as AF12. The functions of NF11 described above are just examples; each NF11 may have other functions, execute the functions of other NF11s, or multiple NF11s may cooperate to execute a single function.

[0040] These FN11s are defined, for example, in 3GPP® TS23.501. DN5 is an external data network (such as the Internet) outside of 5GC. For example, an information processing device 6 is connected to DN5. The information processing device 6 may also be AF12 of 5GC. RAN3 is a wireless access network to 5GC. RAN3 is formed, for example, by base station 3A. Note that the information communication system 100 may not be the entire system shown in Figure 3, but rather a combination of some of the 5GC FN11s illustrated in Figure 3.

[0041] Of the FN11 units, the AMF 11b is the UE location accommodation device in 5GC. The AMF 11b accommodates RAN3 and performs subscriber authentication control, UE2 location (mobility) management, etc.

[0042] NWDAF 11k is an NF11 that provides data analysis for 5G networks. NWDAF 11k notifies other NF11s (e.g., AMF 11b, SMF 11c, PCF 11d, etc.) of the data analysis results, supporting the dynamic network control and management of these NF11s. Examples of data analysis results provided by NWDAF 11k include latency per UE2, UE2 movement path, location information, movement speed, movement direction, and material. Other examples of data analysis results include load levels (resource utilization for each cell or base station), future load predictions, load distribution by area, and resource block or frequency availability information. For more information on NWDAF 11k, see, for example, 3GPP® TS 29.520 or TS 23.288, which define its functions and processing.

[0043] SENSING 11n performs processing including collecting information from UE2 or other external systems, analyzing the collected information, and providing the analysis results to other FN11, UE2, AF12, or other external systems (such as DN5). However, NWDAF 11k may perform the analysis processing of the sensing results by SENSING 11n instead of SENSING 11n.

[0044] SENSING 11n, other NF11, AF12, etc. are formed on a computer according to a computer program. The configurations of SENSING 11n, other NF11, AF12, etc. are virtual and may be formed on different computers or on multiple computers. Alternatively, any multiple of SENSING 11n, other NF11, AF12, etc. may be formed on the same computer. Furthermore, such a computer may have a configuration similar to that of the information processing device 6 or UE2 connected to the data network DN5.

[0045] These computers consist of a Central Processing Unit (CPU61) and main memory. The device 62 and external equipment are included, and information processing and communication processing are performed by a computer program. The CPU 61 is also called a processor. The CPU 61 is not limited to a single processor, but may be a multi-processor configuration. Furthermore, the CPU 61 may include a Graphics Processing Unit (GPU), a Digital Signal Processor (DSP), etc.

[0046] The CPU 61 executes the computer program that has been loaded into the main memory 62 in an executable format, and provides processing to the information processing device 6. The main memory 62 stores the computer program executed by the CPU 61, the data that the CPU 61 processes, etc. The CPU 61 and the main memory 62 are called the control unit 60.

[0047] Examples of external devices include an external storage device 63, an output device 64, an operating device 65, and a communication device 66. The external storage device 63 is used, for example, as a storage area that supplements the main memory 62, and stores computer programs executed by the CPU 61, data processed by the CPU 61, and the like.

[0048] The output device 64 is, for example, a display device such as a liquid crystal display or an electroluminescent panel. However, the output device 64 may also include a speaker or other device that outputs sound. The operating device 65 may be, for example, a touch panel with a touch sensor superimposed on the display of the output device 64. The communication device 66 is, for example, a server that accesses the network N1 provided by the information and communication system 100 and is connected to the network. It communicates with B6A, etc. (see Figure 1).

[0049] (Example of processing) Figure 4 is a flowchart illustrating the event detection process by the information processing device 6 of the first embodiment. This process starts, for example, when the information processing device 6 receives a request to execute a sensing process from a requester. Here, the requester is, for example, UE2-1 mounted on vehicle C1, UE2-2 carried by a person, etc.

[0050] In this process, the information processing device 6 first receives the sensing intent from the requester (S1). The sensing intent may be described in natural language. However, the sensing intent in natural language may be defined by words in a predetermined order. The predetermined order is, for example, a combination of words indicating location, a word indicating the object to be sensed, the situation of the object, and a word indicating the type of event. That is, the format of such word combinations may not be fixed by the Application Programming Interface (API) provided by the information processing device 6 to the requester UE2. good.

[0051] In this case, the sensing intent is, for example, "Intersection A, obstacle, presence or absence." Here, Intersection A is, for example, the name of a point searchable in a map database. An obstacle is, for example, an object with dimensions exceeding a certain standard value. Presence or absence is an example of the type of situation or event being sensed, and is information that specifies whether or not the object being sensed exists.

[0052] The requesting UE2 may, for example, execute an application program (hereinafter referred to as "app") for sending a sensing intent to the information processing device 6. The app may, for example, present the requesting UE2 with pull-down menus in its user interface for selecting a word indicating location, a word indicating the object to be sensed, and a word indicating the status or type of the object. The UE2 can then generate a sensing intent from the combination of multiple words selected by the user in the pull-down menus and send it to the information processing device 6.

[0053] However, UE2 may also receive the sensing intent in natural language from the user's text input or speech, and transmit it to the information processing device 6. In the example in Figure 4, the sensing intent is specified in natural language. For example, the sensing intent is "Tell me if there is an obstacle at intersection A." Processing S1 is an example of receiving a sensing intent in natural language from the sensing requestor, specifying the event to be sensed.

[0054] When the information processing device 6 receives a sensing intent written in Japanese as natural language, it performs morphological analysis (S2). As a result of morphological analysis, morphemes separated into parts of speech, such as "A, intersection, no, obstacle, ga, aru, ka, oshiete" (Please tell me if there is an obstacle at A intersection), are obtained. The same applies when the natural language is Chinese. When the natural language is Japanese, the information processing device 6 may unify the verbs obtained as a result of morphological analysis into their terminal forms. On the other hand, when the natural language is formed by a sequence of words, such as English, the information processing device 6 may omit morphological analysis and simply process the sequence of words as is.

[0055] Next, the information processing device 6 refers to a thesaurus (S3) and identifies words indicating location, words indicating the sensing object, and words indicating the type of target situation or event from the results of morphological analysis (or sequence of words). For example, the thesaurus defines words indicating location as place, point, ahead, before, forward, intersection, etc. Therefore, from the word "intersection" included in the sensing intent, the information processing device 6 can identify the location as "intersection" by combining it with the preceding proper noun "A" to identify the location as "A intersection".

[0056] Furthermore, for example, a thesaurus defines words that indicate an object as "object," "obstacle," "hindrance," "obstacle," etc. Therefore, the information processing device 6 can identify the word indicating the object as "obstacle" from the word "obstacle" included in the sensing intent.

[0057] Furthermore, for example, the thesaurus defines words that indicate a situation, such as situation, type, exist, be, presence, movement, and speed. Therefore, the information processing device 6 can identify from the word "exist" included in the sensing intent that the word indicating the situation (or type of event) of the target is "exist," and can recognize the intent to request the presence or absence of something.

[0058] Furthermore, for example, the thesaurus may define words indicating instructions as such as "instruct," "teach," "instruct," "inform," and "show." In this case, the information processing device 6 can identify the word indicating an instruction as "teach" from the word "teach" included in the sensing intent, and recognize the intent requested by the requester.

[0059] The thesaurus can be built, for example, in the main memory 12 or the external memory 13. Alternatively, the thesaurus may be built on another computer connected to the network N1 (such as server 6A in Figure 1). The thesaurus is an example of a first database that defines the relationship between information contained in sensing intent and information contained in sensing request. The information processing device 6 may also use a rule-based system instead of, or in conjunction with, the thesaurus to identify words indicating location, words indicating sensing object, and words indicating situation or type. Examples of rule-based rules are as follows:

[0060] Rule: "IF the sensing intent includes a place synonym, then fix the place synonym before and after it." "There is a noun, then the combination of a place synonym and a proper noun identifies the sensing area." According to this rule, if the sensing intent expressed in natural language has a synonym for a location in the thesaurus, and proper nouns are present before or after it, the sensing area is determined. For example, if morphological analysis reveals that the proper noun "A" and the word "intersection" (a common noun) are present before and after it, the information processing device 6 processes "A intersection" as a term that identifies the sensing area.

[0061] Next, the information processing device 6 sets a sensing area in which sensing processing will be performed, including a location (point, area) identified by a word indicating a location, and requests the execution of sensing processing (S4). More specifically, the information processing device 6 searches the map database based on the term that identifies the sensing area (e.g., "A intersection") obtained in processing S3, and finds the geographical location (e.g., latitude and longitude) of "A intersection".

[0062] Next, the information processing device 6 determines the sensing area. The sensing area may be identified, for example, by cell identification information (hereinafter referred to as cell ID) that encompasses a point or area specified by a word indicating location, or by a unit of tracking area (TA). The communication area provided by the telecommunications carrier is divided into TAs and identified by a Tracking Area Code (TAC). A TA includes one or more base stations 3A, i.e., cells. Therefore, the TAC can be used as location information. The sensing area may include one or more cell IDs or one or more TACs.

[0063] In this embodiment, the information processing device 6 refers to a base station database containing base station IDs, cell IDs, latitude, longitude, and cell radii for each frequency in order to determine the sensing area. The base station database may be built on, for example, a computer on the network N1 (server 6A in Figure 1). Alternatively, the information processing device 6 may have the base station database internally. The information in the base station database is provided, for example, by a telecommunications company operating the information communication system 100. In this way, the information processing device 6 determines the sensing area obtained in the processing of S3. Find the cell or TA that contains the geographical location (e.g., latitude and longitude) of a term that identifies a single area (e.g., "Intersection A").

[0064] Then, the information processing device 6 requests the SENSING 11n to execute sensing processing via the NEF 11e. Here, the NEF 11e performs processing to receive information securely from the information processing device 1, which is running an external application, to the 5GC. A request to execute sensing processing with a set sensing area is an example of a sensing request corresponding to a sensing intention. Processing S3 and S4 are examples of determining a sensing request corresponding to a sensing intention. Processing S4 is also an example of sending a sensing request corresponding to a sensing intention.

[0065] The information processing device 6 receives a Subscribe message and then receives a SENSING 11n message. You may request the execution of the processing. The Subscribe message indicates that the sensing target will continue to be detected. This is a message requesting that the message be sent. The Subscribe message will send the specified follower For the duration of the subscription, or until an unsubscribe request is received via an Unsubscribe message, even if Then, the sensing process is repeated at the specified frequency. However, the information processing device 6 requests the SENSING 11n to execute the sensing process each time using a Request message. It is also possible.

[0066] The information processing device 6 then acquires sensing data via the NEF 11e (S5). The sensing data includes, for example, the latitude and longitude of the location where the object is detected, the size of the detected object, whether it is moving, the speed of movement, the direction of movement, and the material. The processing in S5 is an example of receiving sensing data in response to a sensing request from the SENSING 11n, which acts as a management device.

[0067] Next, the information processing device 6 refers to the rule base and filters the sensing data (S6). The rule base defines filtering rules, i.e., conditions for selecting sensing data, in the form of IF THEN ELSE. Filtering rules include, for example, conditions about the source of the sensing request, conditions about the location where the object was detected, and conditions about the status or type of the detected object. More specifically, conditions about the status or type of the object are exemplified by, for example, conditions about the size of the object, conditions about whether the object is moving, conditions about the speed of movement, conditions about the material, etc. The rules in the rule base are, for example, as follows:

[0068] Rule: "IF requester is a 'vehicle', THEN IF detection location is an intended 'location', THEN IF request includes an 'obstacle', THEN IF detection size of detected object > SS mm, THEN obstacle detected." In this rule, the source requesting sensing processing is the UE2 mounted on the vehicle. Furthermore, if the detected point of the target corresponds to the intended location, the request is for the detection of an obstacle, and the detected target is larger than SS millimeters, the information processing device 6 determines that an obstacle has been detected. Note that the rules may be modified and applied by replacing words such as "vehicle," "location," and "obstacle" included in these rules with synonyms defined in the thesaurus.

[0069] The rule base can be built, for example, in main memory 12 or external memory 13. Alternatively, the rule base may be built on another computer connected to network N1 (such as server 6A in Figure 1). The rule base is an example of a second database in which rules for identifying events corresponding to sensing intentions are defined. Furthermore, the thesaurus and the rule base may be managed in a database on the same computer. That is, the information processing device 6 or the computer managing the thesaurus and rule base replaces terms included in the rules of the rule base with thesauruses from the thesaurus, references the rules, and performs filtering. It is permissible to act according to reason.

[0070] The information processing device 6 then determines whether the sensing data conforms to the sensing intent according to one of the rule-based rules (S7). The processes in S6 and S7 are examples of detecting an event specified in the sensing intent from the received sensing data. Furthermore, the processes in S6 and S7 are also examples of detecting an event from sensing data according to a rule.

[0071] If the sensing data matches the sensing intent, the information processing device 6 can determine that it has detected an event corresponding to the sensing intent. Therefore, the information processing device 6 notifies the requester of the detected event by transmitting the sensing data (S8). The process in S8 is an example of transmitting the detected event to the requester. On the other hand, if the sensing data does not match the sensing intent, the information processing device 6 proceeds to S9.

[0072] Then, the information processing device 6 determines whether or not to terminate the processing (S9). For example, if the sensing request requests continued sensing processing (Subscribe message If sensing is performed via a message, the information processing device 6 returns the process to S5 and processes the next sensing data. On the other hand, if the sensing request requests a single sensing process (sensing via a Request message), the information processing device 6 terminates the process.

[0073] (Effects of the embodiment) As described above, the information processing device 6 receives sensing intent in natural language from the requester, such as UE2-1, and notifies the management device, SENSING 11n, of a sensing request corresponding to the sensing intent. The information processing device 6 then acquires sensing data in response to the sensing request and detects events corresponding to the sensing intent from the acquired sensing data. Furthermore, the information processing device 6 transmits the detected events to the requester. In this way, the information processing device 6 can grasp the intent of the user, such as UE2-1, and perform sensing processing that conforms to the user's intent. In other words, the information processing device 6 can respond flexibly and accurately to user requests. In addition, the processing performed by SENSING 11n as described above may be performed by other NF11s such as SMF11c, PCF11d, NEF11e, NRF11g, NSSF11h, AUSF11i, UDM11j, and NWDAF 11k, or it may be performed in cooperation with multiple NFs.

[0074] Furthermore, as described above, the thesaurus can be considered an example of a first database in which the relationship between information contained in sensing intent and information contained in sensing requests corresponding to sensing intent is defined. The information processing device 6 then searches the thesaurus based on the sensing intent and determines the sensing request corresponding to the sensing intent. Therefore, the information processing device 6 can receive the user's sensing intent in natural language and process it appropriately.

[0075] Furthermore, as described above, the rule-based system can be considered an example of a second database in which rules are defined for identifying events corresponding to sensing intentions based on sensing results. Therefore, the information processing device 6 can receive the user's sensing intentions in natural language and appropriately detect events from the sensing results according to the rules of the rule-based system.

[0076] Furthermore, in this embodiment, the sensing intent described in natural language includes information to identify at least one of the object on which the event is detected, the location where the event is detected, the circumstances of the object, and the type of event. Therefore, the information processing device 6 can detect events that meet the conditions of "what," "where," and "how."

[0077] <Second Embodiment> The event detection process according to the second embodiment will be described with reference to Figures 5 to 8. In the first embodiment described above, the information processing device 6 detected events that matched the sensing intent in natural language using a synonym dictionary and a rule base. In this embodiment, a more limited sensing intent is processed in the same configuration as the first embodiment (Figures 1 to 3). That is, in this embodiment, the information processing device 6 performs processing assuming that the sensing intent includes "who," "where," "what," and "how."

[0078] The information processing device 6 then refers to the target location definition dictionary to recognize "where" included in the sensing intent. Furthermore, the information processing device 6 refers to the sensing target definition dictionary to recognize "what" included in the sensing intent. In addition, the information processing device 6 refers to the event type definition dictionary to recognize "how" included in the sensing intent.

[0079] These dictionaries, like the dictionaries in the first embodiment, are built on the main memory 12, the external memory 13, or another computer connected to the network N1 (such as server 6A in Figure 1). The configuration of the information processing device 6, other than these dictionaries, is the same as in the first embodiment. Therefore, Figures 1 to 3 are directly applicable to the second embodiment. In addition, the information processing device 6 refers to a rule base similar to that in the first embodiment in order to process sensing intent.

[0080] (Example data) As described above, in this embodiment, the sensing intent expressed in natural language includes information corresponding to "who," "where," "what," and "how." The sensing intent is exemplified as follows:

[0081] Sensing intent: "We want to detect whether or not an obstacle is present in a vehicle 100 meters away." In this example, "who" is "the car." "where" is "100m away." "what" is "the obstacle to the car." "how" is "whether it exists or not." The format of the information corresponding to "who," "where," "what," and "how" may be fixed in the Application Programming Interface (API) provided to the requesting UE2 by the information processing device 6, as in the first embodiment. Also, in the same application as in the first embodiment... UE2 should ideally present pull-down menus for selecting information corresponding to, for example, "who," "where," "what," and "how." The information processing device 6 generates sensing requests based on such sensing intentions and therefore has dictionaries as illustrated in Figures 5 to 7.

[0082] Figure 5 illustrates data for a target location definition dictionary corresponding to "where". This data defines the information corresponding to "where". Examples of target location definition dictionary data include: Ym ahead, Ym forward, Ym behind, Ym behind, C city D town E block F number G, (latitude Tx, longitude Ny), CR intersection, RC railroad crossing. Of this data, Ym, C through G, Tx, Ny, CR, RC, etc., may be parameters that can be set by the requesting UE2. Therefore, the UE2 may accept input from the user for the values ​​to be set for these parameters via a user interface. The UE2 may then set the values ​​received from the user for the above parameters and generate a sensing request.

[0083] On the other hand, the information processing device 6 only needs to recognize "where" included in the sensing intent based on words (common nouns) such as "ahead," "forward," "back," "rear," "city," "latitude," "longitude," "intersection," and "railroad crossing," excluding these parameters. Furthermore, when processing sensing intent in Japanese, the information processing device 6 can use "at," "in," and "at" in these words (common nouns). When morphemes indicating location, such as "where," are attached, it is sufficient to recognize "where." In addition, when processing sensing intent in English, the information processing device 6 recognizes these words (common nouns) When a location is specified with a preposition such as in, at, on, upon, or over, we only need to recognize "where." This process is the same in Chinese.

[0084] Furthermore, the information processing device 6 adds "at," "in," and other place-related prefixes to these words (common nouns). If a morpheme indicating "where" is attached, the rule-based system may specify that "where" should be recognized. Similarly, for English and Chinese, the rule-based system may specify that "where" should be recognized. However, as mentioned above, if the format of the information corresponding to "who," "where," "what," and "how" is fixed in the API, such recognition processing in the information processing device 6 is unnecessary. This is because the requesting UE2-1, etc., will notify the information processing device 6 of "who," "where," "what," and "how" according to the API specifications.

[0085] Figure 6 illustrates the data in the sensing target definition dictionary corresponding to "what". The data in the sensing target definition dictionary includes common nouns such as object, thing, obstacle, person, intersection, and railroad crossing. When processing sensing intent in Japanese, the information processing device 6 only needs to recognize "what" when these words (common nouns) are accompanied by morphemes indicating an object, such as "を". When processing sensing intent in English, the information processing device 6 may also recognize "what" when these words (common nouns) are the object of a verb. The same applies to Chinese.

[0086] Furthermore, the information processing device 6 may define a rule-based mechanism to recognize "what" when a morpheme indicating an object, such as "を," is attached to these words (common nouns). Similarly, a rule-based mechanism may be defined for recognizing "what" in the case of English and Chinese. However, as mentioned above, if the format of information corresponding to "who," "where," "what," and "how" is fixed in the API, such rule-based recognition processing is unnecessary.

[0087] Figure 7 illustrates the data in the event type definition dictionary corresponding to "how". The event type definition dictionary contains words that define the type of event or the situation of the object. As shown in Figure 7, the data in the event type definition dictionary includes common nouns such as existence, presence or absence, being, action, movement, speed, precision, and accuracy. The information processing device 6 only needs to recognize "how" included in the sensing intent based on these words (common nouns). However, as mentioned above, if the format of the information corresponding to "who," "where," "what," and "how" is fixed in the API, then such recognition processing in the information processing device 6 is unnecessary.

[0088] (Example of processing) Figure 8 is a flowchart illustrating the event detection process of the second embodiment. In this process, the information processing device 6 first receives a sensing intention from the requester (S11). Next, the information processing device 6 identifies "who" made the request from the source information of the sensing intention (S12). For example, the information processing device 6 has a user-defined database that records the correspondence between the user identification information of the UE2 accessing the information processing device 6 and the way the UE2 is moving. When the information processing device 6 receives a request from the user of the UE2 to provide information processing by the information processing device 6, it should issue user identification information to identify the user. The information processing device 6 should then have the user input the way the UE2 is moving (e.g., in a vehicle, carried by the user) along with the user identification information. The information processing device 6 should then record the way the UE2 is moving in the user-defined database.

[0089] In this way, when the information processing device 6 receives sensing intent (including user identification information) from the requester, it only needs to identify the method of movement corresponding to the user identification information using the user-defined database. The method of movement is, as described above, in-vehicle, for example, carried by the user. If the method of movement of UE2 is in-vehicle, information identifying vehicle C1, etc., may be registered in the user-defined database. In this way, the information processing device 6 identifies the user identification information Based on this, you should identify the source of the request and specify "who" made it.

[0090] Next, the information processing device 6 identifies "where" from the sensing intent based on the target location definition dictionary and determines the sensing target location. As mentioned above, "where" is, for example, "100 units away". In this case, the information processing device 6 obtains the current position of the vehicle C1, etc. The UE2 of the vehicle C1, etc., obtains its current position using the Global Positioning System (GPS) or Global Navigation Satellite System (GNSS) and provides it to the information processing device 6. That's all that's needed. The information processing device 6 then needs to identify the location information (for example, latitude and longitude) corresponding to "where" from the current position of the vehicle C1, etc., and the relative distance information such as "100 units away" included in the sensing intent.

[0091] The information processing device 6 then sets a sensing area that includes the location information corresponding to "where". The method for setting the sensing area is the same as in S4 of the first embodiment. That is, the information processing device 6 refers to a base station database that includes base station ID, cell ID, latitude, longitude, and a sequence of cell radii for each frequency, and obtains a cell ID or TAC that includes the location information corresponding to "where" (for example, latitude and longitude), and sets it as the sensing area. Furthermore, the information processing device 6 determines the sensing accuracy from the sensing intent based on the event type definition dictionary (S13). However, the sensing accuracy does not have to be included in the sensing intent.

[0092] The information processing device 6 requests SENSING 11n to perform sensing processing, specifying the sensing area and, if necessary, the sensing accuracy (S14). The processing in S14 is the same as the processing in S4 of the first embodiment (Figure 4).

[0093] The information processing device 6 then receives sensing data via the NEF11e (S15). The sensing data, as in the first embodiment, includes, for example, the latitude and longitude of the location where the object was detected, the size of the detected object, whether or not it was moving, the speed of movement, the direction of movement, the material, etc.

[0094] Next, the information processing device 6 refers to the sensing target definition dictionary based on the sensing intent and identifies "what" (e.g., "obstacle"). The information processing device 6 also refers to the event type definition dictionary based on the sensing intent and identifies "how" (e.g., "existence or absence"). Then, similar to the first embodiment, the information processing device 6 extracts the target from the sensing result using rule-based rules (Example 1) (S16). The process in S16 is the same as the process in S6 of the first embodiment (Figure 4). Also, the processes in S17 to S19 in Figure 8 are the same as the processes in S7 to S9 of the first embodiment (Figure 4).

[0095] However, for example, if the object corresponding to "what" is detected in the image, a multimodal Large Language Model (LLM) may be applied in the S16 process to determine "what" (type of object, etc.). An example of a LM (Language Transformer) is a system that combines a Vision Transformer (ViT) with a transformer for language processing.

[0096] ViT decomposes a single image into multiple sub-regions (patches) and places these sub-regions in a space of embedding vectors. ViT processes sub-images similarly to words in a sentence, learning from a large amount of existing images to recognize images. Therefore, by combining ViT with a transformer, it becomes possible to perform processing based on the co-occurrence probability of image patch embedding vectors and word embedding vectors. For example, it can recognize that the co-occurrence probability of an image of an object blocking a road and the word "obstacle" is higher than the co-occurrence probability of an image of a rose and the word "obstacle."

[0097] The information processing device 1 may cooperate with a multimodal LLM built on a computer connected to the network N1 (for example, server 6A in Figure 1) to determine whether the image received in S15 corresponds to "what". Alternatively, the information processing device 1 may be equipped with a multimodal LLM and determine whether the image received in S15 corresponds to "what".

[0098] (Effects of the embodiment) As described above, the information processing device 6 can understand the intentions of users such as UE2 and perform sensing processing that conforms to those intentions. In other words, the information processing device 6 can respond flexibly and accurately to user requests.

[0099] <Third Embodiment> In the first embodiment described above, the information processing device 6 processed sensing intent using a thesaurus and a rule base. In the second embodiment described above, the information processing device 6 processed sensing intent using a target location definition dictionary, a sensing target definition dictionary, an event type definition dictionary, and a rule base. In this embodiment, the information processing device 6 processes sensing intent described in natural language using a generation AI, LLM, or multimodal LLM, etc. The configuration and processing of this embodiment are the same as in the first embodiment, except that LLM, etc., is used. Therefore, for example, the configurations in Figures 1 to 3 can be applied directly to this embodiment.

[0100] (Language model) Language models can be exemplified as models of the probability of generating individual sentences. For example, a language model can be described as a model that determines the probability of simultaneous occurrence of words, or morphemes obtained by separating particles and auxiliary verbs, contained in a sentence (see, for example, Tsuyoshi Okadome, "Deep Learning: Fundamentals of Generative AI," 1st edition, Kyoritsu Shuppan, March 30, 2024 (hereinafter, [Okadome])). Another example of a language model is the sequence transformation model. A sequence transformation model can be described as a model that deals with the probability of transforming a sequence X of a certain word (or morpheme) into another sequence Y. In other words, a sequence transformation model is a model of the probability of sequence Y occurring given that sequence X has occurred.

[0101] In these language models, each word (or morpheme) is represented by a single vector in an N-dimensional vector space called an embedding vector. In the embedding vector space, the embedding vectors of two words that are relatively likely to occur simultaneously are positioned at relatively close angles to each other. Therefore, the larger the dot product of the embedding vectors corresponding to two words, the higher the probability of those two words occurring simultaneously. Furthermore, in the embedding vector space, the difference value of the embedding vectors between two pairs of words with similar relationships will be approximately the same, and linear operations will hold. For example, in the embedding vectors, France-Paris = Japan-Tokyo holds true, where - is the subtraction sign.

[0102] LLM can be described as a model that has been pre-trained on the word (or morpheme) structure of a large amount of text and fine-tuned for a specific application. LLM makes it possible to predict the probability of word and sentence occurrence, or to determine the likelihood of sentences generated by a computer.

[0103] A multimodal LLM can be described as a large-scale language model capable of processing multiple modals (formats, such as text, images, audio, and video) that include information other than text. In other words, a multimodal LLM can accept not only text but also images and audio as input and has the ability to understand and generate that information. For example, by learning data that includes images and their corresponding sentences, a multimodal LLM can learn the relationship between a certain word (or sentence) and images that have a high probability of occurring simultaneously. Therefore, a multimodal LLM can generate the word "obstacle" from a photograph of an object blocking a road. A multimodal LLM integrates information from different modals to understand its meaning and return a response. On the other hand, a generative AI can be described as a multimodal LLM primarily specialized in content generation.

[0104] (Example of processing) Figures 9 and 10 are flowcharts illustrating the event detection process of the third embodiment. In this process, processes S21 and S22 are the same as S11 and S12 in the second embodiment. In this embodiment, the sensing intent is assumed to be, for example, "to detect the presence or absence of an obstacle at a distance of 100m with an error of less than 1m."

[0105] However, in this embodiment, the information processing device 6 cooperates with a generative AI or multimodal LLM built on a computer connected to the network N1 (for example, server 6A in Figure 1). Alternatively, in this embodiment, the information processing device 6 may have a generative AI or multimodal LLM internally. Therefore, in this embodiment, the information processing device 6 may receive natural language sensing intent as a prompt to the generative AI or multimodal LLM.

[0106] In this embodiment, the information processing device 6 decomposes the sensing intent described in natural language into morphemes and converts them into word embedding vectors (S23). The procedure for converting a sentence into a word embedding vector has been reported, for example, in a system called Word2Vec (Mikolov, T. et al. (2013) Efficient estimation of word representation in vector space. arXiv:1301.3781).

[0107] Next, the information processing device 6 uses a multimodal LLM to extract and determine the words that best correspond to "where" (location) and "how" ("accuracy") from the words included in the sensing intent in the word embedding vector space of the language model (S24). However, the information processing device 6 may simply perform the dot product of "where" and the embedding vector from the words included in the sensing intent and determine the word with the largest value as the word corresponding to "where". In this way, the information processing device 1 determines "100m away" as the word corresponding to "where" from the sensing intent. Then, the information processing device 6 sets the sensing area in the same way as in S14 of Figure 8 of the second embodiment.

[0108] Alternatively, the information processing device 6 may simply perform the dot product of the words "how" included in the sensing intent with the embedding vector and determine the word with the largest value to be the word corresponding to "how". Alternatively, the information processing device 6 may simply perform the dot product of the words "precision" included in the sensing intent with the embedding vector and determine the word with the largest value to be the word corresponding to "precision". In this way, the information processing device 1 determines "within an error of 1m" as the word corresponding to "precision" from the sensing intent.

[0109] Then, the information processing device 6 requests SENSING 11n to execute a sensing process specifying the sensing area and sensing accuracy via NEF 11e (S25). The process in S25 is the same as the process in S14 in Figure 8 of the second embodiment.

[0110] The information processing device 6 then acquires sensing data via the NEF11e (S26). The sensing data, as in the first embodiment, includes, for example, the latitude and longitude of the location where the object is detected, the size of the detected object, whether or not it is moving, the speed of movement, the direction of movement, the material, etc.

[0111] Next, the information processing device 6 extracts and determines the words that best correspond to "what" and "how" (type) from the words included in the sensing intent in the embedding vector space of the multimodal LLM (S27). Here, the information processing device 6 determines "obstacle" as "what". The information processing device 6 also determines "how" (type of event) whether it "exists or does not exist."

[0112] However, the information processing device 6 may simply perform the dot product of the words included in the sensing intent, "what," with the embedding vector, and select the word with the largest value as the word corresponding to "what." Alternatively, the information processing device 6 may simply perform the dot product of the words included in the sensing intent, "how" (type of event), with the embedding vector, and select the word with the largest value as the word corresponding to "how" (type of event).

[0113] Next, the information processing device 6 recognizes the object corresponding to "what" from the acquired sensing results (e.g., images) using image recognition (Convolutional Neural Network; CNN, multimodal LLM, etc.) (S28). In this case, the CNN is assumed to have completed deep learning and be able to distinguish objects corresponding to "what" from, for example, an image of an object on a road. The multimodal LLM is assumed to have completed pre-training and fine-tuning of image recognition for wireless sensing images. As described above, the information processing device 6 is assumed to be in cooperation with the CNN, multimodal LLM, etc., or to be equipped with the CNN, LLM, etc. Next, the processing in Figure 9 continues to Figure 10, indicated by symbol A1.

[0114] Next, in the process of S35, when the information processing device 6 repeatedly acquires sensing data, it determines whether there is movement from the time change between the previous sensing result and the current sensing result, and calculates the movement speed if there is movement (S31). Here, in the process of S31, the information processing device 6 may calculate whether there is movement and the movement speed from the change in the image included in the sensing data. In the process of S31, the information processing device 6 may calculate whether there is movement and the movement speed from the change in the position of the object (for example, latitude and longitude) included in the sensing data.

[0115] Next, the information processing device 6 filters the sensing data according to the "what" and "how" of the sensing intent (S32). That is, the information processing device 6 determines whether the size, presence or absence of movement, movement speed, movement direction, material, etc. of the detected object included in the sensing data conform to the "what" and "how" determined by the processing in S27. If the sensing data conforms to the sensing intent as a result of the processing in S32 (YES in S33), the information processing device 6 notifies the requester of the occurrence of the event (S34). On the other hand, if the sensing data does not conform to the sensing intent (NO in S33), the information processing device 6 proceeds to processing in S35.

[0116] As described above, the sensing intent in this embodiment is a prompt to the generating AI or multimodal LLM. Therefore, it can be said that the information processing device 6 detects events corresponding to the sensing intent based on the prompt using the generating AI or multimodal LLM. Furthermore, it can be said that the information processing device 6 filters and outputs sensing data according to the received prompt.

[0117] The subsequent processing is the same as in the first and second embodiments. For example, if the sensing request is a request for continuous sensing processing (by a Subscribe message) If a sensing request is made, the information processing device 6 proceeds to S26 in Figure 9, which is continued by symbol A2, and processes the next sensing data. On the other hand, if the sensing request is for a single sensing process (sensing request by Request message), the information processing device Step 6 terminates the process.

[0118] (Effects of the embodiment) As described above, the information processing device 6 is a language model, LLM or multimodal L The LM (Language Model) allows for the understanding of user intent, such as that of UE2 (User Environment 2), and enables sensing processing that conforms to the user's intent. Furthermore, the information processing device 6 can filter the results of the sensing processing using a language model, LLM (Language Model), or multimodal LLM (Language Model) to obtain sensing results that conform to the sensing intent.

[0119] Furthermore, the sensing intent received in natural language serves as a prompt to the Generative AI, LLM, or multimodal LLM. Therefore, the information processing device 6 uses the Generative AI, LLM, or multimodal LLM to detect events corresponding to the sensing intent based on the prompt. In other words, the information processing device 6 can appropriately filter the sensing data using the Generative AI, LLM, or multimodal LLM and acquire data that matches the sensing intent.

[0120] Furthermore, the information processing device 6 filters and outputs the sensing results according to the received prompt. Therefore, the information processing device 6 can convert requests for generated AI, LLM, or multimodal LLM into requests for SENSING 11n, and appropriately incorporate generated AI, LLM, or multimodal LLM into the sensing process.

[0121] <Fourth Embodiment> The sensing process according to the fourth embodiment will be described with reference to Figure 11. In the first to third embodiments described above, an example of a process for acquiring sensing data that matches a sensing intent described in natural language was shown. In this embodiment, the information processing device 6 aggregates sensing intents from multiple requesters, efficiently notifies the sensing request to SENSING 11n, suppresses resource consumption, and acquires sensing results. In this embodiment as well, the configurations in Figures 1 to 3 are applied as they are, just as in the first to third embodiments described above.

[0122] In this embodiment, the information processing device 6 has already notified SENSING 11n of a sensing request and has stored in the database the parameters to be set for the sensing request corresponding to "where," "what," and "how" for the request that is currently being processed for sensing. Furthermore, the information processing device 6 has also stored in the database the parameters corresponding to "where," "what," and "how" for the sensing data that has already been acquired.

[0123] Figure 11 is a flowchart illustrating the sensing instruction aggregation process A of the fourth embodiment. The process in Figure 11 can be incorporated into the processes of S4 and S5 in Figure 4 of the first embodiment, S14 and S15 in Figure 8 of the second embodiment, or S25 and S26 in Figure 9 of the third embodiment. In this process, the information processing device 6 creates a new sensing request corresponding to the sensing intent received from the requester (S41). The process in S41 is the same as the process described in S4 in Figure 3 of the first embodiment, S14 in Figure 8 of the second embodiment, and S25 in Figure 9 of the third embodiment.

[0124] Next, the information processing device 6 determines whether the sensing area of ​​the newly created sensing request is the same as that of an existing request (S42). That is, suppose the information processing device 6 has sent a request to SENSING 11n and is waiting to receive sensing data. In this case, the information processing device 6 only needs to determine whether the request already sent to SENSING 11n matches the sensing area of ​​the new request. The information processing device 6 also needs to determine whether the sensing area of ​​the available data in the sensing data already acquired matches the sensing area of ​​the new request. In this way, the information processing device 6 determines whether to substitute the new sensing request with an existing sensing request.

[0125] Here, "available" refers to cases where the acquired sensing data is static data with little temporal variation. Static data is, for example, data that detects buildings. On the other hand, the information processing device 6 only needs to avoid including data detecting moving objects or obstacles in the static data. Note that the determination in S42 is not limited to whether or not the sensing areas are common. The information processing device 6 may also determine whether or not to substitute an existing sensing request for a new sensing request based on whether or not the conditions corresponding to "what," "where," and "how" are common.

[0126] If the determination in S42 is YES, the information processing device 6 determines whether the accuracy of the newly created sensing request is less than or equal to the accuracy of the existing request (S43). In other words, it determines whether the accuracy of the existing request satisfies the accuracy of the newly created sensing request.

[0127] If the answer in S43 is YES, the information processing device 6 acquires the sensing results for the existing request (S44). That is, the information processing device 6 waits for the sensing data corresponding to the existing request, acquires it, and sends it to the requester. The information processing device 6 also refers to the acquired sensing data and sends it to the requester. On the other hand, if the answer in S42 or S43 is NO, the information processing device 6 notifies SENSING 11n of a new sensing request and acquires the results (S45).

[0128] (Effects of the embodiment) In sensing processing, if it is assumed that sensing data will be continuously received, at least one of the communications for sensing requests and result acquisition will occur frequently. Furthermore, if the amount of result acquisition is large, constraints may arise on at least one of the communication costs and processing resources. The information processing device 6 of this embodiment efficiently performs sensing according to the sensing intent by aggregating the sensing intent included in multiple sensing requests.

[0129] In other words, in this embodiment, whether or not the sensing conditions included in the second sensing request newly notified to SENSING 11n are satisfied is determined by whether or not the sensing conditions included in the existing first sensing request are satisfied. The existing first sensing request is already managed by the information processing device 6 as a SENSING device. This is a sensing request sent to 11n. In such a case, the information processing device 6 does not send a second sensing request to the management device, but instead detects the event from the sensing data for the first sensing request. Therefore, by aggregating the sensing intentions contained in multiple sensing requests from multiple requesters, sensing according to the sensing intention can be efficiently performed.

[0130] (modified version) Figure 12 is a flowchart illustrating a modified sensing instruction aggregation process B. In the process shown in Figure 11, the information processing device 6 determined whether the sensing area of ​​a newly created sensing request was the same as that of an existing sensing request. However, the information processing device 6 may also determine whether multiple sensing intentions received from multiple UE2s during the same time period are common, using the same procedure as in Figure 11. For example, the information processing device 6 may use the same procedure as in Figure 11 to select only one sensing intention with the highest request accuracy (representative sensing intention) from among multiple sensing intentions received from multiple UE2s during the same time period, and generate a sensing request.

[0131] In other words, in the process shown in Figure 12, the information processing device 6 receives sensing intentions from multiple requesters (UE2, etc.) for a predetermined period (S51). Then, the information processing device 6 groups together sensing intentions that share a common sensing area (S52). The process in S52 is an example of grouping together sensing intentions that share common sensing conditions. Note that the grouping of sensing intentions is not limited to sensing intentions that share a common sensing area. In short, the information processing device 6 groups together sensing intentions that share conditions corresponding to "what," "where," and "how." Multiple sensing intentions can be grouped together. Furthermore, the information processing device 6 selects the sensing intention with the highest required accuracy within the group as the representative sensing intention (S53).

[0132] The information processing device 6 then creates a sensing request corresponding to the representative sensing intent (S54). Furthermore, the information processing device 6 creates a sensing request specifying the sensing area and accuracy and transmits it to SENSING 11n (S55). In this way, the information processing device 6 receives sensing data corresponding to the representative sensing intent from SENSING 11n (S56). Furthermore, if the sensing data corresponding to the representative sensing intent matches the sensing intent, the information processing device 6 transmits the occurrence of the event to the requester (S57). In this way, the information processing device 6 can aggregate multiple sensing intents received from multiple UE2s at the same time using a representative sensing intent. As a result, the information processing device 6 can efficiently utilize wireless resources and acquire sensing data.

[0133] <Other Embodiments> In the first to fourth embodiments and their modifications described above, the information processing device 6 selects sensing data that matches the sensing intent by filtering or the like (F7 in Figure 1, S6 to S7 in Figure 4, S17 in Figure 8, etc.). However, this process of selecting sensing data that matches the sensing intent by filtering or the like may be performed in any FN11 included in the core network (core network) of a mobile communication system such as 5GC.

[0134] When FN11 selects sensing data that matches the sensing intent through filtering or other processes, the information processing device 6 or the UE2 installed in the vehicle C1, etc., simply needs to transmit the sensing intent to FN11 and receive sensing data that matches the sensing intent. In this case, the information processing device 6 or the UE2 installed in the vehicle C1, etc., may receive the sensing intent to specify the event to be sensed from the user in natural language. The information processing device 1 may also receive the sensing intent in natural language from the UE2 installed in the vehicle C1, etc. The information processing device 6 (or the UE2 installed in the vehicle C1, etc.) should then transmit the received natural language to the core network. Then, the core network will detect the event specified in the sensing intent from the sensing data. The information processing device 6 (or the UE2 installed in the vehicle C1, etc.) should receive the event from the core network.

[0135] Furthermore, the embodiments described above are merely examples, and this disclosure may be modified and implemented as appropriate without departing from its essence. Also, the processes and means described in this disclosure can be freely combined and implemented as long as no technical inconsistencies arise. Moreover, processes described as being performed by one device may be divided and executed by multiple devices. Conversely, processes described as being performed by different devices may be executed by a single device. In a computer system, the hardware configuration (server configuration) used to implement each function can be flexibly changed.

[0136] This disclosure can also be realized by supplying a computer program implementing the functions described in the above embodiments to a computer, and having one or more processors in the computer read and execute the program. Such a computer program may be provided to the computer by a non-temporary computer-readable storage medium that can be connected to the computer's system bus, or it may be provided to the computer via a network N1. The non-temporary computer-readable storage medium may be, for example, a magnetic disk, a hard disk drive (HDD), or an optical disk (CD-ROM, DVD disc, Blu-ray disc). This includes any type of disk (such as a disk), read-only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic card, flash memory, or optical card, as well as any other type of media. [Explanation of symbols]

[0137] 2 UE 3A base station 6. Information Processing Device 11 NF 11k NWDAF 11n SENFING 60 Control unit, 61 CPU 62 Main storage 63 External storage device 66 Communication equipment 100 Information and Communication Systems< / url:>

Claims

1. The sensing requester accepts a sensing intent in natural language, specifying the event to be sensed. A sensing request corresponding to the sensing intent is transmitted to a management device that distributes sensing data acquired by a wireless communication device. The sensing data in response to the sensing request is received from the management device. From the received sensing data, the event specified in the sensing intent is detected. An information processing device comprising a control unit that performs the operation of transmitting the detected event to the requesting party.

2. The information processing apparatus according to claim 1, wherein the control unit refers to a first database in which the relationship between the information included in the sensing intent and the information included in the sensing request is defined, and determines a sensing request corresponding to the sensing intent.

3. The information processing apparatus according to claim 1, wherein the control unit refers to a second database in which rules for identifying the event corresponding to the sensing intention are defined, and detects the event from the sensing data in accordance with the rules.

4. The aforementioned natural language is a prompt to a Generative Artificial Intelligence (AI) or a Multimodal Large Language Model (LLM), The information processing apparatus according to claim 1, wherein the control unit detects the event corresponding to the sensing intent based on the prompt using the generating AI or the multimodal LLM.

5. The information processing apparatus according to claim 4, wherein the control unit filters and outputs the sensing data in response to the received prompt.

6. The information processing apparatus according to claim 1, wherein the natural language includes information for identifying at least one of the object on which the event is detected, the location on which the event is detected, the circumstances of the object, and the type of the event.

7. The information processing apparatus according to claim 1, wherein, when the sensing conditions included in the first sensing request already transmitted to the management device are satisfied, the sensing conditions included in the second sensing request newly notified to the management device are satisfied, the control unit does not transmit the second sensing request to the management device, but detects the event from the sensing data for the first sensing request.

8. The control unit groups together sensing intentions that have common sensing conditions from among multiple sensing intentions received from multiple requesters during a predetermined period. A sensing request corresponding to a representative sensing intent is created from a group of the aforementioned sensing intents and transmitted to the management device. The information processing apparatus according to claim 1, which acquires sensing data corresponding to the sensing intent of the representative, detects the event, and transmits the detected event to the plurality of requesters.

9. The computer receives a sensing intent in natural language from the sensing requester, specifying the event to be sensed. A sensing request corresponding to the sensing intent is transmitted to a management device that distributes sensing data acquired by a wireless communication device. The sensing data in response to the sensing request is received from the management device. From the received sensing data, the event specified in the sensing intent is detected. An information processing method that performs the action of transmitting the detected event to the requesting party.

10. The information processing method according to claim 9, wherein the computer refers to a first database in which the relationship between the information contained in the sensing intent and the information contained in the sensing request is defined, and determines a sensing request corresponding to the sensing intent.

11. The information processing method according to claim 9, wherein the computer refers to a second database in which rules for identifying the event corresponding to the sensing intention are defined, and detects the event from the sensing data in accordance with the rules.

12. The aforementioned natural language is a prompt to a generative AI or multimodal LLM, The information processing method according to claim 9, wherein the computer detects the event corresponding to the sensing intent based on the prompt using the generating AI or the multimodal LLM.

13. The information processing method according to claim 12, wherein the computer filters and outputs the sensing data in response to the received prompt.

14. The information processing method according to claim 9, wherein the natural language includes information for identifying at least one of the object on which the event is detected, the location on which the event is detected, the circumstances of the object, and the type of the event.

15. The information processing method according to claim 9, wherein when the sensing conditions included in the first sensing request already transmitted to the management device are satisfied, the sensing conditions included in the second sensing request newly notified to the management device are satisfied, the computer does not transmit the second sensing request to the management device, but detects the event from the sensing data for the first sensing request.

16. The computer groups together sensing intentions that have common sensing conditions from among multiple sensing intentions received from multiple requesters during a predetermined period. A sensing request corresponding to a representative sensing intent is created from a group of multiple sensing intents and transmitted to the management device. The information processing method according to claim 9, comprising acquiring sensing data corresponding to the sensing intent of the representative, detecting the event, and transmitting the detected event to the plurality of requesters.

17. The computer receives a sensing intent in natural language from the sensing requester, specifying the event to be sensed. A sensing request corresponding to the sensing intent is transmitted to a management device that distributes sensing data acquired by a wireless communication device. The sensing data in response to the sensing request is received from the management device. From the received sensing data, the event specified in the sensing intent is detected. A program that causes the detected event to be transmitted to the requesting party.

18. The program according to claim 17 causes the computer to perform the following actions: refer to a first database in which the relationship between the information contained in the sensing intent and the information contained in the sensing request is defined, and determine a sensing request corresponding to the sensing intent. 。

19. The program according to claim 17, which causes a computer to refer to a second database in which rules for identifying the events corresponding to the sensing intent are defined, and to detect the events from the sensing data in accordance with the rules.

20. The system accepts sensing intent in natural language to specify the events to be sensed. The received natural language is transmitted to the core network. An information processing device comprising a control unit that detects the event specified in the sensing intent from sensing data and receives the event from the core network.

Images