Perception data processing method and device, storage medium and program product

Abstract

The invention provides a sensing data processing method and device, a storage medium and a program product, relates to the technical field of wireless communication, and can reduce processing complexity and time delay and greatly reduce sensing data traffic from a base station to a core network. The method comprises the following steps: receiving non-3GPP sensing data of sensing equipment; and processing the non-3GPP (3rd generation partnership project) sensing data.

Description

Technical Field

[0001] This disclosure relates to the field of wireless communication technology, and in particular to methods, apparatus, storage media, and program products for processing sensed data. Background Technology

[0002] Integrated communication and sensing achieves both wireless sensing and wireless communication functions by sharing spectrum and hardware / software resources. It leverages mobile networks to meet sensing performance requirements in various scenarios while simultaneously improving communication performance through sensing capabilities. Besides utilizing 3GPP (3rd Generation Partnership Project) radio spectrum resources for sensing, numerous sensing devices are already in use in the industry, such as roadside stations for vehicle-to-everything (V2X) communication, video surveillance cameras, and IoT sensing units. Integrating these multimodal sensing devices and the processing of sensing information into the 3GPP sensing architecture not only improves overall sensing coverage performance but also expands the forms and modes of sensing.

[0003] However, at present, the processing complexity and latency of non-3GPP sensing information are relatively high. If the core network is used for processing, it will increase the network load of the core network and squeeze communication traffic resources. Summary of the Invention

[0004] This disclosure provides a method, apparatus, storage medium, and program product for processing sensing data, which can reduce processing complexity and latency, and greatly reduce the sensing data traffic from the base station to the core network.

[0005] On the one hand, a method for processing sensing data is provided, applied to a base station, comprising: receiving non-3GPP sensing data from a sensing device; and processing the non-3GPP sensing data.

[0006] On another front, a method for processing sensing data is provided, applied to a terminal device, comprising: receiving non-3GPP sensing data sent by a sensing device; and sending the non-3GPP sensing data to a base station so that the base station processes the non-3GPP sensing data.

[0007] On another front, a method for processing sensing data is provided, applied to a sensing device, comprising: sending non-3GPP sensing data generated by the sensing device to a base station, so that the base station processes the non-3GPP sensing data.

[0008] On another front, a sensing data processing device is provided for use in a base station, comprising: a communication module for receiving non-3GPP sensing data from a sensing device; and a processing module for processing the non-3GPP sensing data.

[0009] On another front, a sensing data processing apparatus is provided, applied to a terminal device, comprising: a receiving module for receiving non-3GPP sensing data sent by a sensing device; and a sending module for sending the non-3GPP sensing data to a base station, so that the base station processes the non-3GPP sensing data.

[0010] In another aspect, a sensing data processing apparatus is provided, applied to a sensing device, comprising: a transmitting module for transmitting non-3GPP sensing data generated by the sensing device to a base station, so that the base station processes the non-3GPP sensing data.

[0011] In another aspect, a communication device is provided, comprising: a memory and a processor; the memory and the processor are coupled; the memory is used to store a computer program; and the processor, when executing the computer program, implements the method for processing sensed data according to any of the above embodiments.

[0012] In another aspect, a computer-readable storage medium is provided, on which computer program instructions are stored, which, when executed by a processor, implement the method for processing perceived data according to any of the above embodiments.

[0013] In another aspect, a computer program product is provided, which includes computer program instructions that, when executed by a processor, implement the method for processing perceived data as described in any of the above embodiments.

[0014] This disclosure provides a method for processing sensing data, which involves receiving non-3GPP sensing data from a sensing device and processing the non-3GPP sensing data. It can be seen that this disclosure can leverage the sensing processing capabilities of the base station to process non-3GPP data at the base station level, directly completing the non-3GPP sensing process in the access network domain, and enabling rapid deployment of the non-3GPP sensing access sensing system. This reduces processing complexity and latency, significantly decreasing the sensing data traffic from the base station to the core network. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in this disclosure, the accompanying drawings used in some embodiments of this disclosure will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings.

[0016] Figure 1 A schematic diagram of the architecture of a sensing system provided in some embodiments of this disclosure. Figure 1 ;

[0017] Figure 2 A schematic diagram of the architecture of a sensing system provided in some embodiments of this disclosure. Figure 2 ;

[0018] Figure 3 A flowchart of a method for processing sensed data provided in some embodiments of this disclosure Figure 1 ;

[0019] Figure 4 A flowchart of a method for processing sensed data provided in some embodiments of this disclosure Figure 2 ;

[0020] Figure 5 A flowchart of a method for processing sensed data provided in some embodiments of this disclosure Figure 3 ;

[0021] Figure 6 A flowchart of a method for processing sensed data provided in some embodiments of this disclosure Figure 4 ;

[0022] Figure 7 A flowchart of a method for processing sensed data provided in some embodiments of this disclosure Figure 5 ;

[0023] Figure 8 A flowchart of a method for processing sensed data provided in some embodiments of this disclosure Figure 6 ;

[0024] Figure 9 A flowchart of a method for processing sensed data provided in some embodiments of this disclosure Figure 7 ;

[0025] Figure 10 A flowchart of a method for processing sensed data provided in some embodiments of this disclosure Figure 8 ;

[0026] Figure 11 A flowchart of a method for processing sensed data provided in some embodiments of this disclosure Figure 9 ;

[0027] Figure 12 A flowchart of a method for processing sensed data provided in some embodiments of this disclosure Figure 10 ;

[0028] Figure 13 A schematic diagram of the structure of a sensing data processing device provided in some embodiments of this disclosure. Figure 1 ;

[0029] Figure 14 A schematic diagram of the structure of a sensing data processing device provided in some embodiments of this disclosure. Figure 2 ;

[0030] Figure 15A schematic diagram of the structure of a sensing data processing device provided in some embodiments of this disclosure. Figure 3 ;

[0031] Figure 16 This is a schematic diagram of the structure of a communication device provided in some embodiments of this disclosure. Detailed Implementation

[0032] The technical solutions of this disclosure will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0033] It should be noted that, in this disclosure, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this disclosure should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0034] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0035] In the description of this disclosure, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. "And / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Furthermore, "at least one" means one or more, and "more than one" means two or more.

[0036] Communication and sensing integration has become an important evolution direction for the next generation of 5G technology (5G Advanced, 5G-A) and the sixth generation mobile communication technology (6G), and is included in the six major application directions of the 6G vision released by IMT-2030, with broad application prospects.

[0037] As mentioned in the background technology, integrated communication and sensing achieves both wireless sensing and wireless communication functions by sharing spectrum and hardware / software resources. Integrated communication and sensing can leverage mobile networks to meet sensing performance requirements in various scenarios, while simultaneously improving communication performance through sensing capabilities. Current industry research focuses on using communication modules to achieve sensing capabilities, such as base station self-transmission and reception, and terminal transmission and base station reception, all of which rely on existing communication modules and bandwidth resources for sensing. In the processes of sensing capability registration, sensing data reporting, sensing data processing, and sensing network collaboration, only the reuse of communication equipment and bandwidth resources is considered to achieve sensing capabilities. Besides utilizing 3GPP wireless spectrum resources for sensing, a large number of sensing devices have already been deployed in the industry, such as roadside stations for vehicle-to-everything (V2X) communication, video surveillance cameras, and IoT sensing units. Integrating these multimodal sensing devices and the processing of sensing information into the 3GPP sensing architecture allows the integrated sensing system to acquire a large amount of sensing information from the existing network of sensing devices and fuse multimodal information to address various sensing needs and improve sensing quality. In this way, not only can the overall coverage performance of synesthesia be improved, but the forms and information patterns of perception can also be expanded.

[0038] Currently, the available methods for integrating non-3GPP sensing information with sensing architecture include: First, sensing information generated by sensing devices is transmitted to the public Internet via a communication link for processing and application. This involves gateway interaction between the 3GPP core network and the public Internet. If integration with 3GPP base station sensing information is required, the processing complexity and latency will be very high. Second, sensing information generated by non-3GPP sensing devices is transmitted to the core network via a communication link, where the core network processes the non-3GPP sensing data. This places high demands on the core network's sensing processing capabilities, increases network load, and consumes communication traffic resources. Furthermore, due to the involvement of multiple network elements, the standardization process is slow.

[0039] It can be seen that the processing of non-3GPP sensing information is currently complex and has high latency. If the core network is used for processing, it will increase the network load of the core network and squeeze communication traffic resources.

[0040] To address the aforementioned technical problems, this disclosure proposes a method for processing sensing data. The method involves receiving non-3GPP sensing data from a sensing device and processing this non-3GPP sensing data. It can be seen that this disclosure leverages the sensing processing capabilities of the base station to process non-3GPP data at the base station level, directly completing the non-3GPP sensing process in the access network domain, thus enabling rapid deployment of the non-3GPP sensing access sensing system. This reduces processing complexity and latency, significantly decreasing the sensing data traffic from the base station to the core network.

[0041] The sensory system provided in the embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0042] The sensing system provided in this disclosure includes: a base station, a terminal device, a sensing device, and a core network device.

[0043] In some embodiments, the aforementioned base station can be any of the following: an evolved NodeB (eNB), a next-generation NodeB (gNB), a transmission receive point (TRP), a transmission point (TP), a relay node, a smart metasurface (RIS), and some other type of access node. Depending on the size of the service coverage area provided, base stations can be further categorized as macro base stations providing macrocell coverage, micro base stations providing microcell coverage, femto base stations providing femtocell coverage, or distributed cell access nodes, etc. As wireless communication technology continues to evolve, future base stations may also adopt other names.

[0044] In some embodiments, a base station may include a communication module and a sensing module. The communication module is an entity within the base station used to perform communication processing. The sensing module is an entity within the base station used to perform sensing processing.

[0045] In some embodiments, the sensing module includes a 3GPP sensing module and a non-3GPP sensing module.

[0046] The 3GPP sensing module must possess at least one of the following capabilities: dedicated scheduling capability related to sensing, dedicated processing capability related to sensing, and the ability to use sensing hardware from a dedicated base station. The 3GPP sensing module does not have the capability to process non-3GPP sensing information such as video information and LiDAR point cloud information.

[0047] Non-3GPP sensing modules possess at least one of the following capabilities: non-3GPP sensor scheduling capability, general processing capability, sensing control capability, and sensing processing capability. For example, a non-3GPP sensing module may employ a server equipped with a Graphics Processing Unit (GPU) to process various types of sensing data.

[0048] To facilitate understanding, the capabilities of non-3GPP sensing modules are illustrated below with examples.

[0049] For example, a non-3GPP sensing module includes sensing control capabilities and sensing processing capabilities.

[0050] 1. Perception and control capabilities include at least one of the following:

[0051] (1) The ability to obtain perceived service requirements from the core network.

[0052] (2) The ability to acquire and report the deployment information and sensing information of the sensing equipment of this base station.

[0053] ① The types of sensing devices include, but are not limited to: cameras, millimeter-wave radar, lidar, sonar, ultra-wideband (UWB), wireless Fidelity (WIFI), and Global Positioning System (GPS) devices.

[0054] ② The deployment information of the sensing devices includes, but is not limited to: the location of the sensing devices, the mobility of the sensing devices, and the identity (ID) of the sensing devices.

[0055] ③ Capability information of sensing devices. The capability information of sensing devices is determined based on the type of sensing device, for example:

[0056] When the type of sensing device includes millimeter-wave radar, the capability information of the sensing device includes at least one of the following: the transmit power range of the millimeter-wave radar, the beam configurable type, the operating frequency band adjustment range, the sensing resolution adjustment range, the sensing angle adjustment range, the sensing distance adjustment range, and the snapshot reporting frame rate adjustment range.

[0057] When the type of sensing device includes LiDAR, the capability information of the sensing device includes at least one of the following: LiDAR scanning mode, laser beam angle adjustment range, angle resolution adjustment range, operating frequency adjustment range, and snapshot reporting frame rate adjustment range.

[0058] When the type of sensing device includes an ultra-wideband device, the capability information of the sensing device includes at least one of the following: the sensing band adjustment range, pulse shape, transmit power adjustment range, pulse repetition frequency adjustment range, and snapshot reporting frame rate adjustment range of the ultra-wideband device.

[0059] When the type of sensing device includes a wirelessly connected device, the capability information of the sensing device includes at least one of the following: the sensing frequency band adjustment range, sensing signal strength, waveform adjustment range, sensing resolution adjustment range, and snapshot reporting frame rate adjustment range of the wirelessly connected device.

[0060] When the type of sensing device includes a GPS device, the capability information of the sensing device includes at least one of the following: positioning accuracy adjustment range, satellite signal strength, navigation path planning capability, real-time speed measurement range, and time synchronization accuracy adjustment range.

[0061] ④ Identify the type of sensing service. The types of sensing services include, but are not limited to: target tracking sensing, area monitoring sensing, environmental reconstruction sensing, micro-deformation sensing, and meteorological sensing.

[0062] (3) The ability to manage and perceive resource allocation.

[0063] ① Indicator Sensing Service Types. These include, but are not limited to: target tracking sensing, area monitoring sensing, environmental reconstruction sensing, micro-deformation sensing, and meteorological sensing.

[0064] ② Instruct each sensing device on the configuration of sensing resources. The configuration of sensing resources includes, but is not limited to, at least one of the following: time domain resources, frequency domain resources, code domain resources, spatial domain resources, and waveform resources.

[0065] ③ Indicate the sensing mode of each sensing device. The sensing modes of the sensing devices include, but are not limited to: sensor sensing, where sensor sensing includes at least one of the following: camera, millimeter-wave radar, lidar, sonar, UWB device, WiFi device, GPS device.

[0066] ④ Indicate the identification information of each sensing device. The identification information of the sensing device includes, but is not limited to, at least one of the following: AAU-ID, BBU-ID, and sensor ID.

[0067] ⑤ Indicate the configuration parameters of each sensing device.

[0068] The camera's configuration parameters include, but are not limited to, at least one of the following: camera resolution, camera reporting frame rate, tilt angle, magnification, focal length, aperture, and calibration parameters.

[0069] The configuration parameters of millimeter-wave radar include, but are not limited to, at least one of the following: transmit power, beam configuration, operating frequency band, sensing resolution, sensing angle range, sensing distance range, and snapshot reporting frame rate.

[0070] The configuration parameters of a lidar include, but are not limited to, at least one of the following: scanning mode, laser beam angle and resolution, operating frequency, and snapshot reporting frame rate.

[0071] The configuration parameters sensed by a WiFi device include, but are not limited to, at least one of the following: sensed frequency band, sensed signal strength, waveform, sensed resolution, and snapshot reporting frame rate.

[0072] The configuration parameters sensed by UWB devices include, but are not limited to, at least one of the following: sensing frequency band, pulse shape, transmit power, pulse repetition frequency, and snapshot reporting frame rate.

[0073] ⑥ Requirements for the sensing indicators of each sensing device.

[0074] The camera's perception metrics requirements include, but are not limited to, at least one of the following: farthest target detection distance, nearest target detection distance, target detection accuracy, target detection resolution, and target recognition type.

[0075] The sensing performance requirements for millimeter-wave radar / WiFi sensing / UWB sensing include, but are not limited to, at least one of the following: maximum sensing distance, speed measurement range, angle measurement range, angular resolution, distance resolution, and speed resolution.

[0076] The sensing performance requirements for lidar include, but are not limited to, at least one of the following: maximum sensing distance, angle measurement range, angular resolution, distance resolution, and data density.

[0077] 2. Perceptual processing capabilities include at least one of the following:

[0078] (1) The ability to process QoS streams into perceptual raw data.

[0079] (2) Perception fusion processing capability. For example, perception fusion processing capability includes at least one of the following: data fusion at the data layer, data fusion at the feature layer, and data fusion at the decision layer.

[0080] (3) Feature extraction capability. For example, the ability to extract features through traditional radar detection, machine learning or deep learning techniques.

[0081] (4) Target recognition capability. For example, the ability to classify, locate, and track targets in order to achieve the ability to identify and analyze targets.

[0082] (5) The ability to standardize perception data. For example, the ability to process perception results from different modalities into perception results in a unified format.

[0083] (6) The ability to calculate sensor feedback information. For example, sensor feedback information includes, but is not limited to: terminal motion control information and sensor parameter adjustment information.

[0084] It is understood that the sensing data processing capability of the sensing device (e.g., sensor) is a capability added to the sensing module in this application. The original sensing module only has the ability to process synsensory data and does not have the ability to process sensing data from other sensors.

[0085] In some embodiments, the 3GPP sensing module and the non-3GPP sensing module can be two independent modules; or, the 3GPP sensing module and the non-3GPP sensing module can be hardware-integrated, with the sensing processing implemented on a processing board based on general processing capabilities; or, the 3GPP sensing module and the non-3GPP sensing module can be integrated at the external interface (without hardware integration), and the processing capabilities are then split onto different hardware within the module for processing.

[0086] In some embodiments, the terminal device described above may be a terminal device with communication capabilities but without sensing capabilities. For example, the terminal device may be a device with wireless transceiver capabilities, such as a mobile phone, tablet computer, wearable device, in-vehicle device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), Internet of Things (IoT) terminal, etc. This disclosure does not limit the specific type of terminal.

[0087] In some embodiments, the sensing device described above is a device that does not have communication capabilities but has sensing capabilities, and is able to acquire sensing data of a target object or environment, such as a sensor. Exemplarily, the sensing devices in the embodiments of this disclosure include, but are not limited to, at least one of the following: a camera, millimeter-wave radar, lidar, sonar, UWB device, WIFI device, and GPS device. The sensing device can also be a handheld mobile device with sensing functions (e.g., a camera or UWB, etc.).

[0088] It should be noted that the sensing devices in this disclosure primarily refer to non-3GPP sensing devices. Currently, non-3GPP sensing devices mainly include two types: standalone devices unrelated to the 5G system and devices deployed dependent on the 5G system. Standalone devices unrelated to the 5G system, such as independently deployed cameras and sensors, do not have the ability to access the 5G network, and the 5G network cannot identify or manage these devices. Devices deployed dependent on the 5G system, such as cameras on mobile phones and sensors deployed on base stations, although they do not have the ability to access the 5G network, can be managed by operators through the management of UEs or base stations. The non-3GPP sensing devices in this disclosure primarily refer to devices deployed dependent on the 5G system.

[0089] In some embodiments, the sensing device is used to generate sensing data. The sensing data includes 3GPP sensing measurement data and non-3GPP sensing measurement data. 3GPP sensing measurement data is sensing measurement data generated by the influence of a target object or environment (e.g., reflection, refraction, diffraction) on 3GPP radio signals. Non-3GPP sensing measurement data is sensing measurement data about a target object or environment provided by non-3GPP sensing devices (e.g., radar, cameras, and other sensor devices). It should be noted that the non-3GPP sensing data mentioned in this disclosure primarily refers to non-3GPP sensing measurement data.

[0090] In some embodiments, the core network equipment is located in the core network domain (i.e., the core network defined by the 3GPP protocol); the base station, terminal equipment and sensing equipment are located in the access network domain (i.e., the access network specified by the 3GPP protocol).

[0091] See Figure 1 This is a schematic diagram of an architecture of a sensing system provided in an embodiment of this disclosure. Figure 1 As shown, the sensing system includes: base station 110, terminal equipment 120, sensing equipment 130 and core network equipment 140.

[0092] The sensing device 130 is connected to the base station 110 through the terminal device 120, and the base station 110 is connected to the core network device 140.

[0093] For example, the sensing device 130 is connected to the terminal device 120 via a non-3GPP communication air interface link (e.g., WiFi, Bluetooth, direct connection, sensor integrated on the terminal device 120); the terminal device 120 is connected to the base station 110 via a 3GPP communication air interface link.

[0094] In some embodiments, base station 110 includes a communication module and a sensing module. Terminal device 120 is connected to the communication module of base station 110 via a 3GPP communication air interface link. The communication module and sensing module of base station 110 are connected via an NS3 interface.

[0095] The NS3 interface has the following functions:

[0096] 1. The sensing module forwards non-3GPP sensing configuration information, non-3GPP sensing registration response messages, and reports capture confirmation flags to the communication module through the NS3 interface.

[0097] 2. The communication module forwards non-3GPP sensing data, non-3GPP sensing registration requests, and non-3GPP sensing data reporting requests to the sensing module through the NS3 interface.

[0098] Understandably, since the sensing module needs to acquire non-3GPP sensing data through the communication module, an NS3 interface has been added between the communication module and the sensing module. This NS3 interface is a special interface for cooperating with non-3GPP sensing signals, and the signaling transmitted through the NS3 interface is special signaling for cooperating with non-3GPP sensing signals. This special signaling for cooperating with non-3GPP sensing signals includes the issued non-3GPP sensing configuration information and the reported non-3GPP sensing data.

[0099] It should be noted that, Figure 1 The embodiments shown are merely examples. In actual use, base station 110 can connect to one or more terminal devices 120, and terminal devices 120 can also connect to one or more sensing devices 130.

[0100] As can be seen, in the sensing system provided in this embodiment, the sensing device can access the sensing system through a terminal-communication air interface link-base station communication module-base station non-3GPP sensing module, realizing the control and processing of non-3GPP sensing data on the base station side. Specifically, the sensing device can register its sensing capabilities with the base station's non-3GPP sensing module through a terminal device. The base station's non-3GPP sensing module can identify and acquire non-3GPP sensing data, and perform processes such as parsing, sensing data processing, and data standardization. Thus, by centralizing all sensing processing processes within the base station's sensing module, base station-side edge computing of non-3GPP sensing data is achieved, reducing processing complexity and latency. Simultaneously, without modifying the sensing-related interfaces from the base station to the core network, the sensing data traffic from the base station to the core network is significantly reduced.

[0101] See Figure 2 This is a schematic diagram of another architecture of the sensing system provided in the embodiments of this disclosure. Figure 2 As shown, the sensing system includes: a base station 110, a sensing device 130, and a core network device 140.

[0102] Among them, the sensing device 130 is directly connected to the base station 110, and the base station 110 is connected to the core network device 140.

[0103] For example, the sensing device 130 is connected to the base station 110 via a non-3GPP communication air interface link (e.g., WiFi, Bluetooth, direct connection, or a sensor integrated on the base station 110).

[0104] In some embodiments, base station 110 includes a communication module and a sensing module. Sensing device 130 is connected to the sensing module of base station 110 via a non-3GPP communication air interface link. The communication module and sensing module of base station 110 are connected via an NS3 interface.

[0105] The function of the NS3 interface can be found in [link to relevant documentation]. Figure 1 The descriptions in the illustrated embodiments will not be repeated here.

[0106] It should be noted that, Figure 2 The embodiments shown are merely examples. In actual use, base station 110 can connect to one or more sensing devices 130.

[0107] As can be seen, in the sensing system provided by this embodiment, the sensing device can access the sensing system through the communication air interface link—the base station's non-3GPP sensing module—to realize the control and processing of non-3GPP sensing data on the base station side. In this way, all sensing processing is concentrated in the sensing module at the base station, realizing base station-side edge computing of non-3GPP sensing data, which reduces processing complexity and latency; at the same time, it eliminates the need to modify the sensing-related interfaces from the base station to the core network, significantly reducing the sensing data traffic from the base station to the core network.

[0108] It should be noted that, in addition to the architecture provided in this disclosure, the sensing data from the sensing device can also be reported and processed through a link between the sensing device, terminal, base station, 3GPP core network, public Internet, and computing center; or, the sensing data from the sensing device can also be reported and processed through a link between the sensor, terminal, and core network. This disclosure does not limit this aspect.

[0109] It is understood that the application scenarios of the embodiments of this disclosure are not limited. The system architecture and business scenarios described in the embodiments of this disclosure are for the purpose of more clearly illustrating the technical solutions of the embodiments of this disclosure, and do not constitute a limitation on the technical solutions provided by the embodiments of this disclosure. As those skilled in the art will know, with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided by the embodiments of this disclosure are also applicable to similar technical problems.

[0110] The processing method for perceived data provided in the embodiments of this disclosure will be described in detail below.

[0111] The method for processing sensing data provided in this disclosure can be applied to scenarios that integrate communication and sensing, such as smart transportation, smart homes, and smart engineering, which simultaneously have sensing devices and communication coverage.

[0112] This disclosure provides a method for processing sensing data, applied to, for example... Figure 1 or Figure 2 The base station in the sensor system shown. For example... Figure 3 As shown, the method includes the following steps:

[0113] S201. The base station receives non-3GPP sensing data from the sensing equipment.

[0114] In some embodiments, the aforementioned non-3GPP sensing data refers to sensing measurement data about target information (e.g., target object, target context, or environment) generated and reported by non-3GPP sensing devices (e.g., millimeter-wave radar devices, camera sensors, etc.). For example, non-3GPP sensing data includes, but is not limited to: camera-recorded video, Wi-Fi sensing reception data, millimeter-wave radar delay Doppler image data, lidar point cloud data, and UWB sensing reception data.

[0115] As one possible implementation, step S201 above can be implemented as: receiving non-3GPP sensing data sent by the sensing device through the terminal device. For example, in Figure 1 In the sensing system shown, the sensing terminal device is connected to the base station. Therefore, the base station can receive non-3GPP sensing data sent by the sensing device through the terminal device.

[0116] For example, the terminal device receives non-3GPP sensing data sent by the sensing device through a non-3GPP communication air interface link, and then sends the non-3GPP sensing data to the base station through a 3GPP communication air interface link.

[0117] As another possible implementation, step S201 above can be implemented as: directly receiving non-3GPP sensing data sent by the sensing device. For example, in Figure 2 In the sensing system shown, the sensing device is directly connected to the base station. Therefore, the base station can directly receive non-3GPP sensing data sent by the sensing device.

[0118] For example, the base station receives non-3GPP sensing data sent by the sensing device through a non-3GPP communication air interface link.

[0119] In some embodiments, the base station includes a communication module and a sensing module, wherein the sensing module is further divided into a 3GPP sensing module and a non-3GPP sensing module. The communication module receives non-3GPP sensing data and then transmits it to the non-3GPP sensing module.

[0120] In some embodiments, when the base station configures the transmission resources used for sensing data reporting for the terminal device, the above step S201 can be implemented as: receiving non-3GPP sensing data on the transmission resources used for sensing data reporting.

[0121] For example, the transmission resources used for reporting sensing data may include time domain resources, frequency domain resources, code domain resources, spatial domain resources, etc.

[0122] In some embodiments, step S201 above can be implemented as: obtaining non-3GPP sensing data based on the received non-3GPP sensing data reporting identifier.

[0123] The sensing data reporting identifier is used to indicate that the terminal device has reported non-3GPP sensing data. For example, the terminal device may simultaneously send the sensing data reporting identifier and both non-3GPP sensing data.

[0124] In some embodiments, the base station includes a communication module and a sensing module, wherein the sensing module is further divided into a 3GPP sensing module and a non-3GPP sensing module. The communication module is used to receive non-3GPP sensing data sent by the terminal device; the non-3GPP sensing module, upon receiving a sensing data reporting identifier, obtains the non-3GPP sensing data from the communication module.

[0125] In some embodiments, the non-3GPP sensing data received by the base station is data processed by the terminal device. For example, the terminal device can encode and modulate the non-3GPP sensing data to convert it into a communication signal before sending it to the base station. Correspondingly, after receiving the communication signal sent by the terminal device, the base station's communication module processes the signal through signal detection, demodulation, and decoding to parse the communication signal into non-3GPP sensing data and forward it to the non-3GPP sensing module.

[0126] S202. The base station processes non-3GPP sensing data.

[0127] In some embodiments, step S202 can be implemented as follows: preprocessing non-3GPP sensing data, processing sensing data, and standardizing data in sequence.

[0128] For example, preprocessing refers to parsing the received raw data and converting it into non-3GPP sensing data. It can be understood that the raw data packets received by the base station are QoS flows, therefore, they need to be parsed and converted into sensing data. The preprocessing process was originally handled by the UPF network element in the core network, but now it is a newly added function of the base station's non-3GPP sensing module.

[0129] For example, perceptual data processing refers to performing perceptual processing on different types of raw perceptual data and converting it into perceptual result data.

[0130] For example, data standardization processing refers to mapping the perception result data and converting it into perception result data in a unified format.

[0131] In some embodiments, the above-mentioned sensing data processing includes sensing data processing and fusion of sensing devices, which may include, for example, data-level fusion, feature-level fusion, and decision-level fusion.

[0132] Data-level fusion includes the following processes:

[0133] a1. Perform time alignment, splicing / weighted averaging, and other operations on the original non-3GPP sensing data and sensing data from other sources (sensing data from other non-3GPP sensing devices accessing at the same time) to obtain data-level fused sensing data.

[0134] a2. Use traditional radar detection, machine learning or deep learning and other technologies to extract features from the data-level fused perception data, extracting basic features including target information, such as time-delay-Doppler maps, target segmentation maps, etc.

[0135] a3. Perform target recognition on the feature extraction results, and perform operations such as classification, recognition, and tracking on the target recognition results to obtain target perception results.

[0136] Feature-level fusion includes the following processing steps:

[0137] b1. Extract basic features, including target information, from raw non-3GPP sensing data.

[0138] b2. By aligning time, splicing, weighted averaging, and feature transformation (e.g., Principal Component Analysis (PCA)), the extracted basic features including the target information are fused to obtain the feature fusion result.

[0139] b3. Perform target recognition on the result of feature fusion, and perform operations such as classification, recognition, and tracking on the target recognition result to obtain the target perception result.

[0140] Decision-level fusion includes the following processes:

[0141] c1. Extract basic features, including target information, from raw non-3GPP sensing data.

[0142] c2. Perform target recognition on the data after feature extraction, and perform operations such as classification, recognition, and tracking on the target recognition results to obtain target perception results.

[0143] c3. By using methods such as voting, decision trees, and ensemble models, non-3GPP sensing results can be fused with other sensing results to obtain the final sensing result; alternatively, fusion can be avoided, and the non-3GPP sensing results can be directly used as the final sensing result.

[0144] In some embodiments, the base station can report non-3GPP sensing data to the core network for further processing. For example, the base station can report non-3GPP sensing data in a standardized format to the core network.

[0145] It is understood that the sensing data processing method proposed in this disclosure allows the base station to receive and process non-3GPP sensing data from sensing devices. It can be seen that this disclosure can leverage the sensing processing capabilities of the base station to process non-3GPP data at the base station level, directly completing the non-3GPP sensing process in the access network domain, thus enabling rapid deployment of the non-3GPP sensing access sensing system. This reduces processing complexity and latency, significantly decreasing the sensing data traffic from the base station to the core network.

[0146] In some embodiments, such as Figure 4 As shown, prior to step S201, the method further includes the following steps:

[0147] S200: The base station sends the configuration information of the sensing device to the sensing device.

[0148] The configuration information of the sensing device includes at least one of the following: sensing mode and parameters used by the sensing device.

[0149] For example, the sensing modes of the sensing device include, but are not limited to, sensor sensing, wherein sensor sensing includes at least one of the following: camera, millimeter-wave radar, lidar, sonar, UWB device, WiFi device, GPS device.

[0150] For example, the parameters used by the sensing device (also known as the configuration parameters of the sensing device) are determined according to the type of sensing device.

[0151] For example, the parameters used by a camera include, but are not limited to, at least one of the following: camera resolution, camera reporting frame rate, turning angle, magnification, focal length, aperture, and correction parameters. The configuration parameters of a millimeter-wave radar include, but are not limited to, at least one of the following: transmit power, beam configuration, operating frequency band, sensing resolution, sensing angle range, sensing distance range, and snapshot reporting frame rate.

[0152] The parameters used by lidar include, but are not limited to, at least one of the following: scanning mode, laser beam angle and resolution, operating frequency, and snapshot reporting frame rate.

[0153] The parameters used by WiFi devices include, but are not limited to, at least one of the following: sensing frequency band, sensing signal strength, waveform, sensing resolution, and snapshot reporting frame rate.

[0154] The parameters used by UWB devices include, but are not limited to, at least one of the following: sensing frequency band, pulse shape, transmit power, pulse repetition frequency, and snapshot reporting frame rate.

[0155] As one possible implementation, step S200 above can be implemented as follows: sending the configuration information of the sensing device to the sensing device through the terminal device. For example, in Figure 1 In the sensing system shown, the sensing terminal device is connected to the base station. Therefore, the base station can send the configuration information of the sensing device to the sensing device through the terminal device.

[0156] For example, sending configuration information of a sensing device to a sensing device via a terminal device includes: sending non-3GPP sensing configuration information to the terminal device, so that the terminal device sends the configuration information of the sensing device to the sensing device based on the non-3GPP sensing configuration information; the configuration information of the sensing device is a subset of the non-3GPP sensing configuration information.

[0157] The non-3GPP sensing configuration information includes at least one of the following: sensing mode, identification information of sensing device, parameters used by sensing device, and transmission resources used for sensing data reporting.

[0158] For example, the identification information of the sensing device can be the ID of the sensing device.

[0159] For example, the transmission resources used for reporting sensing data may include time domain resources, frequency domain resources, code domain resources, spatial domain resources, etc.

[0160] As another possible implementation, step S200 above can be implemented by directly sending the configuration information of the sensing device to the sensing device. For example, in Figure 2 In the sensing system shown, the sensing device is directly connected to the base station. Therefore, the base station can directly send the configuration information of the sensing device to the sensing device.

[0161] Understandably, the base station sends the sensing device's configuration information to the sensing device, so that the sensing device can perform sensing configuration based on the sensing device's configuration information to generate and report non-3GPP sensing data.

[0162] For example, in this embodiment of the disclosure, there are two triggering methods for reporting non-3GPP sensing data. One is active triggering by the base station (or the base station's non-3GPP sensing module). For example, the base station actively sends the sensing device's configuration information to the sensing device to trigger the sensing device to generate and report non-3GPP sensing data. The other is active triggering by the terminal device. For example, the terminal device actively sends a sensing data reporting request to the base station to trigger the base station to send the sensing device's configuration information to the sensing device through the terminal device, thereby triggering the sensing device to generate and report non-3GPP sensing data. See the following embodiments for examples.

[0163] In some embodiments, step S200 can be implemented as follows: when the first triggering condition is met, sending the configuration information of the sensing device to the sensing device.

[0164] The first triggering condition includes at least one of the following:

[0165] The base station triggers the need to sense the static environment;

[0166] The base station triggers the need to sense dynamic targets;

[0167] The base station triggered the need to monitor drone equipment.

[0168] For example, if a base station triggers a need to sense the static environment, the base station can invoke sensing devices (e.g., sensors) carried by the terminal. For example, if the base station needs to sense the surrounding static environment, the base station can invoke a networked camera to collect video data, which, together with the sensing data, is used for environmental sensing near the base station.

[0169] For example, when a base station triggers a need to sense dynamic targets, it can invoke sensing devices (e.g., sensors) carried by the terminal. For example, when the base station senses a vehicle, it invokes a roadside camera or millimeter-wave radar to obtain sensing data about vehicles and pedestrians, and combines this data with sensory data to jointly sense vehicles and pedestrians.

[0170] For example, if a base station triggers a need to monitor a drone, the base station can invoke the sensing devices (e.g., sensors) carried by the connected drone itself. For example, when the base station senses the drone, it invokes the drone's own GPS positioning data to obtain accurate drone location information for calibration of sensing parameters or precise drone location tracking.

[0171] In some embodiments, step S200 can be implemented as follows: upon receiving a first request message, sending configuration information of the sensing device to the sensing device. The first request message is used to request the reporting of non-3GPP sensing data.

[0172] In some embodiments, the first request message includes at least one of the following: sensing mode, identification information of the sensing device, and parameters used by the sensing device.

[0173] In some embodiments, the first request message is sent by the terminal device when the second triggering condition is met.

[0174] The second triggering condition includes at least one of the following:

[0175] The terminal device triggers the need to perceive the surrounding environment;

[0176] Terminal devices trigger the need to report sensing information.

[0177] In some embodiments, after acquiring non-3GPP sensing data, the base station sends a first response message to the terminal device, the first response message including the non-3GPP sensing data.

[0178] For example, if a terminal device needs to sense its surrounding environment, it needs to utilize its own sensing devices (e.g., sensors) in conjunction with the base station's computing power to perform environmental sensing. For example, if the terminal device needs to acquire static environmental data for applications such as smart home systems, a non-3GPP sensing process is triggered. After obtaining the final non-3GPP sensing data, the base station can feed it back to the terminal device.

[0179] For example, if a terminal device triggers a request to report sensing data, it directly reports non-3GPP sensing data. For example, a network-connected camera can be configured to periodically send non-3GPP sensing data to the base station for the base station to perceive; therefore, the terminal device's triggering process needs to be performed periodically.

[0180] In some embodiments, such as Figure 5 As shown, prior to step S201, the method further includes the following steps:

[0181] S301. The base station receives a registration request message, which is used to request that the information of the sensing device be registered with the base station.

[0182] As one possible implementation, step S301 above can be implemented as: receiving a registration request message sent by the sensing device through the terminal device. For example, in Figure 1 In the sensor system shown, the sensing terminal device is connected to the base station. Therefore, the base station can receive the registration request message sent by the sensing device through the terminal device.

[0183] For example, the terminal device receives the registration request message sent by the sensing device through a non-3GPP communication air interface link, and then sends the registration request message to the base station through a 3GPP communication air interface link.

[0184] As another possible implementation, step S301 above can be implemented by directly receiving the registration request message sent by the sensing device. For example, in Figure 2 In the sensor system shown, the sensing device is directly connected to the base station. Therefore, the base station can directly receive the registration request message sent by the sensing device.

[0185] For example, the base station receives the registration request message sent by the sensing device through a non-3GPP communication air interface link.

[0186] In some embodiments, the base station includes a communication module and a sensing module, which is further divided into a 3GPP sensing module and a non-3GPP sensing module. Upon receiving a registration request message, the communication module determines it to be a registration request message from a non-3GPP sensing device through the non-3GPP sensing module. The communication module then forwards the registration request message to the non-3GPP sensing module of the base station.

[0187] In some embodiments, the registration request message includes at least one of the following: registration type, type of sensing device, and capability information of the sensing device.

[0188] Among them, the registration type can be non-3GPP sensing device sensing.

[0189] In some embodiments, the type of sensing device includes at least one of the following: camera, millimeter-wave radar, lidar, UWB device, WIFI device, and GPS device.

[0190] In some embodiments, the capability information of the sensing device is determined based on the type of sensing device. For example, the capabilities of the sensing device may include the following:

[0191] When the type of sensing device includes a camera, the capability information of the sensing device includes at least one of the following: the resolution adjustment range of the camera, the reported frame rate adjustment range, the turning angle, the focal length, the aperture, the magnification, and the correction coefficient.

[0192] When the type of sensing device includes millimeter-wave radar, the capability information of the sensing device includes at least one of the following: the transmit power range of the millimeter-wave radar, the beam configurable type, the operating frequency band adjustment range, the sensing resolution adjustment range, the sensing angle adjustment range, the sensing distance adjustment range, and the snapshot reporting frame rate adjustment range.

[0193] When the type of sensing device includes LiDAR, the capability information of the sensing device includes at least one of the following: LiDAR scanning mode, laser beam angle adjustment range, angle resolution adjustment range, operating frequency adjustment range, and snapshot reporting frame rate adjustment range.

[0194] When the type of sensing device includes an ultra-wideband device, the capability information of the sensing device includes at least one of the following: the sensing band adjustment range, pulse shape, transmit power adjustment range, pulse repetition frequency adjustment range, and snapshot reporting frame rate adjustment range of the ultra-wideband device.

[0195] When the type of sensing device includes a wirelessly connected device, the capability information of the sensing device includes at least one of the following: the sensing frequency band adjustment range, sensing signal strength, waveform adjustment range, sensing resolution adjustment range, and snapshot reporting frame rate adjustment range of the wirelessly connected device.

[0196] When the type of sensing device includes a GPS device, the capability information of the GPS device includes at least one of the following: positioning accuracy adjustment range, satellite signal strength, navigation path planning capability, real-time speed measurement range, and time synchronization accuracy adjustment range.

[0197] S302. The base station sends a registration response message, which indicates that the sensing device has been successfully registered.

[0198] As one possible implementation, step S302 above can be implemented by sending a registration response message to the sensing device via the terminal device. For example, in Figure 1 In the sensor system shown, the sensing terminal device is connected to the base station. Therefore, the base station can send a registration response message to the sensing device through the terminal device.

[0199] For example, the base station sends the registration response message to the terminal device through the 3GPP communication air interface link; then the terminal device sends the registration response message to the sensing device through a non-3GPP communication air interface link.

[0200] As another possible implementation, step S302 above can be implemented by directly sending a registration response message to the sensing device. For example, in Figure 2 In the sensor system shown, the sensing device is directly connected to the base station. Therefore, the base station can directly send a registration response message to the sensing device.

[0201] For example, the base station sends the registration response message to the sensing device via a non-3GPP communication air interface link.

[0202] In some embodiments, the base station registers the sensing device based on the received registration request message. After confirming that the sensing device has been successfully registered, it assigns sensing device identification information to the sensing device. The aforementioned registration response message may include the sensing device identification and information indicating that the sensing device has been successfully registered.

[0203] It should be noted that the registration process provided in steps S301-S302 can be used for any of the following: initial registration, mobility update registration, or periodic registration.

[0204] This disclosure provides a method for processing sensing data, applied to, for example... Figure 1 or Figure 2 The terminal equipment in the sensory system shown. For example... Figure 6 As shown, the method includes the following steps:

[0205] S401. The terminal device receives non-3GPP sensing data sent by the sensing device.

[0206] For example, the terminal device receives non-3GPP sensing data sent by the sensing device through the non-3GPP communication link air interface.

[0207] For an example, a detailed description of non-3GPP sensing data can be found in [reference needed]. Figure 3 The embodiments shown are not described in detail here.

[0208] S402. The terminal device sends non-3GPP sensing data to the base station so that the base station can process the non-3GPP sensing data.

[0209] For example, the terminal device sends non-3GPP sensing data to the base station through the 3GPP communication link air interface.

[0210] In some embodiments, the terminal device simultaneously transmits a sensing data reporting identifier and non-3GPP sensing data. The sensing data reporting identifier indicates that the terminal device has reported non-3GPP sensing data. Correspondingly, the base station's communication module receives the non-3GPP sensing data transmitted by the terminal device; upon receiving the sensing data reporting identifier, the base station's non-3GPP sensing module obtains the non-3GPP sensing data from the communication module.

[0211] For an example, a detailed description of how base stations process non-3GPP sensing data can be found in [reference needed]. Figure 3 The embodiments shown are not described in detail here.

[0212] In some embodiments, prior to step S401 described above, the method further includes:

[0213] S501, The terminal device receives non-3GPP sensing configuration information sent by the base station.

[0214] For example, the terminal device receives non-3GPP aware configuration information sent by the base station through the 3GPP communication link air interface.

[0215] For an example, a detailed description of non-3GPP-aware configuration information can be found in [link to relevant documentation]. Figure 3 The embodiments shown are not described in detail here.

[0216] S502. The terminal device sends the configuration information of the sensing device to the sensing device based on the non-3GPP sensing configuration information.

[0217] The configuration information of the sensing device is a subset of the non-3GPP sensing configuration information; the configuration information of the sensing device is used to enable the sensing device to perform sensing configuration and generate non-3GPP sensing data.

[0218] For example, the terminal device sends the configuration information of the sensing device to the sensing device through a non-3GPP communication link air interface.

[0219] In some embodiments, prior to step S501 described above, the method further includes:

[0220] S500: The terminal device sends a first request message to the base station. The first request message is used to request the reporting of non-3GPP sensing data.

[0221] For an example, a detailed description of the first request message can be found here. Figure 3 The embodiments shown are not described in detail here.

[0222] In some embodiments, step S500 may be implemented as: sending a first request message to the base station when a second triggering condition is met; wherein the second triggering condition includes at least one of the following:

[0223] The terminal device triggers the need to perceive the surrounding environment;

[0224] Terminal devices trigger the need to report sensing information.

[0225] For an example, a detailed description of the second triggering condition can be found in [reference needed]. Figure 3 The embodiments shown are not described in detail here.

[0226] In some embodiments, prior to step S401 described above, the method further includes:

[0227] S601, The terminal device receives the registration request message sent by the sensing device.

[0228] The registration request message is used to request that the information of the sensing device be registered with the base station.

[0229] For example, the terminal device receives the registration request message sent by the sensing device through a non-3GPP communication link air interface.

[0230] In some embodiments, after the terminal device receives the registration request message sent by the sensing device, the method further includes: registering the information of the sensing device in the terminal device.

[0231] S602, The terminal device sends a registration request message to the base station.

[0232] For example, the terminal device sends a registration request message to the base station through the 3GPP communication link air interface.

[0233] For an example, a detailed description of the registration request message can be found here. Figure 3 The embodiments shown are not described in detail here.

[0234] In some embodiments, after step S602, the method further includes:

[0235] S603. The terminal device receives a registration response message sent by the base station. The registration response message is used to indicate that the sensing device has successfully registered.

[0236] For example, the terminal device receives the registration response message sent by the base station through the 3GPP communication link air interface.

[0237] S604. The terminal device sends a registration response message to the sensing device.

[0238] For example, the terminal device sends a registration response message to the sensing device through a non-3GPP communication link air interface.

[0239] For an example, a detailed description of the registration response message can be found here. Figure 3 The embodiments shown are not described in detail here.

[0240] This disclosure provides a method for processing sensing data, applied to, for example... Figure 1 or Figure 2 The sensing device in the sensory system shown. For example... Figure 7 As shown, the method includes the following steps:

[0241] S701. The sensing device sends non-3GPP sensing data generated by the sensing device to the base station so that the base station can process the non-3GPP sensing data.

[0242] As one possible implementation, step S701 above can be implemented as follows: the sensing device sends the non-3GPP sensing data to the base station through the terminal device. For example, in Figure 1 In the sensing system shown, the sensing device's sensing terminal is connected to the base station. Therefore, the sensing device can send the non-3GPP sensing data to the base station through the terminal.

[0243] As another possible implementation, step S701 above can be implemented as follows: the sensing device directly sends the non-3GPP sensing data to the base station. For example, in Figure 2 In the sensing system shown, the sensing device is directly connected to the base station. Therefore, the sensing device can directly send the non-3GPP sensing data to the base station.

[0244] For an example, a detailed description of non-3GPP sensing data can be found in [reference needed]. Figure 3 The embodiments shown are not described in detail here.

[0245] In some embodiments, prior to step S701, the method further includes:

[0246] S801, The sensing device receives the configuration information of the sensing device sent by the base station.

[0247] As one possible implementation, step S801 above can be implemented as follows: the sensing device receives configuration information of the sensing device sent by the base station through the terminal device. For example, in Figure 1 In the sensor system shown, the sensing device's sensing terminal device is connected to the base station. Therefore, the sensing device can receive the configuration information of the sensing device sent by the base station through the terminal device.

[0248] As another possible implementation, step S801 above can be implemented as follows: the sensing device directly receives the configuration information of the sensing device sent by the base station. For example, in Figure 2 In the sensing system shown, the sensing device is directly connected to the base station. Therefore, the sensing device can directly receive the configuration information of the sensing device sent by the base station.

[0249] For an example, a detailed description of the configuration information for the sensing device can be found here. Figure 3 The embodiments shown are not described in detail here.

[0250] S802. The sensing device performs sensing configuration based on the sensing device's configuration information and generates non-3GPP sensing data.

[0251] For example, the sensing device performs sensing configuration based on the sensing mode and parameters used by the sensing device included in the sensing device's configuration information, and generates non-3GPP sensing data.

[0252] In some embodiments, prior to step S701, the method further includes:

[0253] S901. The sensing device sends a registration request message to the base station. The registration request message is used to request that the information of the sensing device be registered with the base station.

[0254] As one possible implementation, step S901 above can be implemented as follows: the sensing device sends the registration request message to the base station through the terminal device. For example, in Figure 1 In the sensor system shown, the sensing device's sensing terminal device is connected to the base station. Therefore, the sensing device can send the registration request message to the base station through the terminal device.

[0255] As another possible implementation, step S901 above can be implemented as follows: the sensing device directly sends the registration request message to the base station. For example, in Figure 2 In the sensor system shown, the sensing device is directly connected to the base station. Therefore, the sensing device can directly send the registration request message to the base station.

[0256] For an example, a detailed description of the registration request message can be found here. Figure 3The embodiments shown are not described in detail here.

[0257] S902. The sensing device receives a registration response message sent by the base station. The registration response message is used to indicate that the sensing device has successfully registered.

[0258] As one possible implementation, step S902 above can be implemented as follows: the sensing device receives the registration response message sent by the base station through the terminal device. For example, in Figure 1 In the sensing system shown, the sensing device's sensing terminal device is connected to the base station. Therefore, the sensing device can receive the registration response message sent by the base station through the terminal device.

[0259] As another possible implementation, step S902 above can be implemented as follows: the sensing device directly receives the registration response message sent by the base station. For example, in Figure 2 In the sensing system shown, the sensing device is directly connected to the base station. Therefore, the sensing device can directly receive the registration response message sent by the base station.

[0260] For an example, a detailed description of the registration response message can be found here. Figure 3 The embodiments shown are not described in detail here.

[0261] Below, based on Figure 1 The sensor system shown illustrates the processing method for sensing data provided in the embodiments of this application through interaction between base stations, terminal devices, and sensing devices.

[0262] For example, such as Figure 8 As shown, when a base station triggers the reporting of non-3GPP sensing data, the sensing data processing method provided in this application embodiment includes the following steps:

[0263] Sd1, the base station meets the first triggering condition.

[0264] For an example, a detailed description of the first triggering condition can be found in [reference needed]. Figure 3 The embodiments shown are not described in detail here.

[0265] Sd2, The base station sends the configuration information of the sensing device to the terminal device; correspondingly, the terminal device receives the configuration information of the sensing device sent by the base station.

[0266] For an example, a detailed description of the configuration information for the sensing device can be found here. Figure 3 The embodiments shown are not described in detail here.

[0267] Sd3, The terminal device sends the configuration information of the sensing device to the sensing device; correspondingly, the sensing device receives the configuration information of the sensing device sent by the terminal device.

[0268] Sd4. The sensing device performs sensing configuration based on the sensing device's configuration information and generates non-3GPP sensing data.

[0269] For an example, a detailed description of non-3GPP sensing data can be found in [reference needed]. Figure 3 The embodiments shown are not described in detail here.

[0270] Sd5, The sensing device sends non-3GPP sensing data to the terminal device; correspondingly, the terminal device receives the non-3GPP sensing data sent by the sensing device.

[0271] Sd6. The terminal device sends non-3GPP sensing data to the base station; correspondingly, the base station receives the non-3GPP sensing data sent by the terminal device.

[0272] Sd7, the base station processes non-3GPP sensing data.

[0273] For an example, a detailed description of how base stations process non-3GPP sensing data can be found in [reference needed]. Figure 3 The embodiments shown are not described in detail here.

[0274] For example, such as Figure 9 As shown, when a terminal device triggers the reporting of non-3GPP sensing data, the sensing data processing method provided in this application embodiment includes the following steps:

[0275] Se1, The terminal device meets the second triggering condition.

[0276] For an example, a detailed description of the second triggering condition can be found in [reference needed]. Figure 3 The embodiments shown are not described in detail here.

[0277] Se2, the terminal device sends the first request message to the base station.

[0278] The first request message is used to request the reporting of non-3GPP sensing data.

[0279] For an example, a detailed description of the first request message can be found here. Figure 3 The embodiments shown are not described in detail here.

[0280] Se3. In response to the first request message, the base station sends the configuration information of the sensing device to the terminal device; correspondingly, the terminal device receives the configuration information of the sensing device sent by the base station.

[0281] For an example, a detailed description of the configuration information for the sensing device can be found here. Figure 3 The embodiments shown are not described in detail here.

[0282] Se4. The terminal device sends the configuration information of the sensing device to the sensing device; correspondingly, the sensing device receives the configuration information of the sensing device sent by the terminal device.

[0283] Se5: Sensing devices perform sensing configuration based on the configuration information of the sensing devices, generating non-3GPP sensing data.

[0284] For an example, a detailed description of how base stations process non-3GPP sensing data can be found in [reference needed]. Figure 3 The embodiments shown are not described in detail here.

[0285] Se6: The sensing device sends non-3GPP sensing data to the terminal device; correspondingly, the terminal device receives the non-3GPP sensing data sent by the sensing device.

[0286] Se7: The terminal device sends non-3GPP sensing data to the base station; correspondingly, the base station receives the non-3GPP sensing data sent by the terminal device.

[0287] Se8, the base station processes non-3GPP sensing data.

[0288] For an example, a detailed description of how base stations process non-3GPP sensing data can be found in [reference needed]. Figure 3 The embodiments shown are not described in detail here.

[0289] For example, such as Figure 10 As shown, before reporting non-3GPP sensing data, the above method also includes registering the sensing device with the base station. For example, this can be implemented through the following steps:

[0290] Sf1. The sensing device sends a registration request message to the terminal device; correspondingly, the terminal device receives the registration request message sent by the sensing device.

[0291] For an example, a detailed description of the registration request message can be found here. Figure 3 The embodiments shown are not described in detail here.

[0292] Sf2, The terminal device registers the sensing device in the terminal device.

[0293] For example, the terminal device registers the sensing capabilities of the sensing device within the terminal device.

[0294] Sf3, The terminal device sends a registration request message to the base station; correspondingly, the base station receives the registration request message sent by the terminal device.

[0295] Sf4. The base station registers the sensing devices in the base station.

[0296] For example, the base station registers the sensing capabilities of the sensing devices within the base station.

[0297] Sf5. The base station sends a registration response message to the terminal device; correspondingly, the terminal device receives the registration response message sent by the base station.

[0298] For an example, a detailed description of the registration response message can be found here. Figure 3 The embodiments shown are not described in detail here.

[0299] Sf6. The terminal device sends a registration response message to the sensing device; correspondingly, the sensing device receives the registration response message sent by the terminal device.

[0300] It should be noted that the above registration process can be used for any of the following: initial registration, mobility update registration, or periodic registration.

[0301] Below, based on Figure 2 The sensor system shown illustrates the processing method for sensing data provided in the embodiments of this application through interaction between a base station and a sensing device.

[0302] For example, such as Figure 11 As shown, the method for processing perceived data provided in this application includes the following steps:

[0303] Sg1, The base station meets the first triggering condition.

[0304] For example, step Sg1 above can be implemented as follows: the non-3GPP sensing module in the base station satisfies the first triggering condition.

[0305] For an example, a detailed description of the first triggering condition can be found in [reference needed]. Figure 3 The embodiments shown are not described in detail here.

[0306] Sg2, The base station sends the configuration information of the sensing device to the sensing device; correspondingly, the sensing device receives the configuration information of the sensing device sent by the base station.

[0307] For an example, a detailed description of the configuration information for the sensing device can be found here. Figure 3 The embodiments shown are not described in detail here.

[0308] Sg3. The sensing device performs sensing configuration based on the sensing device's configuration information and generates non-3GPP sensing data.

[0309] For an example, a detailed description of non-3GPP sensing data can be found in [reference needed]. Figure 3 The embodiments shown are not described in detail here.

[0310] Sg4, The sensing device sends non-3GPP sensing data to the base station; correspondingly, the base station receives the non-3GPP sensing data sent by the sensing device.

[0311] Sg5, the base station processes non-3GPP sensing data.

[0312] For an example, a detailed description of how base stations process non-3GPP sensing data can be found in [reference needed]. Figure 3 The embodiments shown are not described in detail here.

[0313] For example, such as Figure 12 As shown, before reporting non-3GPP sensing data, the above method also includes registering the sensing device with the base station. For example, this can be implemented through the following steps:

[0314] Sh1. The sensing device sends a registration request message to the base station; correspondingly, the base station receives the registration request message sent by the sensing device.

[0315] For an example, a detailed description of the registration request message can be found here. Figure 3 The embodiments shown are not described in detail here.

[0316] Sh2. The base station registers the sensing devices in the base station.

[0317] For example, the base station registers the sensing capabilities of the sensing devices within the base station.

[0318] Sh3. The base station sends a registration response message to the sensing device; correspondingly, the sensing device receives the registration response message sent by the base station.

[0319] For an example, a detailed description of the registration response message can be found here. Figure 3 The embodiments shown are not described in detail here.

[0320] It should be noted that the above registration process can be used for any of the following: initial registration or periodic registration.

[0321] To facilitate understanding, the processing method of perceived data provided in the embodiments of this disclosure will be described below with examples.

[0322] Example 1: In such Figure 1 In the sensor system shown, if the non-3GPP sensing module of the base station triggers sensing; assuming the sensing device is a camera, connected to the terminal device through onboard circuitry; the sensing service is target tracking service, and the sensing processing is decision-level fusion.

[0323] In Example 1, the registration of the sensing capabilities of a sensing device can be achieved through the following steps:

[0324] Step 1: The terminal device acquires the sensing capabilities of the sensing device.

[0325] For example, if the sensing device is a camera, then the terminal device acquires the sensing capabilities of the camera.

[0326] Step 2: The terminal device sends a registration request message to the base station through the 3GPP communication air interface link.

[0327] For example, the registration request message includes:

[0328] (1) Registration type: Non-3GPP sensing device sensing;

[0329] (2) Sensing device type: camera;

[0330] (3) Sensing device capabilities: The camera resolution is 2560×1440; the camera reporting frame rate is 50Hz; the data format is H.265 / H.264; the camera angle is 30 degrees south of west, the height is 10 meters, and the maximum aperture is F1.6; focal length & field of view: 4mm, horizontal field of view: 70°, vertical field of view: 35°, diagonal field of view: 85°6mm; or, horizontal field of view: 46°, vertical field of view: 24°, diagonal field of view: 54°8mm; or, horizontal field of view: 43°, vertical field of view: 24°, diagonal field of view: 50°12mm; or, horizontal field of view: 27°, vertical field of view: 15°, diagonal field of view: 31°.

[0331] Step 3: The base station's communication module receives the registration request sent by the terminal device. The non-3GPP sensing module determines that it is a registration request from a non-3GPP sensing device, and then forwards the registration request message to the base station's non-3GPP sensing module.

[0332] Step 4: The non-3GPP sensing module of the base station receives the registration request message, confirms that the sensing device has been successfully registered as Camera 1, and sends a registration response message (including registration success information and the identification information of the sensing device) to the base station's communication module, which then sends it to the terminal.

[0333] In Example 1, the reporting and processing flow of non-3GPP sensing data generated by the sensing device can be implemented as follows:

[0334] Step 1: The base station's non-3GPP sensing module generates a sensing requirement, which requires non-3GPP sensing.

[0335] For example, if the non-3GPP sensing module of the base station meets the first triggering condition, it is determined that the non-3GPP sensing module of the base station generates a sensing requirement, which requires non-3GPP sensing.

[0336] Step 2: The non-3GPP sensing module of the base station sends the non-3GPP sensing configuration to the base station's communication module through the NS3 interface.

[0337] The non-3GPP awareness configurations include:

[0338] (1) Perception mode: Camera perception;

[0339] (2) Identification information of sensing devices: Camera 1

[0340] (3) Sensor parameters: camera resolution is 2560×1440, camera reporting frame rate is 50Hz, data format is H.265 / H.264, camera angle is 30 degrees south of west, height is 10 meters, focal length is 12mm, horizontal field of view: 27°, vertical field of view: 15°, diagonal field of view: 31°, maximum aperture: F1.6;

[0341] (4) Transmission resources used for reporting sensing data.

[0342] Step 3: The base station's communication module sends the non-3GPP sensing configuration to the terminal device through the 3GPP communication air interface link.

[0343] Step 4: The terminal device receives the sensing configuration information and sends the sensing configuration information to the camera 1 through the transmission interface according to the identification information of the sensing device (camera 1).

[0344] The configuration information of the sensing device includes:

[0345] (1) Perception mode: Camera perception;

[0346] (2) Parameters used by the sensing device: camera resolution is 2560×1440, camera reporting frame rate is 50Hz, data format is H.265 / H.264, camera angle is 30 degrees south of west, height is 10 meters, focal length is 12mm, horizontal field of view: 27°, vertical field of view: 15°, diagonal field of view: 31°, maximum aperture: F1.6.

[0347] Step 5: Camera 1 performs perception configuration based on the configuration information of the sensing device, and generates video data to be recorded by the camera.

[0348] Step 6: Camera 1 transmits the video data recorded by the camera to the terminal device through the onboard circuit.

[0349] Step 7: The terminal device encodes and modulates the video data recorded by the camera into a communication signal, and reports the communication signal to the base station's communication module via the 3GPP air interface link on the designated transmission resources based on the transmission resources used for reporting sensing data in the non-3GPP sensing configuration. Simultaneously, it sends a non-3GPP sensing data reporting identifier.

[0350] Step 8: The base station's non-3GPP sensing module obtains the non-3GPP sensing data reporting identifier and notifies the base station's communication module to forward the non-3GPP sensing data. The base station's communication module parses the communication signal into non-3GPP sensing data through signal detection, demodulation, and decoding, and forwards it to the base station's non-3GPP sensing module.

[0351] Step 9: The base station's non-3GPP sensing module processes the non-3GPP sensing data.

[0352] For example, the non-3GPP sensing module of the base station processes non-3GPP sensing data in the following ways:

[0353] Preprocessing: The QoS stream is parsed into camera video data.

[0354] Perception data processing: (1) Use a YOLO neural network to extract the target category, local coordinate system xyz position, and confidence score from the camera video data. (2) Perform a confidence-weighted average of the target extracted from the camera video data and the target perceived by synesthesia to obtain the final target perception result, which includes the target category, local coordinate system xyz position, and confidence score.

[0355] Data standardization: The local coordinate system xyz position is transformed into longitude, latitude and altitude, and the trajectory of multiple frames is correlated to obtain the object's three-dimensional moving speed (vx, vy, vz) and object ID. Combined with the confidence level and target category extracted from the camera data, a unified format perception result containing ID, category, confidence level, position and speed is obtained.

[0356] Step 10: The non-3GPP sensing modules of the base station report sensing data in a unified format to the core network.

[0357] Example 2: In such Figure 1 In the sensor system shown, if the terminal device triggers sensing; assuming the sensing device is a millimeter-wave radar connected to the terminal device via Ethernet; the sensing service is target tracking service, and the sensing processing is feature-level fusion.

[0358] In Example 2, the registration of the sensing capabilities of the sensing device can be achieved through the following steps:

[0359] Step 1: The terminal device acquires the sensing capabilities of the sensing device.

[0360] For example, if the sensing device is a millimeter-wave radar, then the terminal device acquires the sensing capability of the millimeter-wave radar.

[0361] Step 2: The terminal device sends a registration request message to the base station through the 3GPP communication air interface link.

[0362] For example, the registration request message includes:

[0363] (1) Registration type: Non-3GPP sensing device sensing;

[0364] (2) Sensor type: millimeter-wave radar;

[0365] (3) Sensor capabilities: maximum detection distance 150m, distance resolution 1.2m, speed resolution 0.36km / h, azimuth range 90 degrees, elevation range 13 degrees, refresh rate 12.5Hz, and transmit power 30dB.

[0366] Step 3: The base station's communication module receives the registration request message sent by the terminal. The non-3GPP sensing module determines that it is a registration request for a non-3GPP sensing device, and then forwards the registration request message to the base station's non-3GPP sensing module.

[0367] Step 4: The non-3GPP sensing module of the base station receives the registration request message, confirms that the sensing device has been successfully registered as millimeter-wave radar 1, and sends a registration response message (including registration success information and the identification information of the sensing device) to the base station's communication module, which then sends it to the terminal.

[0368] In Example 2, the reporting and processing flow of non-3GPP sensing data generated by the sensing device can be implemented as follows:

[0369] Step 1: The terminal device generates a sensing requirement, which requires non-3GPP sensing.

[0370] For example, if the terminal device meets the second triggering condition, it is determined that the terminal device generates a sensing requirement, which requires non-3GPP sensing.

[0371] Step 2: The terminal device sends a first request message to the base station, wherein the first request message is used to request the reporting of non-3GPP sensing data.

[0372] The first request message includes the configuration information of the sensing device:

[0373] (1) Sensing mode: Millimeter-wave radar sensing;

[0374] (2) Identification information of sensing devices: millimeter-wave radar 1;

[0375] (3) Parameters used by the sensing equipment: The millimeter-wave radar is oriented 30 degrees west of north, with a maximum detection range of 150m, a range resolution of 1.2m, a speed resolution of 0.36km / h, an azimuth range of 90 degrees, an elevation range of 13 degrees, a refresh rate of 12.5Hz, and a transmit power of 30dB.

[0376] Step 3: After the base station's non-3GPP sensing module captures the first request message, it generates a non-3GPP sensing configuration and sends the non-3GPP sensing configuration to the base station's communication module through the NS3 interface.

[0377] The 3GPP awareness configuration includes:

[0378] (1) Sensing mode: Millimeter-wave radar sensing;

[0379] (2) Identification information of sensing devices: millimeter-wave radar 1;

[0380] (3) Sensor parameters: The millimeter-wave radar is oriented 30 degrees west of north, with a maximum detection range of 150m, a range resolution of 1.2m, a speed resolution of 0.36km / h, an azimuth range of 90 degrees, an elevation range of 13 degrees, a refresh rate of 12.5Hz, and a transmit power of 30dB.

[0381] (4) Transmission resources used for reporting sensing data.

[0382] Step 4: The base station's communication module will send the capture confirmation instruction and non-3GPP awareness configuration to the terminal device through the 3GPP communication air interface link.

[0383] Step 5: The terminal device receives the capture confirmation instruction and the non-3GPP sensing configuration, and sends the sensing configuration information of the sensing device to the millimeter-wave radar 1 through the transmission interface according to the identification information of the sensing device (millimeter-wave radar 1).

[0384] The configuration information of the sensing device includes:

[0385] (1) Sensing mode: Millimeter-wave radar sensing;

[0386] (2) Parameters used by the sensing equipment: The millimeter-wave radar is oriented 30 degrees west of north, with a maximum detection range of 150m, a range resolution of 1.2m, a speed resolution of 0.36km / h, an azimuth range of 90 degrees, an elevation range of 13 degrees, a refresh rate of 12.5Hz, and a transmit power of 30dB.

[0387] Step 6: Millimeter-wave radar 1 performs perception configuration based on sensor configuration information and generates millimeter-wave radar time-delay Doppler image data.

[0388] Step 7: Millimeter-wave radar 1 transmits the time-delay Doppler image data to the terminal device via the Ethernet interface.

[0389] Step 8: The terminal device converts the delay Doppler image data into a communication signal through encoding, modulation, and other processes. Based on the transmission resources used for reporting sensing data in the non-3GPP sensing configuration, it reports the communication signal to the base station's communication module via the 3GPP air interface link on the designated transmission resources. Simultaneously, it sends a non-3GPP sensing data reporting identifier.

[0390] Step 9: The base station's non-3GPP sensing module obtains the non-3GPP sensing data reporting identifier and notifies the base station's communication module to forward the non-3GPP sensing data. The base station's communication module parses the communication signal into non-3GPP sensing data through signal detection, demodulation, and decoding, and forwards it to the base station's non-3GPP sensing module.

[0391] Step 10: The base station's non-3GPP sensing module processes the non-3GPP sensing data.

[0392] For example, the non-3GPP sensing module of the base station processes non-3GPP sensing data in the following ways:

[0393] Preprocessing: The QoS stream is parsed into delay Doppler graph data.

[0394] Sensing data processing:

[0395] (1) Use a CNN neural network to extract perceptual features from time-delay Doppler images.

[0396] (2) The target perception features extracted from the time-delay Doppler image data are concatenated with the features extracted from the synesthesia perception results, and then processed by the transformer neural network to obtain the final target perception results, which include the local coordinate system xyz position, confidence level, and object category.

[0397] Data standardization: The local coordinate system xyz position is transformed into longitude, latitude and altitude, and multiple frames of trajectory are correlated to obtain the object's three-dimensional moving speed (vx, vy, vz) and object ID, resulting in a unified format perception result containing ID, category, confidence level, three-dimensional position and three-dimensional speed.

[0398] Step 11: The non-3GPP sensing modules of the base station report sensing data in a unified format to the core network.

[0399] Example 3: In such Figure 2 In the sensor system shown, if the non-3GPP sensing module of the base station triggers sensing; assuming the sensing device is a lidar connected to the base station via Ethernet; the sensing service is an environmental sensing service, and the sensing processing is data-level fusion.

[0400] In Example 3, the registration of the sensing capabilities of the sensing device can be achieved through the following steps:

[0401] Step 1: LiDAR 1 sends a registration request message to the base station via a non-3GPP communication air interface link.

[0402] Example registration request message:

[0403] (1) Registration type: Non-3GPP sensing device sensing;

[0404] (2) Sensing device type: LiDAR;

[0405] (3) Sensing device capabilities: laser wavelength 905nm, detection distance: 90m@10% reflectivity; 130m@20% reflectivity; 260m@80% reflectivity; detection range 70 degrees, distance measurement error 2cm, angle measurement error 0.1 degrees, output frequency 100Hz.

[0406] Step 2: The base station's non-3GPP sensing module receives the registration request message, confirms that the sensing device has been successfully registered as LiDAR 1, and sends a registration response message (including registration success information and the sensing device's identification information) to the sensing device.

[0407] In Example 3, the reporting and processing flow of non-3GPP sensing data generated by the sensing device can be implemented as follows:

[0408] Step 1: The base station's non-3GPP sensing module generates a sensing requirement, which requires non-3GPP sensing.

[0409] For example, if the non-3GPP sensing module of the base station meets the first triggering condition, it is determined that the non-3GPP sensing module of the base station generates a sensing requirement, which requires non-3GPP sensing.

[0410] Step 2: The non-3GPP sensing module of the base station sends the configuration information of the sensing device to the lidar 1 through the transmission interface.

[0411] For example, the configuration information of the sensing device includes:

[0412] (1) Sensing mode: LiDAR;

[0413] (2) Parameters used by the sensing device: angle facing 60 degrees north of west, laser wavelength 905nm, detection distance 260m, detection range 70 degrees, distance measurement error 2cm, angle measurement error 0.1 degrees, output frequency 100Hz.

[0414] Step 3: LiDAR 1 performs perception configuration based on the configuration information of the sensing device, and generates LiDAR point cloud data.

[0415] Step 4: The lidar 1 transmits the lidar point cloud data to the non-3GPP sensing module of the base station via the Ethernet port.

[0416] Step 5: The non-3GPP sensing module of the base station processes the lidar point cloud data.

[0417] For example, the non-3GPP sensing module of the base station processes lidar point cloud data in the following ways:

[0418] Preprocessing: The QoS stream is parsed into LiDAR point cloud data.

[0419] Perception data processing: (1) The lidar point cloud data is stitched together with the original sensor data. (2) The data is processed using a CNN neural network to obtain the three-dimensional point cloud results of the perception, which includes the three-dimensional position, material, and object category of the environmental points in the environmental point cloud.

[0420] Data standardization: The local coordinate system xyz position of the environmental point cloud is transformed into longitude, latitude and altitude to obtain a unified format perception result of the environmental point cloud perception result containing object category, material and three-dimensional position.

[0421] Step 6: The non-3GPP sensing modules of the base station report sensing data in a unified format to the core network.

[0422] In summary, this disclosure can be applied to integrated communication and sensing scenarios, proposing a method for processing sensing data. Based on this disclosure, sensor information carried by mobile terminals, vehicles, smart transportation systems, smart factories, etc. (taking sensing devices as sensors as an example) can be incorporated into the sensing architecture. This increases the sources of sensing data for the sensing system, improving sensing quality, sensing range, and sensing accuracy.

[0423] Based on the synesthetic networking architecture, sensors distributed in various locations can perform data fusion, breaking the original situation of individual sensor perception. Perceived information can be fused in the synesthetic system, improving the perception quality and range of existing sensors.

[0424] The beneficial effects of the embodiments of this disclosure will be briefly described below.

[0425] In related technologies, besides utilizing 3GPP radio spectrum resources for sensing, numerous sensing devices have been applied in the industry, such as roadside stations for vehicle-to-everything (V2X) communication, video surveillance cameras, and IoT sensing units. The sensing information generated by these devices is transmitted to the public Internet via communication links for data processing and application. This involves gateway interaction between the 3GPP core network and the public Internet, resulting in high processing complexity and latency. Furthermore, currently, base stations, acting as sensing sites, rely on self-transmission and self-reception (A transmits, B receives) in the absence of 3GPP standards, limiting the sensing range and diversity. Therefore, this disclosure integrates the existing sensing architecture through communication links, external sensing devices, and terminal-attached sensing devices. This offers the following advantages:

[0426] 1. It enables non-3GPP sensors to be incorporated into the sensing architecture, allowing the sensing architecture to utilize existing sensing resources, greatly expanding the sensing range and diversity of the sensing system, and improving the sensing effect.

[0427] 2. Similar to the 5G-NSA architecture, non-3GPP sensing information is aggregated and processed at the base station side, simplifying the deployment and evolution of core network sensing function entities. It is easy to implement, adaptable to current field conditions, and facilitates rapid deployment of experimental sites.

[0428] 3. Compared with the communication link-based transmission to the Internet public network transmission link, or the core network processing of non-3GPP sensing data architecture, the sensing architecture provided in this disclosure can reduce the latency and complexity of the original non-3GPP sensing data transmission and processing, and can be adapted to better processing methods such as data-level fusion and feature-level fusion.

[0429] The foregoing primarily describes the solutions of the embodiments of this disclosure from a methodological perspective. It is understood that, in order to achieve the above-mentioned functions, the sensing data processing device includes at least one of the hardware structures and software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, the embodiments of this disclosure can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this disclosure.

[0430] This disclosure embodiment can divide the sensing data processing device into functional modules according to the above method embodiment. For example, each function can be divided into a separate functional module, or two or more functions can be integrated into one functional module. The integrated module can be implemented in hardware or software. It should be noted that the module division in this disclosure embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods. The following description uses the example of dividing each functional module according to each function.

[0431] Figure 13 This is a schematic diagram of a sensing data processing device provided in an embodiment of this disclosure. This sensing data processing device is applied to a base station and can execute the sensing data processing method provided in the above-described method embodiments. Figure 13 As shown, the sensing data processing device 1000 includes a communication module 1001 and a processing module 1002.

[0432] Communication module 1001 is used to receive non-3GPP sensing data from sensing devices;

[0433] The processing module 1002 is used to process non-3GPP sensing data.

[0434] In some embodiments, the communication module 1001 is specifically used to receive non-3GPP sensing data sent by the sensing device through the terminal device; or, to directly receive non-3GPP sensing data sent by the sensing device.

[0435] In some embodiments, the communication module 1001 is further configured to send configuration information of the sensing device to the sensing device.

[0436] In some embodiments, the communication module 1001 is specifically used to directly send the configuration information of the sensing device to the sensing device; or, to send the configuration information of the sensing device to the sensing device through a terminal device.

[0437] In some embodiments, the communication module 1001 is specifically used to send configuration information of the sensing device to the sensing device when a first triggering condition is met; wherein the first triggering condition includes at least one of the following: the base station triggers the need to sense the static environment; the base station triggers the need to sense the dynamic target; the base station triggers the need to monitor the unmanned aerial vehicle (UAV) equipment.

[0438] In some embodiments, the communication module 1001 is specifically used to send configuration information of the sensing device to the sensing device upon receiving a first request message; the first request message is used to request the reporting of non-3GPP sensing data.

[0439] In some embodiments, the first request message is sent by the terminal device when a second triggering condition is met; wherein the second triggering condition includes at least one of the following: the terminal device triggers a need to sense the surrounding environment; the terminal device triggers a need to report sensed information.

[0440] In some embodiments, the first request message includes at least one of the following: sensing mode, identification information of the sensing device, and parameters used by the sensing device.

[0441] In some embodiments, the configuration information of the sensing device includes at least one of the following: sensing mode and parameters used by the sensing device.

[0442] In some embodiments, the communication module 1001 is specifically used to send non-3GPP sensing configuration information to the terminal device, so that the terminal device sends the configuration information of the sensing device to the sensing device based on the non-3GPP sensing configuration information; the configuration information of the sensing device is a subset of the non-3GPP sensing configuration information.

[0443] In some embodiments, non-3GPP sensing configuration information includes at least one of the following: sensing mode, identification information of sensing device, parameters used by sensing device, and transmission resources used for sensing data reporting.

[0444] In some embodiments, the communication module 1001 is specifically used to receive non-3GPP sensing data on the transmission resources used for sensing data reporting.

[0445] In some embodiments, the communication module 1001 is specifically used to obtain non-3GPP sensing data based on the received non-3GPP sensing data reporting identifier.

[0446] In some embodiments, the processing module 1002 is specifically used to sequentially perform preprocessing, sensing data processing, and data standardization processing on non-3GPP sensing data.

[0447] In some embodiments, the communication module 1001 is further configured to receive a registration request message, which requests the information of the sensing device to be registered with the base station; and send a registration response message, which indicates that the sensing device has been successfully registered.

[0448] In some embodiments, the communication module 1001 is specifically used to receive a registration request message sent by the sensing device through the terminal device; or, to directly receive a registration request message sent by the sensing device.

[0449] In some embodiments, the communication module 1001 is specifically used to send a registration response message to the sensing device through the terminal device; or, to send the registration response message directly to the sensing device.

[0450] In some embodiments, the registration request message includes at least one of the following: registration type, type of sensing device, and capability information of the sensing device.

[0451] In some embodiments, the capability information of the sensing device is determined based on the type of sensing device.

[0452] In some embodiments, when the type of sensing device includes a camera, the capability information of the sensing device includes at least one of the following: the camera's resolution adjustment range, reporting frame rate adjustment range, turning angle, focal length, aperture, magnification, and correction coefficient; when the type of sensing device includes millimeter-wave radar, the capability information of the sensing device includes at least one of the following: the millimeter-wave radar's transmit power range, beam configurable type, operating frequency band adjustment range, sensing resolution adjustment range, sensing angle adjustment range, sensing distance adjustment range, and snapshot reporting frame rate adjustment range; when the type of sensing device includes lidar, the capability information of the sensing device includes at least one of the following: the lidar's scanning mode, laser beam angle adjustment range, angle resolution adjustment range, operating frequency adjustment range, and snapshot... The reporting frame rate adjustment range; when the type of sensing device includes ultra-wideband (UWB) devices, the capability information of the sensing device includes at least one of the following: the sensing frequency band adjustment range, pulse shape, transmit power adjustment range, pulse repetition frequency adjustment range, and snapshot reporting frame rate adjustment range of the UWB device; when the type of sensing device includes wireless connectivity devices, the capability information of the sensing device includes at least one of the following: the sensing frequency band adjustment range, sensing signal strength, waveform adjustment range, sensing resolution adjustment range, and snapshot reporting frame rate adjustment range of the wireless connectivity device; when the type of sensing device includes GPS devices, the capability information of the GPS devices includes at least one of the following: positioning accuracy adjustment range, satellite signal strength, navigation path planning capability, real-time speed measurement range, and time synchronization accuracy adjustment range.

[0453] Figure 14 This is a schematic diagram of a sensing data processing device provided in an embodiment of this disclosure. This sensing data processing device is applied to a terminal device and can execute the sensing data processing method provided in the above-described method embodiment. Figure 14 As shown, the sensing data processing device 1100 includes a receiving module 1101 and a transmitting module 1102. In some embodiments, the sensing data processing device 1100 further includes a registration module 1103.

[0454] Receiver module 1101 is used to receive non-3GPP sensing data sent by sensing devices;

[0455] The transmitting module 1102 is used to transmit non-3GPP sensing data to the base station so that the base station can process the non-3GPP sensing data.

[0456] In some embodiments, the receiving module 1101 is further configured to receive non-3GPP sensing configuration information sent by the base station; the sending module 1102 is further configured to send the configuration information of the sensing device to the sensing device based on the non-3GPP sensing configuration information; the configuration information of the sensing device is a subset of the non-3GPP sensing configuration information; the configuration information of the sensing device is used to enable the sensing device to perform sensing configuration and generate non-3GPP sensing data.

[0457] In some embodiments, the sending module 1102 is further configured to send a first request message to the base station, the first request message being used to request the reporting of non-3GPP sensing data.

[0458] In some embodiments, the sending module 1102 is specifically used to send a first request message to the base station when a second triggering condition is met; wherein the second triggering condition includes at least one of the following: the terminal device triggers a need to sense the surrounding environment; the terminal device triggers a need to report sensed information.

[0459] In some embodiments, the receiving module 1101 is further configured to receive a registration request message sent by the sensing device, the registration request message being used to request that the information of the sensing device be registered in the base station; the sending module 1102 is further configured to send the registration request message to the base station.

[0460] In some embodiments, the registration module 1103 is used to register the information of the sensing device in the terminal device.

[0461] In some embodiments, the receiving module 1101 is further configured to receive a registration response message sent by the base station, the registration response message being used to indicate that the sensing device has successfully registered; the sending module 1102 is further configured to send the registration response message to the sensing device.

[0462] Figure 15 This is a schematic diagram of a sensing data processing device provided in an embodiment of this disclosure. This sensing data processing device is applied to a sensing device and can execute the sensing data processing method provided in the above-described method embodiments. Figure 15 As shown, the sensing data processing device 1200 includes a transmitting module 1201. In some embodiments, the sensing data processing device 1200 further includes a receiving module 1202 and a configuration module 1203.

[0463] The transmitting module 1201 is used to transmit non-3GPP sensing data generated by the sensing device to the base station so that the base station can process the non-3GPP sensing data.

[0464] In some embodiments, the receiving module 1202 is used to receive configuration information of the sensing device sent by the base station; the configuration module 1203 is used to perform sensing configuration based on the configuration information of the sensing device and generate non-3GPP sensing data.

[0465] In some embodiments, the sending module 1201 is further configured to send a registration request message to the base station, the registration request message being used to request the information of the sensing device to be registered with the base station; the receiving module 1202 is further configured to receive a registration response message sent by the base station, the registration response message being used to indicate that the sensing device has been successfully registered.

[0466] When implementing the functions of the integrated modules described above in hardware, this disclosure provides a possible structure for the communication device involved in the above embodiments. For example... Figure 16 As shown, the communication device 1300 includes: a processor 1302 and a bus 1304. Optionally, the communication device may also include a memory 1301; alternatively, the communication device 1300 may also include a communication interface 1303.

[0467] Processor 1302 may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with embodiments of this disclosure. Processor 1302 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with embodiments of this disclosure. Processor 1302 may also be a combination of functions implementing computation, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0468] The communication interface 1303 is used to connect to other devices via a communication network. This communication network can be Ethernet, wireless access network, wireless local area network (WLAN), etc.

[0469] The memory 1301 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0470] In one possible implementation, the memory 1301 can exist independently of the processor 1302. The memory 1301 can be connected to the processor 1302 via a bus 1304 and is used to store instructions or program code. When the processor 1302 calls and executes the instructions or program code stored in the memory 1301, it can implement the perceptual data processing method provided in this embodiment. In another possible implementation, the memory 1301 can also be integrated with the processor 1302.

[0471] The 1304 bus can be an extended industry standard architecture (EISA) bus, etc. The 1304 bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 16 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0472] Some embodiments of this disclosure provide a computer-readable storage medium (e.g., a non-transitory computer-readable storage medium) storing computer program instructions that, when executed on a computer, cause the computer to perform a method for processing sensed data as described in any of the above embodiments.

[0473] For example, the computer-readable storage media described above may include, but are not limited to: magnetic storage devices (e.g., hard disks, floppy disks, or magnetic tapes), optical disks (e.g., compact disks (CDs), digital versatile disks (DVDs), etc.), smart cards, and flash memory devices (e.g., erasable programmable read-only memory (EPROMs), cards, sticks, or key drives, etc.). The various computer-readable storage media described in this disclosure may represent one or more devices for storing information and / or other machine-readable storage media. The term "machine-readable storage media" may include, but is not limited to, wireless channels and various other media capable of storing, containing, and / or carrying instructions and / or data.

[0474] This disclosure provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the perceptual data processing method described in any of the above embodiments.

[0475] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any changes or substitutions within the technical scope disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A method for processing sensory data, characterized in that, Applied to a base station, the method includes: Receive non-3GPP sensing data from sensing devices; The non-3GPP sensing data is processed.

2. The method according to claim 1, characterized in that, The non-3GPP sensing data received from the sensing device includes: The terminal device receives the non-3GPP sensing data sent by the sensing device; or... It directly receives the non-3GPP sensing data sent by the sensing device.

3. The method according to claim 1, characterized in that, The method further includes: Send the configuration information of the sensing device to the sensing device.

4. The method according to claim 3, characterized in that, Sending the configuration information of the sensing device to the sensing device includes: Directly send the configuration information of the sensing device to the sensing device; or... The terminal device sends the configuration information of the sensing device to the sensing device.

5. The method according to claim 3, characterized in that, Sending the configuration information of the sensing device to the sensing device includes: Upon meeting a first triggering condition, the configuration information of the sensing device is sent to the sensing device; wherein the first triggering condition includes at least one of the following: The base station is required to sense the static environment. The need to trigger the base station to sense dynamic targets; This triggers the need for the base station to monitor drone equipment.

6. The method according to claim 3, characterized in that, Sending the configuration information of the sensing device to the sensing device includes: Upon receiving the first request message, the configuration information of the sensing device is sent to the sensing device; the first request message is used to request the reporting of non-3GPP sensing data.

7. The method according to claim 6, characterized in that, The first request message is sent by the terminal device when a second triggering condition is met; wherein the second triggering condition includes at least one of the following: The terminal device triggers the need to sense the surrounding environment. The terminal device triggers the requirement to report sensing information.

8. The method according to claim 7, characterized in that, The first request message includes at least one of the following: Sensing mode, identification information of sensing devices, and parameters used by sensing devices.

9. The method according to claim 3, characterized in that, The configuration information of the sensing device includes at least one of the following: sensing mode and parameters used by the sensing device.

10. The method according to claim 4, characterized in that, The step of sending the configuration information of the sensing device to the sensing device through the terminal device includes: The terminal device sends non-3GPP sensing configuration information to the terminal device, so that the terminal device sends the configuration information of the sensing device to the sensing device based on the non-3GPP sensing configuration information; the configuration information of the sensing device is a subset of the non-3GPP sensing configuration information.

11. The method according to claim 10, characterized in that, The non-3GPP sensing configuration information includes at least one of the following: sensing mode, identification information of sensing device, parameters used by sensing device, and transmission resources used for sensing data reporting.

12. The method according to claim 11, characterized in that, The non-3GPP sensing data received from the sensing device includes: The non-3GPP sensing data is received on the transmission resources used for reporting the sensing data.

13. The method according to claim 1, characterized in that, The non-3GPP sensing data received from the sensing device includes: The non-3GPP sensing data is obtained based on the received non-3GPP sensing data reporting identifier.

14. The method according to claim 1, characterized in that, The processing of the non-3GPP sensing data includes: The non-3GPP sensing data is preprocessed, sensing data is processed, and data standardization is performed sequentially.

15. The method according to claim 1, characterized in that, The method further includes: Receive a registration request message, the registration request message being used to request that the information of the sensing device be registered with the base station; A registration response message is sent, which indicates that the sensing device has been successfully registered.

16. The method according to claim 15, characterized in that, The receipt of the registration request message includes: The terminal device receives the registration request message sent by the sensing device; or... The system directly receives the registration request message sent by the sensing device.

17. The method according to claim 15, characterized in that, Sending the registration response message includes: The registration response message is sent from the terminal device to the sensing device; or... The registration response message is sent directly to the sensing device.

18. The method according to claim 15, characterized in that, The registration request message includes at least one of the following: registration type, type of sensing device, and capability information of the sensing device.

19. The method according to claim 18, characterized in that, The capability information of the sensing device is determined based on the type of the sensing device.

20. The method according to claim 19, characterized in that, When the type of sensing device includes a camera, the capability information of the sensing device includes at least one of the following: the resolution adjustment range of the camera, the reporting frame rate adjustment range, the turning angle, the focal length, the aperture, the magnification, and the correction coefficient. When the type of sensing device includes millimeter-wave radar, the capability information of the sensing device includes at least one of the following: the transmit power range of the millimeter-wave radar, the beam configurable type, the operating frequency band adjustment range, the sensing resolution adjustment range, the sensing angle adjustment range, the sensing distance adjustment range, and the snapshot reporting frame rate adjustment range. When the type of sensing device includes LiDAR, the capability information of the sensing device includes at least one of the following: LiDAR scanning mode, laser beam angle adjustment range, angle resolution adjustment range, operating frequency adjustment range, and snapshot reporting frame rate adjustment range; When the type of sensing device includes an ultra-wideband device, the capability information of the sensing device includes at least one of the following: the sensing frequency band adjustment range, pulse shape, transmit power adjustment range, pulse repetition frequency adjustment range, and snapshot reporting frame rate adjustment range of the ultra-wideband device; When the type of sensing device includes a wirelessly connected device, the capability information of the sensing device includes at least one of the following: the sensing frequency band adjustment range, sensing signal strength, waveform adjustment range, sensing resolution adjustment range, and snapshot reporting frame rate adjustment range of the wirelessly connected device. When the type of sensing device includes a GPS device, the capability information of the GPS device includes at least one of the following: positioning accuracy adjustment range, satellite signal strength, navigation path planning capability, real-time speed measurement range, and time synchronization accuracy adjustment range.

21. A method for processing sensory data, characterized in that, Applied to a terminal device, the method includes: Receive non-3GPP sensing data sent by sensing devices; The non-3GPP sensing data is sent to the base station so that the base station can process the non-3GPP sensing data.

22. The method according to claim 21, characterized in that, The method further includes: Receive non-3GPP sensing configuration information sent by the base station; Based on the non-3GPP sensing configuration information, the configuration information of the sensing device is sent to the sensing device; the configuration information of the sensing device is a subset of the non-3GPP sensing configuration information; the configuration information of the sensing device is used to enable the sensing device to perform sensing configuration and generate non-3GPP sensing data.

23. The method according to claim 22, characterized in that, Before receiving the non-3GPP sensing configuration information sent by the base station, the method further includes: A first request message is sent to the base station, the first request message being used to request the reporting of non-3GPP sensing data.

24. The method according to claim 23, characterized in that, Sending the first request message to the base station includes: If a second triggering condition is met, a first request message is sent to the base station; wherein the second triggering condition includes at least one of the following: The terminal device triggers the need to sense the surrounding environment. The terminal device triggers the requirement to report sensing information.

25. The method according to claim 21, characterized in that, The method further includes: The system receives a registration request message sent by a sensing device, the registration request message being used to request that the information of the sensing device be registered with the base station; The registration request message is sent to the base station.

26. The method according to claim 25, characterized in that, The method further includes: The information of the sensing device is registered in the terminal device.

27. The method according to claim 25, characterized in that, The method further includes: The system receives a registration response message sent by the base station, the registration response message indicating that the sensing device has successfully registered. Send the registration response message to the sensing device.

28. A method for processing sensory data, characterized in that, Applied to a sensing device, the method includes: The non-3GPP sensing data generated by the sensing device is sent to the base station so that the base station can process the non-3GPP sensing data.

29. The method according to claim 28, characterized in that, The method further includes: Receive configuration information of the sensing device sent by the base station; Based on the configuration information of the sensing device, sensing configuration is performed to generate non-3GPP sensing data.

30. The method according to claim 28, characterized in that, The method further includes: Send a registration request message to the base station, the registration request message being used to request that the information of the sensing device be registered with the base station; The system receives a registration response message sent by the base station, which indicates that the sensing device has successfully registered.

31. A communication device, characterized in that, include: Memory and processor; Memory and processor are coupled; The memory is used to store instructions that can be executed by the processor; When the processor executes the instructions, it performs the method as described in any one of claims 1 to 30.

32. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1 to 30.

33. A computer program product, characterized in that, The computer program product includes computer instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1 to 30.

Images