Sensing data processing method, communication apparatus, storage medium and program product

Abstract

Provided in the present disclosure are a sensing data processing method, a communication apparatus, a storage medium and a program product. The sensing data processing method comprises: receiving non-3GPP sensing data from a sensing device; and processing the non-3GPP sensing data.

Description

Method for processing perception data, communication device, storage medium and program product

[0001] The present disclosure claims priority to Chinese Patent Application No. 202411079584.6, filed on August 6, 2024, the entire contents of which are incorporated herein by reference. TECHNICAL FIELD

[0002] The present disclosure relates to the technical field of wireless communication, and particularly relates to a method for processing perception data, a communication device, a storage medium and a program product. BACKGROUND

[0003] Communication and perception integration realizes wireless perception function and wireless communication function by sharing spectrum and software and hardware resources. Communication and perception integration can meet the perception performance requirements in various scenarios by means of mobile networks, and can improve communication performance by using perception function. In addition to realizing perception by using 3rd Generation Partnership Project (3GPP) wireless spectrum resources, a large number of perception devices have been applied in the industry, such as vehicle-to-everything roadside stations, video monitoring cameras, Internet of Things perception units, etc. Incorporating these multi-modal perception devices and perception information processing into the 3GPP perception architecture can not only improve the overall coverage performance of communication and perception, but also expand the form and information mode of perception. SUMMARY

[0004] In one aspect, a method for processing perception data is provided, and applied to a base station. The method for processing perception data comprises: receiving non-3GPP perception data of a perception device; and processing the non-3GPP perception data.

[0005] In another aspect, a method for processing perception data is provided, and applied to a terminal device. The method for processing perception data comprises: receiving non-3GPP perception data sent by a perception device; and sending the non-3GPP perception data to a base station, so that the base station processes the non-3GPP perception data.

[0006] In yet another aspect, a method for processing perception data is provided, and applied to a perception device. The method for processing perception data comprises: sending non-3GPP perception data generated by the perception device to a base station, so that the base station processes the non-3GPP perception data.

[0007] In yet another aspect, a perception data processing device is provided, and applied to a base station. The perception data processing device comprises: a communication module configured to receive non-3GPP perception data of a perception device; and a processing module configured to process the non-3GPP perception data.

[0008] In another aspect, a sensing data processing apparatus is provided, which is applied to a terminal device. The sensing data processing apparatus comprises a receiving module configured to receive non-3GPP sensing data sent by a sensing device; and a sending module configured to send the non-3GPP sensing data to a base station, so that the base station processes the non-3GPP sensing data.

[0009] In another aspect, a sensing data processing apparatus is provided, which is applied to a sensing device. The sensing data processing apparatus comprises a sending module configured to send non-3GPP sensing data generated by the sensing device to a base station, so that the base station processes the non-3GPP sensing data.

[0010] In another aspect, a communication apparatus is provided, which comprises a memory and a processor. The memory is coupled to the processor. The memory is configured to store a computer program. The processor is configured to implement the sensing data processing method when executing the computer program.

[0011] In another aspect, a computer readable storage medium is provided, which stores computer program instructions. The computer program instructions are configured to implement the sensing data processing method when running on a computer.

[0012] In another aspect, a computer program product is provided, which comprises computer program instructions. The computer program instructions are configured to implement the sensing data processing method when running on a computer. BRIEF DESCRIPTION OF DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the present disclosure, the following will briefly introduce the drawings needed to be used in some embodiments of the present disclosure. Obviously, the drawings described in the following are only some of the drawings of the present disclosure, and other drawings can also be obtained by those skilled in the art according to these drawings.

[0014] Fig. 1 is a schematic diagram of an architecture of a sensing system according to some embodiments of the present disclosure.

[0015] Fig. 2 is a schematic diagram of another architecture of a sensing system according to some embodiments of the present disclosure.

[0016] Fig. 3 is a flow chart of a sensing data processing method according to some embodiments of the present disclosure.

[0017] Fig. 4 is a flow chart of another sensing data processing method according to some embodiments of the present disclosure.

[0018] Fig. 5 is a flow chart of another sensing data processing method according to some embodiments of the present disclosure.

[0019] FIG. 6 is a flowchart of another method for processing perception data according to some embodiments of the present disclosure.

[0020] FIG. 7 is a flowchart of another method for processing perception data according to some embodiments of the present disclosure.

[0021] FIG. 8 is a flowchart of another method for processing perception data according to some embodiments of the present disclosure.

[0022] FIG. 9 is a flowchart of another method for processing perception data according to some embodiments of the present disclosure.

[0023] FIG. 10 is a flowchart of another method for processing perception data according to some embodiments of the present disclosure.

[0024] FIG. 11 is a flowchart of another method for processing perception data according to some embodiments of the present disclosure.

[0025] FIG. 12 is a flowchart of another method for processing perception data according to some embodiments of the present disclosure.

[0026] FIG. 13 is a structural diagram of a perception data processing apparatus according to some embodiments of the present disclosure.

[0027] FIG. 14 is a structural diagram of another perception data processing apparatus according to some embodiments of the present disclosure.

[0028] FIG. 15 is a structural diagram of another perception data processing apparatus according to some embodiments of the present disclosure.

[0029] FIG. 16 is a structural diagram of a communication apparatus according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0030] The technical solutions in the present disclosure will be described clearly and completely below with reference to the drawings in the present disclosure. Apparently, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, any other embodiments obtained by a person of ordinary skill in the art without creative effort shall fall within the protection scope of the present disclosure.

[0031] It should be noted that in the present disclosure, the words “exemplary” or “for example” are used to describe examples, instances, or illustrations. Any embodiment or design scheme described in the present disclosure by the words “exemplary” or “for example” shall not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the words “exemplary” or “for example” are used to present the relevant concept in a specific manner.

[0032] Hereinafter, the terms "first", "second", and the like are used only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Therefore, the features defined by the terms "first", "second", and the like can be explicitly or implicitly included one or more of the features.

[0033] In the description of the present disclosure, unless otherwise specified, " / " means "or", for example, A / B can mean A or B. "And / or" herein is only used to describe the association relationship of the associated objects, which means that there can be three relationships. For example, A and / or B can mean: only A, only B, and A and B. In addition, "at least one" means one or more, and "multiple" means two or more.

[0034] Communication and perception integration has become an important evolution direction of the next generation 5G technology (5G Advanced, 5G-A) and the sixth generation mobile communication technology (6th Generation Mobile Communication Technology, 6G), and is one of the six application directions of the 6G vision published by IMT-2030 (International Mobile Telecommunications for 2030 and beyond) (International Mobile Telecommunications for 2030 and beyond), and has broad application prospects.

[0035] Communication and perception integration can realize wireless perception function and wireless communication function by sharing spectrum and software and hardware resources. Communication and perception integration can meet the perception performance requirements in various scenarios by means of mobile networks, and can also improve communication performance by using perception function. In the current communication and perception industry research, focus is on using communication modules to realize perception capability, such as base station self-generation and self-reception, terminal generation and base station reception, and other modes, which all rely on original communication modules, bandwidth and other resources to realize perception. In the perception capability registration, perception data reporting, perception data processing, and perception networking collaboration processes, only the perception capabilities realized by reusing communication devices, bandwidth and other resources are considered. In addition to using 3GPP wireless spectrum resources to realize perception, a large number of perception devices have been applied in the industry, such as vehicle networking roadside stations, video monitoring cameras, and Internet of Things perception units. By incorporating these multi-modal perception devices and perception information processing into the 3GPP perception architecture, the communication and perception system can obtain a large amount of perception information by means of the large number of perception devices already deployed, and can also fuse multi-modal information to meet various perception requirements and improve perception quality. In this way, not only the overall coverage performance of communication and perception can be improved, but also the form and information mode of perception can be expanded.

[0036] The current way of fusing non-3GPP awareness information with the awareness architecture includes two categories. The first category: the awareness information generated by the awareness device is transmitted to the Internet public network based on the communication link, and then the processing and application of the awareness data are performed, which involves the gateway interaction of the 3GPP core network to the Internet public network; if it is necessary to fuse with the 3GPP base station awareness information, the processing complexity and latency will be high. The second category: the awareness information generated by the non-3GPP awareness device is transmitted to the core network through the communication link, and the core network processes the non-3GPP awareness data; this puts high requirements on the awareness processing capability of the core network, and also increases the network load of the core network, which occupies the communication traffic resources; at the same time, due to the involvement of multiple network elements, the standardization process is slow.

[0037] It can be seen that the current processing complexity and latency of non-3GPP awareness information are high, and if the core network is used for processing, the network load of the core network will be increased, and the communication traffic resources will be occupied.

[0038] In view of the above technical problems, the disclosure embodiment proposes a processing method of awareness data, the idea of which is that the base station receives the non-3GPP awareness data of the awareness device and processes the non-3GPP awareness data. It can be seen that the disclosure embodiment can utilize the awareness processing capability of the base station side to realize the processing of non-3GPP data on the base station side, directly complete the non-3GPP awareness process in the access network domain, and realize the rapid deployment of the non-3GPP awareness access awareness system. In this way, the processing complexity and latency can be reduced, and the awareness data flow from the base station to the core network is greatly reduced.

[0039] The awareness system provided by the disclosure embodiment will be described in detail below with reference to the accompanying drawings.

[0040] The awareness system provided by the disclosure embodiment includes a base station, a terminal device, an awareness device, and a core network device.

[0041] In some embodiments, the base station described above can be any one of an evolution nodeB (eNB), a next generation nodeB (gNB), a transmission and reception point (TRP), a transmission point (TP), a relay node, a Reconfigurable Intelligent Surface (RIS), and some other access node. According to the size of the service coverage area provided, the base station can be further classified into a macro base station for providing a macro cell, a micro base station for providing a micro cell, and a femto base station for providing a femto cell, or a distributed unit access node, etc. As wireless communication technologies continue to evolve, future base stations can also be referred to by other names.

[0042] In some embodiments, the base station can include a communication module and a perception module. The communication module is an entity in the base station for completing a communication processing process. The perception module is an entity in the base station for completing a perception processing process.

[0043] In some embodiments, the perception module described above includes a 3GPP perception module and a non-3GPP perception module.

[0044] The 3GPP perception module has at least one of the following capabilities: a dedicated scheduling capability related to sensing, a dedicated processing capability related to sensing, and a capability of using a sensing hardware of a dedicated base station. The 3GPP perception module does not have a capability of processing non-3GPP perception information such as video information and lidar point cloud information.

[0045] The non-3GPP perception module has at least one of the following capabilities: a non-3GPP sensor scheduling capability, a general processing capability, a perception control capability, and a perception processing capability. Exemplarily, the non-3GPP perception module can use a server carrying a graphics processing unit (GPU) to process various types of perception data.

[0046] For ease of understanding, the capabilities of the non-3GPP perception module are exemplarily illustrated below.

[0047] Exemplarily, the non-3GPP perception module includes a perception control capability and a perception processing capability.

[0048] 1. The perception control capability includes at least one of the following: (1) the capability of obtaining the perception service requirement from the core network; (2) the capability of obtaining and reporting the deployment information and the perception information of the perception device of the base station; and (3) the capability of managing the perception resource configuration. The capability of obtaining and reporting the deployment information and the perception information of the perception device of the base station involves the following ① to ④.

[0049] ①. The type of the perception device includes but is not limited to: a camera, a millimeter wave radar, a laser radar, a sonar, an ultra-wideband (UWB), a wireless fidelity (WIFI), and a global positioning system (GPS) device.

[0050] ②. The deployment information of the perception device includes but is not limited to: the location of the perception device, the movement capability of the perception device, and the identity (ID) of the perception device.

[0051] ③. The capability information of the perception device. The capability information of the perception device is determined according to the type of the perception device, for example: in the case where the type of the perception device includes a millimeter wave radar, the capability information of the perception device includes at least one of the following: the transmission power range of the millimeter wave radar, the beam configurable type, the working frequency band adjustment range, the perception resolution adjustment range, the perception angle adjustment range, the perception distance adjustment range, and the snapshot reporting frame rate adjustment range; in the case where the type of the perception device includes a laser radar, the capability information of the perception device includes at least one of the following: the scanning mode of the laser radar, the laser beam angle adjustment range, the angle resolution adjustment range, the working frequency adjustment range, and the snapshot reporting frame rate adjustment range; in the case where the type of the perception device includes an ultra-wideband device, the capability information of the perception device includes at least one of the following: the perception frequency band adjustment range of the ultra-wideband device, the pulse shape, the transmission power adjustment range, the pulse repetition frequency adjustment range, and the snapshot reporting frame rate adjustment range; in the case where the type of the perception device includes a wireless connection device, the capability information of the perception device includes at least one of the following: the perception frequency band adjustment range of the wireless connection device, the perception signal strength, the waveform adjustment range, the perception resolution adjustment range, and the snapshot reporting frame rate adjustment range;

[0052] in the case where the type of the perception device includes a GPS device, the capability information of the perception device includes at least one of the following: the positioning accuracy adjustment range, the satellite signal strength, the navigation path planning capability, the real-time speed measurement range, and the time synchronization accuracy adjustment range.

[0053] ④. The type of the perception service. The type of the perception service includes but is not limited to: target tracking perception, area monitoring perception, environment reconstruction perception, micro-deformation perception, and weather perception.

[0054] The capability of managing the sensing resource configuration involves the following ① to ⑥.

[0055] ①, indicating the sensing service type. The sensing service type includes but is not limited to: target tracking sensing, area monitoring sensing, environment reconstruction sensing, micro deformation sensing, meteorological sensing.

[0056] ②, indicating the sensing resource configuration of each sensing device. The sensing resource configuration includes but is not limited to at least one of the following: time domain resource, frequency domain resource, code domain resource, space domain resource, waveform resource.

[0057] ③, indicating the sensing mode of each sensing device. The sensing mode of the sensing device includes but is not limited to: sensor sensing. The sensor sensing includes at least one of the following: camera, millimeter wave radar, laser radar, sonar, UWB device, WiFi device, GPS device.

[0058] ④, indicating the identification information of each sensing device. The identification information of the sensing device includes but is not limited to at least one of the following: AAU (Active Antenna Unit) -ID, BBU (Base Band Unit) -ID, sensor ID.

[0059] ⑤, indicating the configuration parameters of each sensing device.

[0060] The configuration parameters of the camera include but are not limited to at least one of the following: camera resolution, camera reporting frame rate, steering angle, magnification, focal length, aperture, correction parameter.

[0061] The configuration parameters of the millimeter wave radar include but are not limited to at least one of the following: transmission power, beam configuration, working frequency band, sensing resolution, sensing detection angle range, sensing distance range, snapshot reporting frame rate.

[0062] The configuration parameters of the laser radar include but are not limited to at least one of the following: scanning mode, laser beam angle and resolution, working frequency, snapshot reporting frame rate.

[0063] The configuration parameters of the WiFi device sensing include but are not limited to at least one of the following: sensing frequency band, sensing signal strength, waveform, sensing resolution, snapshot reporting frame rate.

[0064] The configuration parameters of the UWB device sensing include but are not limited to at least one of the following: sensing frequency band, pulse shape, transmission power, pulse repetition frequency, snapshot reporting frame rate.

[0065] ⑥, the sensing index requirement for each sensing device.

[0066] The perception index requirements of the camera include, but are not limited to, at least one of the following: farthest target detection distance, nearest target detection distance, target detection accuracy, target detection resolution, target recognition type.

[0067] The perception index requirements of the millimeter wave radar / WiFi perception / UWB perception include, but are not limited to, at least one of the following: farthest perception distance, speed measurement range, angle measurement range, angle resolution, distance resolution, speed resolution.

[0068] The perception index requirements of the laser radar include, but are not limited to, at least one of the following: farthest perception distance, angle measurement range, angle resolution, distance resolution, data density.

[0069] 2. The perception processing capability includes at least one of the following: (1) the capability of processing QoS (Quality of Service) flow into perception raw data; (2) perception fusion processing capability, for example, the perception fusion processing capability includes at least one of the following: data layer data fusion, feature layer data fusion, decision layer data fusion; (3) feature extraction capability, for example, the capability of extracting features through traditional radar detection, machine learning or deep learning technology; (4) target recognition capability, for example, the capability of classifying, positioning, tracking and the like of targets to realize recognition and analysis of the targets; (5) the capability of standardizing perception data, for example, the capability of processing perception results of different modalities into perception results in a unified format; (6) the capability of calculating perception feedback information, for example, the perception feedback information includes, but is not limited to: terminal movement control information, perception parameter adjustment information of the sensor.

[0070] It can be understood that the perception data processing capability of the perception device (for example, the sensor) is a newly added capability of the disclosure on the perception module. The original perception module only has the capability of processing common perception data, and does not have the processing capability of other sensor perception data.

[0071] In some embodiments, the 3GPP perception module and the non-3GPP perception module can be two independent modules; or the 3GPP perception module and the non-3GPP perception module can be implemented by hardware fusion, and the perception processing process is implemented on a processing board based on general processing capability; or the 3GPP perception module and the non-3GPP perception module can be fused on an external interface (without hardware fusion), and the processing capability is then split into different hardware for processing inside the module.

[0072] In some embodiments, the terminal device described above can be a terminal device with communication function but without sensing capability. Illustratively, the terminal device described above can be a device with wireless transceiving function, such as a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), an Internet of Things terminal, and the like. The embodiments of the present disclosure do not limit the type of terminal.

[0073] In some embodiments, the sensing device described above is a device without communication capability but with sensing capability, which can obtain sensing data of a target object or an environment, such as a sensor. Illustratively, the sensing device in the embodiments of the present disclosure includes, but is not limited to, at least one of the following: a camera, a millimeter wave radar, a laser radar, a sonar, a UWB device, a WIFI device, and a GPS device. The sensing device can also be a handheld mobile device with sensing function (e.g., a camera or a UWB, etc.).

[0074] It should be noted that the sensing device in the embodiments of the present disclosure mainly refers to a non-3GPP sensing device. At present, the non-3GPP sensing device mainly includes two types: a standalone device independent of the 5G system and a device dependent on the 5G system deployment. The standalone device independent of the 5G system, such as a standalone camera or a sensor, does not have the capability of accessing the 5G network, and the 5G network cannot identify and manage such devices. The device dependent on the 5G system deployment, such as a camera on a mobile phone or a sensor deployed on a base station, does not have the capability of accessing the 5G network, but the operator can manage the relevant non-3GPP sensing device through the management of the UE (user equipment) or the base station. The non-3GPP sensing device in the embodiments of the present disclosure mainly refers to the device dependent on the 5G system deployment.

[0075] In some embodiments, the sensing device is used to generate sensing data. The sensing data includes 3GPP sensing measurement data and non-3GPP sensing measurement data. The 3GPP sensing measurement data is sensing measurement data generated by the 3GPP radio signal affected by the target object or the environment (e.g., reflection, refraction, diffraction). The non-3GPP sensing measurement data is sensing measurement data about the target object or the environment provided by the non-3GPP sensing device (e.g., a radar, a camera, or a sensor device). It should be noted that the non-3GPP sensing data mentioned in the embodiments of the present disclosure mainly refers to the non-3GPP sensing measurement data.

[0076] In some embodiments, the core network device is located in a core network domain (i.e., a core network defined by 3GPP protocols); the base station, the terminal device, and the sensing device are located in an access network domain (i.e., an access network defined by 3GPP protocols).

[0077] Referring to FIG. 1, an architecture schematic diagram of a sensing system according to an embodiment of the present disclosure is shown. As shown in FIG. 1, the sensing system includes a base station 110, a terminal device 120, a sensing device 130, and a core network device 140.

[0078] The sensing device 130 is connected to the base station 110 through the terminal device 120, and the base station 110 is connected to the core network device 140.

[0079] For example, the sensing device 130 is connected to the terminal device 120 through a non-3GPP communication air interface link (e.g., WiFi, Bluetooth, direct connection, or a sensor integrated in the terminal device 120); and the terminal device 120 is connected to the base station 110 through a 3GPP communication air interface link.

[0080] In some embodiments, the base station 110 includes a communication module and a sensing module, the terminal device 120 is connected to the communication module of the base station 110 through a 3GPP communication air interface link, and the communication module of the base station 110 is connected to the sensing module through an NS3 interface.

[0081] The NS3 interface has the following functions: 1. The sensing module forwards non-3GPP sensing configuration information, a non-3GPP sensing registration response message, and a captured confirmation identifier to the communication module through the NS3 interface; 2. The communication module forwards non-3GPP sensing data, a non-3GPP sensing registration request, and a non-3GPP sensing data reporting request to the sensing module through the NS3 interface.

[0082] It can be understood that, since the sensing module needs to obtain non-3GPP sensing data through the communication module, the NS3 interface is newly added between the communication module and the sensing module. The NS3 interface is a special interface cooperating with non-3GPP sensing signals, and the signaling transmitted through the NS3 interface is special signaling cooperating with non-3GPP sensing signals. The special signaling cooperating with non-3GPP sensing signals includes non-3GPP sensing configuration information and non-3GPP sensing data.

[0083] It should be noted that the embodiment shown in FIG. 1 is only an example, and in actual use, the base station 110 can be connected to one or more terminal devices 120, and the terminal device 120 can be connected to one or more sensing devices 130.

[0084] It can be seen that in the perception system provided by the embodiment of the present disclosure, the perception device can access the perception system through the terminal-communication air interface link-base station communication module-base station non-3GPP perception module, and realize the control and processing of non-3GPP perception data at the base station side. The perception device can register the perception capability in the non-3GPP perception module of the base station through the terminal device. The non-3GPP perception module of the base station can identify and obtain non-3GPP perception data, and perform processes such as analysis, perception data processing, and data standardization. In this way, all perception processing processes are concentrated in the perception module at the base station end, realizing the base station end edge computing of non-3GPP perception data, which can reduce the processing complexity and delay. At the same time, without modifying the perception related interface from the base station end to the core network, the perception data traffic from the base station to the core network is also greatly reduced.

[0085] Referring to FIG. 2, another schematic diagram of the perception system according to an embodiment of the present disclosure is shown. As shown in FIG. 2, the perception system includes a base station 110, a perception device 130, and a core network device 140.

[0086] The perception device 130 is directly connected with the base station 110, and the base station 110 is connected with the core network device 140.

[0087] Exemplarily, the perception device 130 and the base station 110 are connected through a non-3GPP communication air interface link (for example, WiFi, Bluetooth, direct connection, or a sensor integrated on the base station 110).

[0088] In some embodiments, the base station 110 includes a communication module and a perception module, the perception device 130 is connected to the perception module of the base station 110 through a non-3GPP communication air interface link, and the communication module and the perception module of the base station 110 are connected through an NS3 interface.

[0089] The role of the NS3 interface can be referred to the description in the embodiment shown in FIG. 1, which will not be repeated here.

[0090] It should be noted that the embodiment shown in FIG. 2 is only an example, and in actual use, the base station 110 can be connected with one or more perception devices 130.

[0091] It can be seen that in the perception system provided by the embodiment of the present disclosure, the perception device can access the perception system through the communication air interface link-base station non-3GPP perception module, and realize the control and processing of non-3GPP perception data at the base station side. In this way, all perception processing processes are concentrated in the perception module at the base station end, realizing the base station end edge computing of non-3GPP perception data, which can reduce the processing complexity and delay. At the same time, without modifying the perception related interface from the base station end to the core network, the perception data traffic from the base station to the core network is also greatly reduced.

[0092] It should be noted that in addition to the architecture provided by the embodiments of the present disclosure, the perception data of the perception device can also be reported and processed through the link of the perception device-terminal-base station-3GPP core network-Internet public network-computing center; or the perception data of the perception device can also be reported and processed through the link of the sensor-terminal-core network. The embodiments of the present disclosure do not limit this.

[0093] It can be understood that the application scenarios of the embodiments of the present disclosure are not limited. The system architecture and business scenarios described in the embodiments of the present disclosure are used to more clearly illustrate the technical solutions of the embodiments of the present disclosure, and do not constitute a limitation on the technical solutions provided by the embodiments of the present disclosure. It can be known by those skilled in the art that with the evolution of network architecture and the appearance of new business scenarios, the technical solutions provided by the embodiments of the present disclosure are also applicable to similar technical problems.

[0094] The processing method of the perception data provided by the embodiments of the present disclosure will be introduced below.

[0095] The processing method of the perception data provided by the embodiments of the present disclosure can be applied to a communication-perception integrated scenario, for example, a smart traffic, a smart home, a smart engineering, and the like, which simultaneously has a perception device and a communication coverage.

[0096] The present disclosure provides a processing method of perception data, applied to a base station in a communication-perception system as shown in FIG. 1 or FIG. 2. As shown in FIG. 3, the processing method of the perception data includes the following S201 and S202.

[0097] In S201, the base station receives non-3GPP perception data of a perception device.

[0098] In some embodiments, the above-mentioned non-3GPP perception data is the perception measurement data about target information (for example, a target object, a target subject, or an environment) generated and reported by a non-3GPP perception device (for example, a sensor device such as a millimeter wave radar device or a camera). Exemplarily, the non-3GPP perception data includes but is not limited to: camera video recording, WIFI perception receiving data, millimeter wave radar time delay Doppler diagram data, laser radar point cloud data, and UWB perception receiving data.

[0099] As an implementation manner, the above-mentioned S201 can be implemented as: receiving, by a terminal device, the non-3GPP perception data sent by the perception device. For example, in the communication-perception system shown in FIG. 1, the perception device is connected with the base station through the terminal device, and therefore, the base station can receive the non-3GPP perception data sent by the perception device through the terminal device.

[0100] Exemplarily, the terminal device receives the non-3GPP sensing data sent by the sensing device through the non-3GPP communication air interface link, and then sends the non-3GPP sensing data to the base station through the 3GPP communication air interface link.

[0101] As another implementation manner, the S201 can be implemented as: directly receiving the non-3GPP sensing data sent by the sensing device. For example, in the sensing system shown in FIG. 2, the sensing device is directly connected with the base station, and thus the base station can directly receive the non-3GPP sensing data sent by the sensing device.

[0102] Exemplarily, the base station receives the non-3GPP sensing data sent by the sensing device through the non-3GPP communication air interface link.

[0103] In some embodiments, the base station comprises a communication module and a sensing module. The sensing module is divided into a 3GPP sensing module and a non-3GPP sensing module. The communication module receives the non-3GPP sensing data, and then sends it to the non-3GPP sensing module.

[0104] In some embodiments, in the case that the base station configures the terminal device with a transmission resource used for reporting the sensing data, the S201 can be implemented as: receiving the non-3GPP sensing data on the transmission resource used for reporting the sensing data.

[0105] Exemplarily, the transmission resource used for reporting the sensing data can include time domain resource, frequency domain resource, code domain resource, space domain resource, etc.

[0106] In some embodiments, the S201 can be implemented as: obtaining the non-3GPP sensing data based on the received non-3GPP sensing data reporting identifier.

[0107] The sensing data reporting identifier is used to indicate that the terminal device has reported the non-3GPP sensing data. Exemplarily, the terminal device simultaneously sends the sensing data reporting identifier and the non-3GPP sensing data.

[0108] In some embodiments, the base station comprises a communication module and a sensing module. The sensing module is divided into a 3GPP sensing module and a non-3GPP sensing module. The communication module is used to receive the non-3GPP sensing data sent by the terminal device; and the non-3GPP sensing module obtains the non-3GPP sensing data from the communication module in the case that the sensing data reporting identifier is received.

[0109] In some embodiments, the non-3GPP awareness data received by the base station is data processed by the terminal device, for example, the terminal device can encode, modulate, etc. process the non-3GPP awareness data, convert the non-3GPP awareness data into a communication signal, and then send it to the base station. Correspondingly, after receiving the communication signal sent by the terminal device, the communication module of the base station analyzes the communication signal into non-3GPP awareness data through signal detection, demodulation, decoding, etc. processing, and forwards it to the non-3GPP awareness module.

[0110] In S202, the base station processes the non-3GPP awareness data.

[0111] In some embodiments, the above S202 can be implemented by sequentially performing preprocessing, awareness data processing, and data standardization processing on the non-3GPP awareness data.

[0112] Preprocessing refers to parsing the received raw data into non-3GPP awareness data. It can be understood that the raw data packet received by the base station is a QoS flow, so it needs to be parsed and converted into awareness data. The preprocessing process is originally borne by the UPF (User Plane Function) network element of the core network, but in this disclosure, it is a new function added by the non-3GPP awareness module of the base station.

[0113] Awareness data processing refers to performing awareness processing on different types of awareness raw data to convert them into awareness result data.

[0114] Data standardization processing refers to mapping the awareness result data to convert it into awareness result data in a unified format.

[0115] In some embodiments, the above awareness data processing includes awareness data processing and fusion of the sensing device, for example, it can include data level fusion, feature level fusion, and decision level fusion.

[0116] Data level fusion includes the following processing procedures a1 to a3.

[0117] In a1, the original non-3GPP awareness data and other source awareness data (sensing or data sensed by other non-3GPP sensing devices accessed at the same time) are time-aligned, spliced / weighted averaged, etc. to obtain data level fusion awareness data.

[0118] In a2, the feature extraction is performed on the data level fusion awareness data using traditional radar detection, machine learning or deep learning, etc. to extract basic features including target information, such as time delay-Doppler graph, target segmentation graph, etc.

[0119] In a3, target recognition is performed on the result of feature extraction, and classification, recognition, tracking, etc. are performed on the target recognition result to obtain a target perception result.

[0120] Feature-level fusion includes the following processing procedures b1 to b3.

[0121] In b1, basic features including target information are extracted from original non-3GPP perception data.

[0122] In b2, the extracted basic features including target information are fused by time alignment, splicing, weighted average, feature transformation (for example, Principal Component Analysis (PCA)), etc. to obtain a result of feature fusion.

[0123] In b3, target recognition is performed on the result of feature fusion, and classification, recognition, tracking, etc. are performed on the target recognition result to obtain a target perception result.

[0124] Decision-level fusion includes the following processing procedures c1 to c3.

[0125] In c1, basic features including target information are extracted from original non-3GPP perception data.

[0126] In c2, target recognition is performed on the data after feature extraction, and classification, recognition, tracking, etc. are performed on the target recognition result to obtain a target perception result.

[0127] In c3, non-3GPP perception results are fused with other perception results by voting, decision tree, integrated model, etc. to obtain a final perception result; or, the non-3GPP perception results can also be directly taken as the final perception result without fusion.

[0128] In some embodiments, the base station can report the non-3GPP perception data to the core network for further processing. Exemplarily, the base station can report the non-3GPP perception data in a unified format to the core network.

[0129] It can be understood that in the perception data processing method proposed in the embodiments of the present disclosure, the base station can receive non-3GPP perception data of a perception device and process the non-3GPP perception data. As can be seen, the embodiments of the present disclosure can realize the processing of non-3GPP data at the base station side by means of the perception processing capability of the base station side, directly complete the non-3GPP perception process in the access network domain, and realize the rapid deployment of the non-3GPP perception access sensing system. In this way, the processing complexity and latency can be reduced, and the perception data traffic from the base station to the core network is greatly reduced.

[0130] In some embodiments, as shown in FIG. 4, before S201 described above, the method for processing the perception data further comprises the following S200.

[0131] In S200, the base station sends configuration information of the perception device to the perception device.

[0132] The configuration information of the perception device comprises at least one of the following: a perception mode, and a parameter used by the perception device.

[0133] Exemplarily, the perception mode of the perception device comprises but is not limited to sensor perception. The sensor perception comprises at least one of the following: a camera, a millimeter wave radar, a laser radar, a sonar, a UWB device, a WiFi device, and a GPS device.

[0134] Exemplarily, the parameter used by the perception device (which can also be referred to as the configuration parameter of the perception device) is determined according to the type of the perception device.

[0135] For example, the parameter used by the camera comprises but is not limited to at least one of the following: a camera resolution, a camera frame rate, a steering angle, a magnification, a focal length, an aperture, and a correction parameter. The configuration parameter used by the millimeter wave radar comprises but is not limited to at least one of the following: a transmission power, a beam configuration, a working frequency band, a perception resolution, a perception detection angle range, a perception distance range, and a snapshot reporting frame rate.

[0136] The parameter used by the laser radar comprises but is not limited to at least one of the following: a scanning mode, a laser beam angle and resolution, a working frequency, and a snapshot reporting frame rate.

[0137] The parameter used by the WiFi device comprises but is not limited to at least one of the following: a perception frequency band, a perception signal strength, a waveform, a perception resolution, and a snapshot reporting frame rate.

[0138] The parameter used by the UWB device comprises but is not limited to at least one of the following: a perception frequency band, a pulse shape, a transmission power, a pulse repetition frequency, and a snapshot reporting frame rate.

[0139] As an implementation manner, S200 described above can be implemented as: sending, by a terminal device, the configuration information of the perception device to the perception device. For example, in the perception system shown in FIG. 1, the perception device is connected to the base station through the terminal device, and therefore the base station can send the configuration information of the perception device to the perception device through the terminal device.

[0140] Exemplarily, the sending, by the terminal device, of the configuration information of the perception device to the perception device comprises: sending, to the terminal device, non-3GPP perception configuration information, so as to enable the terminal device to send the configuration information of the perception device to the perception device based on the non-3GPP perception configuration information. The configuration information of the perception device is a subset of the non-3GPP perception configuration information.

[0141] The non-3GPP awareness configuration information includes at least one of the following: an awareness mode, identification information of the awareness device, parameters used by the awareness device, and transmission resources used for reporting of awareness data.

[0142] For example, the identification information of the awareness device can be an ID of the awareness device.

[0143] For example, the transmission resources used for reporting of the awareness data can include time domain resources, frequency domain resources, code domain resources, and space domain resources.

[0144] As another implementation manner, the S200 can be implemented as: directly sending the configuration information of the awareness device to the awareness device. For example, in the sensing system shown in FIG. 2, the awareness device is directly connected to the base station, and thus the base station can directly send the configuration information of the awareness device to the awareness device.

[0145] It can be understood that the base station sends the configuration information of the awareness device to the awareness device, so that the awareness device can perform awareness configuration according to the configuration information of the awareness device to generate and report the non-3GPP awareness data.

[0146] For example, in the embodiments of the present disclosure, there are two triggering manners for reporting of the non-3GPP awareness data. One triggering manner is that the base station (or the non-3GPP awareness module of the base station) actively triggers, for example, the base station actively sends the configuration information of the awareness device to the awareness device to trigger the awareness device to generate and report the non-3GPP awareness data. The other triggering manner is that the terminal device actively triggers, for example, the terminal device actively sends a reporting request of the awareness data to the base station to trigger the base station to send the configuration information of the awareness device to the awareness device through the terminal device, so as to trigger the awareness device to generate and report the non-3GPP awareness data. For example, refer to the following embodiments.

[0147] In some embodiments, the S200 can be implemented as: sending the configuration information of the awareness device to the awareness device in a case where a first triggering condition is met.

[0148] The first triggering condition includes at least one of the following: a requirement of the base station for triggering sensing of a static environment; a requirement of the base station for triggering sensing of a dynamic target; and a requirement of the base station for triggering monitoring of a drone device.

[0149] For example, in a case where the base station has a requirement for triggering sensing of a static environment, the base station can call an awareness device (for example, a sensor) carried by the terminal. For example, the base station needs to sense a static environment around the base station, and thus the base station calls a network camera to collect video data, which is used together with the sensing data for environment sensing around the base station.

[0150] Exemplarily, in the case that the base station triggers the demand of perceiving the dynamic target, the base station can invoke the perception device (for example, the sensor) carried by the terminal. Exemplarily, when the base station perceives the vehicle, the base station invokes the roadside camera or the millimeter wave radar to obtain the perception data about the vehicle and the pedestrian, and perceives the vehicle and the pedestrian in combination with the common perception data.

[0151] Exemplarily, in the case that the base station triggers the demand of monitoring the unmanned aerial vehicle device, the base station can invoke the perception device (for example, the sensor) carried by the networking unmanned aerial vehicle itself. Exemplarily, when the base station perceives the unmanned aerial vehicle, the GPS positioning data of the unmanned aerial vehicle itself is invoked to obtain the accurate unmanned aerial vehicle position information, which is used for the calibration of the perception parameter or the accurate unmanned aerial vehicle position tracking.

[0152] In some embodiments, the S200 described above can be implemented as: in the case that the first request message is received, the configuration information of the perception device is sent to the perception device. The first request message is used to request the reporting of the non-3GPP perception data.

[0153] In some embodiments, the first request message includes at least one of the following: the perception mode, the identification information of the perception device, and the parameter used by the perception device.

[0154] In some embodiments, the first request message is sent by the terminal device in the case that the second trigger condition is met.

[0155] The second trigger condition includes at least one of the following: the demand of the terminal device triggering the perception of the surrounding environment; and the demand of the terminal device triggering the reporting of the perception information.

[0156] In some embodiments, after obtaining the non-3GPP perception data, the base station sends the first response message to the terminal device. The first response message includes the non-3GPP perception data.

[0157] Exemplarily, if the terminal device triggers the demand of perceiving the surrounding environment, the self-perception device (for example, the sensor) needs to be invoked to perform the environmental perception in combination with the base station computing power. Exemplarily, in the case that the terminal device needs to obtain the static environmental data for the demand of the smart home, the non-3GPP perception process is triggered. After obtaining the final non-3GPP perception data, the base station can feed back the non-3GPP perception data to the terminal device.

[0158] Exemplarily, if the terminal device triggers the demand of reporting the perception data, the non-3GPP perception data is directly reported. Exemplarily, the networking camera can be set to periodically send the non-3GPP perception data to the base station for the base station to perceive, and therefore, the trigger process of the terminal device needs to be periodically performed.

[0159] In some embodiments, as shown in FIG. 5, before S201, the method further comprises S301 and S302.

[0160] In S301, the base station receives a registration request message. The registration request message is used to request registration of the information of the sensing device in the base station.

[0161] As an implementation manner, S301 can be implemented as: receiving, by the terminal device, the registration request message sent by the sensing device. For example, in the sensing system shown in FIG. 1, the sensing device is connected with the base station through the terminal device, so the base station can receive the registration request message sent by the sensing device through the terminal device.

[0162] For example, the terminal device receives the registration request message sent by the sensing device through a non-3GPP communication air interface link, and then sends the registration request message to the base station through a 3GPP communication air interface link.

[0163] As another implementation manner, S301 can be implemented as: directly receiving the registration request message sent by the sensing device. For example, in the sensing system shown in FIG. 2, the sensing device is directly connected with the base station, so the base station can directly receive the registration request message sent by the sensing device.

[0164] For example, the base station receives the registration request message sent by the sensing device through a non-3GPP communication air interface link.

[0165] In some embodiments, the base station comprises a communication module and a sensing module, and the sensing module is divided into a 3GPP sensing module and a non-3GPP sensing module. After receiving the registration request message, the communication module judges that the registration request message is from a non-3GPP sensing device through the non-3GPP sensing module, and then the communication module forwards the registration request message to the non-3GPP sensing module of the base station.

[0166] In some embodiments, the registration request message comprises at least one of the following: a registration type, a type of the sensing device, and capability information of the sensing device.

[0167] The registration type can be non-3GPP sensing.

[0168] In some embodiments, the type of the sensing device comprises at least one of the following: a camera, a millimeter wave radar, a laser radar, a UWB device, a WIFI device, and a GPS device.

[0169] In some embodiments, the capability information of the perception device is determined according to the type of the perception device. For example, the capability of the perception device has the following cases: in the case that the type of the perception device includes a camera, the capability information of the perception device includes at least one of the following: resolution adjustment range, reporting frame rate adjustment range, steering angle, focal length, aperture, magnification, correction coefficient of the camera; in the case that the type of the perception device includes a millimeter wave radar, the capability information of the perception device includes at least one of the following: transmission power range, beam configurable type, working frequency band adjustment range, perception resolution adjustment range, perception angle adjustment range, perception distance adjustment range, snapshot reporting frame rate adjustment range of the millimeter wave radar; in the case that the type of the perception device includes a laser radar, the capability information of the perception device includes at least one of the following: scanning mode, laser beam angle adjustment range, angle resolution adjustment range, working frequency adjustment range, snapshot reporting frame rate adjustment range of the laser radar; in the case that the type of the perception device includes an ultra-wideband device, the capability information of the perception device includes at least one of the following: perception frequency band adjustment range, pulse shape, transmission power adjustment range, pulse repetition frequency adjustment range, snapshot reporting frame rate adjustment range of the ultra-wideband device; in the case that the type of the perception device includes a wireless connection device, the capability information of the perception device includes at least one of the following: perception frequency band adjustment range, perception signal strength, waveform adjustment range, perception resolution adjustment range, snapshot reporting frame rate adjustment range of the wireless connection device; in the case that the type of the perception device includes a GPS device, the capability information of the GPS device includes at least one of the following: positioning accuracy adjustment range, satellite signal strength, navigation path planning capability, real-time speed measurement range, time synchronization accuracy adjustment range.

[0170] In S302, the base station sends a registration response message, which is used to indicate that the perception device has been successfully registered.

[0171] As an implementation manner, S302 can be implemented by sending the registration response message to the perception device through the terminal device. For example, in the perception system shown in FIG. 1, the perception device is connected to the base station through the terminal device, and therefore, the base station can send the registration response message to the perception device through the terminal device.

[0172] For example, the base station sends the registration response message to the terminal device through a 3GPP communication air interface link, and then the terminal device sends the registration response message to the perception device through a non-3GPP communication air interface link.

[0173] As another implementation manner, S302 can be implemented by directly sending the registration response message to the perception device. For example, in the perception system shown in FIG. 2, the perception device is directly connected to the base station, and therefore, the base station can directly send the registration response message to the perception device.

[0174] Exemplarily, the base station sends the registration response message to the sensing device through the non-3GPP communication air interface.

[0175] In some embodiments, the base station performs the registration of the sensing device based on the received registration request message, and allocates the sensing device identification information to the sensing device after confirming that the sensing device has been successfully registered. The registration response message can include the sensing device identification and the information that the registration of the sensing device is successful.

[0176] It should be noted that the registration procedures provided by S301 and S302 can be used for any one of the following: initial registration, mobility update registration, and periodic registration.

[0177] The present disclosure provides another sensing data processing method, which is applied to a terminal device in a sensing system as shown in FIG. 1 or FIG. 2. As shown in FIG. 6, the sensing data processing method includes the following S401 and S402.

[0178] In S401, the terminal device receives the non-3GPP sensing data sent by the sensing device.

[0179] Exemplarily, the terminal device receives the non-3GPP sensing data sent by the sensing device through the non-3GPP communication air interface.

[0180] Exemplarily, the description of the non-3GPP sensing data can refer to the embodiment shown in FIG. 3, which will not be repeated here.

[0181] In S402, the terminal device sends the non-3GPP sensing data to the base station, so that the base station processes the non-3GPP sensing data.

[0182] Exemplarily, the terminal device sends the non-3GPP sensing data to the base station through the 3GPP communication air interface.

[0183] In some embodiments, the terminal device simultaneously sends the sensing data reporting identifier and the non-3GPP sensing data. The sensing data reporting identifier is used to indicate that the terminal device has reported the non-3GPP sensing data. Correspondingly, the communication module of the base station is configured to receive the non-3GPP sensing data sent by the terminal device, and the non-3GPP sensing module of the base station is configured to acquire the non-3GPP sensing data from the communication module in the case that the sensing data reporting identifier is received.

[0184] Exemplarily, the description of the processing of the non-3GPP sensing data by the base station can refer to the embodiment shown in FIG. 3, which will not be repeated here.

[0185] In some embodiments, before S401, the sensing data processing method further includes S501 and S502. In some embodiments, before S401, the sensing data processing method further includes S501 and S502.

[0186] In S501, the terminal device receives the non-3GPP awareness configuration information sent by the base station.

[0187] Exemplarily, the terminal device receives the non-3GPP awareness configuration information sent by the base station through a 3GPP communication link air interface.

[0188] Exemplarily, the description about the non-3GPP awareness configuration information can refer to the embodiment shown in FIG. 3, which will not be repeated here.

[0189] In S502, the terminal device sends the configuration information of the awareness device to the awareness device based on the non-3GPP awareness configuration information.

[0190] The configuration information of the awareness device is a subset of the non-3GPP awareness configuration information. The configuration information of the awareness device is used to configure the awareness device to generate non-3GPP awareness data.

[0191] Exemplarily, the terminal device sends the configuration information of the awareness device to the awareness device through a non-3GPP communication link air interface.

[0192] In some embodiments, before the above-mentioned S501, the processing method of the awareness data further includes S500.

[0193] In S500, the terminal device sends a first request message to the base station, and the first request message is used to request to report the non-3GPP awareness data.

[0194] Exemplarily, the description about the first request message can refer to the embodiment shown in FIG. 3, which will not be repeated here.

[0195] In some embodiments, S500 can be implemented as: in the case of meeting a second trigger condition, sending the first request message to the base station. The second trigger condition includes at least one of the following: the terminal device triggers the demand of sensing the surrounding environment; the terminal device triggers the demand of reporting the awareness information.

[0196] Exemplarily, the description about the second trigger condition can refer to the embodiment shown in FIG. 3, which will not be repeated here.

[0197] In some embodiments, before the above-mentioned S401, the processing method of the awareness data further includes S601 and S602.

[0198] In S601, the terminal device receives a registration request message sent by the awareness device.

[0199] The registration request message is used to request to register the information of the awareness device in the base station.

[0200] Exemplarily, the terminal device receives the registration request message sent by the sensing device through a non-3GPP communication link air interface.

[0201] In some embodiments, after the terminal device receives the registration request message sent by the sensing device, the processing method of the sensing data further includes: registering the information of the sensing device in the terminal device.

[0202] In S602, the terminal device sends a registration request message to the base station.

[0203] Exemplarily, the terminal device sends the registration request message to the base station through a 3GPP communication link air interface.

[0204] Exemplarily, for the description of the registration request message, please refer to the embodiment shown in FIG. 3, which will not be repeated here.

[0205] In some embodiments, after S602, the processing method of the sensing data further includes S603 and S604.

[0206] In S603, the terminal device receives a registration response message sent by the base station, and the registration response message is used to indicate that the sensing device has been successfully registered.

[0207] Exemplarily, the terminal device receives the registration response message sent by the base station through a 3GPP communication link air interface.

[0208] In S604, the terminal device sends a registration response message to the sensing device.

[0209] Exemplarily, the terminal device sends the registration response message to the sensing device through a non-3GPP communication link air interface.

[0210] Exemplarily, for the description of the registration response message, please refer to the embodiment shown in FIG. 3, which will not be repeated here.

[0211] The present disclosure provides still another processing method of sensing data, applied to a sensing device in a sensing system as shown in FIG. 1 or FIG. 2. As shown in FIG. 7, the processing method of the sensing data includes the following S701.

[0212] In S701, the sensing device sends non-3GPP sensing data generated by the sensing device to the base station, so that the base station processes the non-3GPP sensing data.

[0213] As an implementation manner, the above S701 can be implemented as: the sensing device sends the non-3GPP sensing data to the base station through the terminal device. For example, in the sensing system shown in FIG. 1, the sensing device is connected with the base station through the terminal device, so the sensing device can send the non-3GPP sensing data to the base station through the terminal device.

[0214] As another implementation, S701 above can be implemented as follows: the sensing device directly sends non-3GPP sensing data to the base station. For example, in the sensing system shown in Figure 2, the sensing device is directly connected to the base station; therefore, the sensing device can directly send non-3GPP sensing data to the base station.

[0215] For example, a description of non-3GPP sensing data can be found in the embodiment shown in Figure 3, which will not be repeated here.

[0216] In some embodiments, prior to S701, the method for processing the sensed data further includes S801 and S802.

[0217] In S801, the sensing device receives the configuration information of the sensing device sent by the base station.

[0218] In one implementation, S801 can be implemented as follows: the sensing device receives configuration information of the sensing device sent by the base station through the terminal device. For example, in the sensing system shown in Figure 1, the sensing device is connected to the base station through the terminal device; therefore, the sensing device can receive configuration information of the sensing device sent by the base station through the terminal device.

[0219] As another implementation, S801 described above can be implemented as follows: the sensing device directly receives the configuration information of the sensing device sent by the base station. For example, in the sensing system shown in Figure 2, the sensing device is directly connected to the base station; therefore, the sensing device can directly receive the configuration information of the sensing device sent by the base station.

[0220] For example, a description of the configuration information of the sensing device can be found in the embodiment shown in Figure 3, which will not be repeated here.

[0221] In S802, the sensing device performs sensing configuration based on the sensing device's configuration information and generates non-3GPP sensing data.

[0222] For example, the sensing device performs sensing configuration based on the sensing mode and parameters used by the sensing device included in the sensing device's configuration information, and generates non-3GPP sensing data.

[0223] In some embodiments, prior to S701, the method for processing the sensed data further includes S901 and S902.

[0224] In S901, the sensing device sends a registration request message to the base station, which requests that the sensing device's information be registered with the base station.

[0225] As an implementation form, S901 can be implemented as that the perception device sends a registration request message to the base station through the terminal device. For example, in the perception system shown in FIG. 1, the perception device is connected with the base station through the terminal device, and therefore, the perception device can send the registration request message to the base station through the terminal device.

[0226] As another implementation form, S901 can be implemented as that the perception device directly sends the registration request message to the base station. For example, in the perception system shown in FIG. 2, the perception device is directly connected with the base station, and therefore, the perception device can directly send the registration request message to the base station.

[0227] Exemplarily, the description about the registration request message can refer to the embodiment shown in FIG. 3, which will not be repeated here.

[0228] In S902, the perception device receives a registration response message sent by the base station, and the registration response message is used to indicate that the perception device has been registered successfully.

[0229] As an implementation form, S902 can be implemented as that the perception device receives the registration response message sent by the base station through the terminal device. For example, in the perception system shown in FIG. 1, the perception device is connected with the base station through the terminal device, and therefore, the perception device can receive the registration response message sent by the base station through the terminal device.

[0230] As another implementation form, S902 can be implemented as that the perception device directly receives the registration response message sent by the base station. For example, in the perception system shown in FIG. 2, the perception device is directly connected with the base station, and therefore, the perception device can directly receive the registration response message sent by the base station.

[0231] Exemplarily, the description about the registration response message can refer to the embodiment shown in FIG. 3, which will not be repeated here.

[0232] Next, based on the perception system shown in FIG. 1, the processing method of the perception data provided by the embodiment of the present disclosure is described in the form of interaction between the base station, the terminal device and the perception device.

[0233] Exemplarily, as shown in FIG. 8, in the case that the base station triggers the reporting of the non-3GPP perception data, the processing method of the perception data provided by the embodiment of the present disclosure includes the following: Sd1 to Sd7.

[0234] In Sd1, the base station satisfies a first trigger condition.

[0235] Exemplarily, the description about the first trigger condition can refer to the embodiment shown in FIG. 3, which will not be repeated here.

[0236] In Sd2, the base station sends the configuration information of the sensing device to the terminal device. Accordingly, the terminal device receives the configuration information of the sensing device sent by the base station.

[0237] For example, the description of the configuration information of the sensing device can refer to the embodiment shown in FIG. 3, which will not be repeated here.

[0238] In Sd3, the terminal device sends the configuration information of the sensing device to the sensing device. Accordingly, the sensing device receives the configuration information of the sensing device sent by the terminal device.

[0239] In Sd4, the sensing device performs sensing configuration based on the configuration information of the sensing device, and generates non-3GPP sensing data.

[0240] For example, the description of the non-3GPP sensing data can refer to the embodiment shown in FIG. 3, which will not be repeated here.

[0241] In Sd5, the sensing device sends the non-3GPP sensing data to the terminal device. Accordingly, the terminal device receives the non-3GPP sensing data sent by the sensing device.

[0242] In Sd6, the terminal device sends the non-3GPP sensing data to the base station. Accordingly, the base station receives the non-3GPP sensing data sent by the terminal device.

[0243] In Sd7, the base station processes the non-3GPP sensing data.

[0244] For example, the description of the processing of the non-3GPP sensing data by the base station can refer to the embodiment shown in FIG. 3, which will not be repeated here.

[0245] For example, as shown in FIG. 9, in the case that the terminal device triggers the reporting of the non-3GPP sensing data, the method for processing sensing data provided by the embodiments of the present disclosure includes the following: Se1 to Se8.

[0246] In Se1, the terminal device satisfies a second trigger condition.

[0247] For example, the description of the second trigger condition can refer to the embodiment shown in FIG. 3, which will not be repeated here.

[0248] In Se2, the terminal device sends a first request message to the base station.

[0249] The first request message is used to request the reporting of the non-3GPP sensing data.

[0250] For example, the description of the first request message can refer to the embodiment shown in FIG. 3, which will not be repeated here.

[0251] In Se3, in response to the first request message, the base station sends the configuration information of the sensing device to the terminal device. Accordingly, the terminal device receives the configuration information of the sensing device sent by the base station.

[0252] For example, the description of the configuration information of the sensing device can refer to the embodiment shown in FIG. 3, which will not be repeated here.

[0253] In Se4, the terminal device sends the configuration information of the sensing device to the sensing device. Accordingly, the sensing device receives the configuration information of the sensing device sent by the terminal device.

[0254] In Se5, the sensing device performs sensing configuration based on the configuration information of the sensing device to generate non-3GPP sensing data.

[0255] For example, the description of the processing of the non-3GPP sensing data by the base station can refer to the embodiment shown in FIG. 3, which will not be repeated here.

[0256] In Se6, the sensing device sends the non-3GPP sensing data to the terminal device. Accordingly, the terminal device receives the non-3GPP sensing data sent by the sensing device.

[0257] In Se7, the terminal device sends the non-3GPP sensing data to the base station. Accordingly, the base station receives the non-3GPP sensing data sent by the terminal device.

[0258] In Se8, the base station processes the non-3GPP sensing data.

[0259] For example, the description of the processing of the non-3GPP sensing data by the base station can refer to the embodiment shown in FIG. 3, which will not be repeated here.

[0260] For example, as shown in FIG. 10, before reporting the non-3GPP sensing data, the above-mentioned processing method of the sensing data further includes registering the sensing device in the base station. For example, the registration of the sensing device in the base station can be implemented as follows: Sf1 to Sf6.

[0261] In Sf1, the sensing device sends a registration request message to the terminal device. Accordingly, the terminal device receives the registration request message sent by the sensing device.

[0262] For example, the description of the registration request message can refer to the embodiment shown in FIG. 3, which will not be repeated here.

[0263] In Sf2, the terminal device registers the sensing device in the terminal device.

[0264] For example, the terminal device registers the sensing capability of the sensing device in the terminal device.

[0265] In Sf3, the terminal device sends a registration request message to the base station. Accordingly, the base station receives the registration request message sent by the terminal device.

[0266] In Sf4, the base station registers the sensing device in the base station.

[0267] Exemplarily, the base station registers the sensing capability of the sensing device in the base station.

[0268] In Sf5, the base station sends a registration response message to the terminal device. Accordingly, the terminal device receives the registration response message sent by the base station.

[0269] Exemplarily, for the description of the registration response message, refer to the embodiment shown in FIG. 3, which will not be repeated here.

[0270] In Sf6, the terminal device sends a registration response message to the sensing device. Accordingly, the sensing device receives the registration response message sent by the terminal device.

[0271] It should be noted that the above registration process can be used for any one of the following: initial registration, mobility update registration, periodic registration.

[0272] Next, based on the sensing system shown in FIG. 2, in the form of interaction between the base station and the sensing device, the processing method of the sensing data provided by the embodiment of the disclosure is described.

[0273] Exemplarily, as shown in FIG. 11, the processing method of the sensing data provided by the embodiment of the disclosure includes the following: Sg1 to Sg5.

[0274] In Sg1, the base station satisfies a first trigger condition.

[0275] Exemplarily, the above Sg1 can be implemented as: the non-3GPP sensing module in the base station satisfies the first trigger condition.

[0276] Exemplarily, for the description of the first trigger condition, refer to the embodiment shown in FIG. 3, which will not be repeated here.

[0277] In Sg2, the base station sends the configuration information of the sensing device to the sensing device. Accordingly, the sensing device receives the configuration information of the sensing device sent by the base station.

[0278] Exemplarily, for the description of the configuration information of the sensing device, refer to the embodiment shown in FIG. 3, which will not be repeated here.

[0279] In Sg3, the sensing device performs sensing configuration based on the configuration information of the sensing device to generate non-3GPP sensing data.

[0280] Exemplarily, the description about the non-3GPP awareness data can refer to the embodiment shown in FIG. 3, which will not be repeated here.

[0281] In Sg4, the awareness device sends the non-3GPP awareness data to the base station. Accordingly, the base station receives the non-3GPP awareness data sent by the awareness device.

[0282] In Sg5, the base station processes the non-3GPP awareness data.

[0283] Exemplarily, the description about the processing of the non-3GPP awareness data by the base station can refer to the embodiment shown in FIG. 3, which will not be repeated here.

[0284] Exemplarily, as shown in FIG. 12, before reporting the non-3GPP awareness data, the processing method of the awareness data further includes registering the awareness device in the base station. Exemplarily, the registration of the awareness device in the base station can be implemented as follows: Sh1 to Sh3.

[0285] In Sh1, the awareness device sends a registration request message to the base station. Accordingly, the base station receives the registration request message sent by the awareness device.

[0286] Exemplarily, the description about the registration request message can refer to the embodiment shown in FIG. 3, which will not be repeated here.

[0287] In Sh2, the base station registers the awareness device in the base station.

[0288] Exemplarily, the base station registers the awareness capability of the awareness device in the base station.

[0289] In Sh3, the base station sends a registration response message to the awareness device. Accordingly, the awareness device receives the registration response message sent by the base station.

[0290] Exemplarily, the description about the registration response message can refer to the embodiment shown in FIG. 3, which will not be repeated here.

[0291] It should be noted that the above registration process can be used for any one of the following: initial registration, periodic registration.

[0292] For the convenience of understanding, the processing method of the awareness data provided by the embodiment of the present disclosure will be described in the form of examples.

[0293] Example one

[0294] In the awareness system as shown in FIG. 1, if the non-3GPP awareness module of the base station triggers awareness; assuming that the awareness device is a camera, which is connected with the terminal device through the on-board circuit; the awareness service is a target tracking service, and the awareness processing is decision-level fusion.

[0295] In Example One, the registration of the sensing capability of the sensing device can be implemented as S1 to S4 below.

[0296] In S1, the terminal device acquires the sensing capability of the sensing device.

[0297] Exemplarily, the sensing device is a camera, and the terminal device acquires the sensing capability of the camera.

[0298] In S2, the terminal device sends a registration request message to the base station through a 3GPP communication air interface link.

[0299] Exemplarily, the registration request message includes: (1) registration type: non-3GPP sensing device sensing; (2) sensing device type: camera; (3) sensing device capability: camera resolution is 2560x1440; camera reporting frame rate is 50Hz; data format is H.265 / H.264; camera angle is 30 degrees south by west, height is 10 meters, maximum aperture number F1.6; focal length & field of view angle: 4mm, horizontal field of view angle: 70°, vertical field of view angle: 35°, diagonal field of view angle: 85°6mm; or, horizontal field of view angle: 46°, vertical field of view angle: 24°, diagonal field of view angle: 54°8mm; or, horizontal field of view angle: 43°, vertical field of view angle: 24°, diagonal field of view angle: 50°12mm; or, horizontal field of view angle: 27°, vertical field of view angle: 15°, diagonal field of view angle: 31°.

[0300] In S3, the communication module of the base station receives the registration request sent by the terminal device, judges it as a non-3GPP sensing device registration request through the non-3GPP sensing module, and then forwards the registration request message to the non-3GPP sensing module of the base station.

[0301] In S4, the non-3GPP sensing module of the base station receives the registration request message, confirms that the sensing device has been successfully registered, and registers as camera 1. The registration response message (including registration success information and the identification information of the sensing device) is sent to the communication module of the base station, and then sent to the terminal by the communication module of the base station.

[0302] In Example One, the reporting and processing flow of the non-3GPP sensing data generated by the sensing device can be implemented as S1 to S10 below.

[0303] In S1, the non-3GPP sensing module of the base station generates a sensing demand, which requires non-3GPP sensing.

[0304] For example, in the case that the non-3GPP sensing module of the base station meets the first trigger condition, it is determined that the non-3GPP sensing module of the base station generates a sensing demand, which requires non-3GPP sensing.

[0305] In S2, the non-3GPP awareness module of the base station sends a non-3GPP awareness configuration to the communication module of the base station through the NS3 interface.

[0306] The non-3GPP awareness configuration includes: (1) awareness mode: camera awareness; (2) identification information of the awareness device: camera 1; (3) parameters used by the sensor: camera resolution is 2560x1440, camera reporting frame rate is 50Hz, data format is H.265 / H.264, camera angle is 30 degrees south of west, height is 10 meters, focal length is 12mm, horizontal field of view is 27°, vertical field of view is 15°, diagonal field of view is 31°, maximum aperture number is F1.6; (4) transmission resources used for reporting awareness data.

[0307] In S3, the communication module of the base station sends the non-3GPP awareness configuration to the terminal device through the 3GPP communication air interface link.

[0308] In S4, the terminal device receives the awareness configuration information, and sends the configuration information of the awareness device to camera 1 through the transmission interface according to the identification information of the awareness device (camera 1).

[0309] The configuration information of the awareness device includes: (1) awareness mode: camera awareness; (2) parameters used by the awareness device: camera resolution is 2560x1440, camera reporting frame rate is 50Hz, data format is H.265 / H.264, camera angle is 30 degrees south of west, height is 10 meters, focal length is 12mm, horizontal field of view is 27°, vertical field of view is 15°, diagonal field of view is 31°, maximum aperture number is F1.6.

[0310] In S5, camera 1 performs awareness configuration based on the configuration information of the awareness device, and generates camera recorded video data.

[0311] In S6, camera 1 transmits the camera recorded video data to the terminal device through the on-board circuit.

[0312] In S7, the terminal device converts the camera recorded video data into a communication signal through encoding, modulation and other processes, and reports the communication signal to the communication module of the base station on the specified transmission resource through the 3GPP communication air interface link based on the transmission resources used for reporting awareness data in the non-3GPP awareness configuration; at the same time, a non-3GPP awareness data reporting identifier is sent.

[0313] In S8, the non-3GPP awareness module of the base station acquires the non-3GPP awareness data reporting identifier, and notifies the communication module of the base station to forward the non-3GPP awareness data. The communication module of the base station analyzes the communication signal into non-3GPP awareness data through signal detection, demodulation and decoding, and forwards it to the non-3GPP awareness module of the base station.

[0314] In S9, the non-3GPP awareness module of the base station processes the non-3GPP awareness data.

[0315] For example, the processing of the non-3GPP awareness data by the non-3GPP awareness module of the base station includes preprocessing, awareness data processing, and data standardization.

[0316] Preprocessing: parsing the QoS flow into camera video data.

[0317] Awareness data processing: (1) using a yolo neural network, extracting target categories, local coordinate system xyz positions, and confidence from the camera video data; (2) performing confidence weighted averaging on the targets extracted from the camera video data and the targets sensed by the common sense to obtain the final target sensing result, which contains target categories, local coordinate system xyz positions, and confidence.

[0318] Data standardization: converting the local coordinate system xyz positions into longitude, latitude, and height, and associating multiple frames of trajectories to obtain object three-dimensional moving speed (vx, vy, vz) and object ID, and combining the confidence and target categories extracted from the camera data to obtain a unified format of sensing results containing ID, category, confidence, position, and speed.

[0319] In S10, the non-3GPP awareness module of the base station reports the unified format of awareness data to the core network.

[0320] Example Two

[0321] In the common sense system as shown in FIG. 1, if the terminal device triggers sensing; assuming that the sensing device is a millimeter wave radar, connected to the terminal device through Ethernet; the sensing service is a target tracking service, and the sensing processing is feature-level fusion.

[0322] In Example Two, the sensing capability registration of the sensing device can be implemented as follows.

[0323] In S1, the terminal device obtains the sensing capability of the sensing device.

[0324] Illustratively, the sensing device is a millimeter wave radar, and the terminal device obtains the sensing capability of the millimeter wave radar.

[0325] In S2, the terminal device sends a registration request message to the base station through a 3GPP communication air interface link.

[0326] Exemplarily, the registration request message includes: (1) registration type: non-3GPP aware device awareness; (2) sensor type: millimeter wave radar; (3) sensor capability: maximum detection distance 150 m, distance resolution 1.2 m, speed resolution 0.36 km / h, azimuth angle range 90 degrees, elevation angle range 13 degrees, refresh rate 12.5 Hz, and transmission power 30 dB.

[0327] In S3, the communication module of the base station receives the registration request message sent by the terminal, judges that it is a non-3GPP aware device registration request through the non-3GPP aware module, and then forwards the registration request message to the non-3GPP aware module of the base station.

[0328] In S4, the non-3GPP aware module of the base station receives the registration request message, confirms that the aware device has been successfully registered, and registers as millimeter wave radar 1. A registration response message (including registration success information and identification information of the aware device) is sent to the communication module of the base station, and then sent to the terminal by the communication module of the base station.

[0329] In Example Two, the reporting and processing flow of non-3GPP aware data generated by the aware device can be implemented as follows: S1 to S11.

[0330] In S1, the terminal device generates an awareness requirement and needs non-3GPP awareness.

[0331] For example, in the case where the terminal device meets the second trigger condition, it is determined that the terminal device generates an awareness requirement and needs non-3GPP awareness.

[0332] In S2, the terminal device sends a first request message to the base station. The first request message is used to request reporting of non-3GPP aware data.

[0333] The first request message includes configuration information of the aware device: (1) awareness mode: millimeter wave radar awareness; (2) identification information of the aware device: millimeter wave radar 1; (3) parameters used by the aware device: millimeter wave radar orientation is north by west 30 degrees, maximum detection distance 150 m, distance resolution 1.2 m, speed resolution 0.36 km / h, azimuth angle range 90 degrees, elevation angle range 13 degrees, refresh rate 12.5 Hz, and transmission power 30 dB.

[0334] In S3, after the non-3GPP aware module of the base station captures the first request message, a non-3GPP aware configuration is generated, and the non-3GPP aware configuration is sent to the communication module of the base station through the NS3 interface.

[0335] The 3GPP sensing configuration includes: (1) a sensing mode: millimeter wave radar sensing; (2) identification information of the sensing device: millimeter wave radar 1; (3) parameters used by the sensor: the millimeter wave radar is oriented at 30 degrees west of north, the maximum detection distance is 150 m, the distance resolution is 1.2 m, the speed resolution is 0.36 km / h, the azimuth angle range is 90 degrees, the elevation angle range is 13 degrees, the refresh rate is 12.5 Hz, and the transmission power is 30 dB; and (4) transmission resources used for reporting sensing data.

[0336] In S4, the communication module of the base station sends the reporting capture confirmation indication and the non-3GPP sensing configuration to the terminal device through the 3GPP communication air interface link.

[0337] In S5, the terminal device receives the capture confirmation indication and the non-3GPP sensing configuration, and sends the configuration information of the sensing device to the millimeter wave radar 1 through the transmission interface according to the identification information of the sensing device (millimeter wave radar 1).

[0338] The configuration information of the sensing device includes: (1) a sensing mode: millimeter wave radar sensing; (2) parameters used by the sensing device: the millimeter wave radar is oriented at 30 degrees west of north, the maximum detection distance is 150 m, the distance resolution is 1.2 m, the speed resolution is 0.36 km / h, the azimuth angle range is 90 degrees, the elevation angle range is 13 degrees, the refresh rate is 12.5 Hz, and the transmission power is 30 dB.

[0339] In S6, the millimeter wave radar 1 performs sensing configuration based on the sensor configuration information, and generates millimeter wave radar time delay Doppler figure data.

[0340] In S7, the millimeter wave radar 1 transmits the time delay Doppler figure data to the terminal device through the Ethernet interface.

[0341] In S8, the terminal device converts the time delay Doppler figure data into a communication signal through encoding, modulation and the like, and reports the communication signal to the communication module of the base station on the specified transmission resource through the 3GPP communication air interface link based on the transmission resources used for reporting sensing data in the non-3GPP sensing configuration; at the same time, a non-3GPP sensing data reporting identifier is sent.

[0342] In S9, the non-3GPP sensing module of the base station acquires the non-3GPP sensing data reporting identifier, and informs the communication module of the base station to forward the non-3GPP sensing data. The communication module of the base station analyzes the communication signal into non-3GPP sensing data through signal detection, demodulation and decoding, and forwards the non-3GPP sensing data to the non-3GPP sensing module of the base station.

[0343] In S10, the non-3GPP sensing module of the base station processes the non-3GPP sensing data.

[0344] For example, the non-3GPP awareness module of the base station processes the non-3GPP awareness data, including preprocessing, awareness data processing, and data standardization.

[0345] Preprocessing: parsing the QoS flow into time delay Doppler map data.

[0346] Awareness data processing: (1) extracting awareness features from the time delay Doppler map using a CNN (Convolutional Neural Network) neural network; (2) splicing the target awareness features extracted from the time delay Doppler map data with the features extracted from the common awareness result, and then processing them through a transformer neural network to obtain the final target awareness result, including the local coordinate system xyz position, confidence, and object category.

[0347] Data standardization: converting the local coordinate system xyz position into longitude, latitude, and height, and associating multiple frames of trajectories to obtain object three-dimensional moving speed (vx, vy, vz) and object ID, and obtaining the unified format awareness result containing ID, category, confidence, three-dimensional position, and three-dimensional speed.

[0348] In S11, the non-3GPP awareness module of the base station reports the unified format awareness data to the core network.

[0349] Example Three

[0350] In the common awareness system as shown in FIG. 2, if the non-3GPP awareness module of the base station triggers awareness; assuming that the awareness device is a laser radar, which is connected to the base station through Ethernet; the awareness service is an environmental awareness service, and the awareness processing is data-level fusion.

[0351] In Example Three, the awareness capability registration of the awareness device can be implemented as follows: S1 and S2.

[0352] In S1, the laser radar 1 sends a registration request message to the base station through a non-3GPP communication air interface link.

[0353] Exemplarily, the registration request message includes: (1) registration type: non-3GPP awareness device awareness; (2) awareness device type: laser radar; (3) awareness device capability: laser wavelength 905nm, detection distance: 90m@10% reflectivity; 130m@20% reflectivity; 260m@80% reflectivity; detection range 70 degrees, ranging error 2cm, angle error 0.1 degrees, output frequency 100Hz.

[0354] In S2, the non-3GPP awareness module of the base station receives the registration request message, confirms that the awareness device has been successfully registered, and registers as Lidar 1. A registration response message (including registration success information and identification information of the awareness device) is sent to the awareness device.

[0355] In Example Three, the reporting and processing flow of the non-3GPP awareness data generated by the awareness device can be implemented as follows: S1 to S6.

[0356] In S1, the non-3GPP awareness module of the base station generates awareness needs, requiring non-3GPP awareness.

[0357] For example, in the case where the non-3GPP awareness module of the base station meets the first trigger condition, it is determined that the non-3GPP awareness module of the base station generates awareness needs, requiring non-3GPP awareness.

[0358] In S2, the non-3GPP awareness module of the base station sends the configuration information of the awareness device to Lidar 1 through the transmission interface.

[0359] Exemplarily, the configuration information of the awareness device includes: (1) awareness mode: Lidar; (2) parameters used by the awareness device: angle towards north-west 60 degrees, laser wavelength 905nm, detection distance 260m, detection range 70 degrees, ranging error 2cm, angle error 0.1 degrees, output frequency 100Hz.

[0360] In S3, Lidar 1 performs awareness configuration based on the configuration information of the awareness device, and generates Lidar point cloud data.

[0361] In S4, Lidar 1 transmits the Lidar point cloud data to the non-3GPP awareness module of the base station through the Ethernet port.

[0362] In S5, the non-3GPP awareness module of the base station processes the Lidar point cloud data.

[0363] For example, the processing of the Lidar point cloud data by the non-3GPP awareness module of the base station includes preprocessing, awareness data processing, and data standardization.

[0364] Preprocessing: parsing the QoS flow into Lidar point cloud data.

[0365] Awareness data processing: (1) splicing the Lidar point cloud data with the general awareness raw data; (2) using a CNN neural network to process the data to obtain a three-dimensional point cloud result of the mitigation awareness, including the environmental point three-dimensional position, material, and object category of the environmental point cloud.

[0366] Data standardization: The local coordinate system xyz position of the environmental point cloud is converted into a coordinate system, converted into longitude, latitude and height, and the unified format of the environmental point cloud perception result containing object category, material, three-dimensional position is obtained.

[0367] In S6, the non-3GPP awareness module of the base station reports the unified format of the perception data to the core network.

[0368] In summary, the present disclosure can be applied to the communication and perception integrated scene, and a perception data processing method is proposed. Based on the present disclosure, the sensor perception information carried by mobile terminals, vehicles, intelligent transportation, intelligent factories, etc. (taking the perception device as an example) can be included in the communication and perception architecture, thereby increasing the perception data source of the communication and perception system and improving the perception quality, perception range and perception accuracy.

[0369] Based on the communication and perception networking architecture, the sensors distributed in various places can perform data fusion, breaking the original single sensor perception, and the perception information can be fused in the communication and perception system to improve the perception quality and perception range of the related sensors.

[0370] Next, the beneficial effects of the embodiments of the present disclosure will be briefly introduced.

[0371] In related technologies, in addition to using 3GPP wireless spectrum resources for perception, a large number of perception devices have been applied in the industry, such as vehicle-to-everything roadside stations, video monitoring cameras, and Internet of Things perception units. The perception information generated by these perception devices is transmitted to the Internet public network based on the communication link, and then the perception data is processed and applied, involving gateway interaction from the 3GPP core network to the Internet public network, and the processing complexity and latency will be high. At the same time, the current base station side as a perception site, in the absence of 3GPP standards, the base station side spontaneously sends and receives, and the base station A sends and the base station B receives, and the perception range and diversity are still limited. Therefore, the embodiments of the present disclosure fuse the original communication and perception architecture by the communication link, external perception devices, and terminal external perception devices, which have the following beneficial effects.

[0372] 1. The non-3GPP sensor can be included in the communication and perception architecture, realizing the utilization of related perception resources by the communication and perception architecture, greatly expanding the perception range and diversity of the communication and perception system, and improving the perception effect.

[0373] 2. Similar to the 5G-NSA (Non-standalone, non-independent networking) architecture, the non-3GPP perception information is processed by the base station side, the core network perception function entity can simplify the deployment evolution, the implementation difficulty is low, adapts to the current field situation, and is convenient for rapid deployment of experimental stations.

[0374] 3. Compared with the architecture for transmitting non-3GPP-aware data to an Internet public network transmission link or a core network, the sensing architecture provided by the disclosure can reduce the latency and complexity of the original non-3GPP-aware data transmission and processing, and can adapt to better processing methods such as data-level fusion and feature-level fusion.

[0375] The above describes the solutions of the embodiments of the disclosure mainly from the perspective of the method. It can be understood that, in order to implement the above functions, the sensing data processing apparatus includes at least one of the corresponding hardware structure and software module for executing each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of the examples described in the embodiments disclosed herein, the embodiments of the disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is implemented in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of the disclosure.

[0376] The embodiments of the disclosure can divide the sensing data processing apparatus into functional modules according to the above method embodiments. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one functional module. The above integrated module can be implemented in the form of hardware or software. It should be noted that the division of the modules in the embodiments of the disclosure is illustrative, and is only a logical functional division. In actual implementation, another division method can be used. The following takes the example of dividing each functional module according to each function.

[0377] FIG. 13 is a structural schematic diagram of a sensing data processing apparatus according to an embodiment of the disclosure. The sensing data processing apparatus is applied to a base station, and can execute the processing method of the sensing data provided by the above method embodiments. As shown in FIG. 13, the sensing data processing apparatus 1000 includes a communication module 1001 and a processing module 1002.

[0378] The communication module 1001 is configured to receive non-3GPP-aware data of a sensing device.

[0379] The processing module 1002 is configured to process the non-3GPP-aware data.

[0380] In some embodiments, the communication module 1001 is configured to receive the non-3GPP-aware data sent by the sensing device through a terminal device, or directly receive the non-3GPP-aware data sent by the sensing device.

[0381] In some embodiments, the communication module 1001 is further configured to send, to the sensing device, configuration information of the sensing device.

[0382] In some embodiments, the communication module 1001 is configured to send, to the sensing device, the configuration information of the sensing device directly or through the terminal device.

[0383] In some embodiments, the communication module 1001 is configured to send, to the sensing device, the configuration information of the sensing device when a first trigger condition is met. The first trigger condition includes at least one of the following: a requirement of the base station triggering sensing of a static environment; a requirement of the base station triggering sensing of a dynamic target; a requirement of the base station triggering monitoring of a drone device.

[0384] In some embodiments, the communication module 1001 is configured to send, to the sensing device, the configuration information of the sensing device when a first request message is received. The first request message is used to request reporting of non-3GPP sensing data.

[0385] In some embodiments, the first request message is sent by the terminal device when a second trigger condition is met. The second trigger condition includes at least one of the following: a requirement of the terminal device triggering sensing of a surrounding environment; a requirement of the terminal device triggering reporting of sensing information.

[0386] In some embodiments, the first request message includes at least one of the following: a sensing mode, identification information of the sensing device, and a parameter used by the sensing device.

[0387] In some embodiments, the configuration information of the sensing device includes at least one of the following: a sensing mode and a parameter used by the sensing device.

[0388] In some embodiments, the communication module 1001 is configured to send, to the terminal device, non-3GPP sensing configuration information, so that the terminal device sends, to the sensing device, the configuration information of the sensing device based on the non-3GPP sensing configuration information. The configuration information of the sensing device is a subset of the non-3GPP sensing configuration information.

[0389] In some embodiments, the non-3GPP sensing configuration information includes at least one of the following: a sensing mode, identification information of the sensing device, a parameter used by the sensing device, and a transmission resource used for reporting of sensing data.

[0390] In some embodiments, the communication module 1001 is configured to receive non-3GPP sensing data on the transmission resource used for reporting of sensing data.

[0391] In some embodiments, the communication module 1001 is configured to obtain the non-3GPP sensing data based on a received reporting identifier of the non-3GPP sensing data.

[0392] In some embodiments, the processing module 1002 is configured to sequentially perform preprocessing, perception data processing and data standardization processing on the non-3GPP perception data.

[0393] In some embodiments, the communication module 1001 is configured to receive a registration request message, the registration request message being used to request registration of information of the perception device in the base station; and send a registration response message, the registration response message being used to indicate that the perception device has been successfully registered.

[0394] In some embodiments, the communication module 1001 is configured to receive, by the terminal device, the registration request message sent by the perception device; or directly receive the registration request message sent by the perception device.

[0395] In some embodiments, the communication module 1001 is configured to send, by the terminal device, the registration response message to the perception device; or directly send the registration response message to the perception device.

[0396] In some embodiments, the registration request message comprises at least one of the following: a registration type, a type of the perception device, and capability information of the perception device.

[0397] In some embodiments, the capability information of the perception device is determined according to the type of the perception device.

[0398] In some embodiments, in the case that the type of the perception device includes a camera, the capability information of the perception device includes at least one of the following: a resolution adjustment range, a reporting frame rate adjustment range, a steering angle, a focal length, an aperture, a magnification, a correction coefficient of the camera; in the case that the type of the perception device includes a millimeter wave radar, the capability information of the perception device includes at least one of the following: a transmission power range, a beam configurable type, an operating frequency range adjustment range, a perception resolution adjustment range, a perception angle adjustment range, a perception distance adjustment range, a snapshot reporting frame rate adjustment range of the millimeter wave radar; in the case that the type of the perception device includes a laser radar, the capability information of the perception device includes at least one of the following: a scanning mode, a laser beam angle adjustment range, an angle resolution adjustment range, an operating frequency adjustment range, a snapshot reporting frame rate adjustment range of the laser radar; in the case that the type of the perception device includes an ultra-wideband device, the capability information of the perception device includes at least one of the following: an ultra-wideband device perception frequency range adjustment range, a pulse shape, a transmission power adjustment range, a pulse repetition frequency adjustment range, a snapshot reporting frame rate adjustment range; in the case that the type of the perception device includes a wireless connection device, the capability information of the perception device includes at least one of the following: a wireless connection device perception frequency range adjustment range, a perception signal strength, a waveform adjustment range, a perception resolution adjustment range, a snapshot reporting frame rate adjustment range; in the case that the type of the perception device includes a GPS device, the capability information of the GPS device includes at least one of the following: a positioning accuracy adjustment range, a satellite signal strength, a navigation path planning capability, a real-time speed measurement range, a time synchronization accuracy adjustment range.

[0399] FIG. 14 is a structural schematic diagram of another perception data processing apparatus according to an embodiment of the present disclosure. The perception data processing apparatus is applied to a terminal device, and can perform the perception data processing method provided by the method embodiments. As shown in FIG. 14, the perception data processing apparatus 1100 includes a receiving module 1101 and a sending module 1102. In other embodiments, the perception data processing apparatus 1100 further includes a registration module 1103.

[0400] The receiving module 1101 is configured to receive the non-3GPP perception data sent by the perception device.

[0401] The sending module 1102 is configured to send the non-3GPP perception data to a base station, so that the base station processes the non-3GPP perception data.

[0402] In some embodiments, the receiving module 1101 is further configured to receive non-3GPP awareness configuration information sent by the base station; and the sending module 1102 is further configured to send, based on the non-3GPP awareness configuration information, configuration information of the awareness device to the awareness device. The configuration information of the awareness device is a subset of the non-3GPP awareness configuration information. The configuration information of the awareness device is used to configure the awareness device to perform awareness configuration and generate non-3GPP awareness data.

[0403] In some embodiments, the sending module 1102 is further configured to send, to the base station, a first request message. The first request message is used to request reporting of the non-3GPP awareness data.

[0404] In some embodiments, the sending module 1102 is configured to send, to the base station, the first request message when a second trigger condition is met. The second trigger condition includes at least one of the following: a requirement of the terminal device to trigger awareness of the surrounding environment; and a requirement of the terminal device to trigger reporting of awareness information.

[0405] In some embodiments, the receiving module 1101 is further configured to receive a registration request message sent by the awareness device. The registration request message is used to request registration of information of the awareness device in the base station; and the sending module 1102 is further configured to send, to the base station, the registration request message.

[0406] In some embodiments, the registration module 1103 is configured to register the information of the awareness device in the terminal device.

[0407] In some embodiments, the receiving module 1101 is further configured to receive a registration response message sent by the base station. The registration response message is used to indicate that the awareness device has been successfully registered; and the sending module 1102 is further configured to send, to the awareness device, the registration response message.

[0408] FIG. 15 is a structural schematic diagram of another awareness data processing apparatus provided by an embodiment of the present disclosure. The awareness data processing apparatus is applied to an awareness device and can perform the awareness data processing method provided by the above-mentioned method embodiments. As shown in FIG. 15, the awareness data processing apparatus 1200 includes a sending module 1201. In other embodiments, the awareness data processing apparatus 1200 further includes a receiving module 1202 and a configuration module 1203.

[0409] The sending module 1201 is configured to send, to a base station, non-3GPP awareness data generated by an awareness device, so that the base station processes the non-3GPP awareness data.

[0410] In some embodiments, the receiving module 1202 is configured to receive configuration information of the awareness device sent by the base station; and the configuration module 1203 is configured to perform awareness configuration based on the configuration information of the awareness device and generate non-3GPP awareness data.

[0411] In some embodiments, the sending module 1201 is further configured to send a registration request message to the base station, the registration request message being used to request registration of the information of the sensing device in the base station; and the receiving module 1202 is further configured to receive a registration response message sent by the base station, the registration response message being used to indicate that the sensing device has been successfully registered.

[0412] In the case of implementing the functions of the above-mentioned integrated modules in the form of hardware, the embodiments of the present disclosure provide a structure of the communication apparatus involved in the above-mentioned embodiments. As shown in FIG. 16, the communication apparatus 1300 includes a processor 1302 and a bus 1304. In some embodiments, the communication apparatus can further include a memory 1301. In some embodiments, the communication apparatus 1300 can further include a communication interface 1303.

[0413] The processor 1302 can implement or execute the various exemplary logical blocks, modules, and circuits described in combination with the embodiments of the present disclosure. The processor 1302 can be a central processing unit, a general purpose processor, a digital signal processor, an application specific integrated circuit, a field programmable gate array or other programmable logic device, transistor logic, a hardware component, or any combination thereof. It can implement or execute the various exemplary logical blocks, modules, and circuits described in combination with the embodiments of the present disclosure. The processor 1302 can also be a combination of implementing computing functions, such as a combination of one or more microprocessors, a combination of a DSP (digital signal processor) and a microprocessor, and the like.

[0414] The communication interface 1303 is configured to connect with other devices through a communication network. The communication network can be an Ethernet, a wireless access network, a WLAN (wireless local area network), and the like.

[0415] The memory 1301 can be a ROM (read-only memory) or other type of static storage device that can store static information and instructions, a RAM (random access memory) or other type of dynamic storage device that can store information and instructions, an EEPROM (electrically erasable programmable read-only memory), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to this.

[0416] As an implementation manner, the memory 1301 can exist independently of the processor 1302, or the memory 1301 can be connected with the processor 1302 through the bus 1304, for storing instructions or program codes. When the processor 1302 invokes and executes the instructions or program codes stored in the memory 1301, the processing method of the perception data provided in the embodiments of the present disclosure can be implemented. In another implementation manner, the memory 1301 can also be integrated with the processor 1302.

[0417] The bus 1304 can be an extended industry standard architecture (EISA) bus, etc. The bus 1304 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in FIG. 16, but this does not mean that there is only one bus or only one type of bus.

[0418] Some embodiments of the present disclosure provide a computer readable storage medium (for example, a non-transitory computer readable storage medium), which stores computer program instructions, and the computer program instructions, when running on a computer, cause the computer to execute the processing method of the perception data as described in any of the above embodiments.

[0419] Exemplarily, the above computer readable storage medium can include, but is not limited to, a magnetic storage device (for example, a hard disk, a floppy disk or a magnetic tape, etc.), an optical disc (for example, a compact disk (CD), a digital versatile disk (DVD), etc.), a smart card and a flash memory device (for example, an erasable programmable read-only memory (EPROM), a card, a stick or a key drive, etc.). The various computer readable storage media described in the present disclosure can represent one or more devices and / or other machine readable storage media for storing information. The term "machine readable storage medium" can include, but is not limited to, a wireless channel and various other media capable of storing, containing and / or carrying instructions and / or data.

[0420] The embodiments of the present disclosure provide a computer program product containing instructions, which, when running on a computer, cause the computer to execute the processing method of the perception data as described in any of the above embodiments.

[0421] The above description is only a specific implementation of the present disclosure, but the protection scope of the present disclosure is not limited thereto, and any change or replacement within the technical scope disclosed in the present disclosure should be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. A method for processing sensing data, applied to a base station, comprising: receiving non-3rd Generation Partnership Project (3GPP) sensing data of a sensing device; processing the non-3GPP sensing data.

2. The method of claim 1, wherein, The receiving of the non-3GPP sensing data of the sensing device comprises: receiving the non-3GPP sensing data sent by the sensing device through a terminal device; or directly receiving the non-3GPP sensing data sent by the sensing device.

3. The method of claim 1, further comprising: sending configuration information of the sensing device to the sensing device.

4. The method of claim 3, wherein, The sending of the configuration information of the sensing device to the sensing device comprises: directly sending the configuration information of the sensing device to the sensing device; or sending the configuration information of the sensing device to the sensing device through a terminal device.

5. The method of claim 3, wherein, The sending of the configuration information of the sensing device to the sensing device comprises: sending the configuration information of the sensing device to the sensing device in a case where a first trigger condition is met; wherein the first trigger condition comprises at least one of: a requirement for the base station to sense a static environment; a requirement for the base station to sense a dynamic target; a requirement for the base station to monitor a drone device.

6. The method of claim 3, wherein, The sending of the configuration information of the sensing device to the sensing device comprises: sending the configuration information of the sensing device to the sensing device in a case where a first request message is received; the first request message is used to request reporting of non-3GPP sensing data.

7. The method of claim 6, wherein, The first request message is sent by a terminal device in a case where a second trigger condition is met; wherein the second trigger condition comprises at least one of: a requirement for the terminal device to sense a surrounding environment; a requirement for the terminal device to report sensing information.

8. The method of claim 7, wherein, The first request message comprises at least one of: a sensing mode, identification information of a sensing device, and parameters used by the sensing device.

9. The method of claim 3, wherein, The configuration information of the sensing device comprises at least one of: a sensing mode and parameters used by the sensing device.

10. The method of claim 4, wherein, The sending of the configuration information of the sensing device to the sensing device through the terminal device comprises: sending non-3GPP sensing configuration information to the terminal device, so that the terminal device sends the configuration information of the sensing device to the sensing device based on the non-3GPP sensing configuration information; the configuration information of the sensing device is a subset of the non-3GPP sensing configuration information.

11. The method of claim 10, wherein, The non-3GPP sensing configuration information comprises at least one of: a sensing mode, identification information of a sensing device, parameters used by the sensing device, and transmission resources used for reporting of sensing data.

12. The method of claim 11, wherein, The receiving of the non-3GPP sensing data of the sensing device comprises: receiving the non-3GPP sensing data on transmission resources used for reporting of the sensing data.

13. The method of claim 1, wherein, The receiving of the non-3GPP sensing data of the sensing device comprises: obtaining the non-3GPP sensing data based on a received non-3GPP sensing data reporting identifier.

14. The method of claim 1, wherein, The processing of the non-3GPP sensing data comprises: The non-3GPP awareness data is sequentially pre-processed, awareness data processed and data standardized.

15. The method of claim 1, further comprising: receiving a registration request message, the registration request message being used to request registration of information of the awareness device in the base station; sending a registration response message, the registration response message being used to indicate that the awareness device has been successfully registered.

16. The method of claim 15, wherein, The receiving of the registration request message comprises: receiving the registration request message sent by the awareness device through a terminal device; or directly receiving the registration request message sent by the awareness device.

17. The method of claim 15, wherein, The sending of the registration response message comprises: sending the registration response message to the awareness device through a terminal device; or directly sending the registration response message to the awareness device.

18. The method of claim 15, wherein, The registration request message comprises at least one of the following: a registration type, a type of the awareness device, and capability information of the awareness device.

19. The method of claim 18, wherein the capability information of the awareness device is determined according to the type of the awareness device.

20. The method of claim 19, wherein in a case where the type of the awareness device comprises a camera, the capability information of the awareness device comprises at least one of the following: a resolution adjustment range, a reporting frame rate adjustment range, a turning angle, a focal length, an aperture, a magnification, and a correction coefficient of the camera; in a case where the type of the awareness device comprises a millimeter wave radar, the capability information of the awareness device comprises at least one of the following: a transmission power range, a beam configurable type, an operating frequency range adjustment range, an awareness resolution adjustment range, an awareness angle adjustment range, an awareness distance adjustment range, and a snapshot reporting frame rate adjustment range of the millimeter wave radar; in a case where the type of the awareness device comprises a laser radar, the capability information of the awareness device comprises at least one of the following: a scanning mode, a laser beam angle adjustment range, an angle resolution adjustment range, an operating frequency adjustment range, and a snapshot reporting frame rate adjustment range of the laser radar; in a case where the type of the awareness device comprises an ultra-wideband device, the capability information of the awareness device comprises at least one of the following: an awareness frequency range adjustment range, a pulse shape, a transmission power adjustment range, a pulse repetition frequency adjustment range, and a snapshot reporting frame rate adjustment range of the ultra-wideband device; in a case where the type of the awareness device comprises a wireless connection device, the capability information of the awareness device comprises at least one of the following: an awareness frequency range adjustment range, an awareness signal strength, a waveform adjustment range, an awareness resolution adjustment range, and a snapshot reporting frame rate adjustment range of the wireless connection device; in a case where the type of the awareness device comprises a global positioning system (GPS) device, the capability information of the GPS device comprises at least one of the following: a positioning accuracy adjustment range, a satellite signal strength, a navigation path planning capability, a real-time speed measurement range, and a time synchronization accuracy adjustment range.

21. A processing method of awareness data, applied to a terminal device, comprising: receiving non-third generation partnership project (3GPP) awareness data sent by an awareness device; sending the non-3GPP awareness data to a base station, so that the base station processes the non-3GPP awareness data.

22. The method of claim 21, further comprising: receiving non-3GPP awareness configuration information sent by the base station; sending configuration information of the awareness device to the base station based on the non-3GPP awareness configuration information; the configuration information of the awareness device is a subset of the non-3GPP awareness configuration information; and the configuration information of the awareness device is used to configure the awareness device to generate the non-3GPP awareness data.

23. The method of claim 22, wherein, before the receiving the non-3GPP awareness configuration information sent by the base station, the method further comprises: sending a first request message to the base station, the first request message being used to request reporting the non-3GPP awareness data.

24. The method of claim 23, wherein, the sending the first request message to the base station comprises: sending the first request message to the base station when a second trigger condition is met; the second trigger condition comprises at least one of the following: a requirement of the terminal device to trigger awareness of the surrounding environment; a requirement of the terminal device to trigger reporting of awareness information.

25. The method of claim 21, further comprising: receiving a registration request message sent by the awareness device, the registration request message being used to request registering information of the awareness device in the base station; sending the registration request message to the base station.

26. The method of claim 25, further comprising: registering the information of the awareness device in the terminal device.

27. The method of claim 25, further comprising: receiving a registration response message sent by the base station, the registration response message being used to indicate that the awareness device has been successfully registered; sending the registration response message to the awareness device.

28. A method for processing awareness data, applied to an awareness device, comprising: sending non-3rd Generation Partnership Project (3GPP) awareness data generated by the awareness device to a base station, so that the base station processes the non-3GPP awareness data.

29. The method of claim 28, further comprising: receiving configuration information of the awareness device sent by the base station; configuring the awareness device based on the configuration information of the awareness device to generate the non-3GPP awareness data.

30. The method of claim 28, further comprising: sending a registration request message to the base station, the registration request message being used to request registering information of the awareness device in the base station; receiving a registration response message sent by the base station, the registration response message being used to indicate that the awareness device has been successfully registered.

31. A communications device comprising: a memory and a processor; the memory is coupled to the processor; the memory is used to store instructions executable by the processor; and the processor executes the instructions to perform the method according to any one of claims 1 to 30.

32. A computer readable storage medium, wherein, The computer readable storage medium stores computer instructions, when the computer instructions run on a computer, make the computer execute the method according to any one of claims 1 to 30.

33. A computer program product, wherein, The computer program product comprises computer instructions which, when run on a computer, cause the computer to perform the method according to any one of claims 1 to 30.

Images