Sensing data transmission method and apparatus

Abstract

The present application relates to the technical field of communications, in particular to a sensing data transmission method and apparatus, which aim to improve the flexibility of performing compression processing on sensing data, acquire higher compression efficiency, and reduce the data volume of the sensing data. The method comprises: a first communication apparatus generating compressed data and first indication information, wherein the first indication information indicates the categories of a plurality of pieces of sensing data, and the compressed data is obtained by means of compressing the plurality of pieces of sensing data on the basis of a compression mode corresponding to the categories of the plurality of pieces of sensing data, the sensing data of the same category among the plurality of pieces of sensing data corresponding to the same compression mode in the correspondence between the categories of different sensing data and different compression modes; and sending the compressed data and the first indication information to a second communication apparatus.

Description

A method and device for transmitting perceptual data

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of the People's Republic of China on November 20, 2023, with application number 202311555347.8 and application name "A Method and Device for Transmitting Perceptual Data", the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The embodiments of the present application relate to the field of communication technologies, and in particular to a method and apparatus for perceptual data transmission. Background Art

[0004] As wireless communication applications become increasingly diverse, they can be applied to a wider range of scenarios, such as perception, point cloud, imaging, and environmental reconstruction. For example, in environmental reconstruction, multiple terminal devices can scan the environment and send the acquired perception data to a base station. The base station can then construct a complete environmental map based on the received perception data.

[0005] Since a large amount of perception data is transmitted in scenarios such as environmental reconstruction, how to compress the perception data and reduce its volume is an issue worthy of attention.

[0006] Summary of the Invention

[0007] The present application provides a method and apparatus for transmitting perceptual data, in order to improve the flexibility of compressing perceptual data, obtain higher compression efficiency, and reduce the amount of perceptual data.

[0008] In a first aspect, an embodiment of the present application provides a method for transmitting perceptual data, which can be executed by a first communication device, and the method includes: generating compressed data and first indication information, the first indication information indicating the categories of multiple perceptual data, the compressed data being obtained by compressing multiple perceptual data according to the compression methods corresponding to the categories of the multiple perceptual data, wherein perceptual data of the same category in the multiple perceptual data corresponds to the same compression method in the correspondence between the categories of different perceptual data and different compression methods; and sending the compressed data and the first indication information to a second communication device.

[0009] Exemplarily, the categories of the plurality of perception data may be determined according to the categories of the target objects corresponding to the plurality of perception data.

[0010] In the above-mentioned perceptual data transmission method, the first communication device and the second communication device are different communication devices. The first communication device (or the second communication device) can be a terminal device, an internal component of the terminal device (such as a processor, chip, or chip system, etc.), or a device that is matched with the terminal device. It can also be a network device, an internal component of the network device (such as a processor, chip, or chip system, etc.), or a device that is matched with the network device.

[0011] Through the above method, considering that the importance and characteristics of different categories of perception data may vary, different compression methods can be set for different categories of perception data, which is conducive to improving the flexibility of compressing perception data, obtaining higher compression efficiency, and reducing the amount of perception data.

[0012] In one possible design, the correspondence between different types of sensory data and different compression methods includes one or more of the following: a compression method that quantizes and compresses unsampled sensory data; a compression method that quantizes and compresses sampled sensory data; a compression method that compresses based on the geometric structure corresponding to the sensory data; a compression method that compresses based on the bounding box corresponding to the sensory data; and a compression method that compresses sensory data based on geometric projection. This design allows different compression methods to be set for different types of sensory data, which helps improve the flexibility of sensory data compression processing and achieve higher compression efficiency.

[0013] Exemplary: A compression method of performing quantization compression on the perceptual data without sampling, wherein the quantization compression without sampling may mean performing quantization compression on all data.

[0014] A compression method that performs quantization compression on the sampling of perception data, where sampling can refer to extracting part of the data, such as extracting part of the data based on data downsampling / extracting feature data, and sampling quantization compression quantizes and compresses the extracted part of the data.

[0015] A compression method based on the geometric structure corresponding to the perception data, wherein the geometric structure can be a plane, a curved surface, a polygon, a polyhedron, a cube, a cylinder, a cone, or other structures. Specific compression methods can be used as follows: (1) replacing the original data with geometric structure information, the compressed data includes the geometric structure information, for example, if a cylinder is replaced, the compressed data may include its center point, radius, and height; if a polygon is replaced, the compressed data may include the vertices of the polygon, etc.; or (2) data compression based on geometric structure information, the compressed data may include geometric structure information and / or perception data information contained in the geometric structure, for example, all / part of the sampled perception data within the geometric structure scanning area. In addition, it is understandable that the compressed data as above can also be entropy coded, or quantized and entropy coded to further reduce the amount of data.

[0016] A compression method based on the bounding box corresponding to the perception data, wherein the bounding box can be the bounding box of the target (car / person / object, etc.), and the form of the bounding box can be various, for example, the minimum enclosing bounding box, the enclosing bounding box based on the xyz axis, etc. The specific compression can be carried out in the following ways: (1) the original data is replaced by the bounding box information, and the compressed data includes the bounding box information, and the bounding box information can be the vertex of the bounding box, or the center position and length and width of the bounding box; or (2) data compression is performed based on the bounding box information, and the compressed data includes the bounding box information and / or the data contained in the bounding box, such as all target data / partial sampled target data within the bounding box (such as target feature points, contour points, etc.). In addition, it can be understood that the compressed data as above can also be entropy coded, or quantized, entropy coded to further reduce the amount of data, wherein the quantized boundary can be the range of the data or the range of the bounding box. Exemplary: the quantized boundary can be the boundary of the perception data based on the xyz three-axis coordinate system, or it can be the transformation of the perception data into a coordinate system based on the length, width and height of the border as axes, based on the boundary in this coordinate system. If it is based on the transformed coordinate system, the coordinate system transformation relationship also needs to be sent to the second communication device (such as a base station), such as a translation vector and / or rotation parameters, where the rotation parameters can be in the form of a rotation matrix or a three-axis rotation angle.

[0017] Compression based on geometric projection of sensory data can reduce the dimensionality of the data obtained by projecting it onto spherical, polar, or Cartesian coordinates. For example, two dimensions of the resulting data are used as position information, and the remaining dimensions are compressed. This method can be used for all or part of the data, and can also be used for geometric structure information and / or data contained within the geometric structure, as well as for object bounding box information and / or data contained within the bounding box.

[0018] In one possible design, any perceptual data includes multidimensional data, and the first indication information further indicates location information of the plurality of perceptual data, where the location information of any perceptual data is determined based on two dimensions of data within the multidimensional data of the perceptual data. With this design, the first indication information can be used to indicate the two dimensions of the perceptual data, and only the data of the perceptual data other than the two dimensions corresponding to the location information needs to be compressed, further reducing the amount of the resulting compressed data.

[0019] In one possible design, multiple sensory data items are of the first category, and generating compressed data includes: performing quantization compression on the third-dimensional data of the multiple sensory data items to obtain the quantized compressed data, wherein the third-dimensional data of any sensory data item is the data in the multi-dimensional data of the sensory data item excluding the two-dimensional data corresponding to the position information. With this design, quantization compression can be used for compression processing, further reducing the amount of the generated compressed data.

[0020] In one possible design, the multidimensional data of any perceptual data includes θ, At least three of R, x, y, z, r, where θ represents the vertical angle in the spherical coordinate system, The third dimension of the multidimensional data, excluding the two-dimensional data corresponding to the position information, is selected based on the information entropy or value range of each dimension in the multidimensional data. This design allows the data dimension to be compressed based on information entropy or value range, which helps achieve higher compression efficiency.

[0021] In one possible design, the category of the multiple perception data is the second category, and compressed data is generated, including: determining the geometric structure parameters or geometric structure identification points of at least one geometric structure (such as a plane, a surface, a polygon, a polyhedron, a cube, a cylinder, a cone, etc.) where the multiple perception data are located based on the multiple perception data, wherein the first indication information also indicates the geometric structure where the multiple perception data are located, and the geometric structure parameters or geometric structure identification points of any geometric structure are used to determine the geometric structure, and the geometric structure identification points can be boundary points of the geometric structure, points located in the concave area and / or convex area of ​​the geometric structure, etc. For example, when the perception geometric structure is a plane, the geometric structure identification points can be three points located on the plane that are not on the same straight line, etc. Through this design, only geometric structure parameters or geometric structure identification points can be sent to obtain a greater compression gain.

[0022] In one possible design, the method further includes quantizing and compressing residuals corresponding to the plurality of sensory data to obtain quantized compressed data, wherein the residuals corresponding to the plurality of sensory data are determined based on the plurality of sensory data and a plurality of restored data corresponding to the plurality of sensory data, wherein the restored data corresponding to any sensory data is determined based on position information of the sensory data and a geometric structure within which the sensory data resides. With this design, further transmission of the residual data can meet the performance requirement of high reconstruction accuracy of the sensory data.

[0023] In one possible design, the category of the multiple perception data is the third category, and compressed data is generated, including: determining at least one bounding box corresponding to the multiple perception data, wherein the first indication information also indicates the bounding box where the multiple perception data are located; performing quantization compression on the bounding box endpoints of the multiple perception data and / or the perception data located at the bounding box endpoints to obtain quantized compressed data; or, performing quantization compression on the third-dimensional data of the multiple perception data located at any bounding box to obtain quantized compressed data, wherein the third-dimensional data of any perception data is data in the multi-dimensional data of the perception data excluding the two-dimensional data corresponding to the position information.

[0024] Exemplary: performing quantization compression on the third-dimensional data of multiple perception data located at any border to obtain quantized compressed data, including: performing quantization compression on the border endpoints of multiple perception data (such as the three-dimensional data of the border endpoints or the third-dimensional data of the border endpoints) and / or the third-dimensional data of the perception data located at the border endpoints to obtain quantized compressed data; or, performing quantization compression on the third-dimensional data of multiple sampled perception data in the multiple perception data to obtain quantized compressed data, wherein the number of the multiple sampled perception data is less than the number of the multiple perception data; or, performing quantization compression on the third-dimensional data of all perception data located at the border to obtain quantized compressed data. Through this design, the sampled perception data (feature points) can be selected for compression to obtain compressed data, and the remaining perception data can be interpolated and restored, which can further improve the compression efficiency.

[0025] In one possible design, the method further includes receiving a compression configuration from a second communication device, the compression configuration indicating one or more categories of sensory data to be transmitted, wherein the multiple sensory data categories belong to the categories of sensory data to be transmitted indicated by the compression configuration. With this design, only compressed data of the categories required by the second communication device can be transmitted, further reducing the amount of data transmitted.

[0026] In one possible design, the compression configuration further indicates compression parameters and / or compression performance corresponding to one or more categories of sensory data to be transmitted, where the compressed data satisfies the compression parameter and / or compression performance requirements. This design facilitates the second communication device to adjust the compression parameters and / or compression performance of the first communication device based on the performance requirements for the sensory data.

[0027] In one possible design, the compression configuration further indicates the transmission priorities corresponding to one or more categories of sensory data to be transmitted. The compressed data and the first indication information are sent to the second communication device based on the transmission priorities corresponding to the multiple sensory data categories. With this design, in resource-constrained scenarios, the first communication device can transmit data in layers or hierarchies based on the transmission priorities corresponding to the categories of sensory data to be transmitted, prioritizing data required by the second communication device.

[0028] In one possible design, the compressed data also includes reconstruction performance of multiple sensory data. The method further includes: receiving second indication information from a second communication device, the second indication information instructing to send residuals corresponding to the multiple sensory data, the second indication information being sent by the second communication device when the reconstruction performance does not meet a reconstruction performance threshold; and sending residuals corresponding to the multiple sensory data to the second communication device, the residuals corresponding to the multiple sensory data being determined based on the multiple sensory data and multiple restored data corresponding to the multiple sensory data, wherein the restored data corresponding to any sensory data is determined based on position information corresponding to the sensory data and the geometric structure within which the sensory data is located. With this design, residual data can be sent only when instructed by the second communication device, and performance requirements for high reconstruction accuracy of the sensory data can be met while obtaining a certain compression gain.

[0029] In a second aspect, an embodiment of the present application provides a method for transmitting perceptual data, which can be executed by a second communication device, and the method includes: receiving compressed data and first indication information from a first communication device, the first indication information indicating the categories of multiple perceptual data, and the compressed data is obtained by compressing the multiple perceptual data according to the compression method corresponding to the categories of the multiple perceptual data; generating multiple perceptual data, and the multiple perceptual data are obtained by decompressing the compressed data based on the decompression method corresponding to the categories of the multiple perceptual data, and the perceptual data of the same category in the multiple perceptual data corresponds to the same decompression method in the correspondence between the categories of different perceptual data and different decompression methods.

[0030] Exemplary: The categories of the plurality of perception data are determined according to the categories of the target objects corresponding to the plurality of perception data.

[0031] In one possible design, the correspondence between different categories of perception data and different compression methods includes: a compression method for quantizing and compressing perception data without sampling, a compression method for quantizing and compressing sampling of perception data, a compression method for compressing based on the geometric structure corresponding to the perception data, a compression method for compressing based on the border corresponding to the perception data, a compression method for compressing perception data based on geometric projection, etc. One or more of the following.

[0032] In one possible design, any perception data includes multidimensional data, and the first indication information also indicates location information of multiple perception data, wherein the location information of any perception data is determined based on two-dimensional data in the multidimensional data of the perception data; the multiple perception data are obtained by decompressing the compressed data based on the decompression method corresponding to the category of the multiple perception data and the location information of the multiple perception data.

[0033] In one possible design, the category of the multiple perception data is the first category, and the compressed data includes quantized compressed data obtained by quantizing and compressing the third-dimensional data of the multiple perception data, wherein the third-dimensional data of any perception data is data other than the two-dimensional data corresponding to the position information in the multi-dimensional data of the perception data.

[0034] In one possible design, the category of the multiple perception data is the second category, the compressed data includes geometric structure parameters or geometric structure identification points of at least one geometric structure where the multiple perception data are located, and the first indication information also indicates the geometric structure where the multiple perception data are located. The geometric structure parameters or geometric structure identification points of any geometric structure are used to determine the geometric structure.

[0035] In one possible design, the compressed data also includes quantized compressed data obtained by quantizing and compressing the residuals corresponding to multiple perception data, wherein the residuals corresponding to the multiple perception data are determined based on the multiple perception data and the multiple restored data corresponding to the multiple perception data, wherein the restored data corresponding to any perception data is determined based on the position information corresponding to the perception data and the geometric structure where the perception data is located.

[0036] In one possible design, the category of the multiple perception data is the third category, the first indication information further indicates the border where the multiple perception data are located, and the compressed data includes quantized compressed data obtained by quantizing and compressing the border endpoints of the multiple perception data and / or the perception data located at the border endpoints, or quantized compressed data obtained by quantizing and compressing the third-dimensional data of the multiple perception data located at any border indicated by the first indication information, wherein the third-dimensional data of any perception data is data in the multi-dimensional data of the perception data except the two-dimensional data corresponding to the position information.

[0037] In one possible design, the quantized compressed data obtained by quantizing and compressing the third-dimensional data of multiple perception data located at any border includes: the quantized compressed data obtained by quantizing and compressing the third-dimensional data of all perception data located at the border; or, the quantized compressed data obtained by quantizing and compressing the third-dimensional data of the border endpoints of multiple perception data and / or the perception data located at the border endpoints; or, the quantized compressed data obtained by quantizing and compressing the third-dimensional data of multiple sampled perception data among the multiple perception data, wherein the number of the multiple sampled perception data is less than the number of the multiple perception data.

[0038] In one possible design, the method further includes: sending a compression configuration to the first communication device, where the compression configuration indicates one or more categories of perception data that need to be transmitted, and the categories of multiple perception data belong to the categories of perception data that need to be transmitted indicated by the compression configuration.

[0039] In one possible design, the compression configuration further indicates compression parameters and / or compression performance corresponding to one or more categories of perceptual data that need to be transmitted, where the compressed data meets the requirements of the compression parameters and / or compression performance.

[0040] In one possible design, the compression configuration further indicates transmission priorities corresponding to one or more categories of sensory data that need to be transmitted.

[0041] In one possible design, the compressed data also includes reconstruction performance of multiple perception data, and the method also includes: when the reconstruction performance does not meet the reconstruction performance threshold, sending second indication information to the first communication device, the second indication information indicating sending residuals corresponding to the multiple perception data; receiving residuals corresponding to the multiple perception data from the first communication device, wherein the residuals corresponding to the multiple perception data are determined based on the multiple perception data and the multiple restored data corresponding to the multiple perception data, wherein the restored data corresponding to any perception data is determined based on the position information corresponding to the perception data and the geometric structure where the perception data is located.

[0042] In a third aspect, embodiments of the present application provide a communication device having the functionality to implement the method of the first or second aspect described above. The functionality may be implemented through hardware or through hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the functionality, such as an interface unit and a processing unit.

[0043] In one possible design, the apparatus may be a device, a chip, or an integrated circuit.

[0044] In one possible design, the device includes a memory and a processor, the memory is used to store instructions executed by the processor, and when the instructions are executed by the processor, the device can perform the method of the first aspect or the second aspect.

[0045] In a fourth aspect, an embodiment of the present application provides a communication device, comprising an interface circuit and a processor, wherein the processor and the interface circuit are coupled to each other. The processor implements the method of the first or second aspect described above through a logic circuit or execution instructions. The interface circuit is configured to receive signals from other communication devices outside the communication device and transmit them to the processor, or to transmit signals from the processor to other communication devices outside the communication device. It will be understood that the interface circuit may be a transceiver, a transceiver, a transceiver, or an input / output interface.

[0046] Optionally, the communication device may further include a memory for storing instructions executed by the processor, or storing input data required by the processor to execute instructions, or storing data generated after the processor executes instructions. The memory may be a physically independent unit, or may be coupled to the processor, or the processor may include the memory (i.e., the processor and memory are integrated together).

[0047] In a possible implementation, the communication device may be a device or a chip.

[0048] In a fifth aspect, an embodiment of the present application provides a communication system, which includes a first communication device and a second communication device, wherein the first communication device is used to implement the method of the first aspect above; the second communication device is used to implement the method of the second aspect above.

[0049] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program or instruction is stored. When the computer program or instruction is executed by a processor, the method of the first or second aspect mentioned above can be implemented.

[0050] In the seventh aspect, an embodiment of the present application further provides a computer program product, including a computer program or instructions, which, when executed by a processor, can implement the method of the first or second aspect above.

[0051] In the eighth aspect, an embodiment of the present application also provides a chip system, which includes a processor, the processor is used to couple with a memory, and the memory is used to store programs or instructions. When the program or instruction is executed by the processor, the method of the first or second aspect above can be implemented.

[0052] The technical effects that can be achieved in the second to eighth aspects mentioned above can refer to the technical effects that can be achieved in the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] FIG1 is a schematic diagram of the architecture of a communication system provided in an embodiment of the present application;

[0054] FIG2 is a schematic diagram of an application scenario provided by an embodiment of the present application;

[0055] FIG3 is a schematic diagram of a method for transmitting sensory data according to an embodiment of the present application;

[0056] FIG4 is a schematic diagram of three-dimensional coordinates provided in an embodiment of the present application;

[0057] FIG5 is a 2D indication diagram provided in an embodiment of the present application;

[0058] FIG6 is a schematic diagram showing a 2D indicator image indicating a category of perception data according to an embodiment of the present application;

[0059] FIG7A is a schematic diagram of the distribution probability of x values ​​of the perception data provided by an embodiment of the present application;

[0060] FIG7B is a schematic diagram of the distribution probability of the y value of the perception data provided in an embodiment of the present application;

[0061] FIG8 is a schematic diagram of residual determination provided in an embodiment of the present application;

[0062] FIG9 is a second schematic diagram of a 2D indicator diagram indicating a category of perception data provided by an embodiment of the present application;

[0063] FIG10 is a schematic diagram of the distribution of perception data provided in an embodiment of the present application;

[0064] FIG11 is a schematic diagram of sensing data sampling and interpolation restoration provided in an embodiment of the present application;

[0065] FIG12 is a third schematic diagram of a 2D indicator diagram indicating a category of perception data provided by an embodiment of the present application;

[0066] FIG13 is a second schematic diagram of a method for perceptual data transmission provided in an embodiment of the present application;

[0067] FIG14 is a layered schematic diagram provided in an embodiment of the present application;

[0068] FIG15 is a schematic diagram of layer-increase transmission provided in an embodiment of the present application;

[0069] FIG16 is a third schematic diagram of a method for transmitting perception data provided in an embodiment of the present application;

[0070] FIG17 is a schematic diagram of a first example of a simulation result of perceptual data transmission provided in an embodiment of the present application;

[0071] FIG18 is a second schematic diagram of a simulation result of perceptual data transmission provided in an embodiment of the present application;

[0072] FIG19 is a schematic diagram of a structure of a communication device according to an embodiment of the present application;

[0073] FIG20 is a second schematic diagram of the structure of the communication device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0074] The technical solutions of the embodiments of the present application can be applied to various communication systems, such as: long term evolution (LTE) system, evolved LTE (LTE-advanced, LTE-A) system, universal mobile telecommunications system (UMTS), and fifth generation (5G) mobile communication system, beyond 5G (B5G) mobile communication system, or communication system evolved after 5G (such as 6G mobile communication system). The communication system can also be a device-to-device (D2D) network, a WiFi network, a machine-to-machine (M2M) network, an Internet of Things (IoT) network, or other networks.

[0075] The architecture of the communication system used in the embodiments of the present application can be shown in Figure 1. Communication system 1000 includes a radio access network (RAN) 100 and a core network (CN) 200. Optionally, communication system 1000 may also include the Internet 300. RAN 100 includes at least one network device (such as 110a and 110b in Figure 1, collectively referred to as 110) and at least one terminal device (such as 120a-120j in Figure 1, collectively referred to as 120). RAN 100 may also include other RAN nodes, such as wireless relay devices and / or wireless backhaul devices (not shown in Figure 1). Terminal device 120 is wirelessly connected to network device 110. Network device 110 is wirelessly or wiredly connected to core network 200. The core network devices in core network 200 and network device 110 in RAN 100 may be different physical devices, or they may be the same physical device that integrates core network logical functions and radio access network logical functions.

[0076] The RAN 100 may be a cellular system related to the Third Generation Partnership Project (3GPP), such as a 4G, 5G, or an evolved system beyond 5G (e.g., a 6G mobile communication system). The RAN 100 may also be an open access network (O-RAN or ORAN), a cloud radio access network (CRAN), or a WiFi system. The RAN 100 may also be a communication system that integrates two or more of the above systems.

[0077] The apparatus provided in the embodiment of the present application can be applied to the network device 110 or to the terminal device 120. It is understood that FIG1 only shows a possible communication system architecture to which the embodiment of the present application can be applied, and in other possible scenarios, the communication system architecture may also include other devices.

[0078] The network device 110 is a node in the radio access network (RAN), which can also be called an access network device or a RAN node (or device). The network device 110 is used to help terminal devices achieve wireless access. The multiple network devices 110 in the communication system 1000 can be nodes of the same type or different types. In some scenarios, the roles of the network device 110 and the terminal device 120 are relative. For example, the network element 120i in Figure 1 can be a helicopter or a drone, which can be configured as a mobile base station. For terminal devices 120j that access the RAN 100 through the network element 120i, the network element 120i is a base station; but for the base station 110a, the network element 120i is a terminal device. The network device 110 and the terminal device 120 are sometimes referred to as communication devices. For example, the network elements 110a and 110b in Figure 1 can be understood as communication devices with base station functions, and the network elements 120a-120j can be understood as communication devices with terminal device functions.

[0079] In one possible scenario, a network device can be a base station, an evolved NodeB (eNodeB), a transmitting and receiving point (TRP), a transmitting point (TP), a next-generation NodeB (gNB), a base station in a future mobile communication system, a satellite, an access point (AP) in a WiFi system, an integrated access and backhaul (IAB) node, a mobile switching center, or a network device in a non-terrestrial network (NTN) communication system, i.e., it can be deployed on a high-altitude platform or satellite. The network device can be a macro base station (such as 110a in Figure 1), a micro base station or an indoor station (such as 110b in Figure 1), a relay node or a donor node, or a wireless controller in a CRAN scenario. The network device can also be a device that functions as a base station in device-to-device (D2D) communication, Internet of Vehicles (IoV) communication, drone communication, or machine communication. Optionally, the network device can also be a server, a wearable device, a vehicle, or an onboard device. For example, the access network device in vehicle to everything (V2X) technology may be a road side unit (RSU).

[0080] In another possible scenario, multiple network devices collaborate to assist the terminal device in achieving wireless access, and different network devices respectively implement part of the functions of the base station. For example, the network device can be a centralized unit (CU), a distributed unit (DU), a CU-control plane (CP), a CU-user plane (UP), or a radio unit (RU). The CU and DU can be set separately, or they can also be included in the same network element, such as a baseband unit (BBU). The RU can be included in a radio frequency device or a radio frequency unit, such as a remote radio unit (RRU), an active antenna unit (AAU), or a remote radio head (RRH). It can be understood that the network device can be a CU node, a DU node, or a device including a CU node and a DU node. In addition, the CU can be divided into a network device in the access network RAN, or the CU can be divided into a network device in the core network CN, which is not limited here.

[0081] In different systems, CU (or CU-CP and CU-UP), DU or RU may also have different names, but those skilled in the art can understand their meanings. For example, in the ORAN system, CU may also be called O-CU (Open CU), DU may also be called O-DU, CU-CP may also be called O-CU-CP, CU-UP may also be called O-CU-UP, and RU may also be called O-RU. For the convenience of description, this application uses CU, CU-CP, CU-UP, DU and RU as examples for description. Any unit of CU (or CU-CP, CU-UP), DU and RU in this application can be implemented by a software module, a hardware module, or a combination of a software module and a hardware module.

[0082] In the embodiments of the present application, the form of the network device is not limited. The device used to implement the function of the network device can be a network device; it can also be a device that can support the network device to implement the function, such as a chip system. The device can be installed in the network device or used in conjunction with the network device.

[0083] The terminal device 120, which may also be referred to as a terminal, user equipment (UE), mobile station (MS), or mobile terminal (MT), can be a device for providing voice or data connectivity to a user, an IoT device, or a station (STA) in a WiFi system. For example, the terminal device includes a handheld device or vehicle-mounted device with wireless connectivity. Currently, terminal devices may include: mobile phones, tablet computers, laptop computers, PDAs, mobile internet devices (MIDs), wearable devices (e.g., smart watches, smart bracelets, pedometers, smart glasses, etc.), vehicle-mounted devices (e.g., cars, bicycles, electric vehicles, airplanes, ships, trains, high-speed trains, etc.), satellite terminals, virtual reality (VR) devices, augmented reality (AR) devices, smart point-of-sale (POS) machines, customer-premises equipment (CPE), wireless terminals in industrial control, smart home devices (e.g., refrigerators, televisions, air conditioners, electric meters, etc.), intelligent robots, robotic arms, workshop equipment, wireless terminals in unmanned driving, wireless terminals in telemedicine, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, and flying devices (e.g., intelligent robots, hot air balloons, drones, airplanes), etc. Terminal devices may also be other devices with terminal functions, for example, a terminal device may also be a device that functions as a terminal in D2D communication.

[0084] The embodiments of this application do not limit the device form factor of the terminal device. The device used to implement the functions of the terminal device can be the terminal device; it can also be a device that supports the terminal device to implement the functions, such as a chip system. The device can be installed in the terminal device or used in conjunction with the terminal device. In the embodiments of this application, the chip system can be composed of a chip or include a chip and other discrete components.

[0085] Based on the communication system architecture shown in Figure 1, Figure 2 illustrates an application scenario applicable to the embodiments of the present application, including terminal devices and network devices. Terminal devices (such as mobile phones, computers, cars, airplanes, etc.) can scan the surrounding environment through sensors set on the terminal devices to obtain perception data (also known as point cloud data, imaging data, etc.), and send the compressed perception data to the network devices. The network devices can perform information fusion and environmental map construction based on the perception data reported by the terminal devices.

[0086] In addition, it should be understood that the perception data involved in the embodiments of the present application may refer to data obtained by a communication device (such as a terminal device, a vehicle-mounted device, etc.) scanning the surrounding environment through a sensor (such as a visual sensor, an electromagnetic wave sensor (or antenna), a millimeter wave sensor, etc.). Exemplary: each perception data may correspond to a point in space and may include at least one-dimensional data obtained by scanning the point. For example, one or more of the three-dimensional coordinates of the point (wherein each dimension of the coordinates may correspond to one-dimensional data), the echo signal strength, the round-trip time of the perception signal (such as an electromagnetic wave signal), etc., each of which may be one-dimensional data included in the perception data. The restored data may refer to the perception data restored based on certain information (or data), such as compressed data obtained by compressing the perception data, perception data obtained by decompressing and restoring, etc. The residual corresponding to the perception data may refer to the difference between the perception data and its restored data.

[0087] In the embodiments of this application, ordinal numbers such as "first" and "second" are used to distinguish multiple objects and are not used to define the size, content, order, timing, priority, or importance of multiple objects. For example, "first communication device" and "second communication device" do not indicate a difference in priority or importance between the two messages.

[0088] In the embodiments of the present application, the number of nouns, unless otherwise specified, means "singular noun or plural noun", that is, "one or more". "At least one" means one or more, and "plural" means two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. For example, A / B means: A or B. "At least one of the following items (individuals) or similar expressions refers to any combination of these items, including any combination of single items (individuals) or plural items (individuals). For example, at least one of a, b, or c means: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, c can be single or multiple.

[0089] This application provides a method and apparatus for transmitting sensory data. Taking into account the potential differences in importance and characteristics of different categories of sensory data, different compression methods can be used to compress different categories of sensory data, thereby increasing the flexibility of sensory data compression processing and achieving higher compression efficiency. The following describes an embodiment of this application in detail with reference to the accompanying drawings.

[0090] The perceptual data transmission method provided in the embodiment of the present application can be performed by a first communication device and a second communication device, wherein the first communication device and the second communication device are different communication devices, and the first communication device (or the second communication device) can refer to a terminal device, a component of a terminal device (such as a processor, a chip, or a chip system, etc.), or a device used in conjunction with a terminal device, and can also refer to a network device, a component of a network device (such as a processor, a chip, or a chip system, etc.), or a device used in conjunction with a network device. The first communication device can act as a transmitter of the perceptual data to compress the perceptual data, and the second communication device can act as a receiver to decompress and restore the perceptual data.

[0091] FIG3 is a schematic diagram of a method for transmitting sensory data according to an embodiment of the present application, the method comprising:

[0092] S301: The first communication device generates compressed data and first indication information, where the first indication information indicates categories of a plurality of perception data. The compressed data is obtained by compressing the plurality of perception data according to compression methods corresponding to the categories of the plurality of perception data.

[0093] Among them, the perception data belonging to the same category in the multiple perception data corresponds to the same compression method in the correspondence relationship between the categories of different perception data and different compression methods.

[0094] S302: The first communication device sends compressed data and first indication information to the second communication device. Correspondingly, the second communication device receives the compressed data and the first indication information.

[0095] In an embodiment of the present application, the first communication device can be used as a sensing device or a scanning device to scan a certain area and obtain perception data; or the first communication device can also obtain perception data from other communication devices, which is not limited in this application. For example: each perception data can correspond to a point in space, and the perception data can include at least one-dimensional data obtained by scanning the point. For example, one or more of the three-dimensional coordinates of the point (where each dimension of the coordinates can correspond to one-dimensional data), the echo signal strength, the round-trip time of the perception signal (such as an electromagnetic wave signal), etc.

[0096] 4, the three-dimensional coordinates included in (or recorded in) the sensing data can be represented by the vertical angle θ (also called the pitch angle), the horizontal angle (also known as the yaw angle) and the distance R from the point to the origin to represent the spherical coordinates Alternatively, the three-dimensional coordinates may be expressed as Cartesian coordinates (x, y, z) represented by x values ​​(horizontal coordinates), y values ​​(vertical coordinates), and z values ​​(vertical coordinates) in a Cartesian coordinate system. Alternatively, the three-dimensional coordinates may be expressed as three-dimensional coordinates in other three-dimensional coordinate systems (such as cylindrical coordinate systems). This application does not limit the specific form of the three-dimensional coordinates. It is understood that the three-dimensional coordinates in different coordinate systems can be converted to each other.

[0097] For example, the mapping algorithm from the Cartesian coordinate system to the spherical coordinate system may satisfy the following formula:

[0098] In an embodiment of the present application, taking into account the different characteristics of the data included (or recorded) in different perception data, the multiple perception data obtained can be classified based on the characteristics of a certain dimension or multi-dimensional data of the perception data (such as the range of values, etc.), and different categories of perception data can be compressed using different compression methods.

[0099] Taking the example of perception data including perception signal round-trip time, when a first communication device scans an area, the greater the distance between a target object in the area and the first communication device, the greater the perception signal round-trip time in the perception data obtained by scanning the object. Therefore, in one possible implementation, the perception data can be classified into different categories based on the perception signal round-trip time. For example, perception data with a perception signal round-trip time within round-trip time range 1 can be classified into category 1, perception data with a perception signal round-trip time within round-trip time range 2 can be classified into category 2, and perception data with a perception signal round-trip time within round-trip time range 3 can be classified into category 3.

[0100] Taking the example of perception data including three-dimensional coordinates, when the first communication device scans a certain area, the three-dimensional coordinates corresponding to multiple perception data obtained by scanning a certain object in the area are matched with the three-dimensional coordinates of multiple points on the surface of the object. Therefore, in one possible implementation, the perception data can also be divided into different categories according to the category of the target object corresponding to the perception data.

[0101] As an example: an artificial intelligence (AI) model can be trained to identify the category of target objects corresponding to perception data (such as ground, buildings, vehicles, vegetation, telephone poles, etc.). The first communication device can input the acquired multiple perception data into the AI ​​model for processing to obtain the category of the target object corresponding to each perception data as the category of the corresponding perception data.

[0102] In an embodiment of the present application, different categories of perceptual data correspond to different compression methods, for example, category 1 corresponds to compression method 1, category 2 corresponds to compression method 2, and so on. Different compression methods may refer to different compression schemes, or may indicate different compression parameters (or compression degrees). For example: a Kd tree (KDtree) compression scheme and a compression scheme for sampling and quantizing the perceptual data (unsampled perceptual data is restored based on interpolation) may be different compression methods; a compression scheme for sampling and quantizing the perceptual data with a sampling rate of 1 / 2 and a compression scheme for sampling and quantizing the perceptual data with a sampling rate of 1 / 4 may also be different compression methods.

[0103] The correspondence between different categories of perception data and different compression methods can be pre-configured in the first communication device, and can also be indicated to the first communication device by other communication devices (such as the second communication device). This application does not limit the manner in which the first communication device obtains (or determines) the correspondence between different categories of perception data and different compression methods.

[0104] It is understandable that if the data characteristics of different categories of perception data are the same or similar, for example, the three-dimensional coordinates included in the perception data conform to the distribution of the three-dimensional coordinates of points on a plane, different categories of perception data can also correspond to the same compression method.

[0105] After the first communication device obtains multiple pieces of perception data (a collection of perception data), the first communication device may compress each category of perception data using a compression method corresponding to that category to generate compressed data. After generating the compressed data, the first communication device may transmit the compressed data and first indication information indicating the category of the obtained plurality of perception data to the second communication device.

[0106] For example, a first communication device obtains 35 pieces of perception data by scanning a certain area, of which 20 pieces of perception data are classified as category 1 and 15 pieces of perception data are classified as category 2. The first communication device may use compression method 1 corresponding to category 1 (e.g., a compression scheme that compresses all perception data) to compress the 20 pieces of perception data of category 1, thereby generating compressed data corresponding to the 20 pieces of perception data of category 1; and use compression method 2 corresponding to category 2 (e.g., a compression scheme that compresses half of the sampled perception data and uses interpolation restoration for the remaining unsampled perception data) to compress the 10 pieces of perception data of category 2, thereby generating compressed data corresponding to the 10 pieces of perception data of category 2. After generating the compressed data, the first communication device may send the compressed data of the 20 pieces of perception data of category 1 and the compressed data of the 10 pieces of perception data of category 2, as well as first indication information indicating the categories of the 35 pieces of perception data obtained, to the second communication device.

[0107] In a possible implementation, the first indication information may indicate the categories of the plurality of perception data in a sequence, a bitmap, or the like. As an example, the sequence is 1, 1, 2, 3, ..., 4, where 1, 2, 3, and 4 are indexes of category 1, category 2, category 3, and category 4, respectively. The sequence may indicate that the category of the first perception data is category 1, the category of the second perception data is category 1, the category of the third perception data is category 2, the category of the fourth perception data is category 3, ..., and the category of the last perception data is category 4.

[0108] Of course, the first indication information may also indicate the category of the perception data via a two-dimensional (2D) indication diagram or other means. For example, where the perception data includes three-dimensional data (e.g., three-dimensional coordinates), the first communication device may determine location information (e.g., 2D structure location) based on the two-dimensional data (e.g., two-dimensional coordinates) of any of the acquired perception data, and use the index (or identifier) ​​corresponding to the category of the perception data as a fill value under the location information to design a 2D indication diagram.

[0109] For example, the first communication device can sense the three-dimensional coordinates of any data in After quantization, the 2D structural position of the perception data is determined, and the index (or identifier) ​​of the category corresponding to the perception data is filled in the 2D structural position of the perception data to obtain a 2D indicator map as shown in Figure 5. Each square in the 2D indicator map corresponds to a 2D structural position and represents a perception data, and the horizontal axis represents the horizontal angle. The vertical axis represents the quantized value of the vertical angle θ, and the value filled in each square represents the index (or identifier) ​​of the category corresponding to the perception data represented by the square. A filled value of 1 indicates that the perception data represented by the square corresponds to the category with index 1 (such as category 1), a filled value of 2 indicates that the perception data represented by the square corresponds to the category with index 2 (such as category 2), a filled value of 3 indicates that the perception data represented by the square corresponds to the category with index 3 (such as category 3), and a filled value of 4 indicates that the perception data represented by the square corresponds to the category with index 4 (such as category 4). A filled value of 0 can indicate that there is no perception data for the square (i.e., 2D structure position), or that the 2D structure position corresponding to the square (i.e., 2D structure position) has no corresponding information and is a hole point.

[0110] In some implementations, the first indication information may also indicate the location information (such as 2D structural location) of multiple perception data, and the location information may be used to indicate (or determine) the two-dimensional data of the multiple perception data. For example, the first indication information may indicate the categories of multiple perception data in the form of a 2D indication diagram as shown in FIG5 above, and indicate the 2D structural location of each perception data, and the 2D structural location may be used to indicate (or determine) the three-dimensional coordinates of the perception data. in

[0111] In one implementation, when the perception data includes three-dimensional data (such as three-dimensional coordinates), a compression method based on 2D indication + filling value may be considered.

[0112] The three-dimensional coordinates of multiple perception data are the three-dimensional coordinates in the spherical coordinate system For example, we can use the θ value of each perception data and The 2D structural position of each perception data is determined by a value, and the 2D structural position of each perception data is indicated by first indication information (such as a 2D indication map). The R values ​​of the plurality of perception data constitute an R value sequence, which can be quantized and compressed to obtain quantized compressed data. The quantized compressed data can refer to data obtained by compressing the quantized R value sequence.

[0113] The quantization range corresponding to the quantization may be pre-configured or determined (e.g., determined by negotiation between the first communication device and the second communication device before data compression, or pre-defined by a protocol and stored in the first communication device and the second communication device, etc.), or may be determined by the first communication device. For example, the range of R values ​​in the R value sequence to be quantized is [R min , R max ], the first communication device may determine the quantization range as [R min , R max ] and so on. It is understood that if the quantization range for compressing the perception data is determined by the first communication device, the compressed data sent by the first communication device to the second communication device may include not only the quantized compressed data obtained by quantizing and compressing the perception data, but also the quantization range. When performing quantization, a smaller quantization range, a greater number of quantization bits, and a higher quantization accuracy.

[0114] In order to further improve the compression efficiency, when the perception data includes three-dimensional data (such as three-dimensional coordinates), a compression method based on 2D indication + filling value selection can be considered.

[0115] As an example: Spherical coordinates can be transformed into Where a∈{x, y, z, r, R…} is one or more of the following: θ represents the vertical angle in the spherical coordinate system, θ represents the horizontal angle in the spherical coordinate system, x represents the abscissa in the Cartesian coordinate system, y represents the ordinate in the Cartesian coordinate system, z represents the vertical coordinate in the Cartesian coordinate system, r represents the projection distance on the horizontal plane (xOy plane) in the Cartesian coordinate system, and R represents the distance to the origin in the spherical coordinate system. x, y, z, r, R, etc. can be determined based on the three-dimensional coordinates included (or recorded) in the perception data, and will not be described in detail. It is understood that a can also be the projection distance on the xOz plane in the Cartesian coordinate system, the projection distance on the yOz plane in the Cartesian coordinate system, etc.

[0116] The value of a can be selected based on one or more information entropies or value ranges corresponding to the x-value sequence, y-value sequence, z-value sequence, r-value sequence, R-value sequence, etc. of the plurality of perception data.

[0117] As an example: After quantizing the values ​​in the x-value sequence, y-value sequence, z-value sequence, r-value sequence, and R-value sequence corresponding to multiple perception data using the same quantization performance (such as the same number of quantization levels), the information entropy H(a) corresponding to each sequence can be calculated, and the value corresponding to the sequence with the smallest H(a) (such as x, y, z, r, R...) can be selected as a. Among them, H(x) can be calculated using the following formula:

[0118] H(a) = -∑P(ai)log(2,P(ai)), (i = 1, 2, ..n), where a can be one or more of x, y, z, r, R, etc., n is the number of perception data, i is the index of the median in the sequence, and ai is a symbol in the quantization space (such as the quantization value of the i-th a in the sequence).

[0119] As an example, based on the range g of values ​​in the x-value sequence, y-value sequence, z-value sequence, r-value sequence, and R-value sequence corresponding to the plurality of perception data, the value corresponding to the sequence with the smallest g (e.g., x, y, z, r, R, etc.) can be selected as a. The range g of values ​​in any sequence can be determined based on the difference between the maximum value max and the minimum value min in the sequence.

[0120] In some implementations, for multiple ground perception data (such as the first category: category 1), when compression is performed using a compression scheme based on fill value selection, since the fluctuation of the z values ​​of the multiple ground perception data is the smallest and the range of the z value sequence is the smallest, the three-dimensional data (such as three-dimensional coordinates) of each perception data can be deformed into The z-value sequences of the plurality of perception data are quantized and compressed to obtain a quantized compression range and quantized compression data S1.

[0121] For multiple perception data of the ground, the compressed data corresponding to the ground sent by the first communication device to the second communication device may include a quantization range (optional, such as when the first communication device determines the quantization range according to the value range of the sequence to be quantized, etc.), a quantization step (such as the number of quantization bits, optional, such as when it is determined solely by the first communication device and is not known to the second communication device), and quantized compressed data. It may also include an index for selecting the a value (such as the index of z). Of course, the selection of the a value can also be directly associated with the category, such as when the category is ground, the a value is selected as z.

[0122] 6, after receiving the first indication information (2D indication map) and compressed data from the first communication device, the second communication device can determine multiple perception data of the ground category according to the filling value 1, and obtain the corresponding values ​​of each perception data according to the first indication information (2D indication map). According to the quantization range and quantization step, the quantized compressed data is dequantized to obtain the z of each perception data and restore the three-dimensional coordinates of each perception point.

[0123] The exterior of buildings, etc., typically exists in the form of some geometric structure (e.g., plane, curved surface, etc.). When scanning a building, etc., the 3D coordinate distribution of the multiple perception data obtained typically conforms to the 3D coordinate distribution of multiple points on this geometric structure. Therefore, if the perception data includes 3D coordinates, for multiple perception data of buildings (e.g., the second category: Category 2), a compression method based on 2D indications and fitting of geometric structures (e.g., plane, curved surface, polygon, polyhedron, cube, cylinder, cone, etc.) can be considered.

[0124] Taking the geometric structure as a plane as an example, the first communication device can also extract the fitting perception data based on the distribution probability of one or more of the x values ​​(or y values ​​or z values) of the multiple perception data of the building, and fit at least one plane according to a continuous segment of x values ​​(or y values ​​or z values) with a larger probability. The plane parameters or plane identification points of at least one plane are obtained. Among them, any plane F can be represented by the formula F = Ax + By + Cz + D = 0, and the parameters of plane F can be (A, B, C, D). The plane identification points are points that can determine the plane. For example, the plane identification points can be three points on the plane that are not on the same straight line. The plane can be uniquely determined by the three-dimensional coordinates of the three points.

[0125] Referring to the schematic diagram of the distribution probability of the x values ​​of multiple perception data shown in FIG7A , the horizontal axis in FIG7A represents the x values ​​of the multiple perception data (i.e., the horizontal coordinate), and the vertical axis represents the distribution probability (or proportion) of the perception data. As can be seen from FIG7A , the x values ​​for which the distribution probability of the perception data is greater than the distribution probability threshold (taking the distribution probability threshold as 0.05 as an example) are -19 and 12 respectively. It can be determined that the x value distribution intervals corresponding to -19 and 12, respectively, are [-19.25, -18.75] and [11.75, 12.25], can be used for plane fitting. The first communication device can fit the perception plane F1 by the least squares method based on the multiple perception data with x values ​​located at [-19.25, -18.75]; and fit the perception plane F2 by the least squares method based on the multiple perception data with x values ​​located at [11.75, 12.25].

[0126] Referring to FIG7B , which shows a schematic diagram of the distribution probability of the y values ​​of multiple perception data, where the horizontal axis represents the y values ​​(i.e., the vertical axis) of the perception data, and the vertical axis represents the distribution probability (or proportion) of the perception data, FIG7B shows that the y values ​​for which the distribution probability of the perception data is greater than the distribution probability threshold (taking the distribution probability threshold as 0.05 as an example) are -77, -75, and 47, respectively. It can be determined that the y value distribution intervals corresponding to -77, -75, and 47, respectively, are [-77.25, -76.75], [-75.25, -74.75], and [46.75, 47.25], can be used for plane fitting. The first communication device can fit the perception plane F3 by the least squares method based on multiple perception data with y values ​​located at [-77.25, -76.75]; fit the perception plane F4 by the least squares method based on multiple perception data with y values ​​located at [-75.25, -74.75]; and fit the perception plane F5 by the least squares method based on multiple perception data with y values ​​located at [46.75, 47.25].

[0127] The horizontal axis of the 2D indicator represents the horizontal angle As shown in FIG8 , each perception data corresponds to a 2D structure position in the 2D indicator map, and the vertical angle θ and horizontal angle θ of the perception data can be obtained according to the 2D structure position. Thus, a straight line I starting from the origin is determined: x / m=y / n=z / k=t, where m, n, and k are the angles θ and The components of the spatial vector on the x-, y-, and z-axes are determined, with t being the variable. Combined with the plane parameters (A, B, C, D) of the plane where the perception data resides, the intersection point P′2 of line I and the plane Ax+By+Cz+D=0 can be determined as the restored data (also called the restoration point) for the perception data.

[0128] By further calculating the difference between the projection distances r′2 and r2 of the restored data P′2 and the perceived data (i.e., the real point) P2 on the horizontal plane (or vertical plane or x-axis, or y-axis or z-axis), the residual Δr corresponding to the perceived data can be obtained.

[0129] In a possible implementation, as shown in FIG9 , for multiple perception data of a building, the first communication device may send first indication information (such as a 2D indication diagram) and compressed data including plane parameters of at least one plane where the multiple perception data are located to the second communication device. The category of the corresponding multiple perception data may be indicated as a building by multiple 2D structure positions with a fill value of 2 in the 2D indication diagram. After receiving the first indication information and the plane parameters of at least one plane, the second communication device may calculate the corresponding angle θ and the corresponding angle θ of each 2D structure position in the 2D indication diagram. Determine the straight line I in the 3D space: x / m=y / n=z / k=t; combined with the plane where the perception data is located: Ax+By+Cz+D=0, the intersection point P′2: (x, y, z) of the straight line I and the plane can be calculated as the restored data (i.e., the restoration point) of the perception data.

[0130] It is understandable that if multiple perception data of a building correspond to multiple planes, and if the perception data corresponding to the multiple planes are discontinuous on the 2D indication diagram, each part of the continuous perception data can correspond to a plane. Of course, if there are situations where the perception data corresponding to multiple planes are continuous on the 2D indication diagram, the planes corresponding to the perception data can also be indicated on the 2D indication diagram by different filling values ​​(such as the index of the filled plane), such as indicating different planes by filling values ​​2 and 5; and the planes corresponding to the perception data can also be indicated by filling the value corresponding to the category + the index of the plane, such as indicating that the category corresponding to the perception data is category 2 by 2A and 2B, and the planes corresponding to the perception data are plane A and plane B respectively, etc. This application does not limit the specific manner in which the first indication information (such as the 2D indication diagram) indicates the planes where the multiple perception data are located.

[0131] Furthermore, the first communication device may also quantize and compress the residuals Δr corresponding to the plurality of sensory data, i.e., the residual Δr sequence formed by the residuals Δr corresponding to the plurality of sensory data, to obtain quantized compressed data S2, and transmit the quantization compression range (optional, such as the quantization range may be included when the first communication device determines the quantization range based on the value range of the sequence to be quantized, etc.), the quantization step size (such as the number of quantization bits, optional, such as the quantization step size may be included when the first communication device determines the quantization range and the second communication device does not know the quantization range), and the quantized compressed data S2 to the second communication device. Optionally, the compressed data may also include an index for selecting the residual Δr (e.g., indicating a residual determined based on a projection distance on a horizontal plane, a vertical plane, an x-axis, a y-axis, or a z-axis). Of course, the selection of the residual Δr may be associated with a category. For example, if the category is ground, the residual Δr is fixedly selected as a residual determined based on a projection distance on a horizontal plane, etc.

[0132] After the second communication device recovers the restoration point P′2 of the sensed data, it calculates the projection distance r′2 of P′2, +Δr to restore the true value, and combines θ and The value can be solved to find out that the perception data refers to the real (x, y, z).

[0133] It can be understood that the above r′2+Δr can also be expressed in the form of x+Δr, y+Δr, z+Δr, R+Δr, etc. For example, when the residual Δr is determined based on the difference in projection distance between the restoration point P′2 and the true position (that is, the true point) P2 of the perception data on the x-axis, it can be expressed as x+Δr; when it is determined based on the difference in projection distance between the restoration point P′2 and the true position (that is, the true point) P2 of the perception data on the y-axis, it can be expressed as y+Δr; when it is determined based on the difference in projection distance between the restoration point P′2 and the true position (that is, the true point) P2 of the perception data on the z-axis, it can be expressed as z+Δr, etc.; when it is determined based on the difference in distance from the restoration point P′2 and the true position (that is, the true point) P2 of the perception data to the origin, it can be expressed as R+Δr, etc. Where x, y, z, and R can be the coordinate of the restoration point P′2 on the x-axis (or the projection distance on the x-axis), the coordinate of the restoration point P′2 on the y-axis (or the projection distance on the y-axis), the coordinate of the restoration point P′2 on the z-axis (or the projection distance on the z-axis), and the distance from the restoration point P′2 to the origin, respectively.

[0134] In one possible implementation, for multiple perception data points representing independent objects, a compression scheme can be designed based on the bounding box of the object. For example, if each vehicle has its own bounding box, a compression scheme based on 2D indications and sampled perception data (feature points) within the bounding box can be considered.

[0135] Referring to the perception data schematic diagram shown in Figure 10, the perception data corresponding to each vehicle can be obtained by clustering the perception data classified as vehicles, that is, the perception data corresponding to the borders of each vehicle, where each circle in Figure 10 can represent a cluster of perception data of a vehicle.

[0136] For example, the multiple perception data corresponding to the border can be compressed based on the filling value selection method. For example, the three-dimensional coordinates of the multiple perception data corresponding to the border are deformed into The θ value of each perception data and The 2D structural position of each perception data is determined by a value, and the 2D structural position of each perception data is indicated by first indication information (such as a 2D indication map). The a values ​​of the multiple perception data constitute an a value sequence, which can be quantized and compressed to obtain quantized compressed data. Where a∈{x, y, z, r, R…} is one or more of the following: For the value of a, refer to the description of the compression method based on the 2D indication + padding value selection above, and will not be repeated here.

[0137] In the above implementation, the third-dimensional data of all the perception data located in the border may be quantized and compressed to obtain quantized compressed data.

[0138] In some implementations, in order to further reduce the amount of data, when the third-dimensional data (such as a) of multiple perception data of the border are quantized and compressed to obtain quantized compressed data, only the border endpoints of the multiple perception data (such as the three-dimensional data of the border endpoints or the third-dimensional data of the border endpoints) and / or the third-dimensional data of the perception data located at the border endpoints (such as a) can be quantized and compressed to obtain the quantization range and quantized compressed data.

[0139] Alternatively, the third-dimensional data (such as a) of multiple sampled perception data (ie, feature points) in the multiple perception data are quantized and compressed to obtain quantized compressed data, wherein the number of the multiple sampled perception data is less than the number of the multiple perception data.

[0140] For multiple perception data of a car, the compressed data corresponding to the car sent by the first communication device to the second communication device may include a quantization range (optional, such as when the quantization range is determined by the first communication device according to the value range of the sequence to be quantized, etc.), a quantization step (such as the number of quantization bits, optional, such as when it is determined solely by the first communication device and is not known to the second communication device), quantized compressed data, and may also include an index for selecting the a value (such as the index of z, x, etc.). Of course, the selection of the a value can be associated with the category, for example, if the category is vehicle, the a value is z.

[0141] As shown in Figure 11, when the target shape is relatively fixed and consists of many surfaces / lines, some feature data (also called feature points) can be extracted and the whole image can be restored based on the feature data. For example, for the first segment of the perception data at continuous positions, the a sequence of odd positions is sent, and the a of even positions is restored based on interpolation. i =(a i -1+a i +1) / 2, where i is the position or index of the perception data to be restored.

[0142] 12, after receiving the first indication information (2D indication map) and compressed data from the first communication device, the second communication device can determine multiple perception data of the vehicle category according to the filling value 3, and obtain the corresponding values ​​of each perception data according to the first indication information (2D indication map). According to the quantization range and the quantization step size, the quantized compressed data is dequantized and restored by interpolation to obtain the z of each perception data, and restore the three-dimensional data (such as the three-dimensional coordinates) of each perception data.

[0143] It can be understood that if multiple perception data of a vehicle correspond to multiple borders, the borders corresponding to the perception data can be indicated on the 2D indication map by different filling values ​​(such as the index of the filled border), such as indicating different borders by filling values ​​3 and 6; the borders corresponding to the perception data can also be indicated by the index corresponding to the category + the index of the border, such as using 3A and 3B to indicate that the category corresponding to the perception data is category 3, and the faces corresponding to the perception data are border A and border B, etc. This application does not limit the specific manner in which the first indication information (such as a 2D indication map) indicates the borders where multiple perception data are located.

[0144] Of course, the compression scheme based on border design can also include quantizing and compressing the border endpoints of multiple perception data and / or the perception data located at the border endpoints to obtain quantized compressed data. The receiving end can restore multiple perception data based on the border endpoints and / or the perception data located at the border endpoints through interpolation and other methods.

[0145] S303: The second communication device generates a plurality of perception data, where the plurality of perception data are obtained by decompressing the compressed data based on decompression methods corresponding to categories of the plurality of perception data.

[0146] Among them, the perception data of the same category in the multiple perception data corresponds to the same decompression method in the correspondence relationship between the categories of different perception data and different decompression methods.

[0147] In the embodiment of the present application, different categories of sensory data correspond to different decompression methods. The decompression method corresponding to any category of sensory data can be used to decompress the compressed data of the sensory data of that category. After receiving the compressed data and first indication information from the first communication device, the first communication device can decompress the compressed data using the corresponding decompression method based on the category of the sensory data indicated by the first indication information to obtain multiple sensory data.

[0148] Exemplarily: the compressed data includes compressed data of 20 perception data of category 1 obtained by compressing 20 perception data of category 1 according to compression method 1 corresponding to category 1, and compressed data of 15 perception data of category 2 obtained by compressing 15 perception data of category 2 according to compression method 2 corresponding to category 2. The second communication device can decompress the compressed data of the 20 perception data of category 1 according to decompression method 1 corresponding to category 1 to obtain 20 perception data of category 1; and decompress the compressed data of the 15 perception data of category 2 according to decompression method 2 corresponding to category 2 to obtain 15 perception data of category 2.

[0149] It is understandable that the present application does not limit the correspondence between different categories of perception data and different compression methods, and the correspondence between different categories of perception data and different compression methods can be flexibly configured or adjusted according to the transmission requirements of the perception data. The compression methods that can be used may include but are not limited to the above-mentioned compression method based on 2D indication + filling value, the compression method based on 2D indication + geometric structure fitting, the compression method based on 2D indication + perception data (or feature points) within the frame, and the compression method of quantizing and compressing the perception data without sampling, the compression method of quantizing and compressing the perception data sampling, the compression method based on the geometric structure corresponding to the perception data, the compression method based on the frame corresponding to the perception data, the compression method based on geometric projection of the perception data, etc. One or more of the above.

[0150] Exemplary: A compression method of performing quantization compression on the perceptual data without sampling, wherein the quantization compression without sampling may mean performing quantization compression on all data.

[0151] A compression method that performs quantization compression on the sampling of perception data, where sampling can refer to extracting part of the data, such as extracting part of the data based on data downsampling / extracting feature data, and sampling quantization compression quantizes and compresses the extracted part of the data.

[0152] A compression method based on the geometric structure corresponding to the perception data, wherein the geometric structure can be a plane, a curved surface, a polygon, a polyhedron, a cube, a cylinder, a cone, or other structures. Specific compression methods can be used as follows: (1) replacing the original data with geometric structure information, the compressed data includes the geometric structure information, for example, if a cylinder is replaced, the compressed data may include its center point, radius, and height; if a polygon is replaced, the compressed data may include the vertices of the polygon, etc.; or (2) data compression based on geometric structure information, the compressed data may include geometric structure information and / or perception data information contained in the geometric structure, for example, all / part of the sampled perception data within the geometric structure scanning area. In addition, it is understandable that the compressed data as above can also be entropy coded, or quantized and entropy coded to further reduce the amount of data.

[0153] A compression method based on the bounding box corresponding to the perception data, wherein the bounding box can be the bounding box of the target (car / person / object, etc.). The specific compression can be carried out in the following ways: (1) the original data is replaced by the bounding box information, and the compressed data includes the bounding box information, and the bounding box information can be the vertices of the bounding box, or the center position and length and width of the bounding box; or (2) data compression is performed based on the bounding box information, and the compressed data includes the bounding box information and / or the data contained in the bounding box, such as all target data / partial sampled target data within the bounding box (such as target feature points, contour points, etc.). It can also be understood that the compressed data as above can also be entropy encoded, or quantized, entropy encoded, to further reduce the amount of data, wherein the quantized boundary can be the range of the data or the range of the bounding box.

[0154] Compression based on geometric projection of sensory data can reduce the dimensionality of the data obtained by projecting it onto spherical, polar, or Cartesian coordinates. For example, two dimensions of the resulting data are used as position information, and the remaining dimensions are compressed. This method can be used for all or part of the data, and can also be used for geometric structure information and / or data contained within the geometric structure, as well as for object bounding box information and / or data contained within the bounding box.

[0155] In some implementations, the second communication device may further send a compression configuration to the first communication device, and the first communication device may further generate and send compressed data according to the compression configuration.

[0156] FIG13 is a second method for sensing data transmission provided by this application, which includes:

[0157] S1301: The second communication device sends a compression configuration to the first communication device, and correspondingly, the first communication device receives the compression configuration.

[0158] The compression configuration may indicate one or more categories of perception data that need to be transmitted, and may also indicate the transmission priorities corresponding to the one or more categories of perception data that need to be transmitted.

[0159] As an example, the compression configuration may indicate one or more categories of perception data to be transmitted by category index (or identifier). For example, if the index of category 1 is 1, the index of category 2 is 2, and the index of category 3 is 3, and the second communication device requires perception data of category 2 and category 3, the first indication information may carry the index 2 of category 2 and the index 3 of category 3 to instruct the first communication device to compress and report the perception data of category 2 and category 3, respectively.

[0160] In addition, the second communication device can also sort the indexes of the categories that need to be transmitted according to the corresponding transmission priority (or importance), such as sorting them in descending order according to the transmission priority (or importance), indicating the transmission priority corresponding to each category that needs to be transmitted.

[0161] It is understood that the compression configuration can also indicate the categories that need to be transmitted through a bitmap or other means. For example, there are 10 categories of perception data, and the 10 categories correspond one-to-one to the 10 bits of the bitmap. A bit in the bitmap that is 1 indicates that the category corresponding to the bit needs to be transmitted, and a bit that is 0 indicates that the category corresponding to the bit does not need to be transmitted, etc.

[0162] In some implementations, the compression configuration may further indicate information such as compression parameters and / or compression performance corresponding to one or more categories of sensory data to be transmitted.

[0163] Taking the example where the compression configuration further indicates the compression performance (or performance accuracy) corresponding to one or more categories of perceptual data to be transmitted, the indexes of different compression performance (or performance accuracy) can be as shown in Table 1. The compression performance corresponding to one or more categories of perceptual data to be transmitted can be determined by one or more performance indexes carried by the compression performance field of the compression configuration. For example: the categories of perceptual data to be transmitted are category 2 and category 3, the compression performance field of the compression configuration includes indexes 1 and 2, and the first communication device that receives the compression configuration information can determine that the compression performance corresponding to category 2 is e-04 and the compression performance corresponding to category 2 is e-05.

[0164] Table 1

[0165] In the embodiment of the present application, compression accuracy (or performance accuracy) may refer to information such as the mean-square error (MSE) between data restored based on compressed data and the original data.

[0166] Compression parameters can include information such as the number of quantization bits. The number of quantization bits refers to the number of binary digits required to distinguish all quantization levels. For example, if there are 8 quantization levels, they can be distinguished using a 3-bit binary number. The quantization precision can be expressed as the ratio of the range of values ​​to be quantized (i.e., the quantization range) to the number of quantization levels. If quantization compression is used, the greater the number of quantization levels, the higher the accuracy of the data recovered after decompression. Therefore, the greater the number of quantization bits, the higher the compression precision (or performance accuracy). In some implementations, the compression precision can be mapped to the compression parameters (such as the number of quantization bits), and the number of quantization bits used can be determined based on the compression precision.

[0167] It should be understood that in the embodiment of the present application, sending the compression configuration by the second communication device to the first communication device is an optional operation. If the second communication device does not send the compression configuration to the first communication device, the first communication device can adopt the default compression configuration, such as the default transmission priority of each category, etc.

[0168] S1302: The first communication device generates compressed data and first indication information.

[0169] Specifically, the first communication device may generate compressed data and first indication information based on the compression configuration. For example, the first communication device may compress one or more categories of sensory data to be transmitted as indicated by the compression configuration, and for each category of sensory data to be transmitted, perform compression according to a compression method corresponding to the category to obtain compressed data.

[0170] If the compression configuration also indicates that there are one or more categories of perception data that need to be transmitted and corresponding compression parameters and / or compression performance information, the first communication device compresses each category of perception data that needs to be transmitted according to the compression method corresponding to the category. When the compressed data is obtained, the compression parameters used (such as the number of quantization bits) can be determined based on the compression parameters and / or compression performance information corresponding to the category.

[0171] It is understandable that the compression parameters (such as the number of quantization bits) and other information used by the compression method corresponding to each category can also be configured by default.

[0172] S1303: The first communication device sends compressed data and first indication information to the second communication device. Correspondingly, the second communication device receives the compressed data and the first indication information.

[0173] For example, the sensory data to be transmitted includes category 1 (ground), category 2 (building), category 3 (vehicle), and category 4 (other), where the first indication information (such as a 2D indication map) can indicate the category of each sensory data. The compressed data may include quantized compression parameters and quantized compressed data. For example: for category 1 (ground), a compression method based on 2D indication + filling value selection is adopted, which may include a value index (which may be default), a value boundary min and max (quantization range) + a value quantization data (which may be a value quantization data compressed by entropy coding, etc.); for category 2 (building), a compression method based on 2D indication + surface fitting is adopted, which may include surface parameters, and may also include residual quantization range + residual quantization data; for category 3 (vehicle), a compression method based on 2D indication + sampling perception data (feature points) compression within the frame is adopted, which may include a value index (which may be default), a value boundary min and max (quantization range) + a value quantization data (which may be a value quantization data compressed by entropy coding, etc.); for category 4 (other), a compression method based on 2D indication + filling value selection is adopted, which may include a value index (which may be default), a value boundary min and max (quantization range) + a value quantization data (which may be a value quantization data compressed by entropy coding, etc.);

[0174] In addition, if the first communication device expands the correspondence between other categories and compression methods, it can also report the expanded category information (e.g., index) and the corresponding compression method index to the second communication device. For example, in Table 2, compression method indices 0, 1, 2, 3, 4, and 5 represent different compression methods. If the first communication device expands the correspondence between a certain category and a certain compression method in Table 2, it can report the category information (e.g., index) and the corresponding compression method index to the second communication device.

[0175] Table 2

[0176] S1304: The second communication device generates a plurality of perception data.

[0177] The second communication device can restore multiple perception data based on the received first indication information and compressed data.

[0178] In some implementations, when resources are limited, etc., it is also possible to prioritize the transmission of compressed data of categories with higher transmission priority (or importance) based on the transmission priority (or importance) corresponding to each category, and to increase the transmission of compressed data of categories with lower transmission priority (or importance); or different transmission resources can be allocated to different categories based on the transmission priority (or importance) corresponding to different categories, and so on.

[0179] As an example, you can configure the transmission priority (or importance) for each category based on different scenarios. For example, in an obstacle avoidance scenario, the transmission priority for moving targets (such as vehicles) is higher than the transmission priority for buildings and higher than the transmission priority for the ground, giving priority to the transmission of compressed data for moving targets. In a map building scenario, the transmission priority for buildings is higher than the transmission priority for the ground and higher than the transmission priority for moving targets (such as vehicles), giving priority to the transmission of the surrounding background.

[0180] It can be understood that the transmission priority (or importance) corresponding to each category can be pre-configured in the first communication device, or indicated by the second communication device, such as the second communication device sending it through compression configuration, etc. This application does not limit this.

[0181] For example, referring to the layered diagram shown in FIG14 , category 1, category 2, category 3, …, category m can be divided into layer 1, layer 2, and layer n according to the transmission priorities corresponding to category 1, category 2, category 3, …, category m, respectively, wherein the transmission priority corresponding to any category in layer 1 is greater than the transmission priority corresponding to any category in layer 2, and similarly, the transmission priority corresponding to any category in layer 2 is greater than the transmission priority corresponding to any category in layer 3, …, the transmission priority corresponding to any category in layer n-1 is greater than the transmission priority corresponding to any category in layer n. Referring to the incremental layer transmission diagram shown in FIG15 , the first communication device can first send compressed data corresponding to each category in layer 1 (base layer) to the second communication device. If incremental layer feedback is received from the second communication device, the compressed data corresponding to each category in the next layer will continue to be sent until the data transmission is completed or the incremental layer feedback is no longer received from the second communication device. It is understandable that the first communication device may also send first indication information (such as a 2D indication map) to the second communication device to indicate the category of each acquired perception data before or after sending the compressed data corresponding to each category in layer 1 (base layer) to the second communication device. Of course, the first indication information (such as a 2D indication map) may also be sent simultaneously with the compressed data corresponding to each category in layer 1 (base layer), and this application does not limit this.

[0182] In some implementations, for the compression method based on 2D indication + plane fitting, an incremental layer transmission method may also be used to reduce the amount of data.

[0183] As shown in Figure 16, the first communication device can compress various types of perception data to be transmitted according to the compression configuration to obtain various types of compressed data. For perception data of categories such as buildings that exist in the form of a plane, the corresponding compressed data can include only plane parameters or plane identification points, and can also include reconstruction performance, wherein the reconstruction performance can be determined based on the error between the restoration point of the perception data determined based on the first indication information (such as a 2D indication map) and the plane parameters (or plane identification points) and the real perception data. If the reconstruction performance does not meet the reconstruction performance threshold T, the second communication device feeds back a second indication information (such as continuing to send residual signaling (or resources)) indicating the sending of the residuals corresponding to the multiple perception data to the second communication device. If the reconstruction performance information meets the reconstruction performance threshold T, the interaction can be terminated. If the first communication device receives the second indication information (such as continuing to send residual signaling), it can send the residual data determined based on the real perception data and the restored data to the second communication device, and the second communication device restores the data based on the plane compression with high precision based on the residual data.

[0184] Referring to Figure 17 , one of the schematic diagrams of perceptual data transmission simulation results is a schematic diagram showing the bit rate (Rate) and mean square error (MSE) corresponding to two frames of perceptual data samples of two intersecting streets from the perspective 360 ​​of the first communication device, taking the three-dimensional data included in the perceptual data as three-dimensional coordinates as an example. The Draco scheme uses KDtree compression with a quantization bit rate of 8-16. The projection (Proj) scheme uses a 2D indicator map (two-dimensional data) + R distance sequence (third-dimensional data), where the R distance sequence is quantized and compressed using the LZMA compression algorithm (with a quantization bit rate of 8-13).

[0185] The scheme of the present application can also be called a classification-based perceptual data compression scheme (Proj class-based compression). Taking the four categories as an example, ground: 2D indicator map (two-dimensional data) + z value sequence (third-dimensional data), + where the z value sequence is quantized and compressed by the LZMA compression algorithm (quantization bits are 8-13); building: 2D indicator map (two-dimensional data) + surface-based compression (low-precision parameters or surface identification points, high-precision parameters or surface identification points + residuals); vehicle: 2D indicator map (two-dimensional data) + r value sequence of odd positions in the 2D indicator map as feature points, where the r value sequence is quantized and compressed by the LZMA compression algorithm (quantization bits are 8-13), and the r value of the even position is obtained by linear interpolation based on the r value restored at the odd position; others (telephone poles, vegetation, etc.), 2D indicator map (two-dimensional data) + r value sequence (third-dimensional data), where the r value sequence is quantized and compressed by the LZMA compression algorithm (quantization bits are 8-13). As can be seen from FIG17 , when the present application solution is used to transmit perception data, the data volume (bit rate) is significantly reduced.

[0186] Referring to the schematic diagram of the perceptual data transmission simulation results shown in Figure 18, which is a schematic diagram of the corresponding bit rate (Rate) and mean square error (MSE) when averaging 24 frames of perceptual data samples of two intersecting streets from the perspective 120 of the first communication device, it can be seen that when the present application solution is used to transmit perceptual data, the data volume (bit rate) also decreases significantly.

[0187] The following describes the communication device provided in an embodiment of the present application. Please refer to Figure 19, which is a schematic diagram of the structure of the communication device in an embodiment of the present application. The communication device may include units or modules corresponding to all or part of the steps in the above-mentioned method embodiment, and may be used to execute the steps performed by the first communication device or the second communication device in the above-mentioned method embodiment. For details, please refer to the relevant description in the above-mentioned method embodiment.

[0188] As shown in Figure 19, communication device 1900 includes a processing unit 1910 and an interface unit 1920. Processing unit 1910 may be a processor or processing circuit, and interface unit 1920 may be a transceiver unit or an input / output interface. Communication device 1900 may be used to implement the steps performed by the first communication device or the second communication device in the above-described embodiments.

[0189] When the communication device 1900 is used to implement the steps performed by the first communication device in the above embodiment:

[0190] The processing unit 1910 is used to generate compressed data and first indication information, where the first indication information indicates the categories of multiple perception data. The compressed data is obtained by compressing multiple perception data according to the compression methods corresponding to the categories of the multiple perception data, wherein the perception data of the same category in the multiple perception data corresponds to the same compression method in the correspondence between the categories of different perception data and different compression methods; the interface unit 1920 is used to send the compressed data and the first indication information to the second communication device.

[0191] Exemplary: The categories of the plurality of perception data are determined according to the categories of the target objects corresponding to the plurality of perception data.

[0192] In one possible design, the correspondence between different categories of perception data and different compression methods includes: a compression method for quantizing and compressing perception data without sampling, a compression method for quantizing and compressing sampling of perception data, a compression method for compressing based on the geometric structure corresponding to the perception data, a compression method for compressing based on the border corresponding to the perception data, a compression method for compressing perception data based on geometric projection, etc. One or more of the following.

[0193] In a possible design, any perception data includes multi-dimensional data, and the first indication information further indicates position information of the plurality of perception data, wherein the position information of any perception data is determined based on two-dimensional data in the multi-dimensional data of the perception data.

[0194] In one possible design, multiple perception data are of the first category. When the processing unit 1910 generates compressed data, it is specifically used to quantize and compress the third-dimensional data of the multiple perception data to obtain a quantization range and quantized compressed data, wherein the third-dimensional data of any perception data is the data in the multi-dimensional data of the perception data excluding the two-dimensional data corresponding to the position information.

[0195] In one possible design, the multidimensional data of any perceptual data includes θ, At least three of R, x, y, z, r, where θ represents the vertical angle in the spherical coordinate system, represents the horizontal angle in the spherical coordinate system, x represents the abscissa in the Cartesian coordinate system, y represents the ordinate in the Cartesian coordinate system, z represents the vertical coordinate in the Cartesian coordinate system, r represents the projection distance on the horizontal plane in the Cartesian coordinate system, and R represents the distance to the origin in the spherical coordinate system. The third dimension of the multiple perception data, excluding the two-dimensional data corresponding to the position information, is selected based on the information entropy or value range of each dimension of the multidimensional data excluding the two-dimensional data.

[0196] In one possible design, the category of the multiple perception data is the second category. When the processing unit 1910 generates compressed data, it is specifically used to determine the geometric structure parameters or geometric structure identification points of at least one geometric structure where the multiple perception data are located based on the multiple perception data, wherein the first indication information also indicates the geometric structure where the multiple perception data are located, and the geometric structure parameters or geometric structure identification points of any geometric structure are used to determine the geometric structure.

[0197] In one possible design, the processing unit 1910 is also used to quantize and compress the residuals corresponding to multiple perception data to obtain quantized compressed data, wherein the residuals corresponding to the multiple perception data are determined based on the multiple perception data and the multiple restored data corresponding to the multiple perception data, wherein the restored data corresponding to any perception data is determined based on the position information of the perception data and the geometric structure where the perception data is located.

[0198] In one possible design, the category of multiple perception data is the third category. When the processing unit 1910 generates compressed data, it is specifically used to determine at least one border corresponding to the multiple perception data, wherein the first indication information also indicates the border where the multiple perception data are located; the border endpoints of the multiple perception data and / or the perception data located at the border endpoints are quantized and compressed to obtain quantized compressed data; or, the third-dimensional data of the multiple perception data located at any border is quantized and compressed to obtain quantized compressed data, wherein the third-dimensional data of any perception data is data in the multi-dimensional data of the perception data except the two-dimensional data corresponding to the position information.

[0199] In one possible design, the processing unit 1910 performs quantization compression on the third-dimensional data of multiple perception data located at any border to obtain quantized compressed data, and is specifically used to perform quantization compression on the third-dimensional data of all perception data located at the border to obtain quantized compressed data; or, performs quantization compression on the border endpoints of multiple perception data and / or the third-dimensional data of the perception data located at the border endpoints to obtain quantized compressed data; or, performs quantization compression on the third-dimensional data of multiple sampled perception data in the multiple perception data to obtain quantized compressed data, wherein the number of the multiple sampled perception data is less than the number of the multiple perception data.

[0200] In one possible design, the interface unit 1920 is further used to receive a compression configuration from the second communication device, where the compression configuration indicates one or more categories of perception data that need to be transmitted, and multiple categories of perception data belong to the categories of perception data that need to be transmitted indicated by the compression configuration.

[0201] In one possible design, the compression configuration further indicates compression parameters and / or compression performance corresponding to one or more categories of perceptual data that need to be transmitted, where the compressed data meets the requirements of the compression parameters and / or compression performance.

[0202] In one possible design, the compression configuration also indicates the transmission priorities corresponding to one or more categories of perception data that need to be transmitted, and the compressed data and the first indication information are sent to the second communication device according to the transmission priorities corresponding to the categories of multiple perception data.

[0203] In one possible design, the compressed data also includes reconstruction performance of multiple perception data. The interface unit 1920 is further used to receive second indication information from the second communication device, the second indication information indicating the sending of residuals corresponding to multiple perception data, and the second indication information is sent by the second communication device when the reconstruction performance does not meet the reconstruction performance threshold; the residuals corresponding to the multiple perception data are sent to the second communication device, and the residuals corresponding to the multiple perception data are determined based on the multiple perception data and the multiple restored data corresponding to the multiple perception data, wherein the restored data corresponding to any perception data is determined based on the position information corresponding to the perception data and the geometric structure where the perception data is located.

[0204] When the communication device 1900 is used to implement the steps performed by the second communication device in the above embodiment:

[0205] The interface unit 1920 is used to receive compressed data and first indication information from the first communication device, where the first indication information indicates categories of multiple perception data, and the compressed data is obtained by compressing the multiple perception data according to the compression method corresponding to the categories of the multiple perception data; the processing unit 1910 is used to generate multiple perception data, and the multiple perception data are obtained by decompressing the compressed data based on the decompression method corresponding to the categories of the multiple perception data. The perception data of the same category in the multiple perception data corresponds to the same decompression method in the correspondence between the categories of different perception data and different decompression methods.

[0206] Exemplary: The categories of the plurality of perception data are determined according to the categories of the target objects corresponding to the plurality of perception data.

[0207] In one possible design, the correspondence between different categories of perception data and different compression methods includes: a compression method for quantizing and compressing perception data without sampling, a compression method for quantizing and compressing sampling of perception data, a compression method for compressing based on the geometric structure corresponding to the perception data, a compression method for compressing based on the border corresponding to the perception data, a compression method for compressing perception data based on geometric projection, etc. One or more of the following.

[0208] In one possible design, any perception data includes multidimensional data, and the first indication information also indicates location information of multiple perception data, wherein the location information of any perception data is determined based on two-dimensional data in the multidimensional data of the perception data; the multiple perception data are obtained by decompressing the compressed data based on the decompression method corresponding to the category of the multiple perception data and the location information of the multiple perception data.

[0209] In one possible design, the category of the multiple perception data is the first category, and the compressed data includes quantized compressed data obtained by quantizing and compressing the third-dimensional data of the multiple perception data, wherein the third-dimensional data of any perception data is data other than the two-dimensional data corresponding to the position information in the multi-dimensional data of the perception data.

[0210] In one possible design, the category of the multiple perception data is the second category, the compressed data includes geometric structure parameters or geometric structure identification points of at least one geometric structure where the multiple perception data are located, and the first indication information also indicates the geometric structure where the multiple perception data are located. The geometric structure parameters or geometric structure identification points of any geometric structure are used to determine the geometric structure.

[0211] In one possible design, the compressed data also includes quantized compressed data obtained by quantizing and compressing the residuals corresponding to multiple perception data, wherein the residuals corresponding to the multiple perception data are determined based on the multiple perception data and the multiple restored data corresponding to the multiple perception data, and the restored data corresponding to any perception data is determined based on the position information corresponding to the perception data and the geometric structure where the perception data is located.

[0212] In one possible design, the category of the multiple perception data is the third category, the first indication information further indicates the border where the multiple perception data are located, and the compressed data includes quantized compressed data obtained by quantizing and compressing the border endpoints of the multiple perception data and / or the perception data located at the border endpoints, or quantized compressed data obtained by quantizing and compressing the third-dimensional data of the multiple perception data located at any border indicated by the first indication information, wherein the third-dimensional data of any perception data is data in the multi-dimensional data of the perception data except the two-dimensional data corresponding to the position information.

[0213] In one possible design, the quantized compressed data obtained by quantizing and compressing the third-dimensional data of multiple perception data located at any border includes: the quantized compressed data obtained by quantizing and compressing the third-dimensional data of all perception data located at the border; or, the quantized compressed data obtained by quantizing and compressing the border endpoints and / or the third-dimensional data located at the border endpoints of multiple perception data; or, the quantized compressed data obtained by quantizing and compressing the third-dimensional data of multiple sampled perception data in the multiple perception data, wherein the number of the multiple sampled perception data is less than the number of the multiple perception data.

[0214] In one possible design, the interface unit 1920 is further used to send a compression configuration to the first communication device, where the compression configuration indicates one or more categories of perception data that need to be transmitted, and multiple categories of perception data belong to the categories of perception data that need to be transmitted indicated by the compression configuration.

[0215] In one possible design, the compression configuration further indicates compression parameters and / or compression performance corresponding to one or more categories of perceptual data that need to be transmitted, where the compressed data meets the requirements of the compression parameters and / or compression performance.

[0216] In one possible design, the compression configuration further indicates transmission priorities corresponding to one or more categories of sensory data that need to be transmitted.

[0217] In one possible design, the compressed data also includes reconstruction performance of multiple perception data. The interface unit 1920 is further used to send second indication information to the first communication device when the reconstruction performance does not meet the reconstruction performance threshold, and the second indication information indicates sending residuals corresponding to multiple perception data; receiving residuals corresponding to multiple perception data from the first communication device, wherein the residuals corresponding to the multiple perception data are determined based on the multiple perception data and the multiple restored data corresponding to the multiple perception data, and the restored data corresponding to any perception data is determined based on the position information corresponding to the perception data and the geometric structure where the perception data is located.

[0218] As shown in Figure 20, the present application also provides a communication device 2000, which includes a processor 2010 and may also include a communication interface 2020. The processor 2010 and the communication interface 2020 are coupled to each other. It is understandable that the communication interface 2020 can be a transceiver, an input / output interface, an input interface, an output interface, an interface circuit, etc. Optionally, the communication device 2000 may also include a memory 2030 for storing instructions executed by the processor 2010 or storing input data required by the processor 2010 to execute instructions or storing data generated after the processor 2010 executes instructions. The memory 2030 may be a physically independent unit, or may be coupled to the processor 2010, or the processor 2010 may include the memory 2030.

[0219] When the communication device 2000 is used to implement the steps performed by the first communication device and the second communication device in the above embodiments, the processor 2010 can be used to implement the functions of the above processing unit 1910, and the communication interface 2020 can be used to implement the functions of the above interface unit 1920.

[0220] It is understood that the processor in the embodiments of the present application may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), logic circuits, field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0221] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory, a register, a hard disk, a mobile hard disk, a CD-ROM or any other form of storage medium well known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and the storage medium can be located in an ASIC. In addition, the ASIC can be located in a network device or a terminal device. Of course, the processor and the storage medium can also be present in a network device or a terminal device as discrete components.

[0222] In the above embodiments, all or part of the embodiments can be implemented using software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are performed in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device, or other programmable device. The computer program or instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer program or instructions can be transmitted from one network device, terminal, computer, server, or data center to another network device, terminal, computer, server, or data center via wired or wireless means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; it can also be an optical medium, such as a digital video disk; or it can be a semiconductor medium, such as a solid-state drive. The computer-readable storage medium may be a volatile or nonvolatile storage medium, or may include both volatile and nonvolatile types of storage media.

[0223] In the various embodiments of the present application, unless otherwise specified or there is any logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced by each other. The technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships.

[0224] Furthermore, it should be understood that in the embodiments of this application, the word "exemplary" is used to indicate an example, illustration, or description. Any embodiment or design described in this application as "exemplary" should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of the word "exemplary" is intended to present concepts in a concrete manner.

[0225] It is understood that the various numbers used in the embodiments of this application are merely for ease of description and are not intended to limit the scope of the embodiments of this application. The order of the sequence numbers of the above-mentioned processes does not necessarily imply a specific order of execution; the order of execution of the processes should be determined by their functions and inherent logic.

Claims

1. A method for transmitting perceptual data, characterized in that: Applied to a first communication device, comprising: Generate compressed data and first indication information, wherein the first indication information indicates categories of a plurality of perception data, the compressed data is obtained by compressing the plurality of perception data according to compression methods corresponding to the categories of the plurality of perception data, wherein perception data of the same category in the plurality of perception data corresponds to the same compression method in a correspondence relationship between categories of different perception data and different compression methods; The compressed data and the first indication information are sent to the second communication device.

2. The method according to claim 1, characterized in that The categories of the plurality of perception data are determined according to the categories of the target objects corresponding to the plurality of perception data.

3. The method according to claim 1 or 2, characterized in that The correspondence between the categories of different perception data and different compression methods includes at least one of the following compression methods: A compression method that quantizes and compresses sensory data without sampling; A compression method for quantizing and compressing the perceptual data samples; Compression based on the geometric structure corresponding to the perceived data; A compression method based on compressing the bounding box corresponding to the perceived data; Compression method for perceptual data based on geometric projection.

4. The method according to claim 1 or 2, characterized in that: Any of the perception data includes multi-dimensional data, and the first indication information further indicates position information of the multiple perception data, wherein the position information of any of the perception data is determined based on two-dimensional data in the multi-dimensional data of the perception data.

5. The method according to claim 4, characterized in that The plurality of perception data are of the first category, and the generating of compressed data comprises: The third-dimensional data of the plurality of perception data are quantized and compressed to obtain quantized compressed data, wherein the third-dimensional data of any of the perception data is data other than the two-dimensional data corresponding to the position information in the multi-dimensional data of the perception data.

6. The method according to claim 5, characterized in that Any of the multi-dimensional data of the perception data includes θ, At least three of R, x, y, z, r, where θ represents the vertical angle in the spherical coordinate system, represents the horizontal angle in the spherical coordinate system, R represents the distance to the origin in the spherical coordinate system, x represents the horizontal coordinate in the Cartesian coordinate system, y represents the vertical coordinate in the Cartesian coordinate system, z represents the vertical coordinate in the Cartesian coordinate system, and r represents the projection distance on the horizontal plane in the Cartesian coordinate system. The third-dimensional data in the multiple perception data except the two-dimensional data corresponding to the position information is selected according to the information entropy or value range of each dimensional data in the multidimensional data except the two-dimensional data.

7. The method according to claim 4, characterized in that The category of the plurality of perception data is the second category, and the generating of compressed data includes: Based on the multiple perception data, determine geometric structure parameters or geometric structure identification points of at least one geometric structure where the multiple perception data are located, wherein the first indication information also indicates the geometric structure where the multiple perception data are located, and the geometric structure parameters or geometric structure identification points of any one of the geometric structures are used to determine the geometric structure.

8. The method according to claim 7, characterized in that The method further comprises: The residuals corresponding to the multiple perception data are quantized and compressed to obtain quantized compressed data, wherein the residuals corresponding to the multiple perception data are determined according to the multiple perception data and multiple restored data corresponding to the multiple perception data, wherein the restored data corresponding to any one of the perception data is determined according to the position information of the perception data and the geometric structure where the perception data is located.

9. The method according to claim 4, characterized in that The category of the plurality of perception data is the third category, and the generating of compressed data comprises: Determine at least one border corresponding to the plurality of perception data, wherein the first indication information further indicates the border in which the plurality of perception data are located; Quantizing and compressing the border endpoints of the plurality of perception data and / or the perception data located at the border endpoints to obtain quantized compressed data; or, The third-dimensional data of the plurality of perception data located in any of the borders are quantized and compressed to obtain quantized compressed data, wherein the third-dimensional data of any of the perception data is data other than the two-dimensional data corresponding to the position information in the multi-dimensional data of the perception data.

10. The method according to claim 9, characterized in that The third dimension data of the plurality of perception data located at any of the borders is quantized and compressed to obtain quantized compressed data, including: Quantizing and compressing the third-dimensional data of all the perception data located in the border to obtain quantized compressed data; or, Quantizing and compressing the border endpoints of the plurality of perception data and / or the third dimension data of the perception data located at the border endpoints to obtain quantized compressed data; or, The third dimension data of multiple sampled perception data among the multiple perception data are quantized and compressed to obtain quantized compressed data, wherein the number of the multiple sampled perception data is less than the number of the multiple perception data.

11. The method according to any one of claims 1 to 10, characterized in that The method further comprises: Receive a compression configuration from the second communication device, the compression configuration indicating one or more categories of perception data that need to be transmitted, and compression parameters and / or compression performance corresponding to the one or more categories of perception data that need to be transmitted, wherein the multiple categories of perception data belong to the categories of perception data that need to be transmitted indicated by the compression configuration, and the compressed data meets the requirements of the compression parameters and / or compression performance.

12. The method according to claim 11, characterized in that The compression configuration also indicates the transmission priorities corresponding to the one or more categories of perception data that need to be transmitted, and the compressed data and the first indication information are sent to the second communication device according to the transmission priorities corresponding to the categories of the multiple perception data.

13. The method according to claim 7, characterized in that The compressed data also includes reconstruction performance of the plurality of perception data, and the method further includes: receiving second indication information from the second communication device, where the second indication information indicates to send residuals corresponding to the plurality of perception data, and the second indication information is sent by the second communication device when the reconstruction performance does not meet the reconstruction performance threshold; The residuals corresponding to the multiple perception data are sent to the second communication device, and the residuals corresponding to the multiple perception data are determined based on the multiple perception data and the multiple restored data corresponding to the multiple perception data, wherein the restored data corresponding to any one of the perception data is determined based on the location information of the perception data and the geometric structure where the perception data is located.

14. A method for transmitting perceptual data, characterized in that: Applied to a second communication device, comprising: receiving compressed data and first indication information from a first communication device, wherein the first indication information indicates categories of a plurality of perception data, and the compressed data is obtained by compressing the plurality of perception data according to compression methods corresponding to the categories of the plurality of perception data; The multiple perception data are generated, and the multiple perception data are obtained by decompressing the compressed data based on a decompression method corresponding to the category of the multiple perception data. The perception data of the same category in the multiple perception data corresponds to the same decompression method in the correspondence between the categories of different perception data and different decompression methods.

15. The method according to claim 14, characterized in that The categories of the plurality of perception data are determined according to the categories of the target objects corresponding to the plurality of perception data.

16. The method according to claim 14 or 15, characterized in that The correspondence between the categories of different perception data and different compression methods includes at least one of the following compression methods: A compression method that quantizes and compresses sensory data without sampling; A compression method for quantizing and compressing the perceptual data samples; Compression based on the geometric structure corresponding to the perceived data; A compression method based on compressing the bounding box corresponding to the perceived data; Compression method for perceptual data based on geometric projection.

17. The method according to claim 14 or 15, characterized in that Any of the perception data includes multi-dimensional data, and the first indication information further indicates position information of the plurality of perception data, wherein the position information of any of the perception data is determined according to two-dimensional data in the multi-dimensional data of the perception data; The multiple perception data are obtained by decompressing the compressed data based on the decompression methods corresponding to the categories of the multiple perception data and the location information of the multiple perception data.

18. The method according to claim 17, characterized in that The category of the multiple perception data is the first category, and the compressed data includes quantized compressed data obtained by quantizing and compressing the third dimensional data of the multiple perception data, wherein the third dimensional data of any of the perception data is data in the multidimensional data of the perception data except the two-dimensional data corresponding to the position information.

19. The method according to claim 17, characterized in that The category of the multiple perception data is the second category, the compressed data includes geometric structure parameters or geometric structure identification points of at least one geometric structure where the multiple perception data are located, and the first indication information also indicates the geometric structure where the multiple perception data are located, and the geometric structure parameters or geometric structure identification points of any one of the geometric structures are used to determine the geometric structure.

20. The method of claim 19, wherein: The compressed data also includes quantized compressed data obtained by quantizing and compressing the residuals corresponding to the multiple perception data, wherein the residuals corresponding to the multiple perception data are determined based on the multiple perception data and multiple restored data corresponding to the multiple perception data, wherein the restored data corresponding to any one of the perception data is determined based on the position information of the perception data and the geometric structure where the perception data is located.

21. The method of claim 17, wherein: The category of the multiple perception data is the third category, the first indication information also indicates the border where the multiple perception data are located, and the compressed data includes quantized compressed data obtained by quantizing and compressing the border endpoints of the multiple perception data and / or the perception data located at the border endpoints, or quantized compressed data obtained by quantizing and compressing the third dimensional data of the multiple perception data located at any border indicated by the first indication information, wherein the third dimensional data of any of the perception data is data in the multidimensional data of the perception data except the two-dimensional data corresponding to the position information.

22. The method according to claim 21, characterized in that The quantized compressed data obtained by quantizing and compressing the third-dimensional data of the plurality of perception data located in any of the borders includes: Quantized compressed data obtained by quantizing and compressing the third-dimensional data of all the perception data located in the border; or, Quantized compressed data obtained by quantizing and compressing the border endpoints of the plurality of perception data and / or the third dimension data of the perception data located at the border endpoints; or, The quantized compressed data is obtained by quantizing and compressing the third dimension data of a plurality of sampled perceptual data among the plurality of perceptual data, wherein the number of the plurality of sampled perceptual data is less than the number of the plurality of perceptual data.

23. The method according to any one of claims 14 to 22, characterized in that The method further comprises: A compression configuration is sent to the first communication device, wherein the compression configuration indicates one or more categories of perception data that need to be transmitted, and compression parameters and / or compression performance corresponding to the one or more categories of perception data that need to be transmitted, wherein the multiple categories of perception data belong to the categories of perception data that need to be transmitted indicated by the compression configuration, and the compressed data meets the requirements of the compression parameters and / or compression performance.

24. The method of claim 23, wherein: The compression configuration further indicates transmission priorities corresponding to the one or more categories of the perception data to be transmitted.

25. The method of claim 19, wherein: The compressed data also includes reconstruction performance of the plurality of perception data, and the method further includes: When the reconstruction performance does not meet the reconstruction performance threshold, sending second indication information to the first communication device, where the second indication information indicates sending residuals corresponding to the plurality of perception data; Receive residuals corresponding to the multiple perception data from the first communication device, wherein the residuals corresponding to the multiple perception data are determined based on the multiple perception data and multiple restored data corresponding to the multiple perception data, and the restored data corresponding to any one of the perception data is determined based on location information of the perception data and a geometric structure where the perception data is located.

26. A communication device, characterized in that: The method comprises a module or a unit for executing the method as claimed in any one of claims 1 to 25.

27. A communication device, characterized in that: It includes a processor and an interface circuit, wherein the interface circuit is used to receive signals from other communication devices outside the communication device and transmit them to the processor, or send signals from the processor to other communication devices outside the communication device, and the processor is used to implement the method as described in any one of claims 1-25 through a logic circuit or execution instructions.

28. A computer program product, characterized in that The method comprises a computer program or an instruction, and when the computer program or the instruction is executed by a processor, the method according to any one of claims 1 to 25 is implemented.

29. A computer-readable storage medium, characterized in that: The storage medium stores a computer program or instruction. When the computer program or instruction is executed by a processor, the method according to any one of claims 1 to 25 is implemented.

30. A communication system, characterized in that: The communication system comprises a first communication device and a second communication device; The first communication device is used to implement the method according to any one of claims 1 to 13; The second communication device is used to implement the method as described in any one of claims 14-25.

Images