Data transmission method, electronic equipment and computer readable storage medium

Abstract

The embodiment of the invention provides a data transmission method, electronic equipment and a computer readable storage medium, and the method comprises the steps: determining correction data of a first model based on first point cloud data and the first model; the first point cloud data is real point cloud data of the first moving object set at a first moment; the first moving object set comprises a plurality of moving objects; the first model is used for predicting point cloud data of each moving object in the first moving object set; correcting the first model based on the correction data to obtain a corrected first model; obtaining compressed data corresponding to the first point cloud data based on the first point cloud data and the corrected first model; and sending the correction data of the first model and the compressed data corresponding to the first point cloud data. By implementing the application, the transmission overhead during point cloud data interaction can be effectively reduced.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a data transmission method, an electronic device, and a computer-readable storage medium. Background Art

[0002] Point cloud data is an important native data for future communication perception scenarios, and is suitable for scenarios such as imaging, environment reconstruction, and target detection. In order to perform subsequent tasks such as multi-view observation, motion detection, and behavior prediction, multi-frame time-series point cloud data perceived by different terminals at multiple locations and angles must be sent to the base station for information fusion.

[0003] In order to save the air interface resources occupied by the interaction of multi-frame joint perception point cloud data, the collected point cloud data can be compressed to reduce the amount of data that needs to be transmitted between the terminal and the base station when the point cloud data is interacting. For example, the point cloud data generated by objects that have changed in the previous and next frames (such as due to movement and change in perspective) can be compressed by ordinary differential equation (ODE) modeling. However, ODE modeling is to establish a model for each moving object in the perception scene separately, so each moving object has a corresponding ODE model correction that needs to be sent to the base station. The more moving objects there are in the perception scene, the more corrections to the ODE model need to be transmitted, and each moving object also has a set of compressed point cloud data that needs to be sent. In this way, the terminal still has a large transmission overhead when interacting with the base station for point cloud. How to reduce the transmission overhead when interacting with point cloud data has become one of the problems that need to be solved urgently. Summary of the invention

[0004] The embodiments of the present application provide a data transmission method, an electronic device, and a computer-readable storage medium, which can effectively reduce the transmission overhead during point cloud data interaction.

[0005] In a first aspect, the present application provides a data transmission method, applied to a terminal device, the method comprising: determining corrected data of a first model based on first point cloud data and a first model; the first point cloud data is real point cloud data of a first moving object set at a first moment; the number of moving objects included in the first moving object set is multiple; the first model is used to predict the point cloud data of each moving object in the first moving object set; correcting the first model based on the corrected data to obtain a corrected first model; obtaining compressed data corresponding to the first point cloud data based on the first point cloud data and the corrected first model; and sending the corrected data of the first model and the compressed data corresponding to the first point cloud data.

[0006] In the method described in the first aspect, the first model can be used to predict the point cloud data of each moving object in the first moving object set, so the first model is used to model the first moving object set as a whole. Furthermore, when obtaining the correction amount of the model and compressing the point cloud data, it is not necessary to determine the model correction amount of each moving object separately and obtain a set of corresponding compressed data for each moving object separately, but by determining the correction amount of the first model, the correction amount of the first model is used to represent the model correction amount corresponding to the first moving object set, and the first point cloud data is uniformly processed based on the corrected first model to obtain compressed data corresponding to the first moving object set. Accordingly, sending the correction amount of the first model and the compressed data of the first moving object set during the point cloud data exchange can effectively reduce the number of correction amounts and compressed data to be sent, thereby reducing the transmission overhead during the point cloud data exchange. Secondly, this method of modeling the first moving object set as a whole can effectively improve the processing efficiency, and enable the obtained first model to utilize the correlation between the various moving objects in the first moving object set to obtain more accurate prediction results.

[0007] In a possible implementation, the above-mentioned method of determining the corrected data of the first model based on the first point cloud data and the first model includes: based on the first model and the second point cloud data, initially predicting the point cloud data of the first moving object set at the first moment to obtain third point cloud data, the second point cloud data being the reconstructed point cloud data of the first moving object set at the second moment, the second moment being the moment before the first moment; and obtaining the corrected data of the first model based on the error between the first point cloud data and the third point cloud data.

[0008] In this manner, based on the error between the first point cloud data and the third point cloud data, correction data of the first model can be obtained, so that the first model can be corrected, making the corrected first model more accurate in predicting point cloud data.

[0009] In one possible implementation, the correction data of the first model includes the correction amount of the first coefficient matrix; the first coefficient matrix is ​​used to indicate the change of the point cloud data of the first moving object set within the first time period, and the first time period is the time period between the first moment and the second moment.

[0010] In this method, the first coefficient matrix is ​​used to indicate the change of point cloud data of the first moving object set within the first time period, which is equivalent to that the prediction of the first model on the point cloud data of the first moving object set can be represented by the first coefficient matrix. Therefore, by sending the correction amount of the first coefficient matrix, the device that receives the correction amount of the first coefficient matrix can obtain the corrected first model by correcting the first coefficient matrix.

[0011] In one possible implementation, the correction data of the first model includes the correction amount of the second coefficient matrix and the correction amount of the third coefficient matrix; the second coefficient matrix and the third coefficient matrix are obtained by low-rank decomposition of the first coefficient matrix, and the first coefficient matrix is ​​used to indicate the change of point cloud data of the first moving object set within a first time period, and the first time period is the time period between the first moment and the second moment.

[0012] In this manner, the first coefficient matrix is ​​subjected to low-rank decomposition to obtain a second coefficient matrix and a third coefficient matrix having smaller dimensions than the first coefficient matrix. Accordingly, when sending the correction data, the correction amount of the second coefficient matrix and the correction amount of the third coefficient matrix are sent, which can effectively reduce the transmission overhead of the correction data.

[0013] In one possible implementation, the correction data of the first model includes the correction amount of the second coefficient matrix, the second coefficient matrix is ​​obtained by low-rank decomposition of the first coefficient matrix when the third coefficient matrix is ​​the initial matrix and the third coefficient matrix remains unchanged, the first coefficient matrix is ​​used to indicate the change of point cloud data of the first moving object set within a first time period, and the first time period is the time period between the first moment and the second moment.

[0014] In this manner, when the first coefficient matrix is ​​subjected to low-rank decomposition, the third coefficient matrix can be kept unchanged, so when obtaining the correction data, only the correction amount of the second coefficient matrix needs to be obtained, and the correction amount of the third coefficient matrix does not need to be obtained. Accordingly, when sending the correction data, the transmission overhead of the correction data can be effectively reduced by only sending the correction amount of the coefficient matrix after low-rank decomposition.

[0015] In one possible implementation, the dimension of the first coefficient matrix is ​​M×M, M is the number of moving objects included in the first moving object set, the dimension of the second coefficient matrix is ​​M×X, and the dimension of the third coefficient matrix is ​​X×M, M is a positive integer greater than 3, and X is a positive integer greater than or equal to 3 and less than or equal to M.

[0016] In this way, when the first coefficient matrix is ​​decomposed into the second coefficient matrix and the third coefficient matrix in a low-rank manner, the minimum value of X in the dimension of the second coefficient matrix and the third coefficient matrix is ​​3. The minimum value of X can ensure that when the low-rank decomposition reduces the dimension of the coefficient matrix, the changes in the point cloud data contained in any of the three coordinate dimensions (i.e., the x-axis, the y-axis, and the z-axis) are not lost.

[0017] In a possible implementation, the method of obtaining compressed data corresponding to the first point cloud data based on the first point cloud data and the corrected first model includes: re-predicting the point cloud data of the first moving object set at the first moment based on the corrected first model and the second point cloud data to obtain fourth point cloud data; and compressing the first point cloud data based on the fourth point cloud data to obtain compressed data.

[0018] In this method, the fourth point cloud data can be predicted again based on the corrected first model, and then the first point cloud data can be compressed according to the fourth point cloud data to obtain compressed data, which is the compressed data corresponding to the first moving object set as a whole.

[0019] In a possible implementation, the compressed data includes encoding information of an index set and remaining point information; wherein the index set is a first index set or a second index set, the first index set includes indexes corresponding to point cloud data in the first point cloud data whose errors with associated point cloud data are less than a first error threshold, and the associated point cloud data are point cloud data in the fourth point cloud data that are associated with the point cloud data in the first point cloud data; the second index set is the complement of the first index set in a preset index set; the preset index set is the index set corresponding to the fourth point cloud data; the remaining point information is information obtained by encoding point cloud data in the first point cloud data except for the point cloud data corresponding to the first index set.

[0020] In this manner, the first point cloud data can be divided into two parts, wherein the first part is the point cloud data whose error is less than the first error threshold, and the second part is the remaining part of the first point cloud data except the first part. Therefore, the first part of the first point cloud data can be indicated by sending an index set instead of directly sending the first part, and the second part can be represented by the remaining point information by sending the remaining point information (i.e., the remaining point information is directly encoded by the second part).

[0021] In a possible implementation, the compressed data further includes first indication information, where the first indication information is used to indicate that the index set is a set requiring the smallest number of bits to be transmitted between the first index set and the second index set.

[0022] In this manner, the first indication information may also be sent, and when sending the index set, the set requiring the smallest number of transmission bits between the first index set and the second index set is sent, thereby further reducing the transmission overhead during point cloud data interaction.

[0023] In a possible implementation, the compressed data includes encoding information of an index set, difference information and remaining point information; wherein the index set includes two of a first index set, a third index set and a fourth index set, the first index set includes indexes corresponding to point cloud data in the first point cloud data whose errors with associated point cloud data are less than a first error threshold, the third index set includes indexes corresponding to point cloud data in the first point cloud data whose errors with associated point cloud data are greater than or equal to the first error threshold and less than a second error threshold, and the associated point cloud data are point cloud data in the fourth point cloud data that are associated with point cloud data in the first point cloud data; the fourth index set is a set of indexes in a preset index set other than the first index set and the third index set, and the preset index set is an index set corresponding to the fourth point cloud data; the remaining point information is information obtained by encoding point cloud data in the first point cloud data other than point cloud data corresponding to the first index set and the third index set; the difference information is information obtained by encoding the error corresponding to the third index set, and the error corresponding to the third index set is the error between the point cloud data corresponding to the third index set in the first point cloud data and the associated point cloud data.

[0024] In this method, the first point cloud data can be divided into three parts, wherein the first part is the point cloud data with an error less than the first error threshold, the second part is the point cloud data with an error greater than or equal to the first error threshold and less than the second error threshold, and the third part is the rest of the first point cloud data except the first and second parts. Therefore, the first part of the first point cloud data can be indicated by an index set without directly sending the first part; the second part of the first point cloud data can be indicated by an index set and difference information without directly sending the second part; and the third part can be represented by the remaining point information (that is, the third part is directly encoded to obtain the remaining point information). Based on this method, the point cloud data corresponding to the part of the remaining point information with small error can be indicated by the difference information and the index set, and the remaining point information that needs to be sent will not be too much, which can further reduce the transmission overhead during the interaction of point cloud data.

[0025] In a possible implementation, the compressed data further includes second indication information, where the second indication information is used to indicate that the index set includes two sets of the first index set, the third index set, and the fourth index set that require a smaller number of bits to be transmitted.

[0026] In this manner, a second indication message may also be sent, and when sending index sets, two sets requiring a smaller number of bits to be transmitted among the first index set, the second index set and the third index set are sent, thereby further reducing the transmission overhead during point cloud data interaction.

[0027] In a possible implementation, before determining the corrected data of the first model based on the first point cloud data and the first model, the method also includes: acquiring reconstructed point cloud data at a second moment; acquiring real point cloud data collected at a first moment; acquiring the first point cloud data from the real point cloud data at the first moment, the first motion object set being the intersection of a motion object set corresponding to the reconstructed point cloud data at the second moment and a motion object set corresponding to the real point cloud data at the first moment; and sending third indication information, the third indication information being used to indicate the first motion object set.

[0028] In this method, the first point cloud data and the first moving object set can be determined from the real point cloud data at the first moment, and indication information of the first moving object set can be sent, so that the device that receives the indication information of the first moving object set can determine the first moving object set, and determine the second point cloud data from the reconstructed point cloud data at the second moment.

[0029] In a possible implementation, the method also includes: obtaining selection conditions for the encoding method of compressed data; if the type of the selection conditions is the first type, judging whether to obtain compressed data corresponding to the first point cloud data based on the first point cloud data, the fourth point cloud data and the first threshold value; the first threshold value matches the selection conditions.

[0030] Based on this method, it can be determined whether to obtain compressed data corresponding to the first point cloud data based on the first point cloud data and the corrected first model according to the predicted deviation of the point cloud data. For example, when the predicted deviation is small, the compressed data corresponding to the first point cloud data obtained based on the first point cloud data and the corrected first model is an effective compression method (which can greatly reduce the data overhead of transmission), and compression can be performed based on this effective compression method; when the predicted deviation is large, when the compressed data corresponding to the first point cloud data is obtained based on the first point cloud data and the corrected first model, the data volume before and after compression is not much different, and the point cloud data can be interacted based on the existing point cloud compression method.

[0031] In a possible implementation, the method also includes: obtaining selection conditions for the encoding method of the compressed data; if the type of the selection conditions is the second type, judging whether to obtain compressed data corresponding to the first point cloud data based on the first point cloud data and the corrected first model based on the coefficient matrix of the first model, the coefficient matrix of the corrected first model and the second threshold; the second threshold matches the selection conditions.

[0032] Based on this method, it can be determined whether to obtain compressed data corresponding to the first point cloud data based on the first point cloud data and the corrected first model according to the changes before and after the correction of the first model. For example, when the changes before and after the correction are small, the compressed data corresponding to the first point cloud data obtained based on the first point cloud data and the corrected first model is an effective compression method (which can greatly reduce the data overhead of transmission), and compression can be performed based on this effective compression method. When the changes before and after the correction are large, when the compressed data corresponding to the first point cloud data is obtained based on the first point cloud data and the corrected first model, the data volume before and after compression is not much different, and the point cloud data can be interacted based on the existing point cloud compression method.

[0033] In a possible implementation, before determining the corrected data of the first model based on the first point cloud data and the first model, the method further includes: sending configuration information; wherein the configuration information includes one or more of the configuration information of compression method, coefficient matrix, coefficient matrix dimension, encoding method selection conditions, pre-encoding index matching method, compression parameters, random seed, time step, and quantization method.

[0034] Based on this method, the terminal device can configure various configuration information involved in point cloud data interaction.

[0035] In a second aspect, the present application provides a data transmission method, applied to a base station, the method comprising: receiving corrected data of a first model and compressed data corresponding to first point cloud data; the first point cloud data is the real point cloud data of a first moving object set at a first moment; the number of moving objects included in the first moving object set is multiple; the first model is used to predict the point cloud data of each moving object in the first moving object set; based on the corrected data, the first model is corrected to obtain a corrected first model; based on the corrected first model, the compressed data is decompressed to obtain the reconstructed point cloud data of the first moving object set at the first moment.

[0036] In the method described in the second aspect, the first model can be used to predict the point cloud data of each moving object in the first moving object set, so the first model models the first moving object set as a whole. Furthermore, the model correction corresponding to the first moving object set can be represented by the correction amount of the received first model, without receiving the model correction amount of each moving object. At the same time, the compressed data corresponding to the first moving object set obtained after the first point cloud data is uniformly processed can be received, without receiving a set of compressed data corresponding to each moving object. This method can effectively reduce the number of correction amounts and compressed data to be received, thereby reducing the transmission overhead during the interaction of point cloud data. Secondly, this method of modeling the first moving object set as a whole can improve the processing efficiency of model correction and decompression, and because the first model can utilize the correlation between the movements of each moving object in the first moving object set, a more accurate reconstructed point cloud result of the first moving object set at the first moment can be obtained during decompression.

[0037] In a possible implementation, the above-mentioned decompressing the compressed data based on the corrected first model to obtain the reconstructed point cloud data of the first moving object set at the first moment includes: predicting the point cloud data of the first moving object set at the first moment based on the corrected first model and the second point cloud data to obtain fourth point cloud data; the second point cloud data is the reconstructed point cloud data of the first moving object set at the second moment, and the second moment is the moment before the first moment; based on the fourth point cloud data, decompressing the compressed data to obtain the reconstructed point cloud data of the first moving object set at the first moment.

[0038] In this way, the fourth point cloud data can be first predicted based on the modified first model, and then the compressed data can be decompressed based on the modified first model and the fourth point cloud data to obtain the reconstructed point cloud data of the first moving object set at the first moment.

[0039] In one possible implementation, the correction data of the first model includes the correction amount of the first coefficient matrix; the first coefficient matrix is ​​used to indicate the change of the point cloud data of the first moving object set within the first time period, and the first time period is the time period between the first moment and the second moment.

[0040] In this method, the first coefficient matrix is ​​used to indicate the changes in point cloud data of the first set of moving objects within the first time period, which is equivalent to that the prediction of the first model on the point cloud data of the first set of moving objects can be represented by the first coefficient matrix. Therefore, by receiving the correction amount of the first coefficient matrix, the first coefficient matrix can be updated to obtain the corrected first model.

[0041] In one possible implementation, the correction data of the first model includes the correction amount of the second coefficient matrix and the correction amount of the third coefficient matrix; the second coefficient matrix and the third coefficient matrix are obtained by low-rank decomposition of the first coefficient matrix; the first coefficient matrix is ​​used to indicate the change of point cloud data of the first moving object set within a first time period, and the first time period is the time period between the first moment and the second moment.

[0042] In this manner, the first coefficient matrix is ​​subjected to low-rank decomposition to obtain a second coefficient matrix and a third coefficient matrix having smaller dimensions than the first coefficient matrix. Accordingly, when receiving the correction data, the correction amount of the second coefficient matrix and the correction amount of the third coefficient matrix can be received, effectively reducing the transmission overhead of the correction data.

[0043] In one possible implementation, the correction data of the first model includes the correction amount of the second coefficient matrix, and the second coefficient matrix is ​​obtained by low-rank decomposition of the first coefficient matrix when the third coefficient matrix is ​​the initial matrix and the third coefficient matrix remains unchanged; the first coefficient matrix is ​​used to indicate the change of point cloud data of the first moving object set within a first time period, and the first time period is the time period between the first moment and the second moment.

[0044] In this method, when the first coefficient matrix is ​​subjected to low-rank decomposition, the third coefficient matrix can be kept unchanged. Therefore, when receiving the correction data, only the correction amount of the second coefficient matrix needs to be received, and there is no need to receive the correction amount of the third coefficient matrix, which can effectively reduce the transmission overhead of the correction data.

[0045] In one possible implementation, the dimension of the first coefficient matrix is ​​M×M, M is the number of moving objects included in the first moving object set, the dimension of the second coefficient matrix is ​​M×X, and the dimension of the third coefficient matrix is ​​X×M, M is a positive integer greater than 3, and X is a positive integer greater than or equal to 3 and less than or equal to M.

[0046] In this way, when the first coefficient matrix is ​​decomposed into the second coefficient matrix and the third coefficient matrix in a low-rank manner, the minimum value of X in the dimension of the second coefficient matrix and the third coefficient matrix is ​​3. The minimum value of X can ensure that the low-rank decomposition will not lose the changes in the point cloud data contained in any of the three coordinate dimensions (i.e., the x-axis, the y-axis, and the z-axis) while reducing the dimension of the coefficient matrix.

[0047] In a possible implementation, the compressed data includes encoding information of an index set and remaining point information; wherein the index set is a first index set or a second index set; the first index set includes indexes corresponding to point cloud data in the first point cloud data whose errors with associated point cloud data are less than a first error threshold, and the associated point cloud data are point cloud data in the fourth point cloud data that are associated with the point cloud data in the first point cloud data; the second index set is the complement of the first index set in a preset index set; the preset index set is the index set corresponding to the fourth point cloud data; the remaining point information is information obtained by encoding point cloud data in the first point cloud data except for the point cloud data corresponding to the first index set.

[0048] In this method, the first point cloud data can be divided into two parts, wherein the first part is the point cloud data with an error less than the first error threshold, and the second part is the remaining part of the first point cloud data except the first part, so the first part of the first point cloud data can be indicated by receiving the index set in the compressed data instead of directly receiving the first part, and the second part can be represented by receiving the remaining point information in the compressed data. Furthermore, the first part of the first point cloud data can be reconstructed from the fourth point cloud data based on the index set, and then the reconstructed point cloud data of the first moving object set at the first moment can be obtained based on the remaining point information and the reconstructed first part.

[0049] In a possible implementation, the compressed data further includes first indication information, where the first indication information is used to indicate that the index set is a set requiring the smallest number of bits to be transmitted between the first index set and the second index set.

[0050] In this manner, the first indication information may also be received, and when receiving the index set, the set requiring the smallest number of bits to be transmitted between the first index set and the second index set is received, thereby further reducing the transmission overhead during point cloud data interaction.

[0051] In a possible implementation, the compressed data includes encoding information, difference information and remaining point information of an index set; wherein the index set includes two of a first index set, a third index set and a fourth index set; the first index set includes indexes corresponding to point cloud data in the first point cloud data whose errors with associated point cloud data are less than a first error threshold, the third index set includes indexes corresponding to point cloud data in the first point cloud data whose errors with associated point cloud data are greater than or equal to the first error threshold and less than a second error threshold, and the associated point cloud data are point cloud data in the fourth point cloud data that are associated with point cloud data in the first point cloud data; the fourth index set is a set of indexes in a preset index set other than the first index set and the third index set, and the preset index set is an index set corresponding to the fourth point cloud data; the remaining point information is information obtained by encoding point cloud data in the first point cloud data other than point cloud data corresponding to the first index set and the third index set; the difference information is information obtained by encoding the error corresponding to the third index set, and the error corresponding to the third index set is the error between the point cloud data corresponding to the third index set in the first point cloud data and the associated point cloud data.

[0052] In this method, the first point cloud data can be divided into three parts, wherein the first part is the point cloud data with an error less than the first error threshold, the second part is the point cloud data with an error greater than or equal to the first error threshold and less than the second error threshold, and the third part is the rest of the first point cloud data except the first and second parts. Therefore, the first part of the first point cloud data can be indicated by receiving the index set in the compressed data instead of directly receiving the first part; the second part of the first point cloud data can be indicated by receiving the index set and the difference information instead of directly receiving the second part; and the third part of the first point cloud data can be represented by receiving the remaining point information. Furthermore, the first part can be reconstructed from the fourth point cloud data based on the index set, the second part can be reconstructed from the fourth point cloud data based on the index set and the difference information, and then the reconstructed point cloud data of the first moving object set at the first moment can be obtained based on the first part, the second part and the remaining point information.

[0053] In a possible implementation, the compressed data further includes second indication information, where the second indication information is used to indicate that the index set includes two sets of the first index set, the third index set, and the fourth index set that require a smaller number of bits to be transmitted.

[0054] In this manner, a second indication message may also be received, and when receiving an index set, two sets requiring a smaller number of bits to be transmitted among the first index set, the second index set and the third index set are received, thereby further reducing the transmission overhead during point cloud data interaction.

[0055] In a possible implementation, before receiving the corrected data of the first model and the compressed data corresponding to the first point cloud data, the method also includes: receiving third indication information, the third indication information being used to indicate the first set of motion objects; obtaining reconstructed point cloud data at a second moment; and determining the first set of motion objects and the second point cloud data based on the third indication information and the reconstructed point cloud data at the second moment.

[0056] In this manner, the first moving object set and the second point cloud data may be determined based on the third indication information.

[0057] In a possible implementation, before receiving the corrected data of the first model and the compressed data corresponding to the first point cloud data, the method further includes: sending configuration information; wherein the configuration information includes one or more configuration information of compression method, coefficient matrix, encoding method selection conditions, pre-encoding index matching method, compression parameters, coefficient matrix dimension, random seed, time step, and quantization method.

[0058] In this manner, the base station can configure various configuration information involved in point cloud data interaction.

[0059] In a third aspect, the present application provides a communication system, which includes a terminal device and a base station in any possible implementation of the first aspect or the second aspect, and is used to execute the data transmission method in the first aspect or the second aspect and any possible implementation of the same.

[0060] In a fourth aspect, the present application provides a communication device, which may be a terminal device or a base station, or a device in a terminal device or a base station, or a device that can be used in combination with a terminal device or a base station. Among them, the communication device may also be a chip system. The communication device may execute the method described in the first aspect or the second aspect. The functions of the communication device may be implemented by hardware, or by hardware executing corresponding software implementations. The hardware or software includes one or more units or modules corresponding to the above functions. The unit or module may be software and / or hardware. The operations and beneficial effects performed by the communication device may refer to the methods and beneficial effects in the first aspect or the second aspect above, and the repetitive parts will not be repeated.

[0061] In a fifth aspect, the present application provides a communication device, comprising a processor and a memory, wherein the processor and the memory are coupled; the processor is used to implement a method as described in any one of the first to second aspects.

[0062] In a sixth aspect, the present application provides a chip, comprising a processor and an interface, the interface being used to receive or output signals, and the processor being used to execute code instructions to implement a method as described in any one of the first to second aspects.

[0063] In a seventh aspect, the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is called by a computer, the computer executes a method as described in any one of the first to second aspects.

[0064] In an eighth aspect, the present application provides a computer program product. When a computer reads and executes the computer program product, the computer executes the method of any one of the first to second aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 It is a schematic diagram of the architecture of a communication system provided by an embodiment of the present application;

[0066] Figure 2 It is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0067] Figure 3 It is a flowchart of a data transmission method provided by an embodiment of the present application;

[0068] Figure 4 is a schematic diagram of a matching index set of a moving object provided in an embodiment of the present application;

[0069] Figure 5 is a schematic diagram of a flow chart of data transmission based on a first coefficient matrix provided in an embodiment of the present application;

[0070] Figure 6 It is a schematic diagram of a flow chart of data transmission based on a second coefficient matrix and a third coefficient matrix provided in an embodiment of the present application;

[0071] Figure 7 is a schematic diagram of a flow chart of data transmission based on a second coefficient matrix provided in an embodiment of the present application;

[0072] Figure 8 It is a flowchart of selecting an encoding method for compressing data provided by an embodiment of the present application;

[0073] Fig. 9 is a schematic diagram of a traffic scenario provided in an embodiment of the present application;

[0074] Fig.10 It is a comparative schematic diagram of air interface overhead provided in an embodiment of the present application;

[0075] Fig.11 is a schematic diagram of another traffic scenario provided in an embodiment of the present application;

[0076] Fig.12It is a comparative schematic diagram of another air interface overhead provided in an embodiment of the present application. DETAILED DESCRIPTION

[0077] The embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0078] The terms "first", "second", "third" and "fourth" etc. in the specification and claims of the present application and the drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0079] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0080] To facilitate understanding of the embodiments of the present application, the following first introduces the professional terms involved in the embodiments of the present application:

[0081] 1. Point cloud data

[0082] Point cloud data refers to a set of vectors in a three-dimensional coordinate system. The smallest unit of point cloud data is the point cloud data of a point, and the point cloud data of this point at least contains the three-dimensional coordinates corresponding to this point. Optionally, the point cloud data of this point may also include other attribute information besides the position, such as color, reflectivity, intensity, etc. Point cloud data is usually collected by sensors carried by sensing devices. The types of sensors include but are not limited to laser scanners, cameras, three-dimensional scanners, etc., and the sensing devices collect point cloud data by frame. Each frame of point cloud data collected can be reported to the base station. The base station can receive point cloud data from multiple sensing devices and perform fusion processing on them to meet the needs of subsequent tasks.

[0083] 2. Reporting Methods of Point Cloud Data

[0084] Usually, when the sensing device collects point cloud data in a scene, it will obtain point cloud data of a large number of points in the scene. If the sensing device directly sends all the collected point cloud data to the base station, it will occupy a large amount of air interface resources. Therefore, the sensing device will first compress the collected point cloud data, and then report the compressed data. At present, the way to compress point cloud data includes but is not limited to traditional point cloud compression algorithms and point cloud compression algorithms based on artificial intelligence (AI). Among them, traditional point cloud compression algorithms include two categories. One is to convert point cloud data from three-dimensional signals into two-dimensional signals, and then further compress them using existing two-dimensional compression algorithms, such as the video-based point cloud compression (V-PCC) algorithm provided by the moving pictures experts group MPEG. This process can also be combined with the compression / decompression (lempel-ziv-markov chain-algorithm, LZMA) algorithm for entropy coding to further compress the two-dimensional signal. The other type is to convert the point cloud data into a tree structure and then perform entropy coding, for example, the Draco algorithm based on kd-tree and the geometry-based compression algorithm (geometry point cloud compression, G-PCC) based on octree.

[0085] The AI-based point cloud compression algorithm includes a compression method based on ODE modeling of a single moving object. The principle of this compression method is to establish point cloud data modeling for each moving object in the scene, calculate an ODE model correction and a set of compressed data for each moving object, and send them to the base station. The compressed data can indicate the part of the point cloud data that the ODE model predicts inaccurately (or has deviations). After receiving the correction, the base station can first correct the ODE model, and then combine the corrected ODE model and the compressed data to reconstruct the point cloud data corresponding to each moving object.

[0086] However, for this compression method based on ODE modeling of a single moving object, as the number of moving objects in the scene increases, the number of corrections to the ODE model to be sent and the amount of compressed data also increase, which will increase the transmission overhead when interacting with the base station for point cloud data.

[0087] In order to solve the shortcomings of the above-mentioned compression method based on a single moving object for ODE modeling, the present application provides a data transmission method, a communication device, a communication system, etc.

[0088] The data transmission in the following embodiments of the present application refers to the data transmission involved in the point cloud data interaction between the base station and the terminal device. The terminal device is a perception device that collects point cloud data in the perception scene. The data transmission mainly involves how the terminal device reports the point cloud data collected at a certain moment to the base station. In addition, before the terminal device interacts with the base station for point cloud data, the terminal device side also involves how to model and compress the collected point cloud data. After the base station receives the data sent by the terminal device, it also involves how to model and reconstruct the point cloud data. Among them, reconstructing point cloud data refers to how the base station restores the point cloud data in the scene at a certain moment after receiving the data sent by the terminal device.

[0089] In a data transmission method provided in the following embodiment of the present application, point cloud data can still be compressed based on modeling, but multiple moving objects are modeled simultaneously during modeling, that is, one model is used to indicate the changes in point cloud data corresponding to multiple moving objects, thereby obtaining correction amounts and compressed data of the models corresponding to multiple moving objects. Accordingly, when point cloud data is interacted, the terminal device sends the correction amounts and compressed data of the models corresponding to the multiple moving objects to the base station, and the base station corrects the model based on the correction amounts of the models corresponding to the multiple moving objects, and obtains reconstructed point cloud data based on the corrected model and compressed data. The data transmission method provided in the embodiment of the present application can effectively reduce the amount of correction amounts and compressed data to be sent, thereby reducing the transmission overhead during point cloud data interaction.

[0090] In addition, by modeling multiple moving objects at the same time, the correlation between point cloud data of multiple moving objects can be effectively utilized to improve the accuracy of the model. If subsequent tasks require the prediction of the movement of moving objects, motion feature information can also be provided for them.

[0091] The following is an introduction to the communication system provided by an embodiment of the present application.

[0092] like Figure 1 As shown, an embodiment of the present application provides a communication system, which may include a terminal device 100 and a base station 200. The number of the terminal device 100 and the base station 200 may be one or more, for example Figure 1 The communication system shown includes three terminal devices 100 and one base station 200 .

[0093] The terminal device 100 is a perception device that collects point cloud data in a scene, performs modeling and compression on the point cloud data, and interacts with the base station 200 for point cloud data.

[0094] The base station 200 is a device that interacts with the terminal device 100 on point cloud and models and reconstructs point cloud data.

[0095] The terminal device 100 and the base station 200 both store a first model, which can be modeled based on the first moving object set and used to predict the point cloud data of multiple moving objects in the first moving object set. In addition, the terminal device 100 and the base station 200 may also store reconstructed point cloud data corresponding to some moments before the current moment when point cloud interaction is performed at the current moment. For example, the terminal device 100 and the base station store reconstructed point cloud data corresponding to the previous moment of the current moment, and the first model can predict the point cloud data at the current moment based on the reconstructed point cloud data corresponding to the previous moment.

[0096] The terminal device 100 can be an intelligent terminal in different perception scenarios. Figure 1 Taking the traffic scene shown as an example, the terminal device 100 can be a smart phone, tablet computer, or laptop computer in a vehicle. The terminal device 100 can also be a smart screen, a wearable device, an augmented reality (AR) device, a virtual reality (VR) device, a wireless terminal device in industrial control, a terminal device in unmanned automatic driving, a terminal device in remote medical, a terminal device in a smart grid, a terminal device in transportation safety, a wireless terminal device in a smart city, or a terminal device in smart transportation, etc., and this application does not limit this.

[0097] The base station 200 can be various forms of macro base stations, micro base stations (also called small stations), relay stations, access points (AP), etc. Exemplarily, the base station 200 can be a base station in a new radio (NR). Among them, the base station in the 5G new radio (NR) can also be called a transmission reception point (TRP) or a transmission point (TP) or a next generation Node B (ngNB), or an evolutionary Node B (eNB or eNodeB) in a long term evolution (LTE) system, or a base station in a short-range wireless communication system (such as a Wi-Fi communication system, a UWB communication system, etc.), etc., and the present application is not limited to this.

[0098] Next, the hardware structure of the terminal device 100 and the base station 200 is introduced.

[0099] like Figure 2 As shown, Figure 2 is a hardware structure diagram of an electronic device provided in an embodiment of the present application, and the electronic device can be the above Figure 1 The terminal device 100 or the base station 200 in.

[0100] For example, the electronic device includes one or more processors 110 and one or more memories 120. The one or more memories 120 are coupled to the one or more processors 110. The coupling in the embodiment of the present application is an indirect coupling or communication connection between devices, units or modules, which can be electrical, mechanical or other forms, and is used for information exchange between devices, units or modules.

[0101] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0102] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0103] The memory 120 is used to store program codes and data. In some embodiments, the memory 120 is a cache memory. The memory can store program codes or data that have just been used or are cyclically used by the processor 110. If the processor 110 needs to use the program code or data again, it can be directly called from the memory 120. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0104] The processor 110 may operate in conjunction with the memory 120 , or at least one of the one or more memories 120 may be included in the processor 110 .

[0105] The device may also include a communication interface 130, which may optionally include a standard wired interface, a wireless interface (such as Wi-Fi, a mobile communication interface, etc.), and the communication interface 130 is controlled by the processor 110 to send and receive data; the communication interface 130 may optionally also realize data or signal communication between internal devices of the device.

[0106] It is to be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device. In other embodiments of the present application, the device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0107] Based on the above description, the data transmission method provided by the present application is specifically introduced below.

[0108] like Figure 3 As shown, Figure 3 301 is a flow chart of a data transmission method provided in an embodiment of the present application. The method includes steps 301 to 306. Figure 3 The execution subject of the method shown may be a terminal device and a base station, or the subject may be a chip in the terminal device and a chip in the base station. Figure 3 The method shown may also be performed by other types of products, and those skilled in the art may further expand the method based on the contents disclosed in the specification. Figure 3 The method execution subject shown takes the terminal device and the base station as an example.

[0109] exist Figure 3 The process shown involves the use of a first model, which is used to predict point cloud data of each moving object in a first moving object set, and the number of moving objects included in the first moving object set is multiple.

[0110] Before introducing each step, let’s first introduce the principle of the first model:

[0111] The first model can model the point cloud data corresponding to the first moving object set as a whole by using the ordinary differential equation (ODE) on the left side of the following formula 1 according to the correlation between the point cloud data of the first moving object set at the previous and next moments:

[0112]

[0113] On the left side of Formula 1, is the reconstructed point cloud data of the first moving object set at the j-1th moment, is the reconstructed point cloud data of the first moving object set at the jth moment, A j is the first coefficient matrix of the first model, A j It can represent the change in point cloud data corresponding to the first moving object set from the j-1th moment to the jth moment.

[0114] Solving the left side of Formula 1 can obtain the formula on the right side of Formula 1. The solution method includes but is not limited to the forward Euler algorithm. On the right side of Formula 1, y j is the real point cloud data of the first moving object set at the jth moment, Δt is the time step, and for example, the time step can be the time length between the j-1th moment and the jth moment. According to the formula on the right side of Formula 1, Δt and A j Given the known premise, we can get y j , and y j It can be expressed as the following formula 2:

[0115]

[0116] It can be seen from the form of formula 2 that the first model can obtain the real point cloud data of the first moving object set at the j-1th moment based on the reconstructed point cloud data of the first moving object set at the j-1th moment. It should be noted that formula 2 is under ideal conditions (that is, the first model has no error or is fully converged). Therefore, the point cloud data of the first moving object set at the jth moment predicted by the first model has errors with the real point cloud data of the first moving object set at the jth moment. The point cloud data predicted by the first model will be referred to as predicted point cloud data.

[0117] The first coefficient matrix A of the first model is j To introduce:

[0118] Assume that the number of moving objects in the first moving object set is M, each moving object corresponds to N points of point cloud data, and the point cloud data of each point includes three-dimensional coordinates:

[0119] Then the various types of point cloud data corresponding to the first moving object set (such as y in the above formula j , ) can be expressed as a matrix with a dimension of M×3N. Under the premise that the formula on the right side of Formula 1 is valid, it can be obtained that the dimension of the first coefficient matrix is ​​M×M.

[0120] Optionally, for the first coefficient matrix A, a low-rank approximation may be performed to obtain a coefficient matrix with a lower dimension.

[0121] The process of low-rank decomposition can be referred to the following formula 3. The low-rank decomposition methods include but are not limited to singular value decomposition (SVD):

[0122]

[0123] In formula 3, the low-rank decomposition of the first coefficient matrix A can obtain three matrices U, ∑, V H After the operation of these three matrices, two coefficient matrices B and P are obtained. B is U 1:M,1:X , P is C 1:X,1:M , C 1:X,1:M for The matrix obtained after the operation. That is to say, the low-rank decomposition of the first coefficient matrix A can finally obtain two coefficient matrices: the second coefficient matrix B and the third coefficient matrix P, and the dimensions of B and P are M×X and X×M respectively.

[0124] Optionally, the first model may also be modeled using the coefficient matrix after low-rank decomposition. The modeling process may refer to the following formula 4:

[0125]

[0126] According to the formula on the right side of Formula 4, Δt, B j and P j Given the known premise, we can get y j .

[0127] Optionally, in low-rank decomposition, M is a positive integer greater than 3 (i.e., the first moving object set includes more than 3 moving objects), and X is a positive integer greater than or equal to 3 and less than or equal to M. The value range of M and the value range of X can ensure that the low-rank decomposition will not lose the changes in the point cloud data contained in any of the three coordinate dimensions (i.e., the x-axis, the y-axis, and the z-axis) while reducing the transmission overhead.

[0128] Optionally, for the case where M is less than or equal to 3, the first moving object set includes 3 or less moving objects (such as 3 or 2). In this case, the amount of point cloud data corresponding to the first moving object set is not large. Therefore, low-rank decomposition is not used but A in Formula 1 is used for modeling, or the existing ODE modeling method based on a single moving object is used.

[0129] Based on the above introduction to the first model, the implementation of each step is introduced below:

[0130] 301. A terminal device determines correction data of a first model based on first point cloud data and a first model, where the first model is used to predict point cloud data of each moving object in a first moving object set.

[0131] The first point cloud data is the real point cloud data of the first moving object set at the first moment.

[0132] In one possible implementation, the terminal device determines the correction data of the first model based on the first point cloud data and the first model in a manner that specifically includes: the terminal device performs an initial prediction on the point cloud data of the first moving object set at the first moment based on the first model and the second point cloud data to obtain third point cloud data; and obtains the correction data of the first model based on the error between the first point cloud data and the third point cloud data.

[0133] Among them, the second point cloud data is the reconstructed point cloud data of the first moving object set at the second moment, and the second moment is the moment before the first moment. The terminal device stores the coefficient matrix of the first model at the second moment. The process of the terminal device obtaining the third point cloud data based on the first model and the second point cloud data is specifically as follows: when the coefficient matrix at the second moment, the second point cloud data, and the time step (i.e., the time length between the first moment and the second moment) are known, the terminal device can obtain the initial predicted point cloud data (i.e., the third point cloud data) of the first moving object set at the first moment. Since the coefficient matrix at the second moment is used for prediction when obtaining the third point cloud data, rather than the coefficient matrix at the first moment, there is an error between the third point cloud data and the first point cloud data. In other words, the first model at the moment before the first moment is used to make an initial prediction of the point cloud data, and there is an error between the point cloud data obtained by the initial prediction and the real point cloud data. Further, based on the error between the first point cloud data and the third point cloud data, the corrected data of the first model can be obtained.

[0134] Among them, the coefficient matrix of the first model at the first moment and the coefficient matrix at the second moment both refer to the modifiable coefficient matrix of the first model (hereinafter referred to as the coefficient matrix of the first model).

[0135] Optionally, the coefficient matrix of the first model may include a first coefficient matrix A. In this optional manner, the value of A at the second moment may be substituted into the above formula 1 or 2 to obtain the third point cloud data.

[0136] Optionally, the coefficient matrix of the first model may include a second coefficient matrix B and a third coefficient matrix P. In this optional manner, the values ​​of B and P at the second moment may be substituted into the above formula 4 to obtain the third point cloud data.

[0137] Optionally, the coefficient matrix of the first model may include the second coefficient matrix B. In this optional manner, the third coefficient matrix P remains unchanged and is known, so the coefficient matrix of the first model does not include P, and the value of B at the second moment can be substituted into the above formula 4 to obtain the third point cloud data based on formula 4.

[0138] How to obtain the correction data of the first model in these three cases will be described in subsequent embodiments.

[0139] 302. The terminal device corrects the first model based on the correction data to obtain a corrected first model.

[0140] In the present application, the correction data of the first model obtained by the terminal device is the correction amount of the coefficient matrix of the first model.

[0141] Optionally, the correction data of the first model may include a correction amount of the first coefficient matrix A, such as ΔA.

[0142] Optionally, the correction data of the first model may include a correction amount of the second coefficient matrix B, such as ΔB, and a correction amount of the third coefficient matrix P, such as ΔP.

[0143] Optionally, the correction data of the first model may include a correction amount of the second coefficient matrix B, such as ΔB.

[0144] The terminal device corrects the first model based on the correction data, and obtaining the corrected first model means: correcting the coefficient matrix of the first model at the second moment to obtain the coefficient matrix of the first model at the first moment, and the coefficient matrix at the first moment can indicate the changes in the point cloud data of the first moving object set in the time period from the second moment to the first moment.

[0145] 303. The terminal device obtains compressed data corresponding to the first point cloud data based on the first point cloud data and the corrected first model.

[0146] In one possible implementation, the terminal device obtains compressed data corresponding to the first point cloud data based on the first point cloud data and the corrected first model, specifically including: re-predicting the point cloud data of the first moving object set at the first moment based on the corrected first model and the second point cloud data to obtain fourth point cloud data; and compressing the first point cloud data based on the fourth point cloud data to obtain compressed data.

[0147] Among them, the process of the terminal device obtaining the fourth point cloud data based on the corrected first model and the second point cloud data is specifically as follows: when the coefficient matrix at the first moment, the second point cloud data, and the time step (i.e., the time length between the first moment and the second moment) are known, the terminal device can obtain the re-predicted point cloud data (i.e., the fourth point cloud data) of the first moving object set at the first moment. Since although the coefficient matrix at the first moment is used for calculation, the first model still has a model error, there is still an error between the obtained fourth point cloud data and the actual point cloud data (i.e., the first point cloud data), and the error between the fourth point cloud data and the first point cloud data is less than the error between the third point cloud data and the first point cloud data. Furthermore, based on the error between the fourth point cloud data and the first point cloud data, and the part of the first point cloud data accurately predicted by the corrected first model (i.e., the part of the fourth point cloud data and the first point cloud data that is correctly predicted), the first point cloud data can be compressed to obtain compressed data. The present application has the following two methods for obtaining compressed data:

[0148] Optionally, the first point cloud data is divided into two parts, wherein the first part is point cloud data with an error less than a first error threshold, and the second part is the rest of the first point cloud data except the first part. For the first part, the index corresponding to the point cloud data is used for indication; for the second part, the second part is used for direct indication. Therefore, the compressed data includes information obtained by encoding the index corresponding to the first part and information obtained by encoding the second part.

[0149] Optionally, the first point cloud data can be divided into three parts, wherein the first part is point cloud data with an error less than a first error threshold, the second part is point cloud data with an error greater than or equal to the first error threshold and less than a second error threshold, and the third part is the rest of the first point cloud data except the first and second parts. For the first part, the index corresponding to the point cloud data is used for indication; for the second part, the index corresponding to the point cloud data and the difference information (i.e., the predicted error) are used for indication; for the third part, the third part is used for direct indication. Therefore, the compressed data includes information obtained by encoding the index corresponding to the first part, information obtained by encoding the index corresponding to the second part, information obtained by encoding the difference information, and information obtained by encoding the third part.

[0150] These two forms of compressed data will be explained through specific examples later.

[0151] 304. The terminal device sends the corrected data of the first model and the compressed data corresponding to the first point cloud data to the base station.

[0152] Correspondingly, the base station receives the corrected data of the first model and the compressed data corresponding to the first point cloud data.

[0153] 305. The base station corrects the first model based on the correction data to obtain a corrected first model.

[0154] In the present application, the base station stores a coefficient matrix of the first model at the second moment. The base station corrects the first model based on the correction data to obtain the corrected first model, which means: correcting the coefficient matrix of the first model at the second moment to obtain the coefficient matrix of the first model at the first moment, and the coefficient matrix at the first moment can indicate the change of the point cloud data of the first moving object set in the time period from the second moment to the first moment.

[0155] 306. The base station decompresses the compressed data based on the corrected first model to obtain reconstructed point cloud data of the first moving object set at the first moment.

[0156] In a possible implementation, the base station decompresses the compressed data based on the corrected first model to obtain the reconstructed point cloud data of the first moving object set at the first moment, specifically including: predicting the point cloud data of the first moving object set at the first moment based on the corrected first model and the second point cloud data to obtain fourth point cloud data; decompressing the compressed data based on the fourth point cloud data to obtain the reconstructed point cloud data of the first moving object set at the first moment.

[0157] The process by which the base station obtains the fourth point cloud data is the same as the process by which the terminal obtains the fourth point cloud data in step 303. The base station decompresses the compressed data based on the fourth point cloud data to obtain the reconstructed point cloud data of the first moving object set at the first moment, which is the reverse process of the process in which the terminal compresses the first point cloud data based on the corrected first model and the fourth point cloud data in step 303 to obtain the compressed data. In other words, the base station can restore the first point cloud data as much as possible based on the fourth point cloud data and the compressed data, and the restored data is called the reconstructed point cloud data of the first moving object set at the first moment. The base station can store the reconstructed point cloud data of the first moving object set at the first moment, so that the real point cloud data at moments after the first moment can be compressed later.

[0158] Optionally, the terminal device may also decompress the compressed data based on the corrected first model to obtain the reconstructed point cloud data of the first moving object set at the first moment, and store the reconstructed point cloud data of the first moving object set at the first moment.

[0159] For example, for a method in which compressed data includes information obtained by encoding an index corresponding to a first part and information obtained by encoding a second part: the first part can be obtained from the fourth point cloud data based on the index corresponding to the first part, and then the union of the first part and the second part can be determined as the reconstructed point cloud data of the first moving object set at the first moment.

[0160] For another example, for a method in which compressed data includes information obtained by encoding an index corresponding to a first part, information obtained by encoding an index corresponding to a second part, information obtained by encoding difference information, and information obtained by encoding a third part: the first part can be obtained from the fourth point cloud data based on the index corresponding to the first part, and then based on the difference information, the point cloud data of the index corresponding to the second part in the fourth point cloud data can be supplemented to obtain the second part; finally, the union of the first part, the second part, and the third part is determined as the reconstructed point cloud data of the first moving object set at the first moment.

[0161] Optional, in the above Figure 3In a corresponding embodiment, the terminal device may also send the corrected data of the first model to the base station after determining the corrected data of the first model. Then, the terminal device executes steps 302 to 303 to determine the compressed data, and sends the compressed data to the base station after the compressed data is determined. Correspondingly, the base station may also execute step 305 after receiving the corrected data of the first model, and then execute step 306 after receiving the compressed data later.

[0162] The following is a supplementary explanation of the first moving object set involved in the above embodiment:

[0163] In the present application, before executing the above steps, the terminal device needs to first determine the first moving object set. When the terminal device determines the first moving object set, it also determines the first point cloud data (i.e., the real point cloud data of the first moving object set at the first moment) and the second point cloud data (i.e., the reconstructed point cloud data of the first moving object set at the second moment). Then the terminal device also needs to send the third indication information of the first moving object set to the base station, so that the base station can determine the first moving object set based on the third indication information. When the base station determines the first moving object set, it also determines the second point cloud data.

[0164] Specifically, on the terminal device side: the terminal device obtains the reconstructed point cloud data of the second moment; obtains the real point cloud data collected at the first moment; obtains the first point cloud data from the real point cloud data of the first moment, and the first motion object set is the intersection between the motion object set corresponding to the reconstructed point cloud data of the second moment and the motion object set corresponding to the real point cloud data of the first moment; sends third indication information to the base station, and the third indication information is used to indicate the first motion object set.

[0165] Among them, the terminal device stores the reconstructed point cloud data of the second moment and the real point cloud data collected by the terminal device at the first moment. Then the terminal device can identify the set of moving objects corresponding to the reconstructed point cloud data of the second moment, and the set of moving objects corresponding to the real point cloud data collected at the first moment. Since the present application uses the first model for modeling, it uses the correlation between the point cloud data at the previous and next moments. Therefore, the modeling objects of the first model of the present application need to be the set of moving objects corresponding to the reconstructed point cloud data at the second moment and the real point cloud data collected at the first moment. For example, the set of moving objects corresponding to the reconstructed point cloud data at the second moment is {q 1 ,q 2 ,q 3 ,q 4 ,q 5}, the set of moving objects corresponding to the real point cloud data collected at the first moment is {q 2 ,q 3 ,q 4 ,q 5 ,q6 ,q 7}, then comparing these two sets of moving objects, we can see that from the second moment to the first moment, q 1 To perceive the moving object disappearing in the scene, q 6 ,q 7 is a newly added moving object in the perception scene, then the first moving object set is {q 2 ,q 3 ,q 4 ,q 5}. Accordingly, the first point cloud data is the real point cloud data collected at the first moment {q 2 ,q 3 ,q 4 ,q 5}The real point cloud data corresponding to the four moving objects, the second point cloud data is the reconstructed point cloud data at the second moment {q 2 ,q 3 ,q 4 ,q 5}The reconstructed point cloud data corresponding to the four moving objects.

[0166] On the base station side: the base station receives the third indication information; the base station obtains the reconstructed point cloud data at the second moment; based on the third indication information and the reconstructed point cloud data at the second moment, the first moving object set and the second point cloud data are determined.

[0167] The base station stores the reconstructed point cloud data at the second moment. When receiving the third indication information, the base station may first identify the set of moving objects corresponding to the reconstructed point cloud data at the second moment, such as identifying {q 1 ,q 2 ,q 3 ,q 4 ,q 5}, and then determine the first moving object set as {q 2 ,q 3 ,q 4 ,q 5}. Accordingly, the second point cloud data is the reconstructed point cloud data at the second moment {q 2 ,q 3 ,q 4 ,q 5}The reconstructed point cloud data corresponding to the four moving objects.

[0168] Optionally, the third indication information may be represented as a matching index set of a moving object, etc. Taking the matching index set of a moving object as an example, the matching index set of the moving object may be obtained by matching the moving object set corresponding to the real point cloud data collected at the first moment based on the moving object set corresponding to the reconstructed point cloud data at the second moment.

[0169] For example, Figure 4 As shown, the elements in the matching index set of the motion object are: the arrangement order of the motion objects matched by the motion objects in the motion object set at the previous moment, and the motion objects in the motion object set at the current moment that cannot be matched in the motion object set at the previous moment are recorded as 0. Therefore, the index set of the motion object at the 101st moment is {1, 2, 0, 4, 3}, which means: the first motion object in the motion object set at the 101st moment is the first motion object in the motion object set at the 100th moment; the second motion object in the motion object set at the 101st moment is the second motion object in the motion object set at the 100th moment; the third motion object in the motion object set at the 101st moment does not exist in the motion object set at the 100th moment; the fourth motion object in the motion object set at the 101st moment is the fourth motion object in the motion object set at the 100th moment; the fifth motion object in the motion object set at the 101st moment is the third motion object in the motion object set at the 100th moment. The index set of moving objects at the 102nd moment is {2, 3, 4, 5}, which means: the first moving object in the moving object set at the 102nd moment is the second moving object in the moving object set at the 101st moment; the second moving object in the moving object set at the 102nd moment is the third moving object in the moving object set at the 101st moment; the third moving object in the moving object set at the 102nd moment is the fourth moving object in the moving object set at the 101st moment; the fourth moving object in the moving object set at the 102nd moment is the fifth moving object in the moving object set at the 101st moment.

[0170] Optionally, the third indication information may be included in compressed data and sent.

[0171] It should be noted that the first point cloud data corresponding to the first moving object set can be obtained according to Figure 3 The embodiment shown reports point cloud data, and other existing reporting methods may be used to report point cloud data other than the first point cloud data in the real point cloud data collected at the first moment.

[0172] By implementing Figure 3In the described embodiment, the first model can be used to predict the point cloud data of each moving object in the first moving object set, so the first model is used to model the first moving object set as a whole. Furthermore, when the terminal device obtains the correction amount of the model and compresses the point cloud data, it is not necessary to determine the model correction amount of each moving object separately and obtain a set of corresponding compressed data for each moving object separately. Instead, the correction amount of the first model is determined, and the correction amount of the first model is used to represent the model correction amount corresponding to the first moving object set, and the first point cloud data is uniformly processed based on the corrected first model to obtain compressed data corresponding to the first moving object set. Accordingly, the terminal device and the base station can transmit the correction amount of the first model and the compressed data of the first moving object set when the point cloud data is exchanged, which can effectively reduce the number of correction amounts and compressed data to be transmitted, thereby reducing the transmission overhead when the point cloud data is exchanged. Secondly, this method of modeling the first moving object set as a whole can effectively improve the processing efficiency, and enable the obtained first model to utilize the correlation between the moving objects in the first moving object set to obtain a more accurate prediction result.

[0173] The above embodiment is described in detail below by taking three cases of the coefficient matrix of the first model as examples:

[0174] Assume that the second moment is the j-1th moment, the first moment is the jth moment, j is a positive integer greater than or equal to 3, and the first moving object set is M is the number of moving objects in the first moving object set. The first point cloud data (i.e., the real point cloud data of the first moving object set at the jth moment) in the above embodiment is y j , the second point cloud data (i.e., the reconstructed point cloud data of the first moving object set at the j-1th moment) is The reconstructed point cloud data of the first moving object set at the jth moment is

[0175] Case 1: The coefficient matrix of the first model includes the first coefficient matrix A, and the correction data of the first model includes the correction amount of A, such as ΔA.

[0176] The following describes the steps performed by the terminal device and the steps performed by the base station respectively:

[0177] Terminal device side: The terminal device may execute steps 1-501 to 1-508, where steps 1-501 to 1-503 are specific implementations of step 301, step 1-505 is a specific implementation of step 302, and steps 1-506 to 1-507 are specific implementations of step 303:

[0178] 1-501. Initialize the first coefficient matrix.

[0179] In the embodiment of the present application, the first coefficient matrix may be initialized based on the following formula 5 to obtain A1:

[0180]

[0181] In formula 5, and They are the reconstructed point cloud data at the second moment and the reconstructed point cloud data at the first moment, and It can be obtained based on a decompression method corresponding to an existing compression method of point cloud data, such as based on a Draco algorithm or the like.

[0182] 1-502. Perform initial prediction based on the first coefficient matrix at the j-1th moment.

[0183] Here, j is a positive integer greater than or equal to 2.

[0184] Initial prediction means: based on the first coefficient matrix of the first model at the j-1th time and the second point cloud data, an initial prediction is made on the first moving object set at the jth time to obtain the third point cloud data. Assume that the third point cloud data is The first coefficient matrix at the j-1th moment is A j -1, then based on the following formula 6, we can get

[0185]

[0186] 1-503. Calculate the correction amount of the first coefficient matrix at the j-th moment.

[0187] Based on the following formula 7, the correction value ΔA of the first coefficient matrix at the jth moment can be obtained: j ,in, is the error between the third point cloud data and the first point cloud data:

[0188]

[0189] 1-504. Send the correction value of the first coefficient matrix at the j-th moment.

[0190] 1-505. Modify the first coefficient matrix based on the correction amount of the first coefficient matrix at the j-th moment to obtain the first coefficient matrix at the j-th moment.

[0191] This step can be implemented based on the following formula 8, where the first coefficient matrix at the jth moment is A j :

[0192] A j =Aj -1+ΔA j (Formula 8)

[0193] It should be noted that after the terminal device executes step 1-505, if the value of j is a positive integer greater than or equal to 3, then the following steps 1-506 to 1-509 are executed. Otherwise, the terminal device compresses the point cloud data based on the existing point cloud data compression method and obtains the reconstructed point cloud data. That is to say, the terminal device initializes the first coefficient matrix based on step 1-501 at the first moment, and obtains the reconstructed point cloud data based on the existing point cloud data compression method. At the second moment, the terminal device obtains the point cloud data based on the compression method of the existing point cloud data. Based on steps 1-502 to 1-505, A2 is obtained. At the third moment and the moment after the third moment, A is obtained based on steps 1-502 to 1-505. j and compressing data.

[0194] 1-506. Perform another prediction based on the first coefficient matrix at the j-th moment.

[0195] The second prediction means: based on the first coefficient matrix of the first model at the jth time and the second point cloud data, the first moving object set is predicted again at the jth time to obtain the fourth point cloud data. Assume that the fourth point cloud data is This step can be implemented by referring to the following formula 9:

[0196]

[0197] 1-507. Determine the compressed data.

[0198] Determining the compressed data means: based on the fourth point cloud data, compressing the first point cloud data to obtain compressed data. Assume that the compressed data is u j , this step can be implemented by referring to the following formula 10:

[0199]

[0200] Among them, f j () is the compression process, which will be described in detail in the subsequent embodiments.

[0201] 1-508, send compressed data.

[0202] 1-509. Decompress the compressed data and store the decompression result.

[0203] Decompressing the compressed data means: based on the fourth point cloud data, decompressing the compressed data to obtain the reconstructed point cloud data of the first moving object set at the jth moment; then, the reconstructed point cloud data of the first moving object set at the jth moment can be stored.

[0204] Decompression can be achieved by referring to the following formula 11:

[0205]

[0206] in, The decompression process will be described in detail in the subsequent embodiments.

[0207] Base station side: The base station can execute steps 2-501 to 2-503. Step 2-501 is a specific implementation of step 305, and steps 2-502 to 2-503 are specific implementations of step 306. The base station must execute step 2-501 after the terminal device executes step 1-504, that is, the base station receives the ΔA sent by the terminal device. j After that, step 2-501 can be executed. And the base station executes step 2-503 after the terminal device executes step 1-508, that is, the base station receives the u sent by the terminal device. j Only then can step 2-503 be executed.

[0208] 2-501. Modify the first coefficient matrix based on the correction amount of the first coefficient matrix at the j-th moment to obtain the first coefficient matrix at the j-th moment.

[0209] 2-502. Perform another prediction based on the first coefficient matrix at the j-th moment.

[0210] 2-503. Decompress the compressed data and store the decompression result.

[0211] Among them, the specific implementation method of step 2-501 can refer to the specific implementation method of step 1-505; the specific implementation method of step 2-502 can refer to the specific implementation method of step 1-506; the specific implementation method of step 2-503 can refer to the specific implementation method of step 1-509.

[0212] It should be noted that the base station also needs to initialize the first coefficient matrix and obtain the reconstructed point cloud data based on the compression method of the existing point cloud data. When j is 2, the reconstructed point cloud data is obtained based on the compression method of the existing point cloud data. And execute step 2-501 to obtain A2, and do not execute steps 2-502 to 2-503. When j is a positive integer greater than or equal to 3, execute steps 2-501 to 2-503.

[0213] By implementing Figure 5 In the described embodiment, no matter how many moving objects are included in the first moving object set, the terminal device can send a ΔA j , without sending the correction values ​​of multiple coefficient matrices, so that the base station can correct the first model. And based on the first model, a set of compressed data u corresponding to the first set of moving objects can be obtained j And send it without sending the compressed data corresponding to each moving object. This can effectively reduce the amount of correction and compressed data to be transmitted, thereby reducing the transmission overhead when exchanging point cloud data. Secondly, this method of modeling the first moving object set as a whole can effectively improve the processing efficiency, and enable the obtained first model to utilize the correlation between the various moving objects in the first moving object set to obtain a more accurate prediction result.

[0214] Optionally, the terminal device can also set ΔA j and u j After all are determined, ΔA j and u j Sent to the base station together.

[0215] Case 2: The coefficient matrix of the first model includes the second coefficient matrix B and the third coefficient matrix P, and the correction data of the first model includes the correction amount of B, such as ΔB, and the correction amount of P, such as ΔP.

[0216] The following describes the steps performed by the terminal device and the steps performed by the base station respectively:

[0217] Terminal device side: The terminal device may execute steps 1-601 to 1-612. Steps 1-601 to 1-603 and 1-606 are specific implementations of step 301. Steps 1-605 and 1-608 are specific implementations of step 302. Steps 1-609 to 1-610 are specific implementations of step 303.

[0218] 1-601. Initialize the second coefficient matrix and the third coefficient matrix.

[0219] In the embodiment of the present application, the third coefficient matrix may be initialized based on the following formula 12 to obtain P1, and then the second coefficient matrix may be initialized based on the following formula 13 to obtain B1, and then the terminal device may perform steps 1-602 to 1-612:

[0220]

[0221]

[0222] In Formula 12 and Formula 13, diag(1,1,1) represents a diagonal matrix whose diagonal elements are 1, 1, 1. 3×(M - 3) Represents a matrix whose elements are all 1. and They are the reconstructed point cloud data at the second moment and the reconstructed point cloud data at the first moment, and It can be obtained based on the decompression method corresponding to the compression method of the existing point cloud data, such as based on the Draco algorithm. After the terminal device initializes P1 based on formula 12, P1 can be compared with and Substitute into formula 13 and obtain B1.

[0223] 1-602. Perform initial prediction based on the second coefficient matrix at the j-1th moment and the third coefficient matrix at the j-1th moment.

[0224] Wherein, j is a positive integer greater than or equal to 2. Initial prediction means: based on the second coefficient matrix and the third coefficient matrix of the first model at the j-1th moment, and the second point cloud data, the first moving object set is initially predicted at the jth moment to obtain the third point cloud data. The second coefficient matrix at the j-1th moment is B j -1, the third coefficient matrix at the j-1th moment is P j -1, the third point cloud data can be obtained based on the following formula 14:

[0225]

[0226] After the terminal device executes step 1-602, it is necessary to calculate the correction amount of the second coefficient matrix and the correction amount of the third coefficient matrix. In the first optional method, the correction amount of the second coefficient matrix can be calculated first, and then the correction amount of the third coefficient matrix can be calculated. In the second optional method, the correction amount of the third coefficient matrix can be calculated first, and then the correction amount of the second coefficient matrix can be calculated. The second optional method is used as an example for explanation below:

[0227] 1-603. Calculate the correction amount of the second coefficient matrix at the j-th moment.

[0228] Based on the following formula 15, the correction value ΔB of the second coefficient matrix at the jth moment can be obtained: j .

[0229]

[0230] In formula 15, is the error between the third point cloud data and the first point cloud data. Formula 15 is based on the premise that the third coefficient matrix is ​​still the third coefficient matrix at the j-1th moment, and the correction amount of the second coefficient matrix is ​​calculated based on the error between the third point cloud data and the first point cloud data. In other words, when calculating the correction amount of the second coefficient matrix, it is first assumed that the third coefficient matrix is ​​the third coefficient matrix at the previous moment.

[0231] 1-604. Send the correction value of the second coefficient matrix at the j-th moment.

[0232] 1-605. Modify the second coefficient matrix based on the correction amount of the second coefficient matrix at the j-th moment to obtain the second coefficient matrix at the j-th moment.

[0233] This step can be implemented based on the following formula 16, where the second coefficient matrix at the jth moment is B j :

[0234] B j =B j -1+ΔB j (Formula 16)

[0235] 1-606. Calculate the correction amount of the third coefficient matrix at the j-th moment.

[0236] Based on the following formula 17, the correction value ΔP of the third coefficient matrix at the jth moment can be obtained: j .

[0237]

[0238] In Formula 17, is the error between the third point cloud data and the first point cloud data. Formula 17 calculates the correction amount of the third coefficient matrix based on the error between the third point cloud data and the first point cloud data, under the premise that the second coefficient matrix has been corrected to the second coefficient matrix at the jth moment.

[0239] 1-607. Send the correction value of the third coefficient matrix at the j-th moment.

[0240] 1-608. Modify the third coefficient matrix based on the correction amount of the third coefficient matrix at the j-th moment to obtain the third coefficient matrix at the j-th moment.

[0241] This step can be implemented based on the following formula 18, where the third coefficient matrix at the jth moment is P j :

[0242] P j =P j -1+ΔP j (Formula 18)

[0243] It should be noted that after the terminal device executes step 1-608, if the value of j is a positive integer greater than or equal to 3, the following steps 1-609 to 1-612 are executed. Otherwise, the terminal device compresses the point cloud data based on the existing point cloud data compression method and obtains the reconstructed point cloud data. That is to say, the terminal device initializes the second coefficient matrix and the third coefficient matrix based on step 1-601 at the first moment, and obtains the reconstructed point cloud data based on the existing point cloud data compression method. At the second moment, based on the compression method of the existing point cloud data, Based on steps 1-602 to 1-608, B2 and P2 are obtained. At the third moment and the moment after the third moment, B is obtained based on steps 1-602 to 1-608. j , P j and compressing data.

[0244] 1-609. Perform re-prediction based on the second coefficient matrix at the j-th moment and the third coefficient matrix at the j-th moment.

[0245] The second prediction means: based on the second coefficient matrix of the first model at the jth moment and the third coefficient matrix at the jth moment, and the second point cloud data, the first moving object set is predicted again at the jth moment to obtain the fourth point cloud data. This step can be implemented with reference to the following formula 19:

[0246]

[0247] 1-610. Determine the compressed data.

[0248] 1-611. Send compressed data.

[0249] 1-612. Decompress the compressed data and store the decompression result.

[0250] The specific implementation of steps 1-610 to 1-612 can refer to the above Figure 5 The specific implementation method of steps 1-507 to 1-509.

[0251] Base station side: The base station can execute steps 2-601 to 2-604. Steps 2-601 to 2-602 are specific implementations of step 305, and steps 2-603 to 2-604 are specific implementations of step 306. The base station must execute step 2-601 after the terminal device executes step 1-604, that is, the base station receives the ΔB sent by the terminal device. j After that, step 2-601 can be executed. And the base station executes step 2-602 after the terminal device executes step 1-607, that is, the base station receives the ΔP sent by the terminal device.j After that, step 2-601 can be executed. And the base station executes step 2-604 after the terminal device executes step 1-611, that is, the base station can execute step 2-604 after receiving the compressed data sent by the terminal device.

[0252] 2-601. Modify the second coefficient matrix based on the correction amount of the second coefficient matrix at the j-th moment to obtain the second coefficient matrix at the j-th moment.

[0253] 2-602. Modify the third coefficient matrix based on the correction amount of the third coefficient matrix at the j-th moment to obtain the third coefficient matrix at the j-th moment.

[0254] 2-603. Perform a second prediction based on the second coefficient matrix at the j-th moment and the third coefficient matrix at the j-th moment.

[0255] 2-604. Decompress the compressed data and store the decompression result.

[0256] Among them, the specific implementation method of step 2-601 can refer to the specific implementation method of step 1-605; the specific implementation method of step 2-602 can refer to the specific implementation method of step 1-608; the specific implementation method of step 2-603 can refer to the specific implementation method of step 1-609; the specific implementation method of step 2-604 can refer to the specific implementation method of step 1-612.

[0257] It should be noted that the base station also needs to initialize the second coefficient matrix and the third coefficient matrix, and obtain the reconstructed point cloud data based on the compression method of the existing point cloud data. When j is 2, the reconstructed point cloud data is obtained based on the compression method of the existing point cloud data. And execute step 2-601 to obtain B2, execute step 2-602 to obtain P2, and do not execute steps 2-603 to 2-604. When j is a positive integer greater than or equal to 3, execute steps 2-601 to 2-604.

[0258] By implementing Figure 6 In the described embodiment, no matter how many moving objects are included in the first moving object set, the terminal device can send a ΔB j and ΔP j , without the need to send multiple coefficient matrix corrections, which can reduce the transmission overhead of model corrections during point cloud data interaction. j and ΔP j Ratio ΔA j The smaller dimension can further reduce the transmission overhead. Secondly, based on the first model, a set of compressed data u corresponding to the first set of moving objects is obtained.j In addition, the first moving object set is used as a whole for modeling, which can effectively improve the processing efficiency and enable the obtained first model to utilize the correlation between the moving objects in the first moving object set to obtain a more accurate prediction result.

[0259] Optionally, the terminal device can also set ΔB j , ΔP j and u j After all are determined, ΔB j , ΔP j and u j Sent to the base station together.

[0260] Case 3: The coefficient matrix of the first model includes the second coefficient matrix B, and the correction data of the first model includes the correction amount of B, such as ΔB.

[0261] The following describes the steps performed by the terminal device and the steps performed by the base station respectively:

[0262] Terminal device side: The terminal device may execute steps 1-701 to 1-709, where steps 1-701 to 1-703 are specific implementations of step 301, step 1-705 is a specific implementation of step 302, and steps 1-706 to 1-707 are specific implementations of step 303:

[0263] 1-701. Initialize the second coefficient matrix and the third coefficient matrix. The third coefficient matrix remains unchanged after initialization.

[0264] The specific implementation of step 1-701 may refer to the specific implementation of step 1-601.

[0265] 1-702. Perform initial prediction based on the second coefficient matrix at the j-1th moment.

[0266] Wherein, j is a positive integer greater than or equal to 2. Initial prediction means: based on the second coefficient matrix of the first model at the j-1th moment and the initialized third coefficient matrix, as well as the second point cloud data, initial prediction is performed on the first moving object set at the jth moment to obtain the third point cloud data. The second coefficient matrix at the j-1th moment is B j-1 , the initialized third coefficient matrix is ​​P1, then the third point cloud data can be obtained based on the following formula 20:

[0267]

[0268] 1-703. Calculate the correction amount of the second coefficient matrix at the j-th moment.

[0269] Based on the following formula 21, the correction value ΔB of the second coefficient matrix at the jth moment can be obtained: j .

[0270]

[0271] 1-704. Send the correction value of the second coefficient matrix at the j-th moment.

[0272] 1-705. Modify the second coefficient matrix based on the correction amount of the second coefficient matrix at the j-th moment to obtain the second coefficient matrix at the j-th moment.

[0273] The specific implementation of step 1-705 may refer to the specific implementation of step 1-605.

[0274] It should be noted that after the terminal device executes step 1-705, if the value of j is a positive integer greater than or equal to 3, the following steps 1-706 to 1-709 are executed. Otherwise, the terminal device compresses the point cloud data based on the existing point cloud data compression method and obtains the reconstructed point cloud data. That is to say, the terminal device initializes the second coefficient matrix and the third coefficient matrix based on step 1-701 at the first moment, and obtains the reconstructed point cloud data based on the existing point cloud data compression method. At the second moment, based on the compression method of the existing point cloud data, Based on steps 1-702 to 1-705, B2 is obtained. At the third moment and the moment after the third moment, B is obtained based on steps 1-702 to 1-705. j and compressing data.

[0275] 1-706. Perform another prediction based on the second coefficient matrix at the j-th moment.

[0276] The re-prediction means: based on the second coefficient matrix of the first model at the jth moment and the initialized third coefficient matrix, as well as the second point cloud data, the first moving object set is re-predicted at the jth moment to obtain the fourth point cloud data. This step can be implemented with reference to the following formula 22:

[0277]

[0278] 1-707. Determine the compressed data.

[0279] 1-708. Send compressed data.

[0280] 1-709. Decompress the compressed data and store the decompression result.

[0281] The specific implementation of steps 1-707 to 1-709 can refer to the above Figure 5 The specific implementation method of steps 1-507 to 1-509.

[0282] Base station side: The base station can execute steps 2-701 to 2-703. Step 2-701 is a specific implementation of step 305, and steps 2-702 to 2-703 are specific implementations of step 306. The base station must execute step 2-701 after the terminal device executes step 1-704, that is, the base station receives the ΔB sent by the terminal device. j After that, step 2-701 can be executed. And the base station executes step 2-703 after the terminal device executes step 1-708, that is, the base station can execute step 2-703 after receiving the compressed data sent by the terminal device.

[0283] 2-701. Modify the second coefficient matrix based on the correction amount of the second coefficient matrix at the j-th moment to obtain the second coefficient matrix at the j-th moment.

[0284] 2-702. Perform another prediction based on the second coefficient matrix at the j-th moment.

[0285] 2-703. Decompress the compressed data and store the decompression result.

[0286] Among them, the specific implementation method of step 2-701 can refer to the specific implementation method of step 1-705; the specific implementation method of step 2-702 can refer to the specific implementation method of step 1-706; the specific implementation method of step 2-703 can refer to the specific implementation method of step 1-709.

[0287] It should be noted that the base station also needs to initialize the second coefficient matrix and the third coefficient matrix, and obtain the reconstructed point cloud data based on the compression method of the existing point cloud data. When j is 2, the reconstructed point cloud data is obtained based on the compression method of the existing point cloud data. And execute step 2-701 to obtain B2, and do not execute steps 2-702 to 2-703. When j is a positive integer greater than or equal to 3, execute steps 2-701 to 2-703.

[0288] By implementing Figure 7 The described embodiment can keep P unchanged during low-rank decomposition, so that ΔP j Does not exist, the terminal device only needs to send a ΔB j , compared to Figure 6 Send ΔB j and ΔP jThe solution can further reduce the transmission overhead of model correction during point cloud data interaction.

[0289] Optionally, the terminal device can also set ΔB j and u j After all are determined, ΔB j and u j Sent to the base station together.

[0290] The following is how to obtain compressed data in the above method embodiment, that is, The detailed process is described below. for The reverse process:

[0291] First, define the real point cloud data of the first moving object set as S, and the re-predicted point cloud data of the first moving object set as S is the first point cloud data or y in the above embodiment. j , The fourth point cloud data or The first moving object set includes M moving objects, and each moving object corresponds to point cloud data of N points (if N points are not collected during collection, random sampling can be performed until N points are satisfied), so S can be expressed by the following formula 23: It can be expressed by the following formula 24:

[0292]

[0293]

[0294] in, is the real point cloud data of the first point of the first moving object, It is the real point cloud data of the Nth point of the Mth moving object. The re-predicted point cloud data for the first point of the first moving object, Re-predict the point cloud data of the Nth point of the Mth moving object.

[0295] And, for For each element in for The index corresponding to the point cloud data of all points (including M×N points).

[0296] Method 1:

[0297] ① For each point cloud data in S, determine its The associated point cloud data in The index in .

[0298] Optionally, the associated point cloud data can be The point cloud data in S has the smallest error (i.e., is most correlated) with the point cloud data in S. For example, the error can be represented by Euclidean distance or the like.

[0299] For example, a function π(a) may be predefined, and π(a) may refer to the following formula 25:

[0300]

[0301] Where a is any point cloud data in S, b is any point cloud data in S, D() is used to calculate the Euclidean distance between point cloud data, b i For a The associated point cloud data in b i exist The meaning of formula 25 is that for any point cloud data a in S, the point b with the shortest Euclidean distance to a can be returned. i exist The index i in .

[0302] ②. Determine the first index set and / or the second index set.

[0303] First, the first index set X1 and the second index set X2 in this application are introduced: First, S1 is defined based on the following formula 26:

[0304]

[0305] Among them, ∈ is the first error threshold, b π(a) is the associated point cloud data of a, D(a,b π(a) )<∈ is used to represent a and b π(a) The error between them is less than the first error threshold. Formula 26 means that in the first point cloud data, the set of point cloud data whose error with the associated point cloud data is less than the first error threshold is S1.

[0306] Then, a first index set X1 and a second index set X2 are defined based on S1. X1 can refer to the following formula 27, and X2 can refer to the following formula 28:

[0307] X1={π(a):a∈S1} (Formula 27)

[0308]

[0309] Formula 27 means that X1 includes all point cloud data in S1. The meaning of formula 28 is that X2 includes The indices except X1 in .

[0310] ③. Determine the compressed data.

[0311] In the first method, the compressed data includes the encoding information of the index set and the remaining point information.

[0312] Encoding information of the index collection:

[0313] The encoding information of the index set includes information obtained by encoding the first index set or the second index set.

[0314] Optionally, the index set includes the first index set or the second index set. The index set including the first index set or the second index set may be preconfigured, or may be indicated by sending indication information when sending the encoding information of the index set.

[0315] Optionally, the index set is the set requiring the smallest number of transmission bits between the first index set and the second index set. When sending the encoding information of the index set, first indication information may be sent to indicate that the set with the smallest number of transmission bits is the first index set or the second index set.

[0316] Optionally, the manner of encoding the index set includes but is not limited to: direct entropy encoding, entropy encoding after taking the difference between adjacent elements of the index, etc. Encoding the index set can reduce the amount of data transmitted for the index set.

[0317] Decompression principle of the coded information of the index set: After the base station receives the coded information of the index set, X1 can be determined according to the index set (if the index set includes X1, X1 is directly determined; if the index set includes X2, X2 is obtained for Then, the base station determines the corrected first model according to X1. After that, the base station will The corresponding point cloud data is determined as S1 in the above S. In other words, the base station can find the point cloud data corresponding to X1 in the fourth point cloud data according to the index set, and the found point cloud data corresponding to X1 can represent the reconstructed point cloud data of S1 on the base station side.

[0318] Remaining point information:

[0319] The remaining point information includes information obtained by encoding the point cloud data in the first point cloud data except the point cloud data corresponding to the first index set: that is, the remaining point information is the encoding information of the point cloud data in S except S1.

[0320] Optionally, the encoding method of the residual point information may be quantized entropy coding or an existing coding algorithm such as Draco.

[0321] Decompression principle of the remaining point information: After the base station receives the remaining point information, the base station can represent the point cloud data obtained by decoding the remaining point information as the reconstructed point cloud data on the base station side of the point cloud data other than S1 in S. Furthermore, the union of the reconstructed point cloud data of S1 on the base station side and the reconstructed point cloud data of the point cloud data other than S1 in S on the base station side can be used as the reconstructed point cloud data of the first moving object set at the first moment.

[0322] Method 2:

[0323] ① For each point cloud data in S, determine its The associated point cloud data in The index in .

[0324] ②. Determine the first index set, the third index set, and the fourth index set, or determine two of the three index sets.

[0325] Method 2 defines S, Based on π(a), S1, and X1, S2 is also defined. S2 can be referred to the following formula 29:

[0326]

[0327] Among them, δ is the second error threshold, and Formula 29 means that in the first point cloud data, the set of point cloud data whose error with the associated point cloud data is greater than or equal to the first error threshold and less than the second error threshold is S2.

[0328] Then, based on S1 and S2, a third index set X3 and a fourth index set X4 are further defined. X3 can refer to the following formula 30, and X4 can refer to the following formula 31:

[0329] X3={π(a):a∈S2} (Formula 30)

[0330]

[0331] Formula 30 means that X3 includes each point cloud data in S2. The meaning of formula 31 is that X4 includes Indices other than X1 and X3.

[0332] ③. Determine the compressed data.

[0333] In the second method, the compressed data includes the encoding information of the index set, the difference information and the remaining point information.

[0334] Encoding information of the index collection:

[0335] The encoding information of the index set includes information obtained by encoding two of the first index set, the third index set and the fourth index set determined above.

[0336] Optionally, the index set includes two of the first index set, the third index set, and the fourth index set. These two may be pre-configured (for the pre-configured ones, only the pre-configured two may be determined in ②), or two may be determined arbitrarily, and indication information is sent when sending the encoding information of the index set to indicate which two are determined.

[0337] Optionally, the index sets are two sets of the first index set, the third index set, and the fourth index set that need to transmit a smaller number of bits. The terminal device needs to determine the first index set, the third index set, and the fourth index set in ②, and then select two sets with a smaller number of transmission bits from them, and when sending the coding information of the index set, it can send second indication information to indicate which two sets have a smaller number of transmission bits.

[0338] Optionally, the manner of encoding the index set includes but is not limited to: direct entropy encoding, entropy encoding after taking the difference between adjacent elements of the index, etc. Encoding the index set can reduce the amount of data transmitted for the index set.

[0339] Decompression principle of the coded information of the index set: After the base station receives the coded information of the index set, X1 and X3 can be determined according to the index set. Then, the base station determines After that, the base station will The corresponding point cloud data is determined as S1 in the above S. In other words, the base station can find the point cloud data corresponding to X1 in the fourth point cloud data according to the index set, and the found point cloud data corresponding to X1 can represent the reconstructed point cloud data of S1 on the base station side.

[0340] Difference information:

[0341] The difference information is information obtained by encoding the error corresponding to the third index set X3, and the error corresponding to the third index set is: the point cloud data corresponding to the third index set X3 in the first point cloud data (point cloud data in S2) and the error of the associated point cloud data.

[0342] Decompression principle of difference information: After the base station receives the difference information, it can Find the point cloud data corresponding to X3 in the image, and then decode the error pair obtained based on the difference information. The point cloud data corresponding to X3 in the figure is supplemented, and the supplemented point cloud data can represent the reconstructed point cloud data of S2 on the base station side.

[0343] Optionally, the difference may be encoded by quantization entropy coding or the like.

[0344] Remaining point information:

[0345] The remaining point information includes information obtained by encoding point cloud data in the first point cloud data except the point cloud data corresponding to the first index set and the third index set: that is, the remaining point information is the encoding information of the point cloud data in S except S1 and S2.

[0346] Optionally, the encoding method of the residual point information may be quantized entropy coding or an existing coding algorithm such as Draco.

[0347] Decompression principle of the remaining point information: After the base station receives the remaining point information, the base station can represent the point cloud data obtained by decoding the remaining point information as the reconstructed point cloud data on the base station side of the point cloud data other than S1 and S2 in S. Furthermore, the union of the reconstructed point cloud data of S1 on the base station side, the reconstructed point cloud data of S2 on the base station side, and the reconstructed point cloud data of the point cloud data other than S1 in S on the base station side can be used as the reconstructed point cloud data of the first moving object set at the first moment.

[0348] Compared with method 1, method 2 can make the point cloud data corresponding to the part of the remaining point information of method 1 with small error be indicated by difference information and index set, thereby reducing the remaining point information that needs to be sent and reducing the transmission overhead during point cloud data interaction.

[0349] Before the above embodiment determines the compressed data, the following various selection conditions can be used to select whether to encode the compressed data based on the above method (method one or two) or to encode the compressed data using the algorithm in the existing point cloud reporting method.

[0350] For example, the terminal device may execute Figure 8 The process shown:

[0351] 801. Determine correction data of the first model based on first point cloud data and a first model, where the first model is used to predict point cloud data of each moving object in a first moving object set.

[0352] 802. Correct the first model based on the correction data to obtain a corrected first model.

[0353] 803. Obtain selection conditions for an encoding method for compressing data.

[0354] 804. Determine, based on the selection condition, whether to obtain compressed data corresponding to the first point cloud data based on the first point cloud data and the corrected first model.

[0355] Among them, after executing 804, the terminal device can execute step 805 or step 806 according to the judgment result.

[0356] 805. Encode based on an existing method and send the encoded data.

[0357] Among them, existing methods include but are not limited to various traditional point cloud compression algorithms, such as V-PCC algorithm, LZMA algorithm, G-PCC algorithm, Draco algorithm, etc.

[0358] 806. Based on the first point cloud data and the corrected first model, obtain compressed data corresponding to the first point cloud data, and send the corrected data of the first model and the compressed data corresponding to the first point cloud data.

[0359] The following describes the selection conditions and how to make judgments based on the selection conditions in 804:

[0360] This application proposes two types of selection conditions:

[0361] The first type is to make a judgment based on the error predicted by the modified first model, that is, to make a judgment based on the first point cloud data, the fourth point cloud data and the first threshold. For example, when the predicted error is small, based on the first point cloud data and the modified first model, the compressed data corresponding to the first point cloud data can greatly reduce the data overhead of the transmission, and the above-mentioned method 1 or method 2 for determining the compressed data is executed; when the prediction deviation is large, based on the first point cloud data and the modified first model, when the compressed data corresponding to the first point cloud data is obtained, the data volume before and after compression is not much different, and encoding can be performed based on the existing method.

[0362] For example, the first type includes the following two selection conditions: 1) and 2) :

[0363] 1) Determine the norm h of the difference between the convex hulls of the first point cloud data and the fourth point cloud data. Compare h with a first threshold matching h: if h is less than the first threshold, the terminal device executes step 806; otherwise, the terminal device executes step 805.

[0364] 2) Determine the difference d between the center points of the first point cloud data and the fourth point cloud data. Compare d with a first threshold matching d: if d is less than the first threshold, the terminal device executes step 806; otherwise, the terminal device executes step 805.

[0365] The second type is to make a judgment based on the change of the first model before and after correction, that is, to make a judgment based on the coefficient matrix of the first model, the coefficient matrix of the corrected first model and the first threshold. The first threshold is a threshold that matches the selection condition. For example, when the change before and after correction is small, based on the first point cloud data and the corrected first model, obtaining the compressed data corresponding to the first point cloud data can greatly reduce the data overhead of transmission; otherwise, encoding can be performed based on the existing method.

[0366] For example, the first type includes the following three selection conditions: 3), 4), and 5):

[0367] 3) Determine the covariance Cov (covariance) of the coefficient matrix at the first moment and the coefficient matrix at the second moment. Compare Cov with a second threshold (such as 0.1) that matches Cov: If the absolute values ​​of all elements in Cov are less than the second threshold, execute step 806; otherwise, the terminal device executes step 805.

[0368] 4) Determine the correlation coefficient Cor (correlation coefficient) between the coefficient matrix at the first moment and the coefficient matrix at the second moment. Compare Cor with a second threshold value (such as 0.95) that matches Cor: If the absolute values ​​of all elements in Cor are greater than the second threshold value, execute step 806; otherwise, the terminal device executes step 805.

[0369] 5) Determine the sum of the diagonal elements s(trace) of the coefficient matrix at the first moment and the coefficient matrix at the second moment. Compare s with a first threshold (such as 1) that matches s: if s is greater than the second threshold, execute step 806; otherwise, the terminal device executes step 805.

[0370] Optionally, for the above-mentioned multiple selection conditions (such as the above-mentioned 5 types), judgment can be made according to a pre-agreed selection condition; or, a selection condition can be randomly selected for judgment; or, judgment can be made based on a combination of all the selection conditions.

[0371] Optionally, when executing step 805 or step 806, the terminal device also needs to report to the base station whether step 806 is executed, that is, whether the terminal device obtains compressed data corresponding to the first point cloud data based on the first point cloud data and the corrected first model. The reported information may be included in the compressed data.

[0372] Implementation Figure 8 The described embodiment can determine whether to use the scheme proposed in this application based on the compression effect of the scheme proposed in this application, so that when interacting with point cloud data, a better scheme can be selected from existing methods and the scheme proposed in this application to execute.

[0373] The following is an explanation of the relevant configuration information before interacting with point cloud data:

[0374] Optionally, the configuration information may be indicated by the terminal device each time the point cloud data is interacted with.

[0375] Optionally, the configuration information may be indicated by the terminal device when the solution proposed in the present application is used for the first interaction of point cloud data, and the configuration information indicated for the first time may be used for subsequent interactions of point cloud data.

[0376] Optionally, the configuration information may be indicated by the base station.

[0377] Optionally, configuration information may be indicated as specified in the protocol.

[0378] Configuration information includes one or more of the following:

[0379] Compression method configuration information: used to indicate the encoding method used when compressing data, including one or more of quantization entropy coding / NA (quantization only without entropy coding) / Draco / LZMA / MPEG / PCC, etc.

[0380] Coefficient matrix: when used to indicate the correction data of the first model, it is the correction amount of the first coefficient matrix (corresponding to Figure 5 ), or the correction amount of the second coefficient matrix and the correction amount of the third coefficient matrix (corresponding to Figure 6 ), or the correction amount of the second coefficient matrix (corresponding to Figure 7 ).

[0381] Coefficient matrix dimensions: the values ​​of X in the dimensions of the second coefficient matrix and the third coefficient matrix.

[0382] Coding method selection conditions: such as selecting one or more of conditions 1) to 5).

[0383] Index matching method before encoding: such as method 1 or method 2 to determine compressed data.

[0384] Compression parameters: including compression ratio, first error threshold ∈, and second error threshold δ.

[0385] Random seed: used to sample the corresponding points of each moving object to N points.

[0386] Time step: the length of time from the first moment to the second moment.

[0387] Quantization method: such as uniform quantization or non-uniform quantization.

[0388] The effects of the embodiments of the present application are described below, taking the point cloud data interaction in a traffic scenario as an example:

[0389] For example, if there are multiple straight moving objects with different directions and speeds, and multiple turning moving objects with different directions and speeds in a traffic scene, for example Fig. 9 As shown in Figure 1, the air interface overhead occupied by the three methods at different compression rates, namely, the traditional point cloud compression algorithm (such as the Draco algorithm), the compression method based on ODE modeling of a single moving object (abbreviated as ODE-S), and the compression method based on modeling of the first moving object set in this scheme (abbreviated as ODE-M), can be compared. Fig.10 :exist Fig.10 The horizontal axis is the air interface overhead, and the vertical axis is the NMSE mean square error. The NMSE is related to the compression rate, the first error threshold, and the second error threshold used in the solution. Fig.10 The corresponding relationship of each of the 9 groups of compression ratios, the first error threshold and the second error threshold used in the embodiment is shown in Table 1:

[0390] Table 1

[0391] Compression ∈ δ 15.19 2.3 2.3 14.74 2.5 2.5 14.13 2.5 2.5 13.5 2.3 2.3 13.07 2.3 2.3 12.05 1.5 1.2 10.97 1.5 1.2 10.14 1.5 1.2 8.61 1.2 0.8

[0392] Depend on Fig.10 It can be seen that, under the premise of the same NMSE mean square error, the air interface overhead required by ODE-M is less than that required by ODE-S, and the air interface overhead required by ODE-S is less than that of Draco. For example, compared with ODE-S, ODE-M can reduce the air interface overhead by about 18%.

[0393] For example, if there is a single straight moving object and multiple turning moving objects with different directions and speeds in the traffic scene, for example Fig.11 As shown, the comparison of the air interface overhead occupied by the Draco algorithm, ODE-S and ODE-M at different compression rates can be seen in Fig.12 : Fig.12 The corresponding relationship between the five groups of compression ratios, the first error threshold and the second error threshold used in the embodiment is shown in Table 2:

[0394] Table 2

[0395] Compression ∈ δ 14.37 3 3 12.72 3 3 10.48 1.4 1 8 1 0.5 7.53 1 0.3

[0396] Depend on Fig.12 It can be seen that under the premise of the same NMSE mean square error, the air interface overhead required by ODE-M is less than that required by ODE-S, and the air interface overhead required by ODE-S is less than that of Draco. For example, compared with ODE-S, ODE-M can reduce the air interface overhead by about 27%.

[0397] Based on the above example, when interacting with point cloud data, modeling and compressing multiple moving objects simultaneously can effectively reduce air interface overhead. In addition, this method can utilize the association between moving objects and improve the accuracy of the model's description of the state of each moving object.

[0398] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0399] The terms "component", "module", "system", etc. used in this specification are used to represent computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program and / or a computer. By way of illustration, both applications running on a computing device and a computing device can be components. One or more components may reside in a process and / or an execution thread, and a component may be located on a computer and / or distributed between two or more computers. In addition, these components may be executed from various computer-readable media having various data structures stored thereon. Components may, for example, communicate through local and / or remote processes according to signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system and / or a network, such as the Internet interacting with other systems through signals).

[0400] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0401] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps may be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0402] In the several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the above-mentioned units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0403] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0404] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0405] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc., specifically a processor in a computer device) to perform all or part of the steps of the above-mentioned methods of each embodiment of the present application. Among them, the aforementioned storage medium may include: U disk, mobile hard disk, magnetic disk, optical disk, read-only memory (Read-Only Memory, abbreviated: ROM) or random access memory (Random Access Memory, abbreviated: RAM) and other media that can store program codes.

[0406] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A data transmission method, characterized in that: Applied to a terminal device, the method comprises: Based on first point cloud data and a first model, correction data of the first model is determined; the first point cloud data is real point cloud data of a first moving object set at a first moment; the number of moving objects included in the first moving object set is multiple; the first model is used to predict point cloud data of each moving object in the first moving object set; Correcting the first model based on the correction data to obtain a corrected first model; Based on the first point cloud data and the corrected first model, obtaining compressed data corresponding to the first point cloud data; The corrected data of the first model and the compressed data corresponding to the first point cloud data are sent.

2. The method according to claim 1, characterized in that The step of determining correction data of the first model based on the first point cloud data and the first model includes: Based on the first model and the second point cloud data, an initial prediction is performed on the point cloud data of the first moving object set at the first moment to obtain third point cloud data, wherein the second point cloud data is the reconstructed point cloud data of the first moving object set at the second moment, and the second moment is a moment before the first moment; Based on the error between the first point cloud data and the third point cloud data, corrected data of the first model is obtained.

3. The method according to claim 2, characterized in that The correction data of the first model includes the correction amount of the first coefficient matrix; the first coefficient matrix is ​​used to indicate the change of the point cloud data of the first moving object set within a first time period, and the first time period is the time period between the first moment and the second moment.

4. The method according to claim 2, characterized in that: The correction data of the first model includes the correction amount of the second coefficient matrix and the correction amount of the third coefficient matrix; the second coefficient matrix and the third coefficient matrix are obtained by low-rank decomposition of the first coefficient matrix, and the first coefficient matrix is ​​used to indicate the change of point cloud data of the first moving object set within a first time period, and the first time period is the time period between the first moment and the second moment.

5. The method according to claim 2, characterized in that: The correction data of the first model includes a correction amount of a second coefficient matrix, where the second coefficient matrix is ​​obtained by low-rank decomposing the first coefficient matrix when the third coefficient matrix is ​​an initial matrix and the third coefficient matrix remains unchanged; The first coefficient matrix is ​​used to indicate changes in point cloud data of the first moving object set within a first time period, where the first time period is a time period between the first moment and the second moment.

6. The method according to claim 4 or 5, characterized in that: The dimension of the first coefficient matrix is ​​M×M, M is the number of motion objects included in the first motion object set, the dimension of the second coefficient matrix is ​​M×X, and the dimension of the third coefficient matrix is ​​X×M, M is a positive integer greater than 3, and X is a positive integer greater than or equal to 3 and less than or equal to M.

7. The method according to any one of claims 2 to 5, characterized in that: The obtaining, based on the first point cloud data and the corrected first model, compressed data corresponding to the first point cloud data includes: Based on the modified first model and the second point cloud data, re-predicting the point cloud data of the first moving object set at the first moment to obtain fourth point cloud data; Based on the fourth point cloud data, the first point cloud data is compressed to obtain the compressed data.

8. The method according to claim 7, characterized in that The compressed data includes encoding information of an index set and remaining point information; wherein, the index set is a first index set or a second index set, the first index set includes indexes corresponding to point cloud data in the first point cloud data whose errors with associated point cloud data are less than a first error threshold, and the associated point cloud data are point cloud data in the fourth point cloud data that are associated with point cloud data in the first point cloud data; the second index set is the complement of the first index set in a preset index set; the preset index set is an index set corresponding to the fourth point cloud data; the remaining point information is information obtained by encoding point cloud data in the first point cloud data except for point cloud data corresponding to the first index set.

9. The method according to claim 8, characterized in that The compressed data also includes first indication information, where the first indication information is used to indicate that the index set is the set with the smallest number of bits required to be transmitted between the first index set and the second index set.

10. The method according to claim 7, characterized in that The compressed data includes encoding information, difference information and remaining point information of an index set; wherein the index set includes two of a first index set, a third index set and a fourth index set, the first index set includes indexes corresponding to point cloud data in the first point cloud data whose errors with associated point cloud data are less than a first error threshold, the third index set includes indexes corresponding to point cloud data in the first point cloud data whose errors with associated point cloud data are greater than or equal to the first error threshold and less than a second error threshold, the associated point cloud data is point cloud data in the fourth point cloud data associated with point cloud data in the first point cloud data; the fourth index set is a set of indexes in a preset index set other than the first index set and the third index set, the preset index set is an index set corresponding to the fourth point cloud data; the remaining point information is information obtained by encoding point cloud data in the first point cloud data other than point cloud data corresponding to the first index set and the third index set; the difference information is information obtained by encoding errors corresponding to the third index set, the errors corresponding to the third index set are errors between point cloud data corresponding to the third index set in the first point cloud data and the associated point cloud data.

11. The method according to claim 10, characterized in that The compressed data also includes second indication information, where the second indication information is used to indicate that the index set includes two sets of the first index set, the third index set, and the fourth index set that require a smaller number of bits to be transmitted.

12. The method according to any one of claims 2 to 5, characterized in that: Before determining the correction data of the first model based on the first point cloud data and the first model, the method further includes: Obtaining reconstructed point cloud data at the second moment; Acquire the real point cloud data collected at the first moment; The first point cloud data is acquired from the real point cloud data at the first moment, wherein the first moving object set is an intersection between a moving object set corresponding to the reconstructed point cloud data at the second moment and a moving object set corresponding to the real point cloud data at the first moment; Send third indication information, where the third indication information is used to indicate the first moving object set.

13. The method according to claim 7, characterized in that The method further comprises: Acquire selection conditions of the encoding method of the compressed data; If the type of the selection condition is the first type, based on the first point cloud data, the fourth point cloud data and the first threshold, it is determined whether compressed data corresponding to the first point cloud data is obtained based on the first point cloud data and the corrected first model; and the first threshold matches the selection condition.

14. The method according to claim 7, characterized in that The method further comprises: Acquire selection conditions of the encoding method of the compressed data; If the type of the selection condition is the second type, based on the coefficient matrix of the first model, the coefficient matrix of the corrected first model and the second threshold, it is determined whether compressed data corresponding to the first point cloud data is obtained based on the first point cloud data and the corrected first model; the second threshold matches the selection condition.

15. The method according to any one of claims 1 to 5, characterized in that Before determining the correction data of the first model based on the first point cloud data and the first model, the method further includes: Send configuration information; Among them, the configuration information includes one or more configuration information of compression method, coefficient matrix, coefficient matrix dimension, encoding method selection conditions, pre-encoding index matching method, compression parameters, random seed, time step, and quantization method.

16. A data transmission method, characterized in that: Applied to a base station, the method comprises: Receive the corrected data of the first model and the compressed data corresponding to the first point cloud data; the first point cloud data is the real point cloud data of the first moving object set at the first moment; the number of moving objects included in the first moving object set is multiple; the first model is used to predict the point cloud data of each moving object in the first moving object set; Correcting the first model based on the correction data to obtain a corrected first model; Based on the modified first model, the compressed data is decompressed to obtain reconstructed point cloud data of the first moving object set at the first moment.

17. The method according to claim 16, characterized in that The decompressing the compressed data based on the modified first model to obtain the reconstructed point cloud data of the first moving object set at the first moment includes: Based on the corrected first model and the second point cloud data, predicting the point cloud data of the first moving object set at the first moment, to obtain fourth point cloud data; the second point cloud data is the reconstructed point cloud data of the first moving object set at the second moment, and the second moment is a moment before the first moment; Based on the fourth point cloud data, the compressed data is decompressed to obtain reconstructed point cloud data of the first moving object set at the first moment.

18. The method according to claim 17, characterized in that The correction data of the first model includes the correction amount of the first coefficient matrix; the first coefficient matrix is ​​used to indicate the change of the point cloud data of the first moving object set within a first time period, and the first time period is the time period between the first moment and the second moment.

19. The method according to claim 17, characterized in that The correction data of the first model includes the correction amount of the second coefficient matrix and the correction amount of the third coefficient matrix; the second coefficient matrix and the third coefficient matrix are obtained by low-rank decomposition of the first coefficient matrix, and the first coefficient matrix is ​​used to indicate the change of point cloud data of the first moving object set within a first time period, and the first time period is the time period between the first moment and the second moment.

20. The method according to claim 17, characterized in that The correction data of the first model includes a correction amount of a second coefficient matrix, where the second coefficient matrix is ​​obtained by low-rank decomposing the first coefficient matrix when the third coefficient matrix is ​​an initial matrix and the third coefficient matrix remains unchanged; The first coefficient matrix is ​​used to indicate changes in point cloud data of the first moving object set within a first time period, where the first time period is a time period between the first moment and the second moment.

21. The method according to claim 19 or 20, characterized in that The dimension of the first coefficient matrix is ​​M×M, M is the number of motion objects included in the first motion object set, the dimension of the second coefficient matrix is ​​M×X, and the dimension of the third coefficient matrix is ​​X×M, M is a positive integer greater than 3, and X is a positive integer greater than or equal to 3 and less than or equal to M.

22. The method according to claim 17, characterized in that The compressed data includes encoding information of an index set and remaining point information; wherein the index set is a first index set or a second index set; the first index set includes indexes corresponding to point cloud data in the first point cloud data whose errors with associated point cloud data are less than a first error threshold, and the associated point cloud data are point cloud data in the fourth point cloud data that are associated with point cloud data in the first point cloud data; the second index set is the complement of the first index set in a preset index set; the preset index set is the index set corresponding to the fourth point cloud data; the remaining point information is information obtained by encoding point cloud data in the first point cloud data except for point cloud data corresponding to the first index set.

23. The method according to claim 22, characterized in that The compressed data also includes first indication information, where the first indication information is used to indicate that the index set is the set with the smallest number of bits required to be transmitted between the first index set and the second index set.

24. The method according to claim 17, characterized in that The compressed data includes encoding information, difference information and remaining point information of an index set; wherein the index set includes two of a first index set, a third index set and a fourth index set; the first index set includes indexes corresponding to point cloud data in the first point cloud data whose errors with associated point cloud data are less than a first error threshold, the third index set includes indexes corresponding to point cloud data in the first point cloud data whose errors with associated point cloud data are greater than or equal to the first error threshold and less than a second error threshold, and the associated point cloud data is point cloud data in the fourth point cloud data associated with point cloud data in the first point cloud data; the fourth index set is a set of indexes in a preset index set other than the first index set and the third index set, and the preset index set is an index set corresponding to the fourth point cloud data; the remaining point information is information obtained by encoding point cloud data in the first point cloud data other than point cloud data corresponding to the first index set and the third index set; the difference information is information obtained by encoding errors corresponding to the third index set, and the errors corresponding to the third index set are errors between point cloud data corresponding to the third index set in the first point cloud data and the associated point cloud data.

25. The method according to claim 24, characterized in that The compressed data also includes second indication information, where the second indication information is used to indicate that the index set includes two sets of the first index set, the third index set and the fourth index set that require a smaller number of bits to be transmitted.

26. The method according to claim 17, characterized in that Before receiving the corrected data of the first model and the compressed data corresponding to the first point cloud data, the method further includes: receiving third indication information, where the third indication information is used to indicate the first moving object set; Obtaining reconstructed point cloud data at the second moment; Based on the third indication information and the reconstructed point cloud data at the second moment, the first moving object set and the second point cloud data are determined.

27. The method according to any one of claims 16 to 20, characterized in that Before receiving the corrected data of the first model and the compressed data corresponding to the first point cloud data, the method further includes: Send configuration information; Among them, the configuration information includes one or more configuration information of compression method, coefficient matrix, coefficient matrix dimension, encoding method selection conditions, pre-encoding index matching method, compression parameters, random seed, time step, and quantization method.

28. A communication device, characterized in that: Comprising units for performing the method as claimed in any one of claims 1 to 15 or claims 16 to 27.

29. A communication device, characterized in that: The method comprises a processor and a memory, wherein the processor and the memory are coupled, and the processor is used to implement the method according to any one of claims 1 to 15 or claims 16 to 27.

30. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is called by the computer, the computer executes the method according to any one of claims 1 to 15 or claims 16 to 27.

Images