Data transmission method, electronic device and computer-readable storage medium

Abstract

The embodiments of the present application provide a data transmission method, an electronic device and a computer-readable storage medium. The method comprises: determining correction data of a first model on the basis of first point cloud data and the first model, wherein the first point cloud data is real point cloud data of a first moving object set at a first moment, there are a plurality of moving objects included in the first moving object set, and the first model is used for predicting point cloud data of each moving object in the first moving object set; correcting the first model on the basis of the correction data, so as to obtain a corrected first model; on the basis of the first point cloud data and the corrected first model, obtaining compressed data corresponding to the first point cloud data; and sending the correction data of the first model and the compressed data corresponding to the first point cloud data. By implementing the present application, the transmission overhead during point cloud data interaction can be effectively reduced.

Description

Data transmission method, electronic device, and computer-readable storage medium

[0001] This application claims priority to the Chinese patent application with application number 202311435958.9 filed with the State Intellectual Property Office of China on October 30, 2023, and priority to the Chinese patent application with the invention name “Data transmission method, electronic device and computer-readable storage medium”, all contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of communication technology, and in particular to a data transmission method, an electronic device, and a computer-readable storage medium. Background Art

[0003] Point cloud data is crucial raw data for future communication perception scenarios, suitable for imaging, environmental reconstruction, and target detection. To perform subsequent tasks such as multi-view observation, motion detection, and behavior prediction, multi-frame time-series point cloud data sensed by different terminals at multiple locations and angles must be sent to the base station for information fusion.

[0004] In order to save the air interface resources occupied by the interaction of multi-frame joint perception point cloud data, the collected point cloud data can be compressed, thereby reducing the amount of data that needs to be transmitted between the terminal and the base station when the point cloud data is interacting. For example, the point cloud data generated by objects that have changed between the previous and next frames (such as due to movement or change in perspective) can be compressed by using ordinary differential equation (ODE) modeling. However, ODE modeling is established for each moving object in the perception scene, so each moving object has a corresponding ODE model correction that needs to be sent to the base station. The more moving objects there are in the perception scene, the more ODE model corrections need to be transmitted, and each moving object also has a set of compressed point cloud data that needs to be sent. In this way, the terminal still has a large transmission overhead when interacting with the base station for point cloud data. How to reduce the transmission overhead when interacting with point cloud data has become one of the problems that need to be solved urgently.

[0005] Summary of the Invention

[0006] The embodiments of the present application provide a data transmission method, an electronic device, and a computer-readable storage medium, which can effectively reduce the transmission overhead during point cloud data interaction.

[0007] In a first aspect, the present application provides a data transmission method, applied to a terminal device, the method comprising: determining the corrected data of the first model based on first point cloud data and a first model; the first point cloud data is the real point cloud data of the first moving object set at the first moment; the number of moving objects included in the first moving object set is multiple; the first model is used to predict the point cloud data of each moving object in the first moving object set; correcting the first model based on the corrected data to obtain a corrected first model; obtaining compressed data corresponding to the first point cloud data based on the first point cloud data and the corrected first model; and sending the corrected data of the first model and the compressed data corresponding to the first point cloud data.

[0008] In the method described in the first aspect, the first model can be used to predict point cloud data for each moving object in the first moving object set. Therefore, the first model models the first moving object set as a whole. Furthermore, when obtaining model corrections and compressing point cloud data, there is no need to determine the model corrections for each moving object separately and obtain a corresponding set of compressed data for each moving object. Instead, the corrections for the first model are determined, the model corrections corresponding to the first moving object set are expressed as the corrections for the first model, and the first point cloud data are uniformly processed based on the corrected first model to obtain compressed data corresponding to the first moving object set. Accordingly, sending the corrections for the first model and the compressed data for the first moving object set during point cloud data exchange can effectively reduce the amount of corrections and compressed data to be sent, thereby reducing the transmission overhead during point cloud data exchange. Furthermore, this approach of modeling the first moving object set as a whole can effectively improve processing efficiency and enable the obtained first model to leverage the correlations between the individual moving objects in the first moving object set to obtain more accurate prediction results.

[0009] In one possible implementation, the above-mentioned determination of the corrected data of the first model based on the first point cloud data and the first model includes: performing an initial prediction on the point cloud data of the first moving object set at the first moment based on the first model and the second point cloud data to obtain third point cloud data, the second point cloud data being the reconstructed point cloud data of the first moving object set at the second moment, the second moment being the moment before the first moment; and obtaining the corrected data of the first model based on the error between the first point cloud data and the third point cloud data.

[0010] In this manner, based on the error between the first point cloud data and the third point cloud data, correction data of the first model can be obtained, so that the first model can be corrected so that the corrected first model is more accurate in predicting point cloud data.

[0011] In one possible implementation, the correction data of the first model includes the correction amount of the first coefficient matrix; the first coefficient matrix is ​​used to indicate the changes in the point cloud data of the first moving object set within a first time period, and the first time period is the time period between the first moment and the second moment.

[0012] In this method, the first coefficient matrix is ​​used to indicate the changes in the point cloud data of the first set of moving objects within the first time period, which is equivalent to the prediction of the first model on the point cloud data of the first set of moving objects can be represented by the first coefficient matrix. Therefore, by sending the correction amount of the first coefficient matrix, the device that receives the correction amount of the first coefficient matrix can obtain the corrected first model by correcting the first coefficient matrix.

[0013] In one possible implementation, the correction data of the first model includes the correction amount of the second coefficient matrix and the correction amount of the third coefficient matrix; the second coefficient matrix and the third coefficient matrix are obtained by low-rank decomposition of the first coefficient matrix, and the first coefficient matrix is ​​used to indicate the changes in point cloud data of the first moving object set within a first time period, and the first time period is the time period between the first moment and the second moment.

[0014] In this approach, low-rank decomposition of the first coefficient matrix yields second and third coefficient matrices with smaller dimensions than the first coefficient matrix. Accordingly, when transmitting correction data, the corrections to the second and third coefficient matrices are transmitted, effectively reducing the transmission overhead of the correction data.

[0015] In one possible implementation, the correction data of the first model includes the correction amount of the second coefficient matrix, the second coefficient matrix is ​​obtained by low-rank decomposition of the first coefficient matrix when the third coefficient matrix is ​​the initial matrix and the third coefficient matrix remains unchanged, and the first coefficient matrix is ​​used to indicate the changes in point cloud data of the first moving object set within a first time period, and the first time period is the time period between the first moment and the second moment.

[0016] In this approach, when the first coefficient matrix is ​​low-rank decomposed, the third coefficient matrix remains unchanged. Therefore, when obtaining the corrected data, only the corrections to the second coefficient matrix need be obtained, without obtaining the corrections to the third coefficient matrix. Accordingly, when transmitting the corrected data, the transmission overhead of the corrected data can be effectively reduced by only transmitting the corrections to the coefficient matrix after the low-rank decomposition.

[0017] In one possible implementation, the dimension of the first coefficient matrix is ​​M×M, where M is the number of moving objects included in the first moving object set; the dimension of the second coefficient matrix is ​​M×X; the dimension of the third coefficient matrix is ​​X×M, where M is a positive integer greater than 3, and X is a positive integer greater than or equal to 3 and less than or equal to M.

[0018] In this way, when the first coefficient matrix is ​​decomposed into the second coefficient matrix and the third coefficient matrix in a low-rank manner, the minimum value of X in the dimensions of the second coefficient matrix and the third coefficient matrix is ​​3. The minimum value of X can ensure that when the low-rank decomposition reduces the dimension of the coefficient matrix, the changes in the point cloud data contained in any of the three coordinate dimensions (i.e., the x-axis, the y-axis, and the z-axis) are not lost.

[0019] In one possible implementation, the above-mentioned method of obtaining compressed data corresponding to the first point cloud data based on the first point cloud data and the corrected first model includes: re-predicting the point cloud data of the first moving object set at the first moment based on the corrected first model and the second point cloud data to obtain fourth point cloud data; and compressing the first point cloud data based on the fourth point cloud data to obtain compressed data.

[0020] In this method, the fourth point cloud data can be predicted again based on the corrected first model, and then the first point cloud data can be compressed according to the fourth point cloud data to obtain compressed data, which is the compressed data corresponding to the first moving object set as a whole.

[0021] In one possible implementation, the compressed data includes encoding information of an index set and remaining point information; wherein the index set is a first index set or a second index set, the first index set includes indexes corresponding to point cloud data in the first point cloud data whose error with the associated point cloud data is less than a first error threshold, and the associated point cloud data is point cloud data in the fourth point cloud data that is associated with the point cloud data in the first point cloud data; the second index set is the complement of the first index set in the preset index set; the preset index set is the index set corresponding to the fourth point cloud data; the remaining point information is information obtained by encoding point cloud data in the first point cloud data except for the point cloud data corresponding to the first index set.

[0022] In this approach, the first point cloud data can be divided into two parts: the first part is the point cloud data with an error less than a first error threshold, and the second part is the remaining portion of the first point cloud data excluding the first part. Therefore, by sending an index set, the first portion of the first point cloud data can be indicated by the index set, rather than directly sending the first portion. Furthermore, by sending the remaining point information, the second portion can be represented by the remaining point information (i.e., the remaining point information can be directly encoded by the second portion).

[0023] In a possible implementation, the compressed data further includes first indication information, where the first indication information is used to indicate that the index set is the set requiring the smallest number of transmission bits between the first index set and the second index set.

[0024] In this manner, the first indication information may also be sent, and when sending the index set, the set requiring the least number of transmission bits between the first index set and the second index set is sent, thereby further reducing the transmission overhead during point cloud data interaction.

[0025] In a possible implementation, the compressed data includes encoding information, difference information and remaining point information of an index set; wherein the index set includes two of a first index set, a third index set and a fourth index set, the first index set includes indexes corresponding to point cloud data in the first point cloud data whose errors with associated point cloud data are less than a first error threshold, the third index set includes indexes corresponding to point cloud data in the first point cloud data whose errors with associated point cloud data are greater than or equal to the first error threshold and less than a second error threshold, and the associated point cloud data are point cloud data in the fourth point cloud data that are associated with the point cloud data in the first point cloud data; the fourth index set is a set consisting of indexes in the preset index set other than the first index set and the third index set, and the preset index set is the index set corresponding to the fourth point cloud data; the remaining point information is information obtained by encoding point cloud data in the first point cloud data other than the point cloud data corresponding to the first index set and the third index set; the difference information is information obtained by encoding the error corresponding to the third index set, and the error corresponding to the third index set is the error between the point cloud data corresponding to the third index set in the first point cloud data and the associated point cloud data.

[0026] In this method, the first point cloud data can be divided into three parts, wherein the first part is the point cloud data with an error less than the first error threshold, the second part is the point cloud data with an error greater than or equal to the first error threshold and less than the second error threshold, and the third part is the remaining part of the first point cloud data except the first and second parts. Therefore, the first part of the first point cloud data can be indicated by an index set without directly sending the first part; the second part of the first point cloud data can be indicated by an index set and difference information without directly sending the second part; and the third part can be represented by the remaining point information (that is, the remaining point information is directly encoded by the third part). Based on this method, the point cloud data corresponding to the part of the remaining point information with a small error can be indicated by the difference information and the index set, and the remaining point information that needs to be sent will not be too much, which can further reduce the transmission overhead during point cloud data interaction.

[0027] In a possible implementation, the compressed data further includes second indication information, where the second indication information is used to indicate that the index set includes two sets requiring a smaller number of transmission bits among the first index set, the third index set, and the fourth index set.

[0028] In this method, a second indication message can also be sent, and when sending the index set, two sets requiring smaller number of transmission bits among the first index set, the second index set and the third index set are sent, thereby further reducing the transmission overhead during point cloud data interaction.

[0029] In a possible implementation, before determining the corrected data of the first model based on the first point cloud data and the first model, the method further includes: obtaining the reconstructed point cloud data at the second moment; obtaining the real point cloud data collected at the first moment; obtaining the first point cloud data from the real point cloud data at the first moment, the first motion object set being the intersection between the motion object set corresponding to the reconstructed point cloud data at the second moment and the motion object set corresponding to the real point cloud data at the first moment; and sending third indication information, the third indication information being used to indicate the first motion object set.

[0030] In this method, the first point cloud data and the first moving object set can be determined from the real point cloud data at the first moment, and indication information of the first moving object set can be sent so that the device that receives the indication information of the first moving object set can determine the first moving object set and determine the second point cloud data from the reconstructed point cloud data at the second moment.

[0031] In one possible implementation, the method further includes: obtaining selection conditions for the encoding method of the compressed data; if the type of the selection conditions is the first type, judging whether compressed data corresponding to the first point cloud data is obtained based on the first point cloud data, the fourth point cloud data and the first threshold value; and the first threshold value matches the selection conditions.

[0032] Based on this method, it is possible to determine whether compressed data corresponding to the first point cloud data can be obtained based on the first point cloud data and the corrected first model, based on the predicted deviation of the point cloud data. For example, when the predicted deviation is small, if the compressed data corresponding to the first point cloud data obtained based on the first point cloud data and the corrected first model is an effective compression method (which can greatly reduce the data overhead of transmission), compression can be performed based on this effective compression method. When the predicted deviation is large, if the compressed data corresponding to the first point cloud data obtained based on the first point cloud data and the corrected first model has a small difference in the amount of data before and after compression, point cloud data interaction can be performed based on the existing point cloud compression method.

[0033] In one possible implementation, the method further includes: obtaining selection conditions for the encoding method of the compressed data; if the type of the selection conditions is the second type, then judging whether compressed data corresponding to the first point cloud data is obtained based on the first point cloud data and the corrected first model based on the coefficient matrix of the first model, the coefficient matrix of the corrected first model, and the second threshold; the second threshold matches the selection conditions.

[0034] Based on this method, it is possible to determine whether compressed data corresponding to the first point cloud data can be obtained based on the first point cloud data and the corrected first model, based on the changes before and after correction. For example, when the changes before and after correction are small, if the compressed data corresponding to the first point cloud data obtained based on the first point cloud data and the corrected first model is an effective compression method (which can significantly reduce transmission data overhead), compression can be performed based on this effective compression method. When the changes before and after correction are large, if the compressed data corresponding to the first point cloud data obtained based on the first point cloud data and the corrected first model has a similar amount of data before and after compression, point cloud data interaction can be performed based on existing point cloud compression methods.

[0035] In one possible implementation, before determining the corrected data of the first model based on the first point cloud data and the first model, the method further includes: sending configuration information; wherein the configuration information includes one or more configuration information of compression method, coefficient matrix, coefficient matrix dimension, encoding method selection conditions, pre-encoding index matching method, compression parameters, random seed, time step, and quantization method.

[0036] Based on this method, the terminal device can configure various configuration information involved in point cloud data interaction.

[0037] In second aspect, the present application provides a data transmission method, applied to a base station, the method comprising: receiving corrected data of a first model and compressed data corresponding to first point cloud data; the first point cloud data is the real point cloud data of a first moving object set at the first moment; the number of moving objects included in the first moving object set is multiple; the first model is used to predict the point cloud data of each moving object in the first moving object set; the first model is corrected based on the corrected data to obtain a corrected first model; based on the corrected first model, the compressed data is decompressed to obtain the reconstructed point cloud data of the first moving object set at the first moment.

[0038] In the method described in the second aspect, the first model can be used to predict the point cloud data of each moving object in the first moving object set, so the first model models the first moving object set as a whole. Furthermore, the model correction amount corresponding to the first moving object set can be represented by the correction amount of the received first model, without the need to receive the model correction amount of each moving object. At the same time, the compressed data corresponding to the first moving object set obtained after unified processing of the first point cloud data can be received, without the need to receive a set of compressed data corresponding to each moving object. This method can effectively reduce the number of correction amounts and compressed data to be received, thereby reducing the transmission overhead during point cloud data interaction. Secondly, this method of modeling the first moving object set as a whole can improve the processing efficiency of model correction and decompression, and because the first model can utilize the correlation between the movements of each moving object in the first moving object set, a more accurate reconstructed point cloud result of the first moving object set at the first moment can be obtained during decompression.

[0039] In a possible implementation, the above-mentioned decompression of the compressed data based on the corrected first model to obtain the reconstructed point cloud data of the first moving object set at the first moment includes: based on the corrected first model and the second point cloud data, predicting the point cloud data of the first moving object set at the first moment to obtain fourth point cloud data; the second point cloud data is the reconstructed point cloud data of the first moving object set at the second moment, and the second moment is the moment before the first moment; based on the fourth point cloud data, decompressing the compressed data to obtain the reconstructed point cloud data of the first moving object set at the first moment.

[0040] In this method, the fourth point cloud data can be predicted based on the corrected first model, and then the compressed data can be decompressed based on the corrected first model and the fourth point cloud data to obtain the reconstructed point cloud data of the first moving object set at the first moment.

[0041] In one possible implementation, the correction data of the first model includes the correction amount of the first coefficient matrix; the first coefficient matrix is ​​used to indicate the changes in the point cloud data of the first moving object set within a first time period, and the first time period is the time period between the first moment and the second moment.

[0042] In this method, the first coefficient matrix is ​​used to indicate the changes in the point cloud data of the first moving object set within the first time period, which is equivalent to the first model's prediction of the point cloud data of the first moving object set being represented by the first coefficient matrix. Therefore, by receiving the correction amount of the first coefficient matrix, the first coefficient matrix can be updated to obtain the corrected first model.

[0043] In one possible implementation, the correction data of the first model includes the correction amount of the second coefficient matrix and the correction amount of the third coefficient matrix; the second coefficient matrix and the third coefficient matrix are obtained by low-rank decomposition of the first coefficient matrix; the first coefficient matrix is ​​used to indicate the changes in point cloud data of the first moving object set within a first time period, and the first time period is the time period between the first moment and the second moment.

[0044] In this approach, low-rank decomposition of the first coefficient matrix yields second and third coefficient matrices with smaller dimensions than the first coefficient matrix. Accordingly, when receiving correction data, corrections to the second and third coefficient matrices can be received, effectively reducing transmission overhead for the correction data.

[0045] In one possible implementation, the correction data of the first model includes the correction amount of the second coefficient matrix, and the second coefficient matrix is ​​obtained by low-rank decomposition of the first coefficient matrix when the third coefficient matrix is ​​the initial matrix and the third coefficient matrix remains unchanged; the first coefficient matrix is ​​used to indicate the changes in point cloud data of the first moving object set within a first time period, and the first time period is the time period between the first moment and the second moment.

[0046] In this method, when the first coefficient matrix is ​​subjected to low-rank decomposition, the third coefficient matrix can be kept unchanged. Therefore, when receiving the correction data, only the correction amount of the second coefficient matrix needs to be received, and there is no need to receive the correction amount of the third coefficient matrix, which can effectively reduce the transmission overhead of the correction data.

[0047] In one possible implementation, the dimension of the first coefficient matrix is ​​M×M, where M is the number of moving objects included in the first moving object set; the dimension of the second coefficient matrix is ​​M×X; the dimension of the third coefficient matrix is ​​X×M, where M is a positive integer greater than 3, and X is a positive integer greater than or equal to 3 and less than or equal to M.

[0048] In this way, when the first coefficient matrix is ​​decomposed into the second coefficient matrix and the third coefficient matrix in a low-rank manner, the minimum value of X in the dimensions of the second coefficient matrix and the third coefficient matrix is ​​3. The minimum value of X can ensure that the low-rank decomposition reduces the dimension of the coefficient matrix without losing the changes in the point cloud data contained in any of the three coordinate dimensions (i.e., the x-axis, the y-axis, and the z-axis).

[0049] In one possible implementation, the compressed data includes encoding information of an index set and remaining point information; wherein the index set is a first index set or a second index set; the first index set includes indexes corresponding to point cloud data in the first point cloud data whose error with the associated point cloud data is less than a first error threshold, and the associated point cloud data is point cloud data in the fourth point cloud data that is associated with the point cloud data in the first point cloud data; the second index set is the complement of the first index set in the preset index set; the preset index set is the index set corresponding to the fourth point cloud data; the remaining point information is information obtained by encoding point cloud data in the first point cloud data except for the point cloud data corresponding to the first index set.

[0050] In this method, the first point cloud data can be divided into two parts, where the first part is the point cloud data with an error less than a first error threshold, and the second part is the remaining part of the first point cloud data excluding the first part. Therefore, the first part of the first point cloud data can be indicated by receiving an index set in the compressed data instead of directly receiving the first part, and the second part can be represented by receiving the remaining point information in the compressed data. Furthermore, the first part of the first point cloud data can be reconstructed from the fourth point cloud data based on the index set. Then, based on the remaining point information and the reconstructed first part, the reconstructed point cloud data of the first moving object set at the first moment is obtained.

[0051] In a possible implementation, the compressed data further includes first indication information, where the first indication information is used to indicate that the index set is the set requiring the smallest number of transmission bits between the first index set and the second index set.

[0052] In this manner, the first indication information can also be received, and when receiving the index set, the set requiring the least number of bits to be transmitted between the first index set and the second index set is received, thereby further reducing the transmission overhead during point cloud data interaction.

[0053] In a possible implementation, the compressed data includes encoding information, difference information and remaining point information of an index set; wherein the index set includes two of a first index set, a third index set and a fourth index set; the first index set includes the indexes corresponding to the point cloud data in the first point cloud data whose error with the associated point cloud data is less than a first error threshold, the third index set includes the indexes corresponding to the point cloud data in the first point cloud data whose error with the associated point cloud data is greater than or equal to the first error threshold and less than a second error threshold, and the associated point cloud data is the point cloud data in the fourth point cloud data that is associated with the point cloud data in the first point cloud data; the fourth index set is a set consisting of indexes in the preset index set other than the first index set and the third index set, and the preset index set is the index set corresponding to the fourth point cloud data; the remaining point information is information obtained by encoding the point cloud data in the first point cloud data other than the point cloud data corresponding to the first index set and the third index set; the difference information is information obtained by encoding the error corresponding to the third index set, and the error corresponding to the third index set is the error between the point cloud data corresponding to the third index set in the first point cloud data and the associated point cloud data.

[0054] In this method, the first point cloud data can be divided into three parts: the first part comprises point cloud data with an error less than a first error threshold, the second part comprises point cloud data with an error greater than or equal to the first error threshold and less than a second error threshold, and the third part comprises the remainder of the first point cloud data excluding the first and second parts. Therefore, the first part of the first point cloud data can be indicated by receiving an index set in the compressed data, rather than directly receiving the first part; the second part of the first point cloud data can be indicated by receiving an index set and difference information, rather than directly receiving the second part; and the third part of the first point cloud data can be represented by receiving the remaining point information. Furthermore, the first part can be reconstructed from the fourth point cloud data based on the index set, and the second part can be reconstructed from the fourth point cloud data based on the index set and difference information. Finally, based on the first part, the second part, and the remaining point information, the reconstructed point cloud data of the first moving object set at the first moment can be obtained.

[0055] In a possible implementation, the compressed data further includes second indication information, where the second indication information is used to indicate that the index set includes two sets requiring a smaller number of transmission bits among the first index set, the third index set, and the fourth index set.

[0056] In this method, a second indication information can also be received, and when receiving an index set, two sets requiring a smaller number of bits to be transmitted from the first index set, the second index set, and the third index set are received, thereby further reducing the transmission overhead during point cloud data interaction.

[0057] In one possible implementation, before receiving the corrected data of the first model and the compressed data corresponding to the first point cloud data, the method also includes: receiving third indication information, the third indication information being used to indicate the first set of motion objects; obtaining reconstructed point cloud data at a second moment; and determining the first set of motion objects and the second point cloud data based on the third indication information and the reconstructed point cloud data at the second moment.

[0058] In this manner, the first moving object set and the second point cloud data may be determined based on the third indication information.

[0059] In one possible implementation, before receiving the corrected data of the first model and the compressed data corresponding to the first point cloud data, the method further includes: sending configuration information; wherein the configuration information includes one or more configuration information of compression method, coefficient matrix, encoding method selection conditions, pre-encoding index matching method, compression parameters, coefficient matrix dimension, random seed, time step, and quantization method.

[0060] In this way, the base station can configure various configuration information involved in point cloud data interaction.

[0061] In a third aspect, the present application provides a communication system, which includes a terminal device and a base station in any possible implementation of the above-mentioned first aspect or the above-mentioned second aspect, and is used to execute the data transmission method in the above-mentioned first aspect or the above-mentioned second aspect and any possible implementation of it.

[0062] In a fourth aspect, the present application provides a communication device, which may be a terminal device or a base station, or a device in a terminal device or a base station, or a device that can be used in conjunction with a terminal device or a base station. The communication device may also be a chip system. The communication device may execute the method described in the first aspect or the second aspect. The functions of the communication device may be implemented by hardware, or by hardware executing corresponding software implementations. The hardware or software includes one or more units or modules corresponding to the above functions. The unit or module may be software and / or hardware. The operations and beneficial effects performed by the communication device may refer to the methods and beneficial effects in the first aspect or the second aspect above, and the repeated parts will not be repeated.

[0063] In a fifth aspect, the present application provides a communication device, which includes a processor and a memory, and the processor and the memory are coupled; the processor is used to implement the method as described in any one of the first to second aspects.

[0064] In a sixth aspect, the present application provides a chip, which includes a processor and an interface, the interface being used to receive or output signals, and the processor being used to execute code instructions to implement a method as described in any one of the first to second aspects.

[0065] In a seventh aspect, the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is called by a computer, the computer executes the method of any one of the first to second aspects.

[0066] In an eighth aspect, the present application provides a computer program product. When a computer reads and executes the computer program product, the computer executes the method of any one of the first to second aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] FIG1 is a schematic diagram of the architecture of a communication system provided in an embodiment of the present application;

[0068] FIG2 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0069] FIG3 is a flow chart of a data transmission method provided in an embodiment of the present application;

[0070] FIG4 is a schematic diagram of a matching index set of a moving object provided in an embodiment of the present application;

[0071] FIG5 is a schematic diagram of a process of data transmission based on a first coefficient matrix provided in an embodiment of the present application;

[0072] FIG6 is a schematic diagram of a flow chart of data transmission based on a second coefficient matrix and a third coefficient matrix provided in an embodiment of the present application;

[0073] FIG7 is a schematic diagram of a flow chart of data transmission based on a second coefficient matrix provided in an embodiment of the present application;

[0074] FIG8 is a schematic diagram of a flow chart of selecting an encoding method for compressing data provided by an embodiment of the present application;

[0075] FIG9 is a schematic diagram of a traffic scenario provided in an embodiment of the present application;

[0076] FIG10 is a comparative schematic diagram of an air interface overhead provided in an embodiment of the present application;

[0077] FIG11 is a schematic diagram of another traffic scenario provided in an embodiment of the present application;

[0078] FIG12 is a schematic diagram showing another comparison of air interface overhead provided in an embodiment of the present application. DETAILED DESCRIPTION

[0079] The embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0080] The terms "first," "second," "third," and "fourth," etc., in the specification and claims of this application and the accompanying drawings are used to distinguish between different objects, rather than to describe a specific order. In addition, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements, but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0081] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0082] To facilitate understanding of the embodiments of the present application, the following first introduces the professional terms involved in the embodiments of the present application:

[0083] 1. Point cloud data

[0084] Point cloud data refers to a collection of vectors in a three-dimensional coordinate system. The smallest unit of point cloud data is the point cloud data of a point, and the point cloud data of this point contains at least the three-dimensional coordinates corresponding to this point. Optionally, the point cloud data of this point may also include other attribute information besides position, such as color, reflectivity, intensity, etc. Point cloud data is usually collected by sensors carried by sensing devices. The types of sensors include but are not limited to laser scanners, cameras, three-dimensional scanners, etc., and the sensing devices collect point cloud data in frames. Each frame of point cloud data collected can be reported to the base station. The base station can receive point cloud data from multiple sensing devices and perform fusion processing on them to meet the needs of subsequent tasks.

[0085] 2. Reporting Methods of Point Cloud Data

[0086] Typically, when a sensing device collects point cloud data within a scene, it obtains point cloud data for a large number of points in the scene. If the sensing device directly sends all the collected point cloud data to the base station, it will occupy a large amount of air interface resources. Therefore, the sensing device will first compress the collected point cloud data and then report the compressed data. Currently, methods for compressing point cloud data include but are not limited to traditional point cloud compression algorithms and point cloud compression algorithms based on artificial intelligence (AI). Among them, traditional point cloud compression algorithms include two categories. One category converts point cloud data from three-dimensional signals into two-dimensional signals, and then further compresses them using existing two-dimensional compression algorithms, such as the video-based point cloud compression (V-PCC) algorithm provided by the moving pictures experts group (MPEG). This process can also be combined with the compression / decompression (lempel-ziv-markov chain-algorithm, LZMA) algorithm for entropy coding to further compress the two-dimensional signal. The other type is to convert point cloud data into a tree structure and then perform entropy coding, for example, the Draco algorithm based on kd-tree and the geometry point cloud compression algorithm based on octree (G-PCC).

[0087] The AI-based point cloud compression algorithm includes a compression method based on ODE modeling of a single moving object. The principle of this compression method is to establish a separate point cloud data model for each moving object in the scene. For each moving object, an ODE model correction and a set of compressed data are calculated and sent to the base station. The compressed data can indicate the part of the point cloud data where the ODE model prediction is inaccurate (or biased). After receiving the correction, the base station can first correct the ODE model and then combine the corrected ODE model and the compressed data to reconstruct the point cloud data corresponding to each moving object.

[0088] However, for this compression method based on ODE modeling of a single moving object, as the number of moving objects in the scene increases, the number of corrections to the ODE model to be sent and the amount of compressed data also increase, which will increase the transmission overhead when interacting with the base station for point cloud data.

[0089] In order to address the shortcomings of the above-mentioned compression method for ODE modeling based on a single moving object, the present application provides a data transmission method, a communication device, a communication system, etc.

[0090] The data transmission in the following embodiments of the present application refers to the data transmission involved in the interaction of point cloud data between a base station and a terminal device. The terminal device is a perception device that collects point cloud data in a perception scene. The data transmission mainly involves how the terminal device reports the point cloud data collected at a certain moment to the base station. In addition, before the terminal device interacts with the base station for point cloud data, the terminal device side also involves how to model the collected point cloud data and compress the point cloud data. After the base station receives the data sent by the terminal device, it also involves how to model and reconstruct the point cloud data. Among them, reconstructing point cloud data refers to how the base station restores the point cloud data in the scene at a certain moment after receiving the data sent by the terminal device.

[0091] In a data transmission method provided in the following embodiment of the present application, point cloud data can still be compressed based on modeling, but multiple moving objects are modeled simultaneously during modeling, that is, one model is used to indicate the changes in point cloud data corresponding to multiple moving objects, thereby obtaining correction amounts and compressed data of the models corresponding to multiple moving objects. Accordingly, when point cloud data is exchanged, the terminal device sends the correction amounts and compressed data of the models corresponding to these multiple moving objects to the base station, and the base station corrects the model based on the correction amounts of the models corresponding to these multiple moving objects, and obtains reconstructed point cloud data based on the corrected model and compressed data. The data transmission method provided in the embodiment of the present application can effectively reduce the amount of correction amounts and compressed data to be sent, thereby reducing the transmission overhead during point cloud data exchange.

[0092] Furthermore, by simultaneously modeling multiple moving objects, the correlation between their point cloud data can be effectively utilized, improving model accuracy. This can also provide motion feature information for subsequent tasks requiring prediction of the object's movement.

[0093] The following describes a communication system provided in an embodiment of the present application.

[0094] As shown in Figure 1, an embodiment of the present application provides a communication system, which may include a terminal device 100 and a base station 200. The number of terminal devices 100 and base stations 200 may be one or more. For example, the communication system shown in Figure 1 includes three terminal devices 100 and one base station 200.

[0095] The terminal device 100 is a perception device that collects point cloud data within a scene, models and compresses the point cloud data, and interacts with the base station 200 for point cloud data.

[0096] The base station 200 is a device that interacts with the terminal device 100 on point clouds and models and reconstructs point cloud data.

[0097] Both the terminal device 100 and the base station 200 store a first model. The first model can be built based on the first set of moving objects and used to predict point cloud data for multiple moving objects in the first set of moving objects. Furthermore, the terminal device 100 and the base station 200 may also store reconstructed point cloud data corresponding to moments before the current moment when point cloud interaction is performed. For example, the terminal device 100 and the base station store reconstructed point cloud data corresponding to the moment before the current moment. The first model can predict the point cloud data at the current moment based on the reconstructed point cloud data corresponding to the previous moment.

[0098] The terminal device 100 can be an intelligent terminal in different perception scenarios. Taking the traffic scenario shown in FIG1 as an example, the terminal device 100 can be a smartphone, tablet computer, or laptop computer in a vehicle. The terminal device 100 can also be a smart screen, a wearable device, an augmented reality (AR) device, a virtual reality (VR) device, a wireless terminal device in industrial control, a terminal device in unmanned autonomous driving, a terminal device in remote medical care, a terminal device in a smart grid, a terminal device in transportation safety, a wireless terminal device in a smart city, or a terminal device in smart transportation, etc., and this application does not limit this.

[0099] The base station 200 can be various forms of macro base stations, micro base stations (also called small stations), relay stations, access points (APs), etc. Exemplarily, the base station 200 can be a base station in a new radio (NR). Among them, the base station in the 5G new radio (NR) can also be called a transmission reception point (TRP) or a transmission point (TP) or a next generation Node B (ngNB), or an evolutionary Node B (eNB or eNodeB) in a long term evolution (LTE) system, or a base station in a short-range wireless communication system (such as a Wi-Fi communication system, a UWB communication system, etc.), etc., and this application is not limited to this.

[0100] Next, the hardware structures of the terminal device 100 and the base station 200 are introduced.

[0101] As shown in FIG2 , FIG2 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. The electronic device may be the terminal device 100 or the base station 200 in FIG1 above.

[0102] For example, the electronic device includes one or more processors 110 and one or more memories 120. The one or more memories 120 are coupled to the one or more processors 110. Coupling in the embodiments of the present application is an indirect coupling or communication connection between devices, units, or modules, which can be electrical, mechanical, or other forms, and is used for information exchange between devices, units, or modules.

[0103] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0104] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0105] Memory 120 is used to store program code and data. In some embodiments, memory 120 is a high-speed cache memory. This memory can store program code or data that has just been used or is being recycled by processor 110. If processor 110 needs to use the program code or data again, it can directly call it from memory 120. This avoids repeated accesses, reduces processor 110's waiting time, and thus improves system efficiency.

[0106] The processor 110 may operate in conjunction with the memory 120 , or at least one of the one or more memories 120 may be included in the processor 110 .

[0107] The device may also include a communication interface 130, which may optionally include a standard wired interface, a wireless interface (such as Wi-Fi, a mobile communication interface, etc.), and the communication interface 130 is controlled by the processor 110 to send and receive data; the communication interface 130 may optionally also realize data or signal communication between internal devices of the device.

[0108] It is understood that the structures illustrated in the embodiments of the present application do not constitute specific limitations on the electronic device. In other embodiments of the present application, the device may include more or fewer components than shown, or combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0109] Based on the above description, the data transmission method provided by this application is introduced in detail below.

[0110] As shown in Figure 3, Figure 3 is a flow chart of a data transmission method provided by an embodiment of the present application. The method includes steps 301 to 306. The execution subject of the method shown in Figure 3 can be a terminal device and a base station, or the subject can be a chip in the terminal device and a chip in the base station. Or the execution subject of the method shown in Figure 3 can also be other types of products, and those skilled in the art can make further expansions based on the content disclosed in the specification. The execution subject of the method shown in Figure 3 takes a terminal device and a base station as an example. Among them:

[0111] The process shown in FIG3 involves the use of a first model, which is used to predict point cloud data of each moving object in a first moving object set. The first moving object set includes a plurality of moving objects.

[0112] Before introducing each step, let’s first introduce the principle of the first model:

[0113] The first model can be based on the correlation between the point cloud data of the first moving object set at the previous and next moments, and use the ordinary differential equation (ODE) on the left side of the following formula 1 to model the point cloud data corresponding to the first moving object set as a whole:

[0114] On the left side of Formula 1, is the reconstructed point cloud data of the first moving object set at the j-1th moment, is the reconstructed point cloud data of the first moving object set at the jth moment, A j is the first coefficient matrix of the first model, A j It can represent the change in point cloud data corresponding to the first moving object set from the j-1th moment to the jth moment.

[0115] Solving the left side of Formula 1 can obtain the formula on the right side of Formula 1. The solution method includes but is not limited to the forward Euler algorithm. On the right side of Formula 1, y j is the real point cloud data of the first moving object set at the jth moment, Δt is the time step, for example, the time step can be the time length between the j-1th moment and the jth moment. According to the formula on the right side of Formula 1, Δt and A j Given the known premise, we can get y j , and y j It can be expressed as the following formula 2:

[0116] From the form of Formula 2, it can be seen that the first model can obtain the actual point cloud data of the first moving object set at the j-1th moment based on the reconstructed point cloud data of the first moving object set at the j-1th moment. It should be noted that Formula 2 is under ideal conditions (i.e., when the first model has no error or is fully converged). Therefore, the point cloud data of the first moving object set at the j-th moment predicted by the first model will have errors compared to the actual point cloud data of the first moving object set at the j-th moment. Subsequently, the point cloud data predicted by the first model will be referred to as predicted point cloud data.

[0117] Next, the first coefficient matrix A of the first model above is j Make an introduction:

[0118] Assume that the number of moving objects in the first moving object set is M, each moving object corresponds to N points of point cloud data, and the point cloud data of each point includes three-dimensional coordinates:

[0119] Then the various types of point cloud data corresponding to the first moving object set (such as y in the above formula j 、 ) can be expressed as a matrix with a dimension of M × 3N. Assuming that the formula on the right side of Formula 1 is valid, the dimension of the first coefficient matrix can be obtained to be M × M.

[0120] Optionally, for the first coefficient matrix A, a low-rank approximation may be performed to obtain a coefficient matrix with a lower dimension.

[0121] The process of low-rank decomposition can be referred to the following formula 3. The low-rank decomposition methods include but are not limited to singular value decomposition (SVD):

[0122] In formula 3, the first coefficient matrix A can be decomposed into three matrices U, ∑, V H , after the operation of these three matrices, we get two coefficient matrices B and P. B is U 1:M,1:X , P is C 1:X,1:M , C 1:X,1:M ∑ 1:X,1:X The matrix obtained after the operation. That is to say, the low-rank decomposition of the first coefficient matrix A can finally obtain two coefficient matrices: the second coefficient matrix B and the third coefficient matrix P, and the dimensions of B and P are M×X and X×M respectively.

[0123] Optionally, the first model may also be modeled using the coefficient matrix after low-rank decomposition. The modeling process can be referred to the following formula 4:

[0124] According to the formula on the right side of Formula 4, Δt, B j and P j Under the known premise, we can get t j .

[0125] Optionally, in low-rank decomposition, M is a positive integer greater than 3 (i.e., the first moving object set includes more than 3 moving objects), and X is a positive integer greater than or equal to 3 and less than or equal to M. The value range of M and the value range of X can ensure that the low-rank decomposition does not lose the changes in the point cloud data contained in any of the three coordinate dimensions (i.e., the x-axis, the y-axis, and the z-axis) while reducing the transmission overhead.

[0126] Optionally, for the case where M is less than or equal to 3, the first moving object set includes 3 or fewer moving objects (such as 3 or 2). In this case, the amount of point cloud data corresponding to the first moving object set is not large, so low-rank decomposition is not used but A in Formula 1 is used for modeling, or the existing ODE modeling method based on a single moving object is used.

[0127] Based on the above introduction to the first model, the implementation of each step is introduced below:

[0128] 301. A terminal device determines correction data of a first model based on first point cloud data and a first model, where the first model is used to predict point cloud data of each moving object in a first moving object set.

[0129] The first point cloud data is the real point cloud data of the first moving object set at the first moment.

[0130] In one possible implementation, the terminal device determines the correction data of the first model based on the first point cloud data and the first model in a manner that specifically includes: the terminal device performs an initial prediction on the point cloud data of the first moving object set at the first moment based on the first model and the second point cloud data to obtain third point cloud data; and obtains the correction data of the first model based on the error between the first point cloud data and the third point cloud data.

[0131] The second point cloud data is the reconstructed point cloud data of the first moving object set at a second moment, where the second moment is a moment before the first moment. The terminal device stores the coefficient matrix of the first model at the second moment. The terminal device obtains the third point cloud data based on the first model and the second point cloud data as follows: given the coefficient matrix at the second moment, the second point cloud data, and the time step (i.e., the length of time between the first moment and the second moment), the terminal device can obtain the initial predicted point cloud data (i.e., the third point cloud data) of the first moving object set at the first moment. Because the third point cloud data is predicted using the coefficient matrix at the second moment, rather than the coefficient matrix at the first moment, there is an error between the third point cloud data and the first point cloud data. In other words, the point cloud data obtained by the initial prediction of the first model at the moment before the first moment has an error between the initial predicted point cloud data and the actual point cloud data. Furthermore, based on the error between the first point cloud data and the third point cloud data, corrected data for the first model can be obtained.

[0132] The coefficient matrix of the first model at the first moment and the coefficient matrix at the second moment both refer to the modifiable coefficient matrix of the first model (hereinafter referred to as the coefficient matrix of the first model).

[0133] Optionally, the coefficient matrix of the first model may include a first coefficient matrix A. In this optional manner, the value of A at the second moment may be substituted into the above formula 1 or 2 to obtain the third point cloud data.

[0134] Optionally, the coefficient matrix of the first model may include a second coefficient matrix B and a third coefficient matrix P. In this optional manner, the values ​​of B and P at the second moment may be substituted into the above formula 4 to obtain the third point cloud data.

[0135] Optionally, the coefficient matrix of the first model may include the second coefficient matrix B. In this optional approach, the third coefficient matrix P remains unchanged and is known, so the coefficient matrix of the first model does not include P. The value of B at the second moment can be substituted into the above formula 4 to obtain the third point cloud data based on formula 4.

[0136] How to obtain the correction data of the first model in these three cases will be described in subsequent embodiments.

[0137] 302. The terminal device corrects the first model based on the correction data to obtain a corrected first model.

[0138] In the present application, the correction data of the first model obtained by the terminal device is the correction amount of the coefficient matrix of the first model.

[0139] Optionally, the correction data of the first model may include a correction value of the first coefficient matrix A, such as ΔA.

[0140] Optionally, the correction data of the first model may include a correction value of the second coefficient matrix B, such as ΔB, and a correction value of the third coefficient matrix P, such as ΔP.

[0141] Optionally, the correction data of the first model may include a correction value of the second coefficient matrix B, such as ΔB.

[0142] The terminal device corrects the first model based on the correction data to obtain the corrected first model, which means: correcting the coefficient matrix of the first model at the second moment to obtain the coefficient matrix of the first model at the first moment. The coefficient matrix at the first moment can indicate the changes in the point cloud data of the first moving object set in the time period from the second moment to the first moment.

[0143] 303. The terminal device obtains compressed data corresponding to the first point cloud data based on the first point cloud data and the corrected first model.

[0144] In one possible implementation, the terminal device obtains compressed data corresponding to the first point cloud data based on the first point cloud data and the corrected first model, specifically including: re-predicting the point cloud data of the first moving object set at the first moment based on the corrected first model and the second point cloud data to obtain fourth point cloud data; and compressing the first point cloud data based on the fourth point cloud data to obtain compressed data.

[0145] Among them, the process of the terminal device obtaining the fourth point cloud data based on the corrected first model and the second point cloud data is specifically as follows: when the coefficient matrix at the first moment, the second point cloud data, and the time step (i.e., the time length between the first moment and the second moment) are known, the terminal device can obtain the re-predicted point cloud data (i.e., the fourth point cloud data) of the first moving object set at the first moment. Since although the coefficient matrix at the first moment is used for calculation, the first model still has a model error, there is still an error between the obtained fourth point cloud data and the real point cloud data (i.e., the first point cloud data), and the error between the fourth point cloud data and the first point cloud data is smaller than the error between the third point cloud data and the first point cloud data. Furthermore, based on the error between the fourth point cloud data and the first point cloud data, and the part of the first point cloud data accurately predicted by the corrected first model (i.e., the part of the fourth point cloud data and the first point cloud data that is correctly predicted), the first point cloud data can be compressed to obtain compressed data. The present application has the following two ways to obtain compressed data:

[0146] Optionally, the first point cloud data is divided into two parts, where the first part comprises point cloud data with an error less than a first error threshold, and the second part comprises the remainder of the first point cloud data excluding the first part. The first part is indicated by the index corresponding to the point cloud data; the second part is directly indicated by the second part. Therefore, the compressed data includes information obtained by encoding the index corresponding to the first part and information obtained by encoding the second part.

[0147] Optionally, the first point cloud data can be divided into three parts, wherein the first part is the point cloud data with an error less than a first error threshold, the second part is the point cloud data with an error greater than or equal to the first error threshold and less than a second error threshold, and the third part is the rest of the first point cloud data except the first and second parts. For the first part, the index corresponding to the point cloud data is used for indication; for the second part, the index corresponding to the point cloud data and the difference information (i.e., the predicted error) are used for indication; for the third part, the third part is used directly for indication. Therefore, the compressed data includes information obtained by encoding the index corresponding to the first part, information obtained by encoding the index corresponding to the second part, information obtained by encoding the difference information, and information obtained by encoding the third part.

[0148] These two forms of compressed data will be explained through specific examples later.

[0149] 304. The terminal device sends the corrected data of the first model and the compressed data corresponding to the first point cloud data to the base station.

[0150] Correspondingly, the base station receives the corrected data of the first model and the compressed data corresponding to the first point cloud data.

[0151] 305. The base station corrects the first model based on the correction data to obtain a corrected first model.

[0152] In the present application, the base station stores a coefficient matrix of the first model at the second moment. The base station corrects the first model based on the correction data to obtain the corrected first model, which means: correcting the coefficient matrix of the first model at the second moment to obtain the coefficient matrix of the first model at the first moment. The coefficient matrix at the first moment can indicate changes in point cloud data of the first moving object set during the time period from the second moment to the first moment.

[0153] 306. The base station decompresses the compressed data based on the corrected first model to obtain reconstructed point cloud data of the first moving object set at the first moment.

[0154] In one possible implementation, the base station decompresses the compressed data based on the corrected first model to obtain the reconstructed point cloud data of the first set of moving objects at the first moment, specifically including: predicting the point cloud data of the first set of moving objects at the first moment based on the corrected first model and the second point cloud data to obtain fourth point cloud data; and decompressing the compressed data based on the fourth point cloud data to obtain the reconstructed point cloud data of the first set of moving objects at the first moment.

[0155] The process by which the base station obtains the fourth point cloud data is the same as the process by which the terminal obtains the fourth point cloud data in step 303. Based on the fourth point cloud data, the base station decompresses the compressed data to obtain the reconstructed point cloud data of the first moving object set at the first moment. This is the reverse process of the process in step 303 in which the terminal compresses the first point cloud data based on the corrected first model and the fourth point cloud data to obtain the compressed data. In other words, the base station can restore the first point cloud data as much as possible based on the fourth point cloud data and the compressed data. The restored data is referred to as the reconstructed point cloud data of the first moving object set at the first moment. The base station can store the reconstructed point cloud data of the first moving object set at the first moment so that it can subsequently compress the actual point cloud data at moments after the first moment.

[0156] Optionally, the terminal device may also decompress the compressed data based on the corrected first model to obtain the reconstructed point cloud data of the first moving object set at the first moment, and store the reconstructed point cloud data of the first moving object set at the first moment.

[0157] For example, for a method in which compressed data includes information obtained by encoding the index corresponding to the first part and information obtained by encoding the second part: the first part can be obtained from the fourth point cloud data based on the index corresponding to the first part, and then the union of the first part and the second part can be determined as the reconstructed point cloud data of the first moving object set at the first moment.

[0158] For another example, for a method in which the compressed data includes information obtained by encoding the index corresponding to the first part, information obtained by encoding the index corresponding to the second part, information obtained by encoding the difference information, and information obtained by encoding the third part: the first part can be obtained from the fourth point cloud data based on the index corresponding to the first part, and then based on the difference information, the point cloud data of the index corresponding to the second part in the fourth point cloud data can be supplemented to obtain the second part; finally, the union of the first part, the second part and the third part is determined as the reconstructed point cloud data of the first moving object set at the first moment.

[0159] Optionally, in the embodiment corresponding to FIG. 3 , the terminal device may also send the corrected data of the first model to the base station after determining the corrected data for the first model. The terminal device then performs steps 302-303 to determine compressed data, and after the compressed data is determined, sends it to the base station. Accordingly, the base station may also perform step 305 after receiving the corrected data for the first model, and then perform step 306 after subsequently receiving the compressed data.

[0160] The following is a supplementary explanation of the first moving object set involved in the above embodiment:

[0161] In the present application, before executing the above steps, the terminal device needs to first determine the first moving object set. When the terminal device determines the first moving object set, it also determines the first point cloud data (i.e., the actual point cloud data of the first moving object set at the first moment) and the second point cloud data (i.e., the reconstructed point cloud data of the first moving object set at the second moment). The terminal device then needs to send third indication information of the first moving object set to the base station so that the base station can determine the first moving object set based on the third indication information. When the base station determines the first moving object set, it also determines the second point cloud data.

[0162] Specifically, on the terminal device side: the terminal device obtains the reconstructed point cloud data at the second moment; obtains the real point cloud data collected at the first moment; obtains the first point cloud data from the real point cloud data at the first moment, and the first motion object set is the intersection between the motion object set corresponding to the reconstructed point cloud data at the second moment and the motion object set corresponding to the real point cloud data at the first moment; sends a third indication information to the base station, and the third indication information is used to indicate the first motion object set.

[0163] Among them, the terminal device stores the reconstructed point cloud data of the second moment and the real point cloud data collected by the terminal device at the first moment. Then the terminal device can identify the set of moving objects corresponding to the reconstructed point cloud data at the second moment, and the set of moving objects corresponding to the real point cloud data collected at the first moment. Since the present application uses the first model to model, it uses the correlation between the point cloud data at the previous and next moments. Therefore, the modeling object of the first model of the present application needs to be a set of moving objects corresponding to the reconstructed point cloud data at the second moment and the real point cloud data collected at the first moment. For example, the set of moving objects corresponding to the reconstructed point cloud data at the second moment is {q 1 ,q 2 ,q 3 ,q 4 ,q 5}, the set of moving objects corresponding to the real point cloud data collected at the first moment is {q 2 ,q 3 ,q 4 ,q 5 ,q 6 ,q 7}, then comparing these two sets of moving objects, we can see that from the second moment to the first moment, q 1 To perceive the disappearance of moving objects in the scene, q 6 ,q 7 For the newly added moving objects in the perception scene, the first moving object set is {q 2 ,q 3 ,q 4 ,q 5}. Accordingly, the first point cloud data is {q 2 ,q 3 ,q 4 ,q 5}The real point cloud data corresponding to the four moving objects, the second point cloud data is the reconstructed point cloud data at the second moment {q 2 ,q 3 ,q 4 ,q 5}The reconstructed point cloud data corresponding to these four moving objects.

[0164] On the base station side: the base station receives the third indication information; the base station obtains the reconstructed point cloud data at the second moment; based on the third indication information and the reconstructed point cloud data at the second moment, the first moving object set and the second point cloud data are determined.

[0165] The base station stores the reconstructed point cloud data at the second moment. When the base station receives the third instruction information, it can first identify the set of moving objects corresponding to the reconstructed point cloud data at the second moment, such as {q 1 ,q2 ,q 3 ,q 4 ,q 5}, and then determine the first moving object set as {q 2 ,q 3 ,q 4 ,q 5}. Accordingly, the second point cloud data is the reconstructed point cloud data at the second moment {q 2 ,q 3 ,q 4 ,q 5}The reconstructed point cloud data corresponding to these four moving objects.

[0166] Optionally, the third indication information may be represented as a matching index set of moving objects, etc. Taking the matching index set of moving objects as an example, the matching index set of moving objects may be obtained by matching the moving object set corresponding to the real point cloud data collected at the first moment based on the moving object set corresponding to the reconstructed point cloud data at the second moment.

[0167] For example, as shown in FIG4 , the elements in the matching index set of moving objects are, in order: the order in which each moving object in the moving object set at the current moment matches the moving object in the moving object set at the previous moment. Moving objects in the moving object set at the current moment that could not be matched in the moving object set at the previous moment are marked as 0. Therefore, the index set of the moving object at the 101st moment is {1, 2, 0, 4, 3}, indicating that: the first moving object in the moving object set at the 101st moment is the first moving object in the moving object set at the 100th moment; the second moving object in the moving object set at the 101st moment is the second moving object in the moving object set at the 100th moment; the third moving object in the moving object set at the 101st moment does not exist in the moving object set at the 100th moment; the fourth moving object in the moving object set at the 101st moment is the fourth moving object in the moving object set at the 100th moment; and the fifth moving object in the moving object set at the 101st moment is the third moving object in the moving object set at the 100th moment. The index set of moving objects at the 102nd moment is {2, 3, 4, 5}, which means: the first moving object in the moving object set at the 102nd moment is the second moving object in the moving object set at the 101st moment; the second moving object in the moving object set at the 102nd moment is the third moving object in the moving object set at the 101st moment; the third moving object in the moving object set at the 102nd moment is the fourth moving object in the moving object set at the 101st moment; and the fourth moving object in the moving object set at the 102nd moment is the fifth moving object in the moving object set at the 101st moment.

[0168] Optionally, the third indication information may be included in the compressed data and sent.

[0169] It should be noted that the first point cloud data corresponding to the first set of moving objects can be reported according to the embodiment shown in Figure 3, and the point cloud data other than the first point cloud data in the real point cloud data collected at the first moment can be reported using other existing reporting methods.

[0170] By implementing the embodiment depicted in FIG3 , the first model can be used to predict point cloud data for each moving object in the first moving object set. Therefore, the first model models the first moving object set as a whole. Furthermore, when obtaining model corrections and compressing point cloud data, the terminal device does not need to separately determine model corrections for each moving object and obtain a corresponding set of compressed data for each moving object. Instead, the terminal device determines the corrections for the first model, uses the corrections to represent the model corrections corresponding to the first moving object set, and uniformly processes the first point cloud data based on the corrected first model to obtain compressed data corresponding to the first moving object set. Accordingly, when exchanging point cloud data, the terminal device and the base station can transmit the corrections for the first model and the compressed data for the first moving object set, effectively reducing the amount of corrections and compressed data to be transmitted, thereby reducing transmission overhead during point cloud data exchange. Furthermore, this approach of modeling the first moving object set as a whole can effectively improve processing efficiency and enable the resulting first model to leverage the correlations between the individual moving objects in the first moving object set, resulting in more accurate prediction results.

[0171] The above embodiment is described in detail below by taking three cases of the coefficient matrix of the first model as examples:

[0172] Assume that the second moment is the j-1th moment, the first moment is the jth moment, j is a positive integer greater than or equal to 3, and the first moving object set is M is the number of moving objects in the first moving object set. In the above embodiment, the first point cloud data (i.e., the real point cloud data of the first moving object set at the jth moment) is y j , the second point cloud data (i.e., the reconstructed point cloud data of the first moving object set at the j-1th moment) is The reconstructed point cloud data of the first moving object set at the jth moment is

[0173] Case 1: The coefficient matrix of the first model includes a first coefficient matrix A, and the correction data of the first model includes a correction amount of A, such as ΔA.

[0174] The following describes the steps performed by the terminal device and the steps performed by the base station respectively:

[0175] Terminal device side: The terminal device may execute steps 1-501 to 1-508. Steps 1-501 to 1-503 are specific implementations of step 301, step 1-505 is a specific implementation of step 302, and steps 1-506 to 1-507 are specific implementations of step 303.

[0176] 1-501. Initialize the first coefficient matrix.

[0177] In the embodiment of the present application, the first coefficient matrix can be initialized based on the following formula 5 to obtain A1:

[0178] In formula 5, and They are the reconstructed point cloud data at the second moment and the reconstructed point cloud data at the first moment, and It can be obtained based on a decompression method corresponding to an existing compression method of point cloud data, such as based on the Draco algorithm.

[0179] 1-502. Perform initial prediction based on the first coefficient matrix at the j-1th moment.

[0180] Here, j is a positive integer greater than or equal to 2.

[0181] Initial prediction means: based on the first coefficient matrix of the first model at the j-1th moment and the second point cloud data, the first moving object set is initially predicted at the jth moment to obtain the third point cloud data. Assume that the third point cloud data is The first coefficient matrix at the j-1th moment is A j-1 , then based on the following formula 6, we can get

[0182] 1-503. Calculate the correction amount of the first coefficient matrix at the j-th moment.

[0183] Based on the following formula 7, the correction value ΔA of the first coefficient matrix at the jth moment can be obtained: j ,in, is the error between the third point cloud data and the first point cloud data:

[0184] 1-504. Send the correction value of the first coefficient matrix at the j-th moment.

[0185] 1-505. Modify the first coefficient matrix based on the correction amount of the first coefficient matrix at the j-th moment to obtain the first coefficient matrix at the j-th moment.

[0186] This step can be achieved based on the following formula 8, where the first coefficient matrix at the jth moment is A j : A j =A j-1 +ΔA j (Formula 8)

[0187] It should be noted that after the terminal device executes step 1-505, if the value of j is a positive integer greater than or equal to 3, then the following steps 1-506 to 1-509 are executed. Otherwise, the terminal device compresses the point cloud data based on the existing point cloud data compression method and obtains the reconstructed point cloud data. In other words, the terminal device initializes the first coefficient matrix based on step 1-501 at the first moment, and obtains the reconstructed point cloud data based on the existing point cloud data compression method. The terminal device obtains the point cloud data based on the compression method at the second moment. And based on step 1-502 to step 1-505, A2 is obtained. At the third moment and the moment after the third moment, A is obtained based on 1-502 to 1-505. j and compressed data.

[0188] 1-506. Perform another prediction based on the first coefficient matrix at the j-th moment.

[0189] The second prediction means: based on the first coefficient matrix of the first model at the jth moment and the second point cloud data, the first moving object set is predicted again at the jth moment to obtain the fourth point cloud data. Assume that the fourth point cloud data is This step can be implemented by referring to the following formula 9:

[0190] 1-507. Determine the compressed data.

[0191] Determining the compressed data means: compressing the first point cloud data based on the fourth point cloud data to obtain compressed data. Assume that the compressed data is u j , this step can be implemented by referring to the following formula 10:

[0192] Among them, f j () is the compression process, which will be described in detail in the subsequent examples.

[0193] 1-508. Send compressed data.

[0194] 1-509. Decompress the compressed data and store the decompression result.

[0195] Decompressing the compressed data means: based on the fourth point cloud data, decompressing the compressed data to obtain the reconstructed point cloud data of the first moving object set at the jth moment; then, the reconstructed point cloud data of the first moving object set at the jth moment can be stored.

[0196] Decompression can be achieved by referring to the following formula 11:

[0197] in, The decompression process will be described in detail in the subsequent embodiments.

[0198] Base station side: The base station can execute steps 2-501 to 2-503. Step 2-501 is a specific implementation of step 305, and steps 2-502 to 2-503 are specific implementations of step 306. The base station must execute step 2-501 after the terminal device executes step 1-504, that is, the base station receives the ΔA sent by the terminal device. j After that, step 2-501 can be executed. And the base station executes step 2-503 after the terminal device executes step 1-508, that is, the base station receives the u sent by the terminal device. j Only then can step 2-503 be executed.

[0199] 2-501. Modify the first coefficient matrix based on the correction amount of the first coefficient matrix at the j-th moment to obtain the first coefficient matrix at the j-th moment.

[0200] 2-502. Perform another prediction based on the first coefficient matrix at the j-th moment.

[0201] 2-503. Decompress the compressed data and store the decompression result.

[0202] Among them, the specific implementation method of step 2-501 can refer to the specific implementation method of step 1-505; the specific implementation method of step 2-502 can refer to the specific implementation method of step 1-506; the specific implementation method of step 2-503 can refer to the specific implementation method of step 1-509.

[0203] It should be noted that the base station also needs to initialize the first coefficient matrix and obtain the reconstructed point cloud data based on the compression method of the existing point cloud data. When j is 2, the reconstructed point cloud data is obtained based on the compression method of the existing point cloud data. Then, step 2-501 is executed to obtain A2, and steps 2-502 to 2-503 are not executed. Steps 2-501 to 2-503 are executed only when j is a positive integer greater than or equal to 3.

[0204] By implementing the embodiment described in FIG5 , no matter how many moving objects are included in the first moving object set, the terminal device can send a ΔA j , without sending multiple coefficient matrix corrections, so that the base station can correct the first model. And based on the first model, a set of compressed data u corresponding to the first set of moving objects can be obtained jThe first moving object set is used for modeling and processing, and the first model can be used to obtain more accurate prediction results by utilizing the correlation between the moving objects in the first moving object set.

[0205] Optionally, the terminal device can also set ΔA j and u j After all are determined, ΔA j and u j Sent to the base station together.

[0206] Case 2: The coefficient matrix of the first model includes the second coefficient matrix B and the third coefficient matrix P, and the correction data of the first model includes the correction amount of B, such as ΔB, and the correction amount of P, such as ΔP.

[0207] The following describes the steps performed by the terminal device and the steps performed by the base station respectively:

[0208] Terminal device side: The terminal device may execute steps 1-601 to 1-612. Steps 1-601 to 1-603 and step 1-606 are specific implementations of step 301. Steps 1-605 and 1-608 are specific implementations of step 302. Steps 1-609 to 1-610 are specific implementations of step 303.

[0209] 1-601. Initialize the second coefficient matrix and the third coefficient matrix.

[0210] In an embodiment of the present application, the third coefficient matrix may be initialized based on the following formula 12 to obtain P1, and then the second coefficient matrix may be initialized based on the following formula 13 to obtain B1. Then, the terminal device may execute steps 1-602 to 1-612:

[0211] In formula 12 and formula 13, diag(1,1,1) represents a diagonal matrix whose diagonal elements are 1, 1, 1. 3×(M - 3) Represents a matrix whose elements are all 1. and They are the reconstructed point cloud data at the second moment and the reconstructed point cloud data at the first moment, and It can be obtained based on the decompression method corresponding to the compression method of the existing point cloud data, such as the Draco algorithm. After the terminal device initializes P1 based on formula 12, it can compare P1 with and Substitute into formula 13 and obtain B1.

[0212] 1-602. Perform initial prediction based on the second coefficient matrix at the j-1th moment and the third coefficient matrix at the j-1th moment.

[0213] Wherein, j is a positive integer greater than or equal to 2. Initial prediction means: based on the second coefficient matrix and the third coefficient matrix of the first model at the j-1th moment, and the second point cloud data, the first moving object set is initially predicted at the jth moment to obtain the third point cloud data. The second coefficient matrix at the j-1th moment is B j-1 , the third coefficient matrix at the j-1th moment is P j-1 , then the third point cloud data can be obtained based on the following formula 14:

[0214] After the terminal device completes step 1-602, it is necessary to calculate the correction amount of the second coefficient matrix and the correction amount of the third coefficient matrix. In the first optional method, the correction amount of the second coefficient matrix can be calculated first, and then the correction amount of the third coefficient matrix can be calculated. In the second optional method, the correction amount of the third coefficient matrix can be calculated first, and then the correction amount of the second coefficient matrix can be calculated. The second optional method is used as an example for explanation below:

[0215] 1-603. Calculate the correction amount of the second coefficient matrix at the j-th moment.

[0216] Based on the following formula 15, the correction value ΔB of the second coefficient matrix at the jth moment can be obtained: j .

[0217] In formula 15, is the error between the third point cloud data and the first point cloud data. Formula 15 calculates the correction to the second coefficient matrix based on the error between the third and first point cloud data, assuming that the third coefficient matrix is ​​still the third coefficient matrix at the j-1th moment. In other words, when calculating the correction to the second coefficient matrix, it is assumed that the third coefficient matrix is ​​the third coefficient matrix at the previous moment.

[0218] 1-604. Send the correction value of the second coefficient matrix at the j-th moment.

[0219] 1-605. Modify the second coefficient matrix based on the correction amount of the second coefficient matrix at the j-th moment to obtain the second coefficient matrix at the j-th moment.

[0220] This step can be achieved based on the following formula 16, where the second coefficient matrix at the jth moment is Bj : B j =B j-1 +ΔB j (Formula 16)

[0221] 1-606. Calculate the correction amount of the third coefficient matrix at the j-th moment.

[0222] Based on the following formula 17, the correction value ΔP of the third coefficient matrix at the jth moment can be obtained: j .

[0223] In Formula 17, is the error between the third point cloud data and the first point cloud data. Formula 17 calculates the correction amount of the third coefficient matrix based on the error between the third point cloud data and the first point cloud data, under the premise that the second coefficient matrix has been corrected to the second coefficient matrix at the j-th moment.

[0224] 1-607. Send the correction value of the third coefficient matrix at the j-th moment.

[0225] 1-608. Modify the third coefficient matrix based on the correction amount of the third coefficient matrix at the j-th moment to obtain the third coefficient matrix at the j-th moment.

[0226] This step can be achieved based on the following formula 18, where the third coefficient matrix at the jth moment is P j : P j =P j-1 +ΔP j (Formula 18)

[0227] It should be noted that after the terminal device executes step 1-608, if the value of j is a positive integer greater than or equal to 3, then the following steps 1-609 to 1-612 are executed. Otherwise, the terminal device compresses the point cloud data based on the existing point cloud data compression method and obtains the reconstructed point cloud data. In other words, the terminal device initializes the second coefficient matrix and the third coefficient matrix based on step 1-601 at the first moment, and obtains the reconstructed point cloud data based on the existing point cloud data compression method. At the second moment, based on the compression method of the existing point cloud data, Based on steps 1-602 to 1-608, B2 and P2 are obtained. At the third moment and the moment after the third moment, B is obtained based on steps 1-602 to 1-608. j 、P j and compressed data.

[0228] 1-609. Perform re-prediction based on the second coefficient matrix at the j-th moment and the third coefficient matrix at the j-th moment.

[0229] The second prediction is to predict the first moving object set at the jth moment based on the second coefficient matrix of the first model at the jth moment and the third coefficient matrix at the jth moment, as well as the second point cloud data, to obtain the fourth point cloud data. This step can be implemented by referring to the following formula 19:

[0230] 1-610. Determine the compressed data.

[0231] 1-611. Send compressed data.

[0232] 1-612. Decompress the compressed data and store the decompression result.

[0233] The specific implementation of steps 1-610 to 1-612 may refer to the specific implementation of steps 1-507 to 1-509 in FIG. 5 .

[0234] Base station side: The base station can execute steps 2-601 to 2-604. Steps 2-601 to 2-602 are specific implementations of step 305, and steps 2-603 to 2-604 are specific implementations of step 306. The base station must execute step 2-601 after the terminal device executes step 1-604, that is, the base station receives the ΔB sent by the terminal device. j After that, step 2-601 can be executed. And the base station must execute step 2-602 after the terminal device executes step 1-607, that is, the base station receives the ΔP sent by the terminal device. j After that, step 2-601 can be executed. And the base station must execute step 2-604 after the terminal device executes step 1-611, that is, the base station can execute step 2-604 after receiving the compressed data sent by the terminal device.

[0235] 2-601. Modify the second coefficient matrix based on the correction amount of the second coefficient matrix at the j-th moment to obtain the second coefficient matrix at the j-th moment.

[0236] 2-602. Modify the third coefficient matrix based on the correction amount of the third coefficient matrix at the j-th moment to obtain the third coefficient matrix at the j-th moment.

[0237] 2-603. Perform a second prediction based on the second coefficient matrix at the j-th moment and the third coefficient matrix at the j-th moment.

[0238] 2-604. Decompress the compressed data and store the decompression result.

[0239] Among them, the specific implementation method of step 2-601 can refer to the specific implementation method of step 1-605; the specific implementation method of step 2-602 can refer to the specific implementation method of step 1-608; the specific implementation method of step 2-603 can refer to the specific implementation method of step 1-609; the specific implementation method of step 2-604 can refer to the specific implementation method of step 1-612.

[0240] It should be noted that the base station also needs to initialize the second coefficient matrix and the third coefficient matrix, and obtain the reconstructed point cloud data based on the compression method of the existing point cloud data. When j is 2, the reconstructed point cloud data is obtained based on the compression method of the existing point cloud data. Then, execute step 2-601 to obtain B2, execute step 2-602 to obtain P2, and do not execute steps 2-603 to 2-604. Only when j is a positive integer greater than or equal to 3, execute steps 2-601 to 2-604.

[0241] By implementing the embodiment described in FIG6 , no matter how many moving objects are included in the first moving object set, the terminal device can send a ΔB j and ΔP j , without the need to send multiple coefficient matrix corrections, which can reduce the transmission overhead of model corrections during point cloud data interaction. j and ΔP j Ratio ΔA j The smaller dimension can further reduce the transmission overhead. Secondly, based on the first model, a set of compressed data u corresponding to the first set of moving objects is obtained. j The compressed data corresponding to each moving object is sent simultaneously, without the need to send compressed data corresponding to each moving object. This effectively reduces the amount of compressed data to be transmitted, thereby reducing transmission overhead. Furthermore, this method of modeling the first moving object set as a whole can effectively improve processing efficiency and enable the resulting first model to utilize the correlation between the moving objects in the first moving object set to obtain more accurate prediction results.

[0242] Optionally, the terminal device can also set ΔB j , ΔP j and u j After all are determined, ΔB j , ΔP j and u j Sent to the base station together.

[0243] Case 3: The coefficient matrix of the first model includes the second coefficient matrix B, and the correction data of the first model includes the correction amount of B, such as ΔB.

[0244] The following describes the steps performed by the terminal device and the steps performed by the base station respectively:

[0245] Terminal device side: The terminal device may execute steps 1-701 to 1-709. Steps 1-701 to 1-703 are specific implementations of step 301, step 1-705 is a specific implementation of step 302, and steps 1-706 to 1-707 are specific implementations of step 303.

[0246] 1-701. Initialize the second coefficient matrix and the third coefficient matrix. The third coefficient matrix remains unchanged after initialization.

[0247] The specific implementation of step 1-701 can refer to the specific implementation of step 1-601.

[0248] 1-702. Perform initial prediction based on the second coefficient matrix at the j-1th moment.

[0249] Wherein, j is a positive integer greater than or equal to 2. Initial prediction means: based on the second coefficient matrix of the first model at the j-1th moment and the initialized third coefficient matrix, as well as the second point cloud data, the first moving object set is initially predicted at the jth moment to obtain the third point cloud data. The second coefficient matrix at the j-1th moment is B j-1 , the initialized third coefficient matrix is ​​P1, then the third point cloud data can be obtained based on the following formula 20:

[0250] 1-703. Calculate the correction amount of the second coefficient matrix at the j-th moment.

[0251] Based on the following formula 21, the correction value ΔB of the second coefficient matrix at the jth moment can be obtained: j .

[0252] 1-704. Send the correction value of the second coefficient matrix at the j-th moment.

[0253] 1-705. Modify the second coefficient matrix based on the correction amount of the second coefficient matrix at the j-th moment to obtain the second coefficient matrix at the j-th moment.

[0254] The specific implementation of step 1-705 can refer to the specific implementation of step 1-605.

[0255] It should be noted that after the terminal device executes step 1-705, if the value of j is a positive integer greater than or equal to 3, then the following steps 1-706 to 1-709 are executed. Otherwise, the terminal device compresses the point cloud data based on the existing point cloud data compression method and obtains the reconstructed point cloud data. In other words, the terminal device initializes the second coefficient matrix and the third coefficient matrix based on step 1-701 at the first moment, and obtains the reconstructed point cloud data based on the existing point cloud data compression method. At the second moment, based on the compression method of the existing point cloud data, And based on step 1-702 to step 1-705, B2 is obtained. At the third moment and the moment after the third moment, B is obtained based on 1-702 to 1-705. j and compressed data.

[0256] 1-706. Perform another prediction based on the second coefficient matrix at the j-th moment.

[0257] The second prediction is to predict the first moving object set at the jth moment based on the second coefficient matrix of the first model at the jth moment and the initialized third coefficient matrix, as well as the second point cloud data, to obtain the fourth point cloud data. This step can be implemented by referring to the following formula 22:

[0258] 1-707. Determine the compressed data.

[0259] 1-708. Send compressed data.

[0260] 1-709. Decompress the compressed data and store the decompression result.

[0261] The specific implementation of steps 1-707 to 1-709 may refer to the specific implementation of steps 1-507 to 1-509 in FIG. 5 .

[0262] Base station side: The base station can execute steps 2-701 to 2-703. Step 2-701 is a specific implementation of step 305, and steps 2-702 to 2-703 are specific implementations of step 306. The base station must execute step 2-701 after the terminal device executes step 1-704, that is, the base station receives the ΔB sent by the terminal device. j After that, step 2-701 can be executed. And the base station must execute step 2-703 after the terminal device executes step 1-708, that is, the base station can execute step 2-703 after receiving the compressed data sent by the terminal device.

[0263] 2-701. Modify the second coefficient matrix based on the correction amount of the second coefficient matrix at the j-th moment to obtain the second coefficient matrix at the j-th moment.

[0264] 2-702. Perform another prediction based on the second coefficient matrix at the j-th moment.

[0265] 2-703. Decompress the compressed data and store the decompression result.

[0266] Among them, the specific implementation method of step 2-701 can refer to the specific implementation method of step 1-705; the specific implementation method of step 2-702 can refer to the specific implementation method of step 1-706; the specific implementation method of step 2-703 can refer to the specific implementation method of step 1-709.

[0267] It should be noted that the base station also needs to initialize the second coefficient matrix and the third coefficient matrix, and obtain the reconstructed point cloud data based on the compression method of the existing point cloud data. When j is 2, the reconstructed point cloud data is obtained based on the compression method of the existing point cloud data. Then, step 2-701 is executed to obtain B2, and steps 2-702 to 2-703 are not executed. Steps 2-701 to 2-703 are executed only when j is a positive integer greater than or equal to 3.

[0268] By implementing the embodiment described in FIG. 7 , P can be kept unchanged during low-rank decomposition, thereby making ΔP j Does not exist, the terminal device only needs to send a ΔB j , compared to the ΔB sent in Figure 6 j and ΔP j The solution can further reduce the transmission overhead of model correction during point cloud data interaction.

[0269] Optionally, the terminal device can also set ΔB j and u j After all are determined, ΔB j and u j Sent to the base station together.

[0270] The following describes how to obtain compressed data in the above method embodiment, namely The detailed process of decompression is described below. for The reverse process:

[0271] First, define the real point cloud data of the first moving object set as S, and the re-predicted point cloud data of the first moving object set as S is the first point cloud data or y in the above embodiment j , The fourth point cloud data mentioned above or The first moving object set includes M moving objects, and each moving object corresponds to N points of point cloud data (if N points are not collected during acquisition, random sampling can be performed until N points are met), so S can be expressed by the following formula 23: It can be expressed by the following formula 24:

[0272] in, is the real point cloud data of the first point of the first moving object, is the real point cloud data of the Nth point of the Mth moving object. Re-predict the point cloud data for the first point of the first moving object, Re-predict the point cloud data for the Nth point of the Mth moving object.

[0273] And, for For each element in for The index corresponding to the point cloud data of all points (including M×N points).

[0274] Method 1:

[0275] ①, for each point cloud data in S, determine its The associated point cloud data in The index in .

[0276] Optionally, the associated point cloud data can be The point cloud data in S has the smallest error (ie, the most correlated) with the point cloud data in S. For example, the error can be represented by a method such as Euclidean distance.

[0277] For example, a function π(a) may be predefined, and π(a) may refer to the following formula 25:

[0278] Among them, a is any point cloud data in S, b is any point cloud data in S, D() is used to calculate the Euclidean distance between point cloud data, b i For a The associated point cloud data in b i exist The meaning of formula 25 is that for any point cloud data a in S, the b with the shortest Euclidean distance to a can be returned. i exist The index i in .

[0279] ②. Determine the first index set and / or the second index set.

[0280] First, the first index set X1 and the second index set X2 in this application are introduced: First, S1 is defined based on the following formula 26:

[0281] Among them, ε is the first error threshold, b π(a) is the associated point cloud data of a, D(a,b π(a) )<ε is used to represent a and b π(a) The error between them is less than the first error threshold. Formula 26 means that in the first point cloud data, the set of point cloud data whose error with the associated point cloud data is less than the first error threshold is S1.

[0282] Then, based on S1, a first index set X1 and a second index set X2 are defined. X1 can refer to the following formula 27, and X2 can refer to the following formula 28: X1 = {π(a): a∈S1} (Formula 27)

[0283] Formula 27 means that X1 includes the point cloud data in S1. The index in Formula 28 means that X2 includes The indices except X1 in .

[0284] ③. Determine the compressed data.

[0285] In the first approach, the compressed data includes the encoding information of the index set and the remaining point information.

[0286] Encoding information of the index collection:

[0287] The encoding information of the index set includes information obtained by encoding the first index set or the second index set.

[0288] Optionally, the index set includes the first index set or the second index set. Whether the index set includes the first index set or the second index set may be preconfigured, or may be indicated by sending indication information when sending encoding information of the index set.

[0289] Optionally, the index set is the set requiring the smallest number of transmission bits between the first index set and the second index set. When sending the coding information of the index set, a first indication information can be sent to indicate that the set requiring the smallest number of transmission bits is the first index set or the second index set.

[0290] Optionally, the index set may be encoded in a manner including but not limited to: direct entropy encoding, entropy encoding after taking the difference between adjacent elements of the index, etc. Encoding the index set can reduce the amount of data transmitted for the index set.

[0291] Decompression principle of the coded information of the index set: After the base station receives the coded information of the index set, X1 can be determined according to the index set (if the index set includes X1, X1 is directly determined; if the index set includes X2, X2 is obtained for Then, the base station determines the corrected first model based on the complement of X1. After that, the base station will The corresponding point cloud data is determined to be S1 in the above S. In other words, the base station can find the point cloud data corresponding to X1 in the fourth point cloud data according to the index set, and the found point cloud data corresponding to X1 can represent the reconstructed point cloud data of S1 on the base station side.

[0292] Remaining point information:

[0293] The remaining point information includes information obtained by encoding the point cloud data in the first point cloud data except the point cloud data corresponding to the first index set: that is, the remaining point information is the encoding information of the point cloud data in S except S1.

[0294] Optionally, the encoding method of the residual point information may be quantized entropy coding or an existing coding algorithm such as Draco.

[0295] Decompression principle of the remaining point information: After the base station receives the remaining point information, it can represent the point cloud data obtained by decoding the remaining point information as the reconstructed point cloud data on the base station side for the point cloud data other than S1 in S. Furthermore, the union of the reconstructed point cloud data of S1 on the base station side and the reconstructed point cloud data of the point cloud data other than S1 in S on the base station side can be used as the reconstructed point cloud data of the first moving object set at the first moment.

[0296] Method 2:

[0297] ①, for each point cloud data in S, determine its The associated point cloud data in The index in .

[0298] ②. Determine the first index set, the third index set, and the fourth index set, or determine two of the three index sets.

[0299] Method 2 defines S, Based on π(a), S1, and X1, S2 is also defined. S2 can be referred to the following formula 29:

[0300] Wherein, δ is the second error threshold, and Formula 29 means that in the first point cloud data, the set consisting of point cloud data whose error with the associated point cloud data is greater than or equal to the first error threshold and less than the second error threshold is S2.

[0301] Then, based on S1 and S2, a third index set X3 and a fourth index set X4 are further defined. X3 can refer to the following formula 30, and X4 can refer to the following formula 31: X3 = {π(a): a∈S2} (Formula 30)

[0302] Formula 30 means that X3 includes the point cloud data in S2. The index in Formula 31 means that X4 includes Indices other than X1 and X3.

[0303] ③. Determine the compressed data.

[0304] In the second method, the compressed data includes the encoding information of the index set, the difference information and the remaining point information.

[0305] Encoding information of the index collection:

[0306] The encoding information of the index set includes information obtained by encoding two of the first index set, the third index set, and the fourth index set determined above.

[0307] Optionally, the index set includes two of the first index set, the third index set, and the fourth index set. These two may be preconfigured (for preconfigured ones, only the preconfigured two may be determined in ②), or two may be arbitrarily determined, and indication information is sent when the encoding information of the index set is sent to indicate which two are determined.

[0308] Optionally, the index sets are two sets of the first index set, the third index set, and the fourth index set that require a smaller number of transmitted bits. The terminal device needs to determine the first index set, the third index set, and the fourth index set in ②, then select two sets with a smaller number of transmitted bits from them. When sending the coding information of the index sets, the terminal device may send second indication information to indicate which two sets have a smaller number of transmitted bits.

[0309] Optionally, the index set may be encoded in a manner including but not limited to: direct entropy encoding, entropy encoding after taking the difference between adjacent elements of the index, etc. Encoding the index set can reduce the amount of data transmitted for the index set.

[0310] Decompression principle of the coded information of the index set: After the base station receives the coded information of the index set, X1 and X3 can be determined according to the index set. Then, the base station determines X1 and X3 according to the modified first model. After that, the base station will X1 The corresponding point cloud data is determined to be S1 in the above S. In other words, the base station can find the point cloud data corresponding to X1 in the fourth point cloud data according to the index set, and the found point cloud data corresponding to X1 can represent the reconstructed point cloud data of S1 on the base station side.

[0311] Difference information:

[0312] The difference information is information obtained by encoding the error corresponding to the third index set X3. The error corresponding to the third index set is: the error between the point cloud data corresponding to the third index set X3 in the first point cloud data (the point cloud data in S2) and the associated point cloud data.

[0313] Decompression principle of difference information: After the base station receives the difference information, it can Find the point cloud data corresponding to X3 in the image, and then decode the error pair obtained based on the difference information. The point cloud data corresponding to X3 in the figure is supplemented, and the supplemented point cloud data can represent the reconstructed point cloud data of S2 on the base station side.

[0314] Optionally, the difference may be encoded using quantization entropy coding or the like.

[0315] Remaining point information:

[0316] The remaining point information includes information obtained by encoding the point cloud data in the first point cloud data except the point cloud data corresponding to the first index set and the third index set: that is, the remaining point information is the encoding information of the point cloud data in S except S1 and S2.

[0317] Optionally, the encoding method of the residual point information may be quantized entropy coding or an existing coding algorithm such as Draco.

[0318] Decompression principle of the remaining point information: After the base station receives the remaining point information, it can represent the point cloud data obtained by decoding the remaining point information as the reconstructed point cloud data on the base station side of the point cloud data other than S1 and S2 in S. Furthermore, the union of the reconstructed point cloud data of S1 on the base station side, the reconstructed point cloud data of S2 on the base station side, and the reconstructed point cloud data of the point cloud data other than S1 in S on the base station side can be used as the reconstructed point cloud data of the first moving object set at the first moment.

[0319] Compared with method 1, method 2 can enable the point cloud data corresponding to the part of the remaining point information with small error in method 1 to be indicated by difference information and index set, thereby reducing the remaining point information that needs to be sent and reducing the transmission overhead during point cloud data interaction.

[0320] Before determining the compressed data in the above embodiment, the following various selection conditions can be used to select whether to obtain the compressed data by encoding based on the above method (method one or two) or to obtain the compressed data by encoding using the algorithm in the existing point cloud reporting method.

[0321] For example, the terminal device may execute the process shown in FIG8 :

[0322] 801. Determine correction data of the first model based on first point cloud data and a first model, where the first model is used to predict point cloud data of each moving object in a first moving object set.

[0323] 802. Correct the first model based on the correction data to obtain a corrected first model.

[0324] 803. Obtain selection conditions for an encoding method for compressed data.

[0325] 804 : Determine based on the selection condition whether to obtain compressed data corresponding to the first point cloud data based on the first point cloud data and the corrected first model.

[0326] After executing step 804, the terminal device may execute step 805 or step 806 according to the judgment result.

[0327] 805. Encode based on an existing method and send the encoded data.

[0328] Among them, existing methods include but are not limited to various traditional point cloud compression algorithms, such as V-PCC algorithm, LZMA algorithm, G-PCC algorithm, Draco algorithm, etc.

[0329] 806. Based on the first point cloud data and the corrected first model, obtain compressed data corresponding to the first point cloud data, and send the corrected data of the first model and the compressed data corresponding to the first point cloud data.

[0330] The following describes the selection conditions and how to make judgments based on the selection conditions in 804:

[0331] This application proposes two types of selection conditions:

[0332] The first type involves determining the prediction error based on the corrected first model, specifically, the first point cloud data, the fourth point cloud data, and the first threshold. For example, if the prediction error is small, and compressed data corresponding to the first point cloud data obtained based on the first point cloud data and the corrected first model significantly reduces transmission overhead, then either Method 1 or Method 2 described above for determining compressed data can be used. If the prediction error is large, and the compressed data corresponding to the first point cloud data obtained based on the first point cloud data and the corrected first model is similar in size before and after compression, encoding can be performed using existing methods.

[0333] For example, the first type includes the following two selection conditions: 1) and 2)

[0334] 1) Determine the norm h of the difference between the convex hulls of the first point cloud data and the fourth point cloud data. Compare h with a first threshold value that matches h. If h is less than the first threshold value, the terminal device executes step 806; otherwise, the terminal device executes step 805.

[0335] 2) Determine the difference d between the center points of the first point cloud data and the fourth point cloud data. Compare d with a first threshold value that matches d: If d is less than the first threshold value, the terminal device executes step 806; otherwise, the terminal device executes step 805.

[0336] The second type of judgment is based on the change in the first model before and after correction. Specifically, the judgment is based on the coefficient matrix of the first model, the coefficient matrix of the corrected first model, and a first threshold. The first threshold is a threshold that matches the selection criteria. For example, if the change before and after correction is small, compressing the data corresponding to the first point cloud data based on the first point cloud data and the corrected first model can significantly reduce transmission data overhead. Otherwise, encoding can be performed using existing methods.

[0337] For example, the first type includes the following three selection conditions: 3), 4), and 5):

[0338] 3) Determine the covariance Cov between the coefficient matrix at the first moment and the coefficient matrix at the second moment. Compare Cov with a second threshold value (e.g., 0.1) that matches Cov. If the absolute values ​​of all elements in Cov are less than the second threshold, execute step 806. Otherwise, the terminal device executes step 805.

[0339] 4) Determine the correlation coefficient Cor (Cor) between the coefficient matrix at the first moment and the coefficient matrix at the second moment. Compare Cor with a second threshold value (e.g., 0.95) that matches Cor. If the absolute values ​​of all elements in Cor are greater than the second threshold, execute step 806. Otherwise, execute step 805.

[0340] 5) Determine the sum of the diagonal elements s(trace) of the coefficient matrix at the first moment and the coefficient matrix at the second moment. Compare s with a first threshold value (e.g., 1) that matches s. If s is greater than the second threshold value, execute step 806; otherwise, the terminal device executes step 805.

[0341] Optionally, for the above-mentioned multiple selection conditions (such as the above-mentioned 5 types), judgment can be made according to a pre-agreed selection condition; or, a selection condition can be randomly selected for judgment; or, judgment can be made based on a combination of all the selection conditions.

[0342] Optionally, when executing step 805 or step 806, the terminal device may also report to the base station whether step 806 has been executed, that is, whether the terminal device has obtained compressed data corresponding to the first point cloud data based on the first point cloud data and the corrected first model. This reported information may be included in the compressed data.

[0343] By implementing the embodiment described in FIG8 , it is possible to determine whether to use the scheme proposed in this application based on the compression effect of the scheme proposed in this application, thereby selecting a better scheme from existing methods and the scheme proposed in this application for execution when interacting with point cloud data.

[0344] The following describes the relevant configuration information before interacting with point cloud data:

[0345] Optionally, the configuration information may be indicated by the terminal device each time the terminal device interacts with the point cloud data.

[0346] Optionally, the configuration information may be indicated by the terminal device when the solution proposed in this application is used for the first interaction of point cloud data, and the configuration information indicated for the first time may be used in subsequent interactions of point cloud data.

[0347] Optionally, the configuration information may be indicated by the base station.

[0348] Optionally, the configuration information may be indicated in accordance with protocol specifications.

[0349] Configuration information includes one or more of the following:

[0350] Compression mode configuration information: used to indicate the encoding mode used when compressing data, including one or more of quantization entropy coding / NA (quantization only without entropy coding) / Draco / LZMA / MPEG / PCC, etc.

[0351] Coefficient matrix: When used to indicate the reported correction data of the first model, it is the reported correction amount of the first coefficient matrix (corresponding to Figure 5), or the correction amount of the second coefficient matrix and the correction amount of the third coefficient matrix (corresponding to Figure 6), or the correction amount of the second coefficient matrix (corresponding to Figure 7).

[0352] Coefficient matrix dimensions: The values ​​of X in the dimensions of the second and third coefficient matrices.

[0353] Coding method selection conditions: such as selecting one or more of conditions 1) to 5).

[0354] Index matching method before encoding: such as method 1 or method 2 for determining compressed data.

[0355] Compression parameters: including compression ratio, first error threshold ε, and second error threshold δ.

[0356] Random seed: used to sample the corresponding points of each moving object to N points.

[0357] Time step: the length of time from the first moment to the second moment.

[0358] Quantization method: such as uniform quantization or non-uniform quantization.

[0359] The effects of the embodiments of the present application are described below, taking the interaction of point cloud data in a traffic scenario as an example:

[0360] For example, if there are multiple straight-moving objects of different directions and speeds, and multiple turning moving objects of different directions and speeds in a traffic scene, as shown in Figure 9, the comparison of the air interface overhead occupied by the three methods at different compression rates, namely, the traditional point cloud compression algorithm (such as the Draco algorithm), the compression method based on ODE modeling of a single moving object (abbreviated as ODE-S), and the compression method based on modeling of the first set of moving objects in this scheme (abbreviated as ODE-M), can be seen in Figure 10: In Figure 10, the horizontal axis is the air interface overhead, and the vertical axis is the NMSE mean square error. The NMSE is related to the compression rate, the first error threshold, and the second error threshold used in the scheme. The 9 groups of compression rates, the first error threshold, and the second error threshold used in Figure 10, and the corresponding relationship of each group are shown in Table 1:

[0361] Table 1

[0362] As shown in Figure 10, under the same NMSE, ODE-M requires less air interface overhead than ODE-S, and ODE-S requires less air interface overhead than Draco. For example, compared to ODE-S, ODE-M can reduce air interface overhead by approximately 18%.

[0363] For example, if a traffic scenario contains a single straight-moving object and multiple turning objects with different directions and speeds, as shown in Figure 11, a comparison of the air interface overhead occupied by the Draco algorithm, ODE-S, and ODE-M at different compression rates can be seen in Figure 12. The five groups of compression rates, first error thresholds, and second error thresholds used in Figure 12 are shown in Table 2.

[0364] Table 2

[0365] As shown in Figure 12, under the same NMSE, ODE-M requires less air interface overhead than ODE-S, and ODE-S requires less air interface overhead than Draco. For example, compared to ODE-S, ODE-M can reduce air interface overhead by approximately 27%.

[0366] Based on the above example, when exchanging point cloud data, simultaneously modeling and compressing multiple moving objects can effectively reduce air interface overhead. Furthermore, this approach leverages the relationships between moving objects, improving the accuracy of the model's description of each object's state.

[0367] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0368] As used in this specification, the terms "component," "module," "system," and the like are used to represent computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. By way of illustration, both an application running on a computing device and a computing device can be a component. One or more components can reside in a process and / or an execution thread, and a component can be located on a computer and / or distributed between two or more computers. In addition, these components can be executed from various computer-readable media having various data structures stored thereon. Components can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component on a local system, a distributed system, and / or a network, such as the Internet interacting with other systems via signals).

[0369] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0370] It should be noted that for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps may be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0371] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the above-mentioned units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0372] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0373] In addition, the functional units in the embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0374] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc., specifically a processor in a computer device) to execute all or part of the steps of the above-mentioned methods of each embodiment of the present application. Among them, the aforementioned storage medium may include: U disk, mobile hard disk, magnetic disk, optical disk, read-only memory (Read-Only Memory, abbreviated: ROM) or random access memory (Random Access Memory, abbreviated: RAM) and other media that can store program codes.

[0375] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A data transmission method, characterized in that: Applied to a terminal device, the method comprises: Based on first point cloud data and a first model, correction data of the first model is determined; the first point cloud data is real point cloud data of a first moving object set at a first moment; the number of moving objects included in the first moving object set is multiple; the first model is used to predict point cloud data of each moving object in the first moving object set; Correcting the first model based on the correction data to obtain a corrected first model; Based on the first point cloud data and the corrected first model, obtaining compressed data corresponding to the first point cloud data; The corrected data of the first model and the compressed data corresponding to the first point cloud data are sent.

2. The method according to claim 1, characterized in that The step of determining correction data of the first model based on the first point cloud data and the first model includes: Based on the first model and the second point cloud data, an initial prediction is performed on the point cloud data of the first moving object set at the first moment to obtain third point cloud data, wherein the second point cloud data is the reconstructed point cloud data of the first moving object set at the second moment, and the second moment is a moment before the first moment; Based on the error between the first point cloud data and the third point cloud data, corrected data of the first model is obtained.

3. The method according to claim 2, characterized in that The correction data of the first model includes the correction amount of the first coefficient matrix; the first coefficient matrix is ​​used to indicate the change of the point cloud data of the first moving object set within a first time period, and the first time period is the time period between the first moment and the second moment.

4. The method according to claim 2, characterized in that: The correction data of the first model includes the correction amount of the second coefficient matrix and the correction amount of the third coefficient matrix; the second coefficient matrix and the third coefficient matrix are obtained by low-rank decomposition of the first coefficient matrix, and the first coefficient matrix is ​​used to indicate the change of point cloud data of the first moving object set within a first time period, and the first time period is the time period between the first moment and the second moment.

5. The method according to claim 2, characterized in that: The correction data of the first model includes a correction amount of a second coefficient matrix, where the second coefficient matrix is ​​obtained by low-rank decomposing the first coefficient matrix when the third coefficient matrix is ​​an initial matrix and the third coefficient matrix remains unchanged; The first coefficient matrix is ​​used to indicate changes in point cloud data of the first moving object set within a first time period, where the first time period is a time period between the first moment and the second moment.

6. The method according to claim 4 or 5, characterized in that: The dimension of the first coefficient matrix is ​​M×M, M is the number of motion objects included in the first motion object set, the dimension of the second coefficient matrix is ​​M×X, and the dimension of the third coefficient matrix is ​​X×M, M is a positive integer greater than 3, and X is a positive integer greater than or equal to 3 and less than or equal to M.

7. The method according to any one of claims 2 to 5, characterized in that: The obtaining, based on the first point cloud data and the corrected first model, compressed data corresponding to the first point cloud data includes: Based on the modified first model and the second point cloud data, re-predicting the point cloud data of the first moving object set at the first moment to obtain fourth point cloud data; Based on the fourth point cloud data, the first point cloud data is compressed to obtain the compressed data.

8. The method according to claim 7, characterized in that The compressed data includes encoding information of an index set and remaining point information; wherein, the index set is a first index set or a second index set, the first index set includes indexes corresponding to point cloud data in the first point cloud data whose errors with associated point cloud data are less than a first error threshold, and the associated point cloud data are point cloud data in the fourth point cloud data that are associated with point cloud data in the first point cloud data; the second index set is the complement of the first index set in a preset index set; the preset index set is an index set corresponding to the fourth point cloud data; the remaining point information is information obtained by encoding point cloud data in the first point cloud data except for point cloud data corresponding to the first index set.

9. The method according to claim 8, characterized in that The compressed data also includes first indication information, where the first indication information is used to indicate that the index set is the set with the smallest number of bits required to be transmitted between the first index set and the second index set.

10. The method according to claim 7, characterized in that The compressed data includes encoding information, difference information and remaining point information of an index set; wherein the index set includes two of a first index set, a third index set and a fourth index set, the first index set includes indexes corresponding to point cloud data in the first point cloud data whose errors with associated point cloud data are less than a first error threshold, the third index set includes indexes corresponding to point cloud data in the first point cloud data whose errors with associated point cloud data are greater than or equal to the first error threshold and less than a second error threshold, the associated point cloud data is point cloud data in the fourth point cloud data associated with point cloud data in the first point cloud data; the fourth index set is a set of indexes in a preset index set other than the first index set and the third index set, the preset index set is an index set corresponding to the fourth point cloud data; the remaining point information is information obtained by encoding point cloud data in the first point cloud data other than point cloud data corresponding to the first index set and the third index set; the difference information is information obtained by encoding errors corresponding to the third index set, the errors corresponding to the third index set are errors between point cloud data corresponding to the third index set in the first point cloud data and the associated point cloud data.

11. The method according to claim 10, characterized in that The compressed data also includes second indication information, where the second indication information is used to indicate that the index set includes two sets of the first index set, the third index set, and the fourth index set that require a smaller number of bits to be transmitted.

12. The method according to any one of claims 2 to 5, characterized in that: Before determining the correction data of the first model based on the first point cloud data and the first model, the method further includes: Obtaining reconstructed point cloud data at the second moment; Acquire the real point cloud data collected at the first moment; The first point cloud data is acquired from the real point cloud data at the first moment, wherein the first moving object set is an intersection between a moving object set corresponding to the reconstructed point cloud data at the second moment and a moving object set corresponding to the real point cloud data at the first moment; Send third indication information, where the third indication information is used to indicate the first moving object set.

13. The method according to claim 7, characterized in that The method further comprises: Acquire selection conditions of the encoding method of the compressed data; If the type of the selection condition is the first type, based on the first point cloud data, the fourth point cloud data and the first threshold, it is determined whether compressed data corresponding to the first point cloud data is obtained based on the first point cloud data and the corrected first model; and the first threshold matches the selection condition.

14. The method according to claim 7, characterized in that The method further comprises: Acquire selection conditions of the encoding method of the compressed data; If the type of the selection condition is the second type, based on the coefficient matrix of the first model, the coefficient matrix of the corrected first model and the second threshold, it is determined whether compressed data corresponding to the first point cloud data is obtained based on the first point cloud data and the corrected first model; the second threshold matches the selection condition.

15. The method according to any one of claims 1 to 5, characterized in that Before determining the correction data of the first model based on the first point cloud data and the first model, the method further includes: Send configuration information; Among them, the configuration information includes one or more configuration information of compression method, coefficient matrix, coefficient matrix dimension, encoding method selection conditions, pre-encoding index matching method, compression parameters, random seed, time step, and quantization method.

16. A data transmission method, characterized in that: Applied to a base station, the method comprises: Receive the corrected data of the first model and the compressed data corresponding to the first point cloud data; the first point cloud data is the real point cloud data of the first moving object set at the first moment; the number of moving objects included in the first moving object set is multiple; the first model is used to predict the point cloud data of each moving object in the first moving object set; Correcting the first model based on the correction data to obtain a corrected first model; Based on the modified first model, the compressed data is decompressed to obtain reconstructed point cloud data of the first moving object set at the first moment.

17. The method according to claim 16, characterized in that The decompressing the compressed data based on the modified first model to obtain the reconstructed point cloud data of the first moving object set at the first moment includes: Based on the corrected first model and the second point cloud data, predicting the point cloud data of the first moving object set at the first moment, to obtain fourth point cloud data; the second point cloud data is the reconstructed point cloud data of the first moving object set at the second moment, and the second moment is a moment before the first moment; Based on the fourth point cloud data, the compressed data is decompressed to obtain reconstructed point cloud data of the first moving object set at the first moment.

18. The method according to claim 17, characterized in that The correction data of the first model includes the correction amount of the first coefficient matrix; the first coefficient matrix is ​​used to indicate the change of the point cloud data of the first moving object set within a first time period, and the first time period is the time period between the first moment and the second moment.

19. The method according to claim 17, characterized in that The correction data of the first model includes the correction amount of the second coefficient matrix and the correction amount of the third coefficient matrix; the second coefficient matrix and the third coefficient matrix are obtained by low-rank decomposition of the first coefficient matrix, and the first coefficient matrix is ​​used to indicate the change of point cloud data of the first moving object set within a first time period, and the first time period is the time period between the first moment and the second moment.

20. The method according to claim 17, characterized in that The correction data of the first model includes a correction amount of a second coefficient matrix, where the second coefficient matrix is ​​obtained by low-rank decomposing the first coefficient matrix when the third coefficient matrix is ​​an initial matrix and the third coefficient matrix remains unchanged; The first coefficient matrix is ​​used to indicate changes in point cloud data of the first moving object set within a first time period, where the first time period is a time period between the first moment and the second moment.

21. The method according to claim 19 or 20, characterized in that The dimension of the first coefficient matrix is ​​M×M, M is the number of motion objects included in the first motion object set, the dimension of the second coefficient matrix is ​​M×X, and the dimension of the third coefficient matrix is ​​X×M, M is a positive integer greater than 3, and X is a positive integer greater than or equal to 3 and less than or equal to M.

22. The method according to claim 17, characterized in that The compressed data includes encoding information of an index set and remaining point information; wherein the index set is a first index set or a second index set; the first index set includes indexes corresponding to point cloud data in the first point cloud data whose errors with associated point cloud data are less than a first error threshold, and the associated point cloud data are point cloud data in the fourth point cloud data that are associated with point cloud data in the first point cloud data; the second index set is the complement of the first index set in a preset index set; the preset index set is the index set corresponding to the fourth point cloud data; the remaining point information is information obtained by encoding point cloud data in the first point cloud data except for point cloud data corresponding to the first index set.

23. The method according to claim 22, characterized in that The compressed data also includes first indication information, where the first indication information is used to indicate that the index set is the set with the smallest number of bits required to be transmitted between the first index set and the second index set.

24. The method according to claim 17, characterized in that The compressed data includes encoding information, difference information and remaining point information of an index set; wherein the index set includes two of a first index set, a third index set and a fourth index set; the first index set includes indexes corresponding to point cloud data in the first point cloud data whose errors with associated point cloud data are less than a first error threshold, the third index set includes indexes corresponding to point cloud data in the first point cloud data whose errors with associated point cloud data are greater than or equal to the first error threshold and less than a second error threshold, and the associated point cloud data is point cloud data in the fourth point cloud data associated with point cloud data in the first point cloud data; the fourth index set is a set of indexes in a preset index set other than the first index set and the third index set, and the preset index set is an index set corresponding to the fourth point cloud data; the remaining point information is information obtained by encoding point cloud data in the first point cloud data other than point cloud data corresponding to the first index set and the third index set; the difference information is information obtained by encoding errors corresponding to the third index set, and the errors corresponding to the third index set are errors between point cloud data corresponding to the third index set in the first point cloud data and the associated point cloud data.

25. The method according to claim 24, characterized in that The compressed data also includes second indication information, where the second indication information is used to indicate that the index set includes two sets of the first index set, the third index set and the fourth index set that require a smaller number of bits to be transmitted.

26. The method according to claim 17, characterized in that Before receiving the corrected data of the first model and the compressed data corresponding to the first point cloud data, the method further includes: receiving third indication information, where the third indication information is used to indicate the first moving object set; Obtaining reconstructed point cloud data at the second moment; Based on the third indication information and the reconstructed point cloud data at the second moment, the first moving object set and the second point cloud data are determined.

27. The method according to any one of claims 16 to 20, characterized in that Before receiving the corrected data of the first model and the compressed data corresponding to the first point cloud data, the method further includes: Send configuration information; Among them, the configuration information includes one or more configuration information of compression method, coefficient matrix, coefficient matrix dimension, encoding method selection conditions, pre-encoding index matching method, compression parameters, random seed, time step, and quantization method.

28. A communication device, characterized in that: Comprising units for performing the method as claimed in any one of claims 1 to 15 or claims 16 to 27.

29. A communication device, characterized in that: The method comprises a processor and a memory, wherein the processor and the memory are coupled, and the processor is used to implement the method according to any one of claims 1 to 15 or claims 16 to 27.

30. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is called by the computer, the computer executes the method according to any one of claims 1 to 15 or claims 16 to 27.

Images