Data processing method and device
Through point cloud data processing, the target feature point and direction information are determined, the direction of the three-dimensional scene model is corrected, and the problem of offsetting the three-dimensional scene model in the target scene is solved, improving the accuracy of image display.
Patent Information
- Application Number
- CN202411379784.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-29
- Publication Date
- 2025-01-03
Smart Images
Figure CN119251441A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a data processing method and device. Background Art
[0002] When an image corresponding to a 3D scene model is displayed in a specific target scene, the 3D scene model may be offset to a certain extent relative to the target scene. For example, the ground of the 3D scene model may be offset by a certain angle relative to a horizontal plane specified in the target scene, resulting in a deviation of the image corresponding to the 3D scene model displayed in the target scene relative to the target scene. Summary of the invention
[0003] To this end, this application discloses the following technical solutions:
[0004] The first aspect of the present application provides a data processing method, comprising:
[0005] Obtaining point cloud data of a three-dimensional scene model, the point cloud data comprising a plurality of feature points in the three-dimensional scene model, each of the feature points having corresponding image rendering information;
[0006] Determining a target feature point belonging to a target object in the point cloud data, wherein a shape of the target object and a first direction of the three-dimensional scene model conform to a target relationship;
[0007] Determining direction information of the first direction according to the target feature point;
[0008] The three-dimensional scene model is corrected according to the direction information of the first direction and the direction information of the second direction in the target scene, and the first direction and the second direction satisfy a corresponding relationship.
[0009] Optionally, determining the target feature points belonging to the target object in the point cloud data includes:
[0010] Performing semantic segmentation processing on the point cloud data according to the target processing model to obtain a segmentation result;
[0011] Target feature points belonging to the target object are determined according to the segmentation result.
[0012] Optionally, determining the target feature points belonging to the target object according to the segmentation result includes:
[0013] Obtaining target text information of the object to which the target object belongs, wherein the target text information is used to characterize the composition structure of the object, and the target object is a component of the object;
[0014] The target feature points belonging to the target object are determined according to the segmentation result and the target text information.
[0015] Optionally, determining the direction information of the first direction according to the target feature point includes:
[0016] Obtaining normal information of a plurality of target feature points belonging to the target object, wherein the normal information of the target feature points is related to the shape of the target object;
[0017] The direction information of the first direction is determined according to the normal information of the plurality of target feature points.
[0018] Optionally, the three-dimensional scene model includes multiple target objects;
[0019] The determining the direction information of the first direction according to the normal information of the plurality of target feature points comprises:
[0020] For each of the target objects, determining normal information corresponding to the target object according to normal information of a plurality of target feature points belonging to the target object;
[0021] The direction information of the first direction is determined according to the normal information corresponding to the plurality of target objects.
[0022] Optionally, determining the normal information corresponding to the target object according to the normal information of a plurality of target feature points belonging to the target object includes:
[0023] In a set of target feature points belonging to the target object, determining a proportion of target feature points corresponding to each normal line information;
[0024] The normal information corresponding to the target feature points whose proportion meets the preset conditions is determined as the normal information corresponding to the target object.
[0025] Optionally, the correcting the three-dimensional scene model according to the direction information of the first direction and the direction information of the second direction in the target scene includes:
[0026] Determining a rotation parameter of the three-dimensional scene model according to the direction information of the first direction and the direction information of the second direction in the target scene;
[0027] The three-dimensional scene model is rotated according to the rotation parameters.
[0028] Optionally, obtaining point cloud data of the three-dimensional scene model includes:
[0029] Obtaining initial point cloud data, wherein the initial point cloud data includes a plurality of Gaussian feature points;
[0030] The initial point cloud data is processed according to the scene image of the three-dimensional scene model to obtain point cloud data of the three-dimensional scene model.
[0031] Optionally, determining the direction information of the first direction according to the target feature point includes:
[0032] Performing fitting processing according to the target feature points to obtain shape information of the target object;
[0033] Direction information of the first direction is determined according to shape information of the target object.
[0034] A second aspect of the present application provides a data processing device, including:
[0035] An obtaining unit, configured to obtain point cloud data of a three-dimensional scene model, wherein the point cloud data includes a plurality of feature points in the three-dimensional scene model, and each of the feature points has corresponding image rendering information;
[0036] a segmentation unit, configured to determine, in the point cloud data, target feature points belonging to a target object, wherein a shape of the target object and a first direction of the three-dimensional scene model conform to a target relationship;
[0037] a determining unit, configured to determine direction information of the first direction according to the target feature point;
[0038] The correction unit is used to correct the three-dimensional scene model according to the direction information of the first direction and the direction information of the second direction in the target scene, and the first direction and the second direction satisfy a corresponding relationship. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.
[0040] Figure 1 is a flow chart of a data processing method provided in an embodiment of the present application;
[0041] Figure 2 is a flow chart of another data processing method provided in an embodiment of the present application;
[0042] Figure 3 is a schematic diagram of a first direction of a three-dimensional scene model and a second direction of a target scene provided in an embodiment of the present application;
[0043] Figure 4It is a structural diagram of a data processing device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0044] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0045] This application embodiment provides a data processing method, see Figure 1 , the method may include the following steps.
[0046] S101, obtaining point cloud data of a three-dimensional scene model, where the point cloud data includes a plurality of feature points in the three-dimensional scene model, and each feature point has corresponding image rendering information.
[0047] The three-dimensional scene model can be constructed based on any real physical scene and used to simulate the scene model of the real physical scene. For example, it can be a three-dimensional scene model constructed based on an office scene or a three-dimensional scene model constructed based on a classroom scene.
[0048] Each feature point in the point cloud data may correspond to a specific local area of a specific object in a real physical scene, and the feature point may have image rendering information that reflects the appearance attributes of the local area, such as position, shape, color, texture, etc. Therefore, based on the image rendering information contained in each feature point in the point cloud data, a corresponding 3D scene model can be generated in the target scene.
[0049] As an example, a table leg in a real physical scene may correspond to multiple feature points in the point cloud data. These feature points may have image rendering information of various areas of the table leg. Based on these feature points, the table leg in the three-dimensional scene model may be rendered.
[0050] The point cloud data of the three-dimensional scene model can be obtained in a variety of ways. For example, the real physical scene for which the three-dimensional scene model needs to be constructed can be photographed from multiple angles to obtain multiple frames of images of the scene, and then the point cloud data of the three-dimensional scene model can be obtained based on these images.
[0051] In some optional embodiments, the feature points contained in the point cloud data may be Gaussian feature points. In this case, the point cloud data of the three-dimensional scene model may be obtained by:
[0052] Obtaining initial point cloud data, the initial point cloud data including a plurality of Gaussian feature points;
[0053] The initial point cloud data is processed according to the scene image of the three-dimensional scene model to obtain point cloud data of the three-dimensional scene model.
[0054] The initial point cloud data may include a preset number of Gaussian feature points, and a Gaussian feature point may include a set of data that can represent image rendering information. In the initial point cloud data, the data of each Gaussian feature point may be preset initial data.
[0055] The scene image can be obtained by photographing a real physical scene from multiple angles. For example, a classroom scene can be photographed from multiple angles to obtain multiple frames of scene images corresponding to the classroom scene.
[0056] When processing the initial point cloud data, the scene image obtained by processing according to the three-dimensional reconstruction method can be adjusted based on the processing result. After the data of each Gaussian feature point is adjusted once, multiple frames of two-dimensional images representing the three-dimensional scene model can be generated based on the point cloud data at this time, and the similarity between the generated image and the actual captured scene image can be determined. According to the similarity, the above-mentioned steps of processing the scene image according to the three-dimensional reconstruction method and adjusting the data contained in each Gaussian feature point based on the processing result are performed again until the similarity between the generated image and the scene image meets the condition, for example, until the similarity is greater than a certain threshold.
[0057] The three-dimensional reconstruction method for processing scene images may be a structure from motion (SFM) method, or other methods capable of achieving three-dimensional reconstruction in the relevant technical field, without limitation.
[0058] S102, determining target feature points belonging to the target object in the point cloud data, wherein the shape of the target object and the first direction of the three-dimensional scene model conform to the target relationship.
[0059] The first direction of the 3D scene model may be a direction with a specific physical meaning in the 3D scene model. As an example, the first direction may be the gravity direction of the 3D scene model, or may be a direction in the 3D scene model parallel to the gravity direction.
[0060] The shape of the target object and the first direction of the three-dimensional scene model conform to a target relationship, which may include that a specific surface of the target object is parallel to or perpendicular to the first direction, or that an extending direction of the target object is parallel to or perpendicular to the first direction.
[0061] As some examples, the target object may be a table leg of a table in a three-dimensional scene model, the extension direction of the table leg being parallel to the gravity direction of the three-dimensional scene model; the target object may be a table top of a table in a three-dimensional scene model, the upper surface of the table top being perpendicular to the gravity direction of the three-dimensional scene model; the target object may be a side frame of a door frame in a three-dimensional scene model, the extension direction of the side frame being parallel to the gravity direction of the three-dimensional scene model.
[0062] The target feature points belonging to the target object are equivalent to the target feature points having the image rendering information of the target object in the point cloud data. For example, when the target object includes a table leg, multiple feature points having the image rendering information of the table leg can be determined as the target feature points belonging to the table leg, and when the target object includes a side frame of a door frame, multiple feature points having the image rendering information of the side frame can be determined as the target feature points belonging to the side frame of the door frame.
[0063] S103: Determine direction information of a first direction according to the target feature point.
[0064] The direction information of the first direction is information used to characterize the first direction of the three-dimensional scene model in the target scene. As an example, the direction information of the first direction can be a vector in the coordinate system of the target scene, denoted as P1(x1, y1, z1), and the direction of the vector is the first direction.
[0065] S104, performing correction processing on the three-dimensional scene model according to the direction information of the first direction and the direction information of the second direction in the target scene, and the first direction and the second direction satisfy a corresponding relationship.
[0066] The direction information of the second direction is information used to characterize the second direction of the target scene in the target scene. As an example, the direction information of the second direction can also be a vector in the coordinate system of the target scene, denoted as P2(x2, y2, z2), and the direction of the vector is the second direction.
[0067] The first direction and the second direction satisfy the corresponding relationship, which may include that the first direction and the second direction have the same physical meaning in their respective scenes. For example, the first direction may be the gravity direction in the three-dimensional scene model, and the second direction may be the gravity direction in the target scene, or the first direction may be the direction perpendicular to the gravity direction in the three-dimensional scene model (e.g., the horizontal direction), and the second direction may be the direction perpendicular to the gravity direction in the target scene (e.g., the horizontal direction).
[0068] Since there is a correspondence between the first direction and the second direction, if the direction information of the first direction and the direction information of the second direction do not satisfy the above correspondence, it can be explained that the three-dimensional scene model is offset in the target scene, and the degree and direction of the deviation between the direction information of the first direction and the direction information of the second direction from the above correspondence can characterize the degree and direction of the offset of the three-dimensional scene model in the target scene.
[0069] Based on this, the three-dimensional scene model can be corrected based on the direction information of the first direction and the direction information of the second direction, so that the direction information of the first direction and the direction information of the second direction of the corrected three-dimensional scene model satisfy the above correspondence.
[0070] Exemplarily, assuming that the first direction and the second direction are the gravity directions of their respective scenes, if the vector corresponding to the first direction and the vector corresponding to the second direction in the coordinate system of the target scene are not parallel, it can be said that the three-dimensional scene model has shifted in the target scene, and the angle between the two vectors reflects the direction and degree of the shift. Therefore, the three-dimensional scene model can be corrected based on the angle between the vector corresponding to the first direction and the vector corresponding to the second direction, so that the vector in the first direction of the corrected three-dimensional scene model is parallel.
[0071] The beneficial effects of this embodiment are:
[0072] After obtaining the point cloud data of the three-dimensional scene model, the direction information of the first direction of the three-dimensional scene model can be determined based on the target feature points of the target object in the point cloud data that satisfy the target relationship with the first direction, and then the offset of the three-dimensional scene model relative to the target scene can be corrected based on the direction information of the first direction and the direction information of the second direction in the target scene, thereby solving the problem of the three-dimensional scene model being offset relative to the target scene.
[0073] In some optional embodiments, see Figure 2 , the method of determining the target feature points belonging to the target object in the point cloud data can be:
[0074] Perform semantic segmentation processing on the point cloud data according to the target processing model to obtain the segmentation result;
[0075] The target feature points belonging to the target object are determined according to the segmentation results.
[0076] The target processing model may be any data processing model that can perform semantic segmentation processing on point cloud data. As an example, the target processing model may be a large language model or other neural network models.
[0077] When performing semantic segmentation processing, in order to improve processing efficiency and accuracy of processing results, the point cloud data of the three-dimensional scene model can be first encoded using an encoder to obtain a feature vector corresponding to each feature point in the point cloud data, and then the feature vectors corresponding to multiple feature points in the point cloud data are input into the target processing model to obtain the segmentation result output by the target processing model.
[0078] The segmentation result is used to indicate which feature points in the point cloud data belong to the same object and which feature points belong to different objects. The segmentation result can be used to determine how many objects there are in the 3D scene model, and the multiple feature points contained in the point cloud data can be divided into the determined multiple objects. For example, the segmentation result can be used to determine that there are 10 objects in the 3D scene model, and it can be determined which feature points belong to object 1, which feature points belong to object 2, which feature points belong to object 3...which feature points belong to object 10.
[0079] The processing of the segmentation results can also be achieved through a model with corresponding processing capabilities, which can be the aforementioned target processing model or other models different from the aforementioned target processing model.
[0080] When processing the segmentation result, it is possible to determine which objects are target objects based on the multiple objects that have been divided in the segmentation result, thereby determining the feature points belonging to the target objects as target feature points.
[0081] In some embodiments, the model used to process the segmentation results may include a pre-trained processing model specifically used to identify specific target objects, such as a model specifically used to identify table legs and a model specifically used to identify side frames of door frames. In this case, the segmentation results can be directly input into the corresponding model, and after processing by the corresponding model, it can be determined which objects are specific target objects.
[0082] In some optional embodiments, the segmentation result may also be directly processed using the aforementioned target processing model to determine the target feature points.
[0083] When the target processing model is a large language model, the text information and segmentation results used to describe the target object can be input into the target processing model, so that the target processing model can identify which objects are target objects that conform to the text information based on the text information.
[0084] The input text information may only describe the target object itself. For example, the input text information may include the name of the target object. As an example, the target object may include table legs, stool legs, and side frames of a door frame. In this case, the input text information may include "table legs", "stool legs", and "door frame sides".
[0085] In some optional embodiments, the text information provided to the target processing model may also be target text information used to describe the target object and the object to which the target object belongs. In this case, the method of determining the target feature points according to the segmentation result may be:
[0086] Obtain target text information of the object to which the target object belongs, the target text information is used to characterize the composition structure of the object, and the target object is a component of the object;
[0087] The target feature points belonging to the target object are determined based on the segmentation results and the target text information.
[0088] For example, if the target object includes table legs, the table is the object to which the target object belongs, and the target text information describing the composition structure of the table may be "the table includes a table top, a beam and table legs."
[0089] The above target text information may be manually input by a relevant user or selected from a plurality of preset text information before determining the target feature point.
[0090] Optionally, the above-mentioned target text information can be directly input into the target processing model, or the target text information can be first encoded using a pre-built text encoder to obtain a hierarchical text tree corresponding to the target text information, and then the hierarchical text tree is input into the target processing model, so that the target processing model determines the target feature points based on the segmentation results and the hierarchical text tree corresponding to the target text information.
[0091] The method of generating a hierarchical text tree based on a text encoder can be referred to in the related art, and this embodiment will not be repeated. As an example, the above target text information "a table includes a table top, a beam and table legs" can be processed to obtain a hierarchical text tree of "table->1 tabletop; 2 beams; 3 table legs".
[0092] The benefits of determining target feature points based on target text information are:
[0093] When the target feature points are determined only based on the text information representing the target object, the target processing model may identify non-target objects with similar shapes to the target object as target objects, and then identify the feature points belonging to these non-target objects as target feature points. However, identifying the target feature points in combination with the composition structure of the object to which the target object belongs can avoid the above situation to a certain extent, thereby improving the accuracy of the recognition results.
[0094] For example, when determining the target feature point only based on the text information "table leg", the target processing model may also identify the wooden stick leaning against the wall as a table leg, resulting in the determination of the wrong target feature point. However, based on the composition structure of the table in the target text information, the target processing model can exclude the wooden stick leaning against the wall, thereby determining the correct target feature point.
[0095] In some optional embodiments, instead of adopting the above-mentioned method of first obtaining the segmentation results and then determining the target feature points, the target text information and the point cloud data of the three-dimensional scene model can be directly input into the target processing model, and the target processing model can directly determine which feature points are the target feature points belonging to the target object.
[0096] In this embodiment, various methods may be used to determine the direction information of the first direction according to the target feature point.
[0097] In some embodiments, the direction information of the first direction may be determined according to the target feature point by:
[0098] Obtaining normal information of a plurality of target feature points belonging to the target object, wherein the normal information of the target feature points is related to the shape of the target object;
[0099] Direction information of the first direction is determined according to normal information of a plurality of target feature points.
[0100] Each feature point in the point cloud data has corresponding normal information. The normal information of a feature point can be a vector representing the normal direction of the feature point in the coordinate system of the target scene. The normal information of different feature points can be the same or different.
[0101] The three-dimensional scene model may include one target object or multiple target objects.
[0102] If the three-dimensional scene model includes multiple objects, the method of determining the direction information of the first direction according to the normal information of multiple target feature points may be:
[0103] For each target object, determining normal information corresponding to the target object according to normal information of a plurality of target feature points belonging to the target object;
[0104] Direction information of a first direction is determined according to normal information corresponding to a plurality of target objects.
[0105] The normal information corresponding to the target object may be a vector representing the target direction of the target object in the coordinate system of the target scene. The target direction of the target object may be a direction that satisfies the following conditions:
[0106] The size of the target object in the target direction is significantly larger than the size of the target object in other non-target directions.
[0107] For example, see Figure 3 (1), the size of the table leg in the length direction is significantly larger than the sizes in other directions, so the target direction of the table leg can be its length direction, and the normal information corresponding to the table leg can be a vector representing the length direction of the table leg in the coordinate system of the target scene.
[0108] After obtaining normal information corresponding to a plurality of target objects, the normal information of the plurality of target objects may be fused, and the fused information may be determined as direction information of the first direction.
[0109] Alternatively, the normal information of multiple target objects may be fused, and based on the fused information, a piece of direction information that meets the target relationship with the fused information is determined as the direction information of the first direction.
[0110] As an example, Figure 3 The three-dimensional scene model shown in (1) may include two target objects, namely, table legs and chair legs. The first direction may be the gravity direction of the three-dimensional scene model. Therefore, the target direction vector of the table legs and the target direction vector of the chair legs may be determined. Then, the two vectors are averaged, and the obtained mean vector is used as the direction information of the first direction.
[0111] Alternatively, the first direction may be a horizontal direction perpendicular to the gravity direction of the three-dimensional scene model. In this case, after determining the above-mentioned mean vector, a vector perpendicular to the mean vector may be determined, and this vector may be determined as the direction information of the first direction.
[0112] If the three-dimensional scene model includes only one target object, after determining the normal information of the target object, the normal information can be directly determined as the direction information of the first direction, or a representation direction perpendicular to the target direction can be determined based on the normal information, and the direction information can be determined as the direction information of the first direction.
[0113] For each target object, according to the normal information of multiple target feature points belonging to the target object, the normal information corresponding to the target object may be determined in the following manner:
[0114] In a set of target feature points belonging to the target object, determining a proportion of target feature points corresponding to each normal information;
[0115] The normal information of the corresponding target feature points whose proportion meets the preset conditions is determined as the normal information corresponding to the target object.
[0116] Generally, for any object, if the size of the object in a certain direction is significantly larger than the size of the object in other directions, then among the feature points belonging to the object, the normal direction of most of the feature points is consistent with the direction of the larger size.
[0117] by Figure 3 For example, the size of a table leg in the length direction is significantly larger than the sizes in other directions, so among the multiple feature points belonging to the table leg, the normal direction of most of the feature points is the length direction of the table leg.
[0118] Based on the above characteristics, for each target object, the normal information of each target feature point belonging to the target object can be obtained first to determine which target feature points have the same corresponding normal information and which target feature points have different corresponding normal information. Then, for the set of all target feature points belonging to the target object, the proportion of target feature points corresponding to each type of normal information in the set is determined, and the normal information with the largest corresponding proportion is determined as the normal information corresponding to the target object.
[0119] In some embodiments, the direction information of the first direction may be determined according to the target feature point by:
[0120] Perform fitting processing according to target feature points to obtain shape information of the target object;
[0121] Direction information of the first direction is determined according to shape information of the target object.
[0122] In this embodiment, for each target object, the coordinate value of each target feature point belonging to the target object in the coordinate system of the target scene can be determined first, and then the coordinate values corresponding to the multiple target feature points are processed based on the curve fitting method to obtain the shape information of the target object. The shape information of the target object can represent a straight line extending along the largest dimension of the target object.
[0123] by Figure 3 (1) is an example. After determining multiple target feature points belonging to the target object table leg, the coordinate values of these target feature points in the coordinate system of the target scene can be processed based on the curve fitting method to obtain the shape information of the table leg. The shape information of the table leg can represent a straight line extending along the length direction of the table leg in the coordinate system of the target scene. For example, the shape information of the table leg can be a function expression corresponding to the straight line in the coordinate system of the target scene.
[0124] After the shape information is obtained, the direction information represented and parallel to the straight line represented by the shape information can be determined as the direction information of the first direction. Figure 3 (1) is an example. If the first direction is the gravity direction of the three-dimensional scene model, it can be determined that the straight line represented by the shape information of the table leg in the three-dimensional scene model is parallel to the gravity direction of the three-dimensional scene model. Therefore, the direction information that is parallel to the straight line represented by the shape information can be determined as the direction information of the gravity direction.
[0125] Alternatively, the direction information representing the direction perpendicular to the straight line represented by the shape information may be determined as the direction information of the first direction.
[0126] Optionally, if there are multiple target objects in the three-dimensional scene model, shape information of the multiple target objects may be fused, and based on the fused shape information, the direction information of the first direction may be determined in the above manner.
[0127] Optionally, the three-dimensional scene model may be corrected according to the direction information of the first direction and the direction information of the second direction in the target scene by:
[0128] Determining a rotation parameter of the three-dimensional scene model according to the direction information of the first direction and the direction information of the second direction in the target scene;
[0129] The 3D scene model is rotated according to the rotation parameters.
[0130] For any two non-parallel vectors in the same coordinate system, a set of rotation parameters can be determined based on the two vectors. The rotation parameters can be a rotation matrix or other forms of parameters. Based on the rotation parameters, any one of the vectors can be rotated to be parallel to the other vector.
[0131] Based on this, after obtaining the direction information of the first direction and the direction information of the second direction, a rotation parameter for rotating the first direction to be parallel to the second direction is determined, and the three-dimensional scene model is rotated based on the rotation parameter to obtain a corrected three-dimensional scene model.
[0132] As an example, see Figure 3 , the coordinate system of the 3D scene model before correction and the target scene can be expressed as Figure 3 As shown in (1), it can be seen that the three-dimensional scene model has a certain degree of inclination relative to the coordinate system of the target scene. In this embodiment, the direction information of the first direction (i.e., the gravity direction of the three-dimensional scene model) can be determined according to the above method, and the rotation parameter can be determined based on the direction information of the first direction and the direction information of the second direction (i.e., the gravity direction of the target scene). Figure 3 The three-dimensional scene model shown in (1) is rotated to obtain Figure 3 The three-dimensional scene model shown in (2).
[0133] The direction information of the second direction of the target scene can be obtained as follows:
[0134] The second direction of the target scene generally satisfies a specific relationship with the coordinate axis of the coordinate system of the target scene, and the direction information of the corresponding coordinate axis in the coordinate system of the target scene can be obtained, and the direction information of the second direction of the target scene is determined based on this direction information. Figure 3 In (1), the gravity direction of the target scene is parallel to the Z axis of the coordinate system of the target scene. Therefore, the direction information of the Z axis of the coordinate system of the target scene can be obtained and the direction information is determined as the direction information of the gravity direction of the target scene.
[0135] The present application embodiment provides a data processing device, see Figure 4 , is a schematic diagram of the structure of the device, and the device may include the following units.
[0136] An obtaining unit 401 is used to obtain point cloud data of a three-dimensional scene model, where the point cloud data includes a plurality of feature points in the three-dimensional scene model, and each feature point has corresponding image rendering information;
[0137] A segmentation unit 402 is used to determine a target feature point belonging to a target object in the point cloud data, and the shape of the target object and the first direction of the three-dimensional scene model meet the target relationship;
[0138] A determination unit 403, configured to determine direction information of a first direction according to a target feature point;
[0139] The correction unit 404 is used to correct the three-dimensional scene model according to the direction information of the first direction and the direction information of the second direction in the target scene, and the first direction and the second direction meet the corresponding relationship.
[0140] Optionally, when the segmentation unit 402 determines the target feature points belonging to the target object in the point cloud data, it can be used to:
[0141] Perform semantic segmentation processing on the point cloud data according to the target processing model to obtain the segmentation result;
[0142] The target feature points belonging to the target object are determined according to the segmentation results.
[0143] Optionally, when the segmentation unit 402 determines the target feature points belonging to the target object according to the segmentation result, it can be used to:
[0144] Obtain target text information of the object to which the target object belongs, the target text information is used to characterize the composition structure of the object, and the target object is a component of the object;
[0145] The target feature points belonging to the target object are determined based on the segmentation results and the target text information.
[0146] Optionally, when the determining unit 403 determines the direction information of the first direction according to the target feature point, it can be used to:
[0147] Obtaining normal information of a plurality of target feature points belonging to the target object, wherein the normal information of the target feature points is related to the shape of the target object;
[0148] Direction information of the first direction is determined according to normal information of a plurality of target feature points.
[0149] Optionally, the three-dimensional scene model includes a plurality of target objects;
[0150] When the determining unit 403 determines the direction information of the first direction according to the normal information of the plurality of target feature points, it can be used to:
[0151] For each target object, determining normal information corresponding to the target object according to normal information of a plurality of target feature points belonging to the target object;
[0152] Direction information of a first direction is determined according to normal information corresponding to a plurality of target objects.
[0153] Optionally, when the determining unit 403 determines the normal information corresponding to the target object according to the normal information of the plurality of target feature points belonging to the target object, it can be used to:
[0154] In a set of target feature points belonging to the target object, determining a proportion of target feature points corresponding to each normal information;
[0155] The normal information of the corresponding target feature points whose proportion meets the preset conditions is determined as the normal information corresponding to the target object.
[0156] Optionally, when the correction unit 404 corrects the three-dimensional scene model according to the direction information of the first direction and the direction information of the second direction in the target scene, it can be used to:
[0157] Determining a rotation parameter of the three-dimensional scene model according to the direction information of the first direction and the direction information of the second direction in the target scene;
[0158] The 3D scene model is rotated according to the rotation parameters.
[0159] Optionally, when the obtaining unit 401 obtains the point cloud data of the three-dimensional scene model, it can be used to:
[0160] Obtaining initial point cloud data, the initial point cloud data including a plurality of Gaussian feature points;
[0161] The initial point cloud data is processed according to the scene image of the three-dimensional scene model to obtain point cloud data of the three-dimensional scene model.
[0162] Optionally, when the determining unit 403 determines the direction information of the first direction according to the target feature point, it can be used to:
[0163] Perform fitting processing according to target feature points to obtain shape information of the target object;
[0164] Direction information of the first direction is determined according to shape information of the target object.
[0165] The working principle of the data processing device provided in this embodiment can refer to the relevant steps of the data processing method provided in any embodiment of the present application, and will not be repeated here.
[0166] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.
[0167] For the convenience of description, the above system or device is described by dividing it into various modules or units according to its functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.
[0168] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the relevant technology in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.
[0169] Finally, it should be noted that, in this article, relational terms such as first, second, third and fourth are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0170] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
Claims
1. A data processing method, comprising: Obtaining point cloud data of a three-dimensional scene model, the point cloud data comprising a plurality of feature points in the three-dimensional scene model, each of the feature points having corresponding image rendering information; Determining a target feature point belonging to a target object in the point cloud data, wherein a shape of the target object and a first direction of the three-dimensional scene model conform to a target relationship; Determining direction information of the first direction according to the target feature point; The three-dimensional scene model is corrected according to the direction information of the first direction and the direction information of the second direction in the target scene, and the first direction and the second direction satisfy a corresponding relationship.
2. According to the method of claim 1, determining the target feature points belonging to the target object in the point cloud data comprises: Performing semantic segmentation processing on the point cloud data according to the target processing model to obtain a segmentation result; Target feature points belonging to the target object are determined according to the segmentation result.
3. The method according to claim 2, wherein determining the target feature points belonging to the target object according to the segmentation result comprises: Obtaining target text information of the object to which the target object belongs, wherein the target text information is used to characterize the composition structure of the object, and the target object is a component of the object; The target feature points belonging to the target object are determined according to the segmentation result and the target text information.
4. The method according to claim 1, wherein determining the direction information of the first direction according to the target feature point comprises: Obtaining normal information of a plurality of target feature points belonging to the target object, wherein the normal information of the target feature points is related to the shape of the target object; The direction information of the first direction is determined according to the normal information of the plurality of target feature points.
5. The method according to claim 4, wherein the three-dimensional scene model includes a plurality of target objects; The determining the direction information of the first direction according to the normal information of the plurality of target feature points comprises: For each of the target objects, determining normal information corresponding to the target object according to normal information of a plurality of target feature points belonging to the target object; The direction information of the first direction is determined according to the normal information corresponding to the plurality of target objects.
6. The method according to claim 5, wherein determining the normal information corresponding to the target object according to the normal information of the plurality of target feature points belonging to the target object comprises: In a set of target feature points belonging to the target object, determining a proportion of target feature points corresponding to each normal line information; The normal information corresponding to the target feature points whose proportion meets the preset conditions is determined as the normal information corresponding to the target object.
7. The method according to claim 1, wherein the correction processing of the three-dimensional scene model according to the direction information of the first direction and the direction information of the second direction in the target scene comprises: Determining a rotation parameter of the three-dimensional scene model according to the direction information of the first direction and the direction information of the second direction in the target scene; The three-dimensional scene model is rotated according to the rotation parameters.
8. The method according to claim 1, wherein obtaining point cloud data of a three-dimensional scene model comprises: Obtaining initial point cloud data, wherein the initial point cloud data includes a plurality of Gaussian feature points; The initial point cloud data is processed according to the scene image of the three-dimensional scene model to obtain point cloud data of the three-dimensional scene model.
9. The method according to claim 1, wherein determining the direction information of the first direction according to the target feature point comprises: Performing fitting processing according to the target feature points to obtain shape information of the target object; Direction information of the first direction is determined according to shape information of the target object.
10. A data processing device, comprising: An obtaining unit, configured to obtain point cloud data of a three-dimensional scene model, wherein the point cloud data includes a plurality of feature points in the three-dimensional scene model, and each of the feature points has corresponding image rendering information; a segmentation unit, configured to determine, in the point cloud data, target feature points belonging to a target object, wherein a shape of the target object and a first direction of the three-dimensional scene model conform to a target relationship; a determining unit, configured to determine direction information of the first direction according to the target feature point; The correction unit is used to correct the three-dimensional scene model according to the direction information of the first direction and the direction information of the second direction in the target scene, and the first direction and the second direction satisfy a corresponding relationship.