Filtering method, projection mapping method, electronic device and storage medium
The resolution of point cloud is adjusted through filtering methods, which solves the problem of different point cloud resolution differences in different devices, and realizes the unity of point cloud resolution and simplification of processing.
Patent Information
- Application Number
- CN202111619623.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-27
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2041-12-27
AI Technical Summary
There are large differences in the resolution of point clouds collected by different acquisition devices, which leads to different parameters that need to be set when operating the same point cloud, which increases the processing difficulty.
Provide a filtering method, by determining the relationship between the resolution of the point cloud to be processed and the target resolution, adjusting the filtered side length, and dividing the point cloud into cubes, using preset points instead of the origin point cloud until the preset resolution is met Required to realize point cloud filtering.
The resolution difference between the acquisition point clouds after filtering is reduced by different acquisition devices, so that their resolution with the target resolution meets the preset requirements, and simplifies the processing process.
Smart Images

Figure CN114463226B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of filtering technology, and in particular to a filtering method, a projection mapping method, an electronic device and a storage medium. Background Art
[0002] The resolutions of point clouds collected by different acquisition devices usually vary greatly, which means that when performing the same operation on point clouds collected by different acquisition devices, different parameters need to be set, which increases the difficulty of processing. Summary of the invention
[0003] The present application provides a filtering method, a projection mapping method, an electronic device and a storage medium, which can reduce the difference in resolution between point clouds collected by different collection devices after filtering.
[0004] A first aspect of an embodiment of the present application provides a filtering method, which includes: determining the resolution of a point cloud to be processed; in response to the relationship between the resolution of the point cloud to be processed and the target resolution not meeting the preset requirements, determining a filtering side length according to the resolution of the point cloud to be processed, wherein the filtering side length is positively correlated with the resolution of the point cloud to be processed; creating a minimum three-dimensional packaging box surrounding the point cloud to be processed; dividing the minimum three-dimensional packaging box into a plurality of cubes, wherein the size of the cubes is determined based on the filtering side length; replacing all points included in each of the cubes with preset points in each of the cubes to obtain a filtered point cloud; using the filtered point cloud as the point cloud to be processed, and returning to execute the step of determining the resolution of the point cloud to be processed until the relationship between the resolution of the point cloud to be processed and the target resolution meets the preset requirements.
[0005] A second aspect of an embodiment of the present application provides a projection mapping method, which includes: obtaining a point cloud to be processed of a target object; filtering the point cloud to be processed using the above-mentioned filtering method to obtain a target point cloud; determining a first pose of the target object in a first coordinate system of a projector based on the target point cloud; adjusting a target model of the target object in a virtual scene based on the first pose; and controlling the projector to project the target model so as to project the target model onto the target object.
[0006] A third aspect of an embodiment of the present application provides an electronic device, comprising a processor, a memory, and a communication circuit, wherein the processor is coupled to the memory and the communication circuit, respectively, and program data is stored in the memory. The processor implements the steps in the above method by executing the program data in the memory.
[0007] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program can be executed by a processor to implement the steps in the above method.
[0008] The beneficial effect is: when the relationship between the resolution of the point cloud and the target resolution does not meet the preset requirements, the filtering method of the present application filters the point cloud until the relationship between the resolution of the filtered point cloud and the target resolution meets the preset requirements, so that the point clouds collected by different acquisition devices after filtering, the relationship between the resolution and the target resolution all meet the preset requirements, and the difference in resolution will be reduced. BRIEF DESCRIPTION OF THE DRAWINGS
[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:
[0010] Figure 1 It is a flowchart of an implementation method of the projection mapping method of the present application;
[0011] Figure 2 yes Figure 1 Schematic diagram of the process of step S130;
[0012] Figure 3 It is a schematic diagram of the relative positions of the projector, camera and target object of the present application;
[0013] Figure 4 yes Figure 1 Schematic diagram of the process of step S120;
[0014] Figure 5 It is a structural schematic diagram of an embodiment of the electronic device of the present application;
[0015] Figure 6 It is a structural schematic diagram of another embodiment of the electronic device of the present application;
[0016] Figure 7 It is a structural schematic diagram of another embodiment of the electronic device of the present application;
[0017] Figure 8 It is a structural schematic diagram of an implementation method of a computer-readable storage medium of the present application. DETAILED DESCRIPTION
[0018] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0019] See also Figure 1 , Figure 1 : is a flowchart of an implementation method of the projection mapping method of the present application, the method comprising:
[0020] S110: Obtain the point cloud to be processed of the target object.
[0021] In this embodiment, the cultural relic is mounted on a scanning turntable and a scanner, such as a laser scanner, is used to perform a 360-degree scan of the cultural relic to obtain a three-dimensional point cloud of the cultural relic, and then the three-dimensional point cloud is connected to form a complete surface to obtain a three-dimensional model of the cultural relic.
[0022] Usually, in order to protect cultural relics from being damaged, museums store them in display cabinets, so that visitors cannot see the details of the cultural relics up close or touch them. Therefore, after obtaining the three-dimensional model of the cultural relic, the cultural relic model is printed out using 3D printing technology (in order to distinguish it from the virtual cultural relic model below, the cultural relic model is called a real cultural relic model), and a movable and rotatable exhibition stand is also printed, and the cultural relic model is placed on the exhibition stand. The target object in step S110 includes the real cultural relic model and the exhibition stand.
[0023] In other embodiments, the target object may be a cultural relic instead of a model of the cultural relic. Or in other embodiments, the target object may be any physical object, such as a table, a refrigerator, a vehicle, etc.
[0024] After the target object is determined, the point cloud of the target object, i.e. the point cloud to be processed, is collected.
[0025] S120: Filter the point cloud to be processed to obtain a target point cloud.
[0026] The point cloud to be processed is filtered to reduce its resolution, so as to reduce the amount of subsequent calculations and the complexity of the processing process. The filtering process can be found below.
[0027] S130: Determine the first pose of the target object in the first coordinate system of the projector according to the target point cloud.
[0028] Among them, considering that the target point cloud may contain background point clouds that do not belong to the target object, for example, the background point cloud is the point cloud of visitors, so in order to reduce the subsequent calculation amount and improve the accuracy, refer to Figure 2 , step S130 includes:
[0029] S131: performing clustering processing on the target point cloud to obtain a plurality of sub-point clouds of different categories.
[0030] S132: Searching for a sub-point cloud whose size matches the target object among a plurality of sub-point clouds of different categories.
[0031] S133: Determine the first pose of the target object in the first coordinate system according to the matched sub-point cloud.
[0032] Among them, any clustering method (using the clustering method based on Euclidean distance) can be used to cluster the target point cloud to obtain sub-point clouds of multiple categories, and sub-point clouds of different categories belong to different objects.
[0033] A sub-point cloud whose size is closest to the target object is searched among multiple sub-point clouds to obtain a sub-point cloud matching the target object, and the matching sub-point cloud is determined as the point cloud of the target object.
[0034] Finally, based on the matched sub-point cloud, the first pose of the target object is determined.
[0035] In one application scenario, the ratios of the length of each sub-point cloud to the length of the target object, the width of each sub-point cloud to the width of the target object, the height of each sub-point cloud to the height of the target object, and the volume of each sub-point cloud to the volume of the target object are calculated respectively, and then the corresponding ratios of each sub-point cloud are multiplied, and finally the sub-point cloud with the largest product is determined to match the target object.
[0036] In other application scenarios, other methods may be used to determine the sub-point cloud that matches the target object, such as finding the sub-point cloud whose volume is closest to the volume of the target object.
[0037] It should be noted that, in other implementations, clustering processing may not be performed on the target point cloud, but the first pose of the target object may be determined directly based on the target point cloud.
[0038] Meanwhile, in this embodiment, step S130 determines the first pose of the target object in the first coordinate system of the projector, and may also include:
[0039] (a1) determining a posture transformation relationship between a first coordinate system of the projector and a second coordinate system of the depth camera, wherein the depth camera is controlled to shoot a target object to obtain a point cloud to be processed.
[0040] (b1) Determine a second pose of the target object in a second coordinate system of the depth camera according to the target point cloud.
[0041] (c1) According to the posture conversion relationship and the second posture, the first posture is obtained.
[0042] Since the final effect of the projection mapping method of this embodiment is to project the target model onto the target object, in order not to affect the viewing effect, the target object cannot be installed with a tracker or other equipment, nor can additional markers be attached. Therefore, this embodiment uses a camera, specifically a depth camera, to determine the first pose of the target object in the first coordinate system of the projector.
[0043] First construct Figure 3 The system structure shown in Figure 3 101, 102, and 103 represent a projector, a depth camera, and a target object, respectively.
[0044] Then the projector and the depth camera are calibrated to obtain the pose transformation relationship of the two coordinate systems, which includes the rotation vector R p and the translation vector T p The camera and projector can be calibrated by using a camera calibration method based on structured light. This method belongs to the prior art and will not be introduced in detail here. The internal parameters of the projector can also be solved by structured light calibration, including the focal length f (the distance from the focus of the projector to the mapping plane), the units occupied by a pixel point in the horizontal and vertical directions on the projected image, etc.
[0045] Then, during the projection process, the depth camera shoots the target object in step S110 to obtain a point cloud to be processed. After obtaining the point cloud to be processed, the second pose of the target object in the coordinate system of the depth camera is determined, and then the first pose of the target object in the coordinate system of the projector is determined according to the pose conversion relationship between the projector coordinate system and the camera coordinate system.
[0046] In the process of step (c1), on the one hand, the coordinates of the target object in the first coordinate system are determined, and on the other hand, the rotation axis and rotation angle of rotating the target object from its own coordinate system to the second coordinate system, and then the rotation axis and rotation angle of rotating the target object from the second coordinate system to the first coordinate system are determined, wherein the process can be determined using the Rodrigues rotation formula. Finally, the first position of the target object in the first coordinate system is determined by combining the various results.
[0047] Among them, the process of obtaining the first posture according to the posture conversion relationship and the second posture belongs to the existing technology and will not be introduced in detail here.
[0048] In order to improve the accuracy in step (b1), the target point cloud is first coarsely registered using a three-dimensional normal distribution transformation point cloud registration algorithm to obtain an initial position and posture of the target object in the second coordinate system; then, based on the initial position and posture, the target point cloud is finely registered using an iterative point nearest registration algorithm to obtain a second position and posture of the target object in the second coordinate system.
[0049] Specifically, the three-dimensional normal distribution transform (3D NDT) point cloud registration algorithm is first used to obtain the pose of the target object in the second coordinate system. This pose is a rough pose with low accuracy. Then the rough pose is used as the initial matrix of the iterative closest point (ICP) registration algorithm. The ICP registration algorithm is used to perform fine registration on the target point cloud to obtain the pose of the target object in the second coordinate system. This pose is an accurate pose, which is the final second pose.
[0050] Of course, in other implementations, only one of the 3D NDT registration algorithm and the ICP registration algorithm may be used to obtain the second posture of the target object in the second coordinate system.
[0051] Or in other implementations, when factors such as appearance do not need to be considered, the first pose of the target object in the first coordinate system can be directly determined by installing a tracker on the target object, affixing a marker, etc., without using a depth camera.
[0052] S140: Adjusting a target model of the target object in the virtual scene according to the first pose.
[0053] Specifically, the model of the target model is adjusted according to the first posture so that the target model in the virtual scene keeps synchronous movement with the target object. Specifically, when the target object is translated or rotated under the operation of the user, the target model in the virtual scene also moves or rotates synchronously.
[0054] A virtual model of the target object, namely the target model, is pre-constructed in the virtual scene. In this embodiment, the target model includes a virtual cultural relic model and a virtual exhibition stand model.
[0055] In this embodiment, the process of constructing the target model includes:
[0056] (a2) Construct a three-dimensional virtual model of the target object.
[0057] Among them, building a virtual scene in Unity 3D includes building a virtual camera that is consistent with the internal parameters of the projector to render the virtual scene, a virtual cultural relic model that is exactly the same size and shape as the real cultural relic model, and a virtual exhibition stand model that is exactly the same size and shape as the exhibition stand.
[0058] (b2) Create an animation of the target object’s evolution.
[0059] The texture image of the cultural relic is collected and preprocessed, the preprocessing includes at least one of smoothing and dirt removal, and then according to different damage conditions, a layered or edge detection method is used to obtain a repair object from the preprocessed texture image.
[0060] Then, for the restoration objects, according to the dynasty, theme, content, style and painting techniques of the cultural relics, cultural relic restoration research experts will judge their discoloration, fading, shedding and damage, etc., and according to the corresponding situations, the experts will perform color restoration reasoning to obtain the original color, and then simulate the original color on the computer, and use the computer color mixing model to obtain the intermediate color. Based on the original color and the intermediate color, the whole process of the cultural relic color evolution is realistically simulated.
[0061] Finally, an animation of the color evolution of the cultural relics was created in 3DSMax, showing the entire process of the cultural relics gradually changing to their current state under the influence of external environmental factors.
[0062] It is understandable that the animation introduced above shows the evolution process of the color of cultural relics. When the above method is applied to other application scenarios, the animation can also show the evolution process of other aspects of a certain target, for example, showing the movement of the earth's crust.
[0063] (c2) Fusion of 3D virtual model and animation to obtain the target model.
[0064] The animation is pasted onto the three-dimensional virtual model as a texture. When the three-dimensional virtual model moves or rotates, the texture will also move or rotate accordingly.
[0065] In this embodiment, the animation is specifically pasted onto the virtual cultural relic model, and text including the age background of the cultural relic, the place of discovery, etc. are drawn on the virtual exhibition stand model to introduce the cultural relic.
[0066] It should be noted that in other implementations, animation may not be produced, and the three-dimensional virtual model may be directly used as the target model.
[0067] S150: Controlling the projector to project the virtual model, so as to project the target model onto the target object.
[0068] After the target model is constructed, the target model is accurately projected onto the target object through a projector, so that all the contents displayed by the target model can be displayed on the target object, including: after projection, the real cultural relic model can show a gradual process of the cultural relic from the past to the current state, and the exhibition stand can also display text introducing the cultural relic.
[0069] From the above content, it can be seen that the projection mapping method of this embodiment can display the process of color evolution of the real cultural relic model regardless of whether it is still or moving, so as to innovate the way of displaying museum collections, without interactive load, and the scene is realistic and vivid, without relying on display media such as screens or glasses, and provide a reference for manual copying, color restoration and cultural relic protection workers.
[0070] It should be noted that the above method is introduced in the field of cultural relics, but the present application is not limited to this, and can also be applied in other fields that are beneficial to teaching. For example, the target object is an earth model, and the target model in the virtual scene is a virtual earth model. The texture on it shows the changes in the earth's crust. Then, according to the posture of the target object, the posture of the virtual earth model is adjusted, and finally the virtual earth model is projected onto the target object.
[0071] The filtering process in the above step S120 is described below:
[0072] S121: Determine the resolution of the point cloud to be processed.
[0073] Determine the resolution of the point cloud to be processed according to the following formula :
[0074] , where n is the number of points in the point cloud to be processed, is the i-th point in the point cloud to be processed, is the nearest point in the neighborhood of the i-th point.
[0075] S122: Determine whether the relationship between the resolution of the point cloud to be processed and the target resolution meets the preset requirements.
[0076] If the conditions are met, the process ends; if not, step S123 is executed.
[0077] If the relationship between the resolution of the point cloud to be processed and the target resolution meets the preset requirements, it means that the resolution of the point cloud to be processed has reached the expected level. Otherwise, it means that the point cloud to be processed needs to be filtered.
[0078] In this embodiment, the preset requirement includes: the ratio of the target resolution to the resolution of the point cloud to be processed is less than a first preset value, that is, the product of the resolution of the point cloud to be processed and the first preset value is greater than the target resolution. For example, the preset requirement includes: ,in, is the target resolution, 1.03 is the first preset value, and of course the first preset value may also be other values such as 1, 1.5 or 2, which are not limited here.
[0079] It can be understood that the ratio of the target resolution to the resolution of the point cloud to be processed is less than the first preset value, indicating that the resolution of the point cloud to be processed is close to the target resolution.
[0080] In other implementations, the preset requirement may also be that the difference between the target resolution and the resolution of the point cloud to be processed is within a preset range.
[0081] S123: Determine the filter side length according to the resolution of the point cloud to be processed, wherein the filter side length is positively correlated with the resolution.
[0082] Among them, the greater the resolution of the point cloud to be processed, the greater the filter side length, ensuring that after multiple filtering, the number of points in the point cloud becomes smaller and smaller.
[0083] In this embodiment, step S123 specifically includes: determining the filter side length according to the target resolution and the resolution of the point cloud to be processed, for example, determining the filter side length according to the following formula: :
[0084] That is to say, first determine the sum of the target resolution and the resolution of the point cloud to be processed, then determine the ratio of the sum to the target resolution, and finally determine the sum of the target resolution and the ratio to obtain the filter edge length.
[0085] In other implementations, other methods may be used to determine the filter side length according to other formulas. ,For example .
[0086] Alternatively, in other implementations, the filter side length may be determined only based on the resolution of the point cloud to be processed. For example, the filter side length may be obtained by summing the resolution of the point cloud to be processed and a second preset value, where the second preset value is greater than zero.
[0087] S124: Create a minimum three-dimensional packaging box that encloses the point cloud to be processed.
[0088] S125: Divide the minimum three-dimensional packaging box into a plurality of cubes, wherein the size of the cubes is determined based on the filtered side length.
[0089] In this embodiment, the cube is a cube, and the side length of the cube is the filtering side length determined above.
[0090] In other implementations, the cube may also be a cuboid, the ratio of the length, width, and height of the cuboid is predetermined, and the above-determined filtering side length is the length, width, or height of the cuboid.
[0091] S126: Replacing all points included in each cube with preset points in each cube to obtain a filtered point cloud.
[0092] It can be understood that the points included in the cube are the points in the point cloud to be processed that are located within the cube.
[0093] In an application scenario, before step S126, cubes that do not contain data points are removed, and then step S126 is performed on the remaining cubes. Of course, this step is not a necessary step.
[0094] In step S126, all points in the cube are replaced by the centroid of the cube or the data point closest to the centroid.
[0095] After this step, the data points in the point cloud to be processed can be greatly reduced, and the density of points in the point cloud to be processed can be reduced.
[0096] S127: Using the filtered point cloud as the point cloud to be processed.
[0097] Then return to execute step S121.
[0098] The filtered point cloud is used as the point cloud to be processed, and the above steps are repeated until the relationship between the resolution of the point cloud to be processed and the target resolution meets the preset requirements.
[0099] In the prior art, there are large differences in the resolution of point clouds collected by different acquisition devices. When the same operation is performed on point clouds collected by different acquisition devices, different parameters need to be set, and the process is cumbersome. However, through the filtering method in this embodiment, the resolution of the filtered point cloud and the target resolution can meet the preset requirements, so that the difference in resolution of point clouds collected by different acquisition devices will be reduced after filtering.
[0100] In this embodiment, before step S110 , the process further includes: removing noise points in the point cloud to be processed, wherein at least one component of the noise points is not in the corresponding value range.
[0101] Each point in the point cloud to be processed includes three components, and the directions corresponding to the three components are perpendicular to each other. For example, each point includes components in the three directions of X, Y, and Z, and each component corresponds to a value range. If at least one of the three components of a point is not in the corresponding value range, the point is determined to be a noise point and is deleted.
[0102] Through the above settings, invalid background data in the point cloud to be processed can be eliminated, and the accuracy of subsequent processing can be improved.
[0103] See also Figure 5 , Figure 51 is a schematic diagram of the structure of an embodiment of an electronic device of the present application. The electronic device 200 includes a processor 210, a memory 220 and a communication circuit 230. The processor 210 is coupled to the memory 220 and the communication circuit 230 respectively. The memory 220 stores program data. The processor 210 implements the steps in any of the above-mentioned implementation methods by executing the program data in the memory 220. The detailed steps can be found in the above-mentioned implementations and will not be repeated here.
[0104] The electronic device 200 may be any device with algorithm processing capability, such as a computer or a mobile phone, and is not limited here.
[0105] See also Figure 6 , Figure 6 The electronic device 300 includes a first determination module 310 , a second determination module 320 , a creation module 330 , a division module 340 and a replacement module 350 .
[0106] The first determination module 310 is used to determine the resolution of the point cloud to be processed.
[0107] The second determination module 320 is connected to the first determination module 310, and is used to determine the filter side length according to the resolution of the point cloud to be processed when the relationship between the resolution of the point cloud to be processed and the target resolution does not meet the preset requirements, wherein the filter side length is positively correlated with the resolution of the point cloud to be processed.
[0108] The creation module 330 is connected to the second determination module 320 and is used to create a minimum three-dimensional packaging box surrounding the point cloud to be processed.
[0109] The dividing module 340 is connected to the creating module 330 and is used to divide the minimum three-dimensional packaging box into a plurality of cubes, wherein the size of the cubes is determined based on the filtered side length.
[0110] The replacement module 350 is connected to the division module 340 and is used to replace all points included in each cube with preset points in each cube to obtain a filtered point cloud.
[0111] The first determination module 310 uses the filtered point cloud as the point cloud to be processed, and continues to determine the resolution of the point cloud to be processed until the relationship between the resolution of the point cloud to be processed and the target resolution meets the preset requirement.
[0112] In one implementation, the second determination module 320 is specifically configured to determine the filter side length according to the target resolution and the resolution of the point cloud to be processed.
[0113] In one embodiment, the second determination module 320 is specifically used to: determine the sum of the target resolution and the resolution of the point cloud to be processed; determine the ratio of the sum to the target resolution; determine the sum of the target resolution and the ratio to obtain the filtering edge length.
[0114] In one embodiment, the preset requirement includes: a ratio of the target resolution to the resolution of the point cloud to be processed is less than a first preset value.
[0115] In one embodiment, the cube is a rectangular solid, and the side length of the rectangular solid is the filtering side length.
[0116] In one embodiment, each point in the point cloud to be processed is included in three components, and the directions corresponding to the three components are perpendicular to each other. Before determining the resolution of the point cloud to be processed, the first determination module 310 is also used to: remove noise points in the point cloud to be processed, wherein at least one component of the noise point is not in the corresponding value range.
[0117] The electronic device 300 may be any device with algorithm processing capability such as a computer or a mobile phone, and is not limited here. The electronic device 300 performs the steps of the clustering method in any of the above embodiments when working. The detailed steps can be found in the above embodiments and will not be repeated here.
[0118] See also Figure 7 , Figure 7 The electronic device 400 includes an acquisition module 410 , a filtering module 420 , a determination module 430 , an adjustment module 440 and a projection module 450 .
[0119] The acquisition module 410 is used to acquire the to-be-processed point cloud of the target object.
[0120] The filtering module 420 is connected to the acquisition module 410 and is used to perform filtering processing on the point cloud to be processed by using the above filtering method to obtain the target point cloud.
[0121] The determination module 430 is connected to the filtering module 420 and is used to determine the first pose of the target object in the first coordinate system of the projector according to the target point cloud.
[0122] The adjustment module 440 is connected to the determination module 430 and is used to adjust the target model of the target object in the virtual scene according to the first pose.
[0123] The projection module 450 is connected to the adjustment module 440 and is used to control the projector to project the target model so as to project the target model onto the target object.
[0124] In one embodiment, the determination module 430 is specifically used to: determine the posture transformation relationship between the first coordinate system of the projector and the second coordinate system of the depth camera, wherein the depth camera is controlled to shoot the target object to obtain a point cloud to be processed; the target point cloud is coarsely aligned with the three-dimensional normal distribution transformation point cloud alignment algorithm to obtain the initial posture of the target object in the second coordinate system; based on the initial posture, the target point cloud is finely aligned with the iterative point nearest alignment algorithm to obtain the second posture of the target object in the second coordinate system; the first posture is obtained according to the posture transformation relationship and the second posture.
[0125] In one embodiment, the determination module 430 is specifically used to: perform clustering processing on the target point cloud to obtain multiple sub-point clouds of different categories; search for sub-point clouds whose sizes match the target object among the multiple sub-point clouds of different categories; and determine the first pose of the target object in the first coordinate system based on the matching sub-point clouds.
[0126] In one embodiment, the electronic device 400 further includes a creation module for constructing a three-dimensional virtual model of the target object; creating an animation of the evolution of the target object; and fusing the three-dimensional virtual model and the animation to obtain the target model.
[0127] The electronic device 400 may be any device with algorithm processing capability such as a computer or a mobile phone, and is not limited here. The electronic device 400 performs the steps of the projection mapping method in any of the above embodiments when working. The detailed steps can be found in the above embodiments and will not be repeated here.
[0128] See also Figure 8 , Figure 8 A schematic diagram of the structure of an embodiment of a computer-readable storage medium of the present application. The computer-readable storage medium 500 stores a computer program 510, which can be executed by a processor to implement the steps in any of the above methods.
[0129] The computer-readable storage medium 500 may specifically be a device that can store the computer program 510, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or may be a server that stores the computer program 510. The server may send the stored computer program 510 to other devices for execution, or may execute the stored computer program 510 by itself.
[0130] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.
Claims
1. A filtering method, characterized in that: The method comprises: Determine the resolution of the point cloud to be processed; In response to the relationship between the resolution of the point cloud to be processed and the target resolution not satisfying a preset requirement, determining a filter side length according to the resolution of the point cloud to be processed, wherein the filter side length is positively correlated with the resolution of the point cloud to be processed; Creating a minimum three-dimensional packaging box that encloses the point cloud to be processed; Dividing the minimum three-dimensional packaging box into a plurality of cubes, wherein the size of the cubes is determined based on the filtered side length; Replacing all points included in each cube with preset points in each cube to obtain a filtered point cloud; The filtered point cloud is used as the point cloud to be processed, and the step of determining the resolution of the point cloud to be processed is returned to be executed until the relationship between the resolution of the point cloud to be processed and the target resolution meets the preset requirement; Wherein, the step of determining the filter side length according to the resolution of the point cloud to be processed includes: Determining the filtering side length according to the target resolution and the resolution of the point cloud to be processed; Wherein, the step of determining the filter side length according to the target resolution and the resolution of the point cloud to be processed includes: Determine a sum of the target resolution and the resolution of the point cloud to be processed; determining a ratio of the sum value to the target resolution; The sum of the target resolution and the ratio is determined to obtain the filtering side length.
2. The method according to claim 1, characterized in that: The preset requirement includes: a ratio of the target resolution to the resolution of the point cloud to be processed is less than a first preset value.
3. The method according to claim 1, characterized in that The cube is a cube, and the side length of the cube is the filtering side length.
4. The method according to claim 1, characterized in that: Each point in the point cloud to be processed is included in three components, and the directions corresponding to the three components are perpendicular to each other; Before determining the resolution of the point cloud to be processed, the method further includes: Noise points in the point cloud to be processed are removed, wherein at least one component of the noise points is not in a corresponding value range.
5. A projection mapping method, characterized in that: The method comprises: Get the point cloud to be processed of the target object; Using the filtering method described in any one of claims 1 to 4 to filter the point cloud to be processed, to obtain a target point cloud; Determine a first pose of the target object in a first coordinate system of the projector according to the target point cloud; According to the first posture, adjusting the target model of the target object in the virtual scene; The projector is controlled to project the target model, so as to project the target model onto the target object.
6. The method according to claim 5, characterized in that The step of determining the first pose of the target object in the coordinate system of the projector according to the target point cloud comprises: Determine a posture conversion relationship between the first coordinate system of the projector and the second coordinate system of the depth camera, wherein the depth camera is controlled to shoot the target object to obtain the point cloud to be processed; Performing a coarse registration process on the target point cloud using a three-dimensional normal distribution transformation point cloud registration algorithm to obtain an initial position and posture of the target object in the second coordinate system; Based on the initial pose, performing fine registration processing on the target point cloud using an iterative point closest registration algorithm to obtain a second pose of the target object in the second coordinate system; The first posture is obtained according to the posture conversion relationship and the second posture.
7. The method according to claim 5, characterized in that The step of determining the first pose of the target object in the first coordinate system of the projector according to the target point cloud comprises: Clustering the target point cloud to obtain a plurality of sub-point clouds of different categories; Searching, among the plurality of sub-point clouds of different categories, a sub-point cloud whose size matches the target object; The first position and posture of the target object in the first coordinate system is determined according to the matched sub-point cloud.
8. The method according to claim 5, characterized in that The method further comprises: Constructing a three-dimensional virtual model of the target object; Animating the evolution of the target object; The three-dimensional virtual model and the animation are fused to obtain the target model.
9. An electronic device, characterized in that: The electronic device includes a processor, a memory and a communication circuit, the processor is coupled to the memory and the communication circuit respectively, the memory stores program data, and the processor implements the steps in the method as described in any one of claims 1-8 by executing the program data in the memory.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program can be executed by a processor to implement the steps in the method according to any one of claims 1 to 8.
