Customized toy and user interaction method, printing method and 3D printer thereof

Abstract

The invention relates to the technical field of computer software, and provides a customized toy, a user interaction method thereof, a splicing generation method, a printing method and a 3D printer, and the method comprises the following steps: (1) inputting description of an expected customized toy on an interaction interface by a user; (2) the description of the customized toy input by the user is converted into a three-dimensional model; (3) dividing the three-dimensional model through a dividing method; the partitioning mode is decided by a default partitioning strategy or selected by a user on a user interface; and (4) generating sub-model bodies with splicing structures according to a partitioning result, and if the same sub-model bodies exist, classifying the sub-model bodies into a sub-model group. According to the building block toy, the possibility of customizing the toy is provided, the interaction performance in the toy customizing process is improved, implementation modes are configured for various types of customizing toys, and the application range of the building block toy is expanded.

Description

Customized toy, user interaction method, printing method and 3D printer thereof Technical Field

[0001] The present invention relates to the field of computer software technology, and in particular to a customized toy and a user interaction method, a printing method and a 3D printer thereof. Background Art

[0002] Traditional toys are typically designed with fixed shapes and lack interactivity. The advent of modular building blocks has made custom toys possible. By combining basic units such as blocks, custom shapes can be assembled. LEGO toys are one such example. The importance of toy sets lies in their ability to help players unleash their creativity and imagination, cultivate design and construction skills, and encourage self-expression and self-exploration. Through customized toy sets, players can design and build unique objects such as robots, cities, and vehicles, which can stimulate their interest and curiosity while also improving their spatial cognition and hand-eye coordination. Furthermore, customized toy sets can promote interaction and cooperation within families, helping to build closer bonds between family members. In short, customized toy sets are important for players' physical and mental development and learning process.

[0003] However, current customized toys, such as Lego building blocks, typically have a pre-defined assembly sequence, which is then assembled by the user. This results in a lack of customization. Without a pre-defined assembly sequence, the sheer number of block styles makes it difficult to create the desired custom design.

[0004] Summary of the Invention

[0005] In order to solve the above problems, the present invention provides the following technical solutions:

[0006] The present invention provides a user interaction method for a customized toy set, comprising the following steps:

[0007] (1) The user enters a description of the desired customized toy in the interactive interface;

[0008] (2) Convert the description of the customized toy input by the user into a three-dimensional model;

[0009] (3) dividing the three-dimensional model into parts by a parting method; the parting method is determined by a default parting strategy or selected by the user on the user interface,

[0010] (4) Generate a sub-model with a splicing structure based on the segmentation results;

[0011] (5) Group the same sub-model bodies into a sub-model group.

[0012] Furthermore, the user can edit and modify the three-dimensional model in step (2), including adjusting the three-dimensional model as a whole and inputting additional text or sketches; wherein, adjusting the three-dimensional model as a whole includes scaling, rotating, etc. the model.

[0013] Furthermore, the user interaction method further includes:

[0014] Numbering each sub-model body in step (4) and attaching the number to the sub-model body; wherein the attachment method includes adding silk screen printing, engraving the three-dimensional number using Boolean operation, and adding it to the surface of the sub-model body; the number is added at the splicing point;

[0015] Display all sub-models and sub-model groups one by one in the order in which they are assembled; gradually display the first sub-model or sub-model group, the combination of the first sub-model or sub-model group and the second sub-model or sub-model group, the combination of the first sub-model or sub-model group and the second sub-model or sub-model group and the third sub-model or sub-model group, until all sub-models have been traversed;

[0016] Furthermore, the user interaction method also includes: using a pre-trained text or image model to output text guidance for the splicing process; or displaying each building block and the position of the building block in a complete customized toy combination in a three-dimensional interactive manner.

[0017] Furthermore, in step (1), the user's description of the desired customized toy is in at least one form of text information, audio information, video information, picture information, image information, two-dimensional sketch information, three-dimensional sketch information, three-dimensional model information, emotion vector, smell information, EEG information, point cloud information, key pair, code book, and numerical value.

[0018] Furthermore, when there are multiple forms of description of the desired customized toy, the multiple forms of description are converted into each other.

[0019] Furthermore, the specific method of converting into a three-dimensional model in step (2) includes:

[0020] The description of the desired customized toy is mapped to a latent code through a deep learning algorithm. The latent code searches for the corresponding 3D model in the latent space, and the decoder decodes it to obtain the 3D model.

[0021] or,

[0022] The description of the desired customized toy is retrieved from the database through a retrieval algorithm, and the corresponding three-dimensional model is fuzzy or exactly matched;

[0023] or,

[0024] The description of the desired customized toy is optimized into a neural radiation field, and the Marching Cube algorithm is used to convert the neural radiation field into a three-dimensional model represented by triangular faces.

[0025] Furthermore, the segmentation method in step (3) is based on the voxel block segmentation strategy, and the specific segmentation steps are:

[0026] (3.1) Predefine a batch of building block patterns and store the building block patterns as a building block dictionary; for example, N×M Lego-shaped building blocks, where N and M are integers;

[0027] (3.2) Voxelize the 3D model to generate a voxel block matrix; convert it into a voxel representation;

[0028] (3.3) Use nested while loops to traverse the rows and columns of the voxel block matrix to obtain updated combinations of building blocks in the building block dictionary.

[0029] Furthermore, the specific steps for traversing the rows and columns of the three-dimensional model in step (3.3) are:

[0030] (3.3.1) Check whether the current voxel is colored and has not been processed before (traversal processing); if the voxel is colored, increment the block counter; otherwise, start traversing the next voxel in the next row or column; where the 3D model itself has color, check whether the voxel after the 3D model is backed up has color;

[0031] (3.3.2) Sort the blocks in the block dictionary in descending order by volume, traverse the sorted block dictionary, and for each block size, create a block with the color of the current voxel;

[0032] (3.3.3) Extract the submatrix in the horizontal and vertical voxel block directions from the voxel block matrix. If the submatrix matches the block in the block dictionary, install the block at the current position in the horizontal or vertical direction and update the volume dataset; otherwise, continue traversing the block dictionary.

[0033] (3.3.4) When the voxel block matrix is ​​completely updated by the building blocks in the building block dictionary, exit the loop and obtain the updated combination composed of the building blocks in the building block dictionary.

[0034] Alternatively, the segmentation method in step (3) is based on a text prompt input by the user, which is used to guide the segmentation of the 3D model into a set of triangular facets corresponding to the category mark. Based on the segmentation results of the triangular facet set, the surfaces of different segmented areas are formed into individual segmented block entities, and all the segmented block entities can be assembled into a complete model.

[0035] A segmented block entity can be generated by selecting a set of triangular facets from a segmented surface and patching the set of triangular facets to form a closed and / or complete manifold. A Boolean subtraction operation is performed on the complete model and the patched closed and / or complete manifold to obtain two model parts. This operation is then repeated for each remaining unsegmented part until all segments corresponding to the segmented facet surfaces have been segmented.

[0036] Furthermore, the method for generating the category mark is:

[0037] Rendering the three-dimensional model from a random viewpoint by a renderer, wherein the renderer may be a SoftRas renderer;

[0038] The rendered image is processed by a pre-trained image segmentation model or image detection model to detect the part to be segmented or the bounding box of the part to be segmented in the image, each bounding box corresponds to a prompt (segmentation), and the bounding box or detection area of ​​each perspective is fused or reprojected onto the three-dimensional model from multiple perspectives; the fusion process is to use a voting mechanism for each facet, or to optimize the color of each model surface vertex or facet using a reconstruction loss method to make it closer to the result of each rendering perspective, or for each bounding box, use the area-weighted average of all vertices of all faces in the bounding box, project the average point onto the bounding box, and take the face where the projection is located as a main facet, and use the geodesic distance of the path length from one face to another along the surface to calculate the geodesic distance from other faces in the bounding box to the face;

[0039] Next, fit a Gaussian distribution to the geodesic distance and calculate the probability density value of each face as a weight;

[0040] Finally, the weights in multiple views are aggregated into a geodesic score vector, and the maximum value of the geodesic score vector is used as a marker of whether it belongs to the category. Specifically, it is compared with a set threshold to obtain the marker of whether it belongs to the category. If it is greater than the threshold, it belongs to the category.

[0041] Furthermore, for each visible face in the bounding box, its local neighborhood is calculated, where neighbors are determined by mesh connectivity: if two faces f and g share at least one vertex, then face g is a neighbor of face f; to this end, a q-ranked neighborhood is constructed as follows:

[0042] For a face f in the bounding box n , we consider a face g∈bounding box area as a neighbor if there exists a path on the graph with no more than q vertices between them;

[0043] Then, for each face f n∈ bounding box area, calculate the neighborhood visibility score vector by calculating the ratio of the number of visible faces in its neighborhood to the overall neighborhood size;

[0044] Next, the weights of multiple views are aggregated into a visibility score vector; the visibility score vector is multiplied by the above-mentioned geodesic score vector, and the maximum value obtained is used as a mark of whether it belongs to the category.

[0045] The present invention also provides a customized toy, comprising a plurality of building blocks, wherein the building blocks are a combination selected from a building block dictionary based on a user's description of the toy, the building blocks comprising a building block body and building block splicing parts, and the building blocks are connected to each other using the splicing parts using a splicing method output according to any one of the splicing methods described in claims 1-11 to obtain a three-dimensional customized toy.

[0046] Furthermore, the user's description of the toy is in at least one form of text information, audio information, video information, picture information, image information, two-dimensional sketch information, three-dimensional sketch information, three-dimensional model information, emotion vector, smell information, EEG information, point cloud information, key pair, code book, and numerical value.

[0047] Furthermore, the building block is provided with an expandable portion.

[0048] Furthermore, the customized toy also includes a building block base, and the building block base is provided with a second protrusion for the building blocks to be placed and attached.

[0049] Furthermore, the building blocks and the building block base are processed layer by layer by a 3D printer, and the 3D printing layer patterns have regular inter-layer diameter changes, and the vertical roughness Ra is 5-50 μm.

[0050] The present invention also provides a method for printing customized toys, comprising:

[0051] The first step is to use the sub-model body generated by the above-mentioned splicing method according to user needs;

[0052] The second step is to print the sub-models of the same color in a platter.

[0053] Furthermore, when the sub-model body performs platter printing, after completing N layers of platter, it searches for building blocks whose size is less than or equal to that of the N layer of platter and has not yet been arranged for printing. After finding a suitable building block, a detachable support layer is generated on top of the corresponding position of the building block of the N layer. The distance between the support layer and the building block of the N layer is greater than or equal to 0.15 mm for easy disassembly and separation. The thickness of the support layer is at least 0.15 mm. After the support layer is printed, the N+1 layer of building blocks continues to be printed until the K layer. The total height is less than or equal to the total processing height of the processing device, where N is greater than or equal to 1.

[0054] Preferably, the support layer material is different from the building block material, for example, the support layer is made of PETG material and the building block is made of PLA material.

[0055] The present invention also provides a 3D printer, comprising a motion drive assembly, a print head, a heatable print base plate, and a camera for monitoring the printing process;

[0056] The motion drive assembly drives the print head to print building blocks according to a preset trajectory and the above-mentioned printing method, and the camera monitors the building block printing process.

[0057] The present invention has the following beneficial effects:

[0058] The present invention provides the possibility of customizing toys and improves the interactive performance during the process of customizing toys. In addition, the present invention configures implementation methods for various types of custom toys, thereby expanding the application scope of building block toys. DETAILED DESCRIPTION

[0059] The specific implementation methods of the present invention are described in detail below in conjunction with specific examples. It should be pointed out that the examples are only specific explanations of the invention and should not be regarded as limitations of the invention. The purpose of the examples is to enable those skilled in the art to better understand and reproduce the technical solutions of the present invention. The scope of protection of the present invention shall still be based on the scope defined by the claims.

[0060] Example 1

[0061] This embodiment provides a user interaction method for a customized toy set, comprising the following steps:

[0062] (1) The user enters a description of the desired customized toy in the interactive interface;

[0063] In step (1), the user's description of the desired customized toy is in at least one of the following forms:

[0064] Text information can come from a structured selection system or from freely input text content. If it comes from freely input text content, a text encoder can be used for encoding, such as the BERT encoder based on the Transformer structure or the CLIP text encoder.

[0065] Audio information, which may be a speech audio segment describing an item. The speech audio segment (audio information) is pre-trained through a speech feature extraction module, and then a multi-layer neural network is used to encode the features.

[0066] Video information is encoded through an image feature extraction network combined with a temporal transformer structure.

[0067] Image information, which may be color or grayscale image information, is encoded using a pre-trained CLIP image encoder or a ResNet network pre-trained on a large dataset such as ImageNet.

[0068] Image information, the image information may be one or more of thermal imaging information, depth map, normal map, and contour projection information, wherein the depth map can be read by LiDAR, ToF sensor, structured light sensor, etc., and the image information is encoded using a convolutional neural network.

[0069] Two-dimensional sketch information, including two-dimensional sketches and hand-drawn sketches with partial incompleteness, is encoded using a convolutional neural network or a graph neural network pre-trained on a sketch dataset.

[0070] 3D sketch information. A 3D sketch can be a sketch drawn in AR or VR. The sketch information is encoded through a graph neural network.

[0071] Three-dimensional model information, where the three-dimensional model can be a sparse version of a three-dimensional model or a three-dimensional model of a primitive. The three-dimensional model information is encoded by a convolutional neural network composed of 3D convolutions.

[0072] The emotion vector is obtained from speech intonation and video, and is encoded by an emotion vector encoder obtained through pre-training.

[0073] Odor information, wherein the odor information encodes data read out by a gas sensor through a graph network or a multi-layer perceptron.

[0074] EEG information, such as a specific EEG sequence, encodes the data read out by the EEG sensor through a graph network or a multi-layer perceptron.

[0075] Point cloud information, such as a full or partial point cloud of a three-dimensional model.

[0076] The key pair can be encoded by a multi-layer perceptron.

[0077] The codebook can be encoded by a multi-layer perceptron.

[0078] The numerical value, which can be a data reading from a physical sensor or an asset quantity, is encoded using a multi-layer perceptron or one or more 1×1 convolutions.

[0079] When there are multiple forms of descriptions of the desired customized toy, the multiple forms are converted to each other. (2) The description of the customized toy input by the user is converted into a three-dimensional model;

[0080] The description of the desired customized toy is mapped to a latent code through a deep learning algorithm. The latent code searches for the corresponding 3D model in the latent space, and the decoder decodes it to obtain the 3D model.

[0081] or,

[0082] The description of the desired customized toy is retrieved from the database through a retrieval algorithm, and the corresponding three-dimensional model is fuzzy or exactly matched;

[0083] or,

[0084] The description of the desired customized toy is optimized into a neural radiation field, and the Marching Cube algorithm is used to convert the neural radiation field into a three-dimensional model represented by triangular faces.

[0085] Users can edit and modify the 3D model, including overall adjustment of the 3D model and input of additional text or sketches; among which, overall adjustment of the 3D model includes scaling, rotating, etc. the model.

[0086] (3) dividing the three-dimensional model into parts by a parting method; the parting method is determined by a default parting strategy or selected by the user on the user interface;

[0087] The segmentation method is based on the voxel block segmentation strategy, and its specific segmentation steps are:

[0088] (3.1) Predefine a batch of building block patterns and store the building block patterns as a building block dictionary; for example, N×M Lego-shaped building blocks, where N and M are integers;

[0089] (3.2) Voxelize the 3D model to generate a voxel block matrix; convert it into a voxel representation;

[0090] (3.3) Use nested while loops to traverse the rows and columns of the volume dataset and obtain updated combinations consisting of building blocks in the building block dictionary.

[0091] Furthermore, the specific steps for traversing the rows and columns of the three-dimensional model in step (3.3) are:

[0092] (3.3.1) Check if the current voxel is colored and has not been processed before (traversal process); if the voxel is colored, increment the block counter, otherwise, start traversing the next voxel in the next row or column;

[0093] (3.3.2) Sort the blocks in the block dictionary in descending order by volume, traverse the sorted block dictionary, and for each block size, create a block with the color of the current voxel;

[0094] (3.3.3) Extract the submatrix in the horizontal and vertical voxel block directions from the voxel block matrix. If the submatrix matches the block in the block dictionary, install the block at the current position in the horizontal or vertical direction and update the volume dataset; otherwise, continue traversing the block dictionary.

[0095] (3.3.4) When the voxel block matrix is ​​completely updated by the building blocks in the building block dictionary, exit the loop and obtain the updated combination composed of the building blocks in the building block dictionary.

[0096] As another method, the segmentation method is based on the text prompt input by the user, and the text prompt is used to guide the segmentation of the triangular facet set corresponding to the category mark of the 3D model; for example, if the user inputs "head of a portrait", the triangular facet set corresponding to the category mark of the portrait head area will be segmented. The method for generating the category mark is as follows:

[0097] Rendering the three-dimensional model from a random viewpoint by a renderer, wherein the renderer may be a SoftRas renderer;

[0098] The rendered image is processed by a pre-trained image segmentation model or image detection model to detect the part to be segmented or the bounding box of the part to be segmented in the image, each bounding box corresponds to a prompt (segmentation), and the bounding box or detection area of ​​each perspective is fused or reprojected onto the three-dimensional model from multiple perspectives; the fusion process is to use a voting mechanism for each facet, or to optimize the color of each model surface vertex or facet using a reconstruction loss method to make it closer to the result of each rendering perspective, or for each bounding box, use the area-weighted average of all vertices of all faces in the bounding box, project the average point onto the bounding box, and take the face where the projection is located as a main facet, and use the geodesic distance of the path length from one face to another along the surface to calculate the geodesic distance from other faces in the bounding box to the face;

[0099] Next, fit a Gaussian distribution to the geodesic distance and calculate the probability density value of each face as a weight;

[0100] Finally, the weights in multiple views are aggregated into a geodesic score vector, and the maximum value of the geodesic score vector is used as a marker of whether it belongs to the category. Specifically, it is compared with a set threshold to obtain the marker of whether it belongs to the category. If it is greater than the threshold, it belongs to the category.

[0101] In some preferred schemes, for each visible face in the bounding box, its local neighborhood is computed, where neighbors are determined by mesh connectivity: if two faces f and g share at least one vertex, then face g is a neighbor of face f; to this end, a q-ranked neighborhood is constructed as follows:

[0102] For a face f in the bounding box n , we consider a face g∈bounding box area as a neighbor if there exists a path on the graph with no more than q vertices between them;

[0103] Then, for each face f n ∈ bounding box area, calculate the neighborhood visibility score vector by calculating the ratio of the number of visible faces in its neighborhood to the overall neighborhood size;

[0104] Next, the weights of multiple views are aggregated into a visibility score vector; the visibility score vector is multiplied by the above-mentioned geodesic score vector, and the maximum value obtained is used as a mark of whether it belongs to the category.

[0105] (4) Generate sub-model bodies with splicing structures based on the segmentation results. If there are identical sub-model bodies, they are grouped into sub-model groups;

[0106] (5) Use pre-trained text or image models to output text guidance for the splicing process, or provide a three-dimensional interactive display of each building block and its position in the complete customized toy set.

[0107] Example 2

[0108] This embodiment provides a user interaction method for a customized toy set, comprising the following steps:

[0109] (1) The user enters a description of the desired customized toy in the interactive interface;

[0110] In step (1), the user's description of the desired customized toy is in at least one of the following forms:

[0111] Text information can come from a structured selection system or from freely input text content. If it comes from freely input text content, a text encoder can be used for encoding, such as the BERT encoder based on the Transformer structure or the CLIP text encoder.

[0112] Audio information, which may be a speech audio segment describing an item. The speech audio segment (audio information) is pre-trained through a speech feature extraction module, and then a multi-layer neural network is used to encode the features.

[0113] Video information is encoded through an image feature extraction network combined with a temporal transformer structure.

[0114] Image information, which may be color or grayscale image information, is encoded using a pre-trained CLIP image encoder or a ResNet network pre-trained on a large dataset such as ImageNet.

[0115] Image information, the image information may be one or more of thermal imaging information, depth map, normal map, and contour projection information, wherein the depth map can be read by LiDAR, ToF sensor, structured light sensor, etc., and the image information is encoded using a convolutional neural network.

[0116] Two-dimensional sketch information, including two-dimensional sketches and hand-drawn sketches with partial incompleteness, is encoded using a convolutional neural network or a graph neural network pre-trained on a sketch dataset.

[0117] 3D sketch information. A 3D sketch can be a sketch drawn in AR or VR. The sketch information is encoded through a graph neural network.

[0118] Three-dimensional model information, where the three-dimensional model can be a sparse version of a three-dimensional model or a three-dimensional model of a primitive. The three-dimensional model information is encoded by a convolutional neural network composed of 3D convolutions.

[0119] The emotion vector is obtained from speech intonation and video, and is encoded by an emotion vector encoder obtained through pre-training.

[0120] Odor information, wherein the odor information encodes data read out by a gas sensor through a graph network or a multi-layer perceptron.

[0121] EEG information, such as a specific EEG sequence, encodes the data read out by the EEG sensor through a graph network or a multi-layer perceptron.

[0122] Point cloud information, such as a full or partial point cloud of a three-dimensional model.

[0123] The key pair can be encoded by a multi-layer perceptron.

[0124] The codebook can be encoded by a multi-layer perceptron.

[0125] The numerical value, which can be a data reading from a physical sensor or an asset quantity, is encoded using a multi-layer perceptron or one or more 1×1 convolutions.

[0126] When there are multiple forms of description of the desired customized toy, the multiple forms are converted into each other.

[0127] (2) Convert the description of the customized toy input by the user into a three-dimensional model;

[0128] The description of the desired customized toy is mapped to a latent code through a deep learning algorithm. The latent code searches for the corresponding 3D model in the latent space, and the decoder decodes it to obtain the 3D model.

[0129] or,

[0130] The description of the desired customized toy is retrieved from the database through a retrieval algorithm, and the corresponding three-dimensional model is fuzzy or exactly matched;

[0131] or,

[0132] The description of the desired customized toy is optimized into a neural radiation field, and the Marching Cube algorithm is used to convert the neural radiation field into a three-dimensional model represented by triangular faces.

[0133] Users can edit and modify the 3D model, including overall adjustment of the 3D model and input of additional text or sketches; among which, overall adjustment of the 3D model includes scaling, rotating, etc. the model.

[0134] (3) dividing the three-dimensional model into parts by a parting method; the parting method is determined by a default parting strategy or selected by the user on the user interface;

[0135] The segmentation method is based on the voxel block segmentation strategy, and its specific segmentation steps are:

[0136] (3.1) Predefine a batch of building block patterns and store the building block patterns as a building block dictionary; for example, N×M Lego-shaped building blocks, where N and M are integers;

[0137] (3.2) Voxelize the 3D model to generate a voxel block matrix; convert it into a voxel representation;

[0138] (3.3) Use nested while loops to traverse the rows and columns of the volume dataset and obtain updated combinations consisting of building blocks in the building block dictionary.

[0139] Furthermore, the specific steps for traversing the rows and columns of the three-dimensional model in step (3.3) are:

[0140] (3.3.1) Check whether the current voxel is colored and has not been processed before (traversal processing); if the voxel is colored, increment the block counter; otherwise, start traversing the next voxel in the next row or column; where the 3D model itself has color, check whether the voxel after the 3D model is backed up has color;

[0141] (3.3.2) Sort the blocks in the block dictionary in descending order by volume, traverse the sorted block dictionary, and for each block size, create a block with the color of the current voxel;

[0142] (3.3.3) Extract the submatrix in the horizontal and vertical voxel block directions from the voxel block matrix. If the submatrix matches the block in the block dictionary, install the block at the current position in the horizontal or vertical direction and update the volume dataset; otherwise, continue traversing the block dictionary.

[0143] (3.3.4) When the voxel block matrix is ​​completely updated by the building blocks in the building block dictionary, exit the loop and obtain the updated combination composed of the building blocks in the building block dictionary.

[0144] As another method, the segmentation method is based on the text prompt input by the user. The text prompt guides the segmentation of the triangular facet set corresponding to the category mark of the 3D model. For example, if the user inputs "head of a portrait", the triangular facet set corresponding to the category mark of the portrait head area will be segmented. The method for generating the category mark is as follows:

[0145] Rendering the three-dimensional model from a random viewpoint by a renderer, wherein the renderer may be a SoftRas renderer;

[0146] The rendered image is processed by a pre-trained image segmentation model or image detection model to detect the part to be segmented or the bounding box of the part to be segmented in the image. Each bounding box corresponds to a prompt (segmentation), and the bounding box or detection area of ​​each view is fused or reprojected onto the three-dimensional model from multiple perspectives. The fusion process adopts a voting mechanism for each facet, or optimizes the color of each model surface vertex or facet using a reconstruction loss method to make it closer to the result of each rendering perspective, or for each bounding box, uses the area-weighted average of all vertices of all faces in the bounding box, projects the average point onto the bounding box, and takes the face where the projection is located as a main facet. The geodesic distance from other faces in the bounding box to the face is calculated using the geodesic distance of the path length from one face to another along the surface.

[0147] Next, fit a Gaussian distribution to the geodesic distance and calculate the probability density value of each face as a weight;

[0148] Finally, the weights in multiple views are aggregated into a geodesic score vector, and the maximum value of the geodesic score vector is used as a marker of whether it belongs to the category. Specifically, it is compared with a set threshold to obtain the marker of whether it belongs to the category. If it is greater than the threshold, it belongs to the category.

[0149] In some preferred schemes, for each visible face in the bounding box, its local neighborhood is computed, where neighbors are determined by mesh connectivity: if two faces f and g share at least one vertex, then face g is a neighbor of face f; to this end, a q-ranked neighborhood is constructed as follows:

[0150] For a face f in the bounding box n , we consider a face g∈bounding box area as a neighbor if there exists a path on the graph with no more than q vertices between them;

[0151] Then, for each face f n ∈ bounding box area, calculate the neighborhood visibility score vector by calculating the ratio of the number of visible faces in its neighborhood to the overall neighborhood size;

[0152] Next, the weights of multiple views are aggregated into a visibility score vector; the visibility score vector is multiplied by the above-mentioned geodesic score vector, and the maximum value obtained is used as a mark of whether it belongs to the category.

[0153] (4) Generate sub-model bodies with splicing structures based on the segmentation results. If there are identical sub-model bodies, they are grouped into one sub-model group;

[0154] (5) Each sub-model body is numbered and the number is attached to the sub-model body; wherein the attachment method includes adding silk screen printing, using Boolean operation to carve the three-dimensional number, adding it to the surface of the sub-model body; the number is added at the splicing point;

[0155] (6) Display all sub-models and sub-model groups one by one in the order of splicing; gradually display the first sub-model or sub-model group, the combination of the first sub-model or sub-model group and the second sub-model or sub-model group, the combination of the first sub-model or sub-model group and the second sub-model or sub-model group and the third sub-model or sub-model group, until all sub-models are traversed;

[0156] (7) Use pre-trained text or image models to output text guidance for the splicing process, or provide a three-dimensional interactive display of each building block and its position in the complete customized toy set.

[0157] Example 3

[0158] This embodiment provides a customized toy.

[0159] The customized toy includes a plurality of building blocks, each of which is a combination selected from a building block dictionary based on a user's description of the toy. The building blocks include a building block body and building block splicing parts. The building blocks are connected to each other using the splicing parts according to the splicing method output in Example 1 or Example 2 to obtain a three-dimensional customized toy.

[0160] The user's description of the toy is in at least one form of text information, audio information, video information, picture information, image information, two-dimensional sketch information, three-dimensional sketch information, three-dimensional model information, emotion vector, smell information, EEG information, point cloud information, key pair, code book, and numerical value.

[0161] Furthermore, the building block is provided with an expandable portion. Preferably, the expandable portion can be an expandable square unit, each square unit including a first protrusion. During the 3D printing process, the layer height of the first protrusion is configured to be 0.2mm-0.6mm; each square unit of the building block includes a groove on the other side of the first protrusion, the groove is matched with the circular first protrusion, and the groove can just accommodate the circular first protrusion. The side length of the groove to accommodate the first protrusion is ±0.04-0.2mm of the side length of the first protrusion. In order to provide a good damping feeling when the user slides the first protrusion of the building block into the groove during the assembly process, the regular interlayer diameter variation of the building block in the 3D printing layer pattern (vertical roughness Ra is controlled at 5-50um) also increases the friction between the first protrusion and the recessed space, so that the building blocks are better interlocked relative to smooth injection molded parts.

[0162] The building block can be expanded and configured to be composed of several identical square units, or it can be configured to be composed of several independent first protrusions and one or several shared interconnected grooves. When the square unit is used as a 1*1 unit, the building block can be an M*N building block combination, where M and N are integers greater than or equal to 1. For example, M and N can be a building block combination of 1*2, 1*3, 1*4, 1*5, 1*6, 1*7, 1*8, 1*9 and 1*10, and can also include a building block combination of 2*2, 2*3, 2*4, 2*5, 2*6, 2*7, 2*8, 2*9 and 2*10. All combinations of M and N constitute a building block library.

[0163] For a customized toy composed of building blocks, it is assembled from K layers of building blocks, where K is greater than or equal to 2. Except for the first layer, the building blocks are arranged in parallel and do not cross. The parallel arrangement direction of all building blocks in the even-numbered layers is orthogonal to the parallel arrangement direction of all building blocks in the odd-numbered layers, that is, the even-numbered layers are arranged horizontally and the odd-numbered layers are arranged vertically, or the odd-numbered layers are arranged vertically and the even-numbered layers are arranged horizontally.

[0164] For a customized toy composed of building blocks, there is at least one connection position between the building blocks on the N+1th layer and the Nth layer.

[0165] For customized toys composed of building blocks, use the largest building blocks as much as possible in each layer. For example, when a row in a layer needs to be filled with a 1*15 building block combination and the maximum P value of 1*P in the building block library is 10, it is decomposed into 1*10 and 1*5 instead of other combinations.

[0166] The customized toy also includes a building block base, which is provided with a second protrusion for attaching building blocks for placement, and the second protrusion is also matched with the groove. The second protrusion and the grooves of all building blocks in the second layer are detachably connected. The building block base can be an irregular configuration and not necessarily an M*N rectangular unit combination. The building block base can also be a K*L rectangular unit combination, where K and L are greater than the number of building blocks in the second layer in the horizontal and vertical directions.

[0167] The building blocks and the building block base are both processed layer by layer by a 3D printer. If the building block base exceeds the processing range of the 3D printer, the building block base is split into several small blocks for processing.

[0168] Example 3

[0169] This embodiment provides a method for printing customized toys, including:

[0170] The first step is to use the sub-model body generated by the above-mentioned splicing method according to user needs;

[0171] The second step is to print the sub-models of the same color in a platter. The platter order can be: in a bottom-up assembly order, or in a random order;

[0172] When the sub-model body performs platter printing, after completing the N-layer platter, it searches for building blocks whose sizes are less than or equal to the N-layer platter and have not yet been arranged for printing. After finding a suitable building block, a detachable support layer is generated on top of the corresponding position of the building block of the N-layer. The distance between the support layer and the building block of the N-layer is greater than or equal to 0.15 mm to facilitate disassembly and separation. The thickness of the support layer is at least 0.15 mm. After the support layer is printed, the N+1-layer building blocks are continued to be printed until the K-th layer, and the total height is less than or equal to the total processing height of the processing device, where N is greater than or equal to 1.

[0173] Preferably, the support layer material is different from the building block material, for example, the support layer is made of PETG material and the building block is made of PLA material.

[0174] Because building block printing requires high precision, R calibration patterns are pre-printed on the printing plate before the printing process. The R calibration patterns are distributed in the area of ​​the printing plate where the building blocks are to be printed. The calibration patterns are rectangular patterns with the same size as the building blocks or circular patterns with the same size as the raised parts. The sizes of the R calibration patterns are checked by a camera. Based on the relationship between the actual size rl' and the expected size rl of each r-th calibration pattern, the building block size ratio in the area where the r-th calibration pattern is located is scaled to rl / rl'*rp, where rp is the original size of the building block.

[0175] Preferably, during the block printing process, the camera compares the printed result with the expected printing result. If there is a large difference between the printing result of a certain area of ​​the block and the expected printing result, the area is skipped in the subsequent printing process, and the block assembly result to be printed is dynamically adjusted during the subsequent assembly.

[0176] Preferably, during the printing process of the building blocks, a peelable raft layer and / or an extended Brim layer is added to the bottom of each building block to increase the adhesion between the building block bottom and the base plate.

[0177] Example 4

[0178] This embodiment provides a 3D printer.

[0179] The 3D printer includes a motion drive assembly, a print head, a heatable print base plate, and a camera for monitoring the printing process;

[0180] The motion drive assembly drives the print head to print building blocks according to a preset trajectory and the above-mentioned printing method, and the camera monitors the building block printing process.

[0181] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0182] It should be noted that the technical features not described in detail in the present invention can be implemented by any existing technology.

Claims

1. A user interaction method for a customized toy combination, characterized in that: The following steps are involved: (1) The user enters a description of the desired customized toy in the interactive interface; (2) converting the description of the customized toy input by the user into a three-dimensional model; (3) Divide the three-dimensional model into parts by using a part-by-part method; (4) Generate a sub-model body with a splicing structure based on the segmentation results; (5) Group the same sub-model bodies into a sub-model group.

2. A user interaction method for a customized toy combination according to claim 1, characterized in that: The user can edit and modify the 3D model in step (2), including overall adjustment of the 3D model and input of additional text or sketches.

3. The user interaction method of a customized toy combination according to claim 1, characterized in that: The user interaction method further comprises: Number each sub-model body in step (4) and attach the number to the sub-model body; display all sub-model bodies and sub-model groups one by one in the splicing order; the combination of the first sub-model or sub-model group and the second sub-model or sub-model group, the combination of the first sub-model or sub-model group and the second sub-model or sub-model group and the third sub-model or sub-model group, until all sub-models are traversed.

4. A user interaction method for a customized toy combination according to claim 1 or 3, characterized in that: The user interaction method also includes: using a pre-trained text or image model to output text guidance for the splicing process; or displaying each building block and the position of the building block in a complete customized toy combination in a three-dimensional interactive manner.

5. The user interaction method of a customized toy combination according to claim 1, characterized in that: In step (1), the user's description of the desired customized toy is in at least one of the following forms: text information, audio information, video information, picture information, image information, two-dimensional sketch information, three-dimensional sketch information, three-dimensional model information, emotion vector, smell information, EEG information, point cloud information, key pair, code book, and numerical value.

6. The user interaction method of a customized toy combination according to claim 5, characterized in that: When there are multiple forms of description of the desired customized toy, the multiple forms of description are converted into each other.

7. The user interaction method of a customized toy combination according to claim 1, characterized in that: The specific method of converting into a three-dimensional model in step (2) includes: The description of the desired customized toy is mapped to a latent code through a deep learning algorithm. The latent code searches for the corresponding three-dimensional model in the latent space, and the three-dimensional model is decoded by a decoder. or, The description of the desired customized toy is searched in the database through a search algorithm, and the corresponding three-dimensional model is fuzzily or exactly matched; or, The description of the desired customized toy is optimized into a neural radiation field, and the Marching Cube algorithm is used to convert the neural radiation field into a three-dimensional model represented by triangular faces.

8. The user interaction method of a customized toy combination according to claim 1, characterized in that: The segmentation method in step (3) is based on the voxel block segmentation strategy, and the specific segmentation steps are: (3.1) Predefine a batch of building block styles and store the building block styles as a building block dictionary; (3.2) voxelizing the three-dimensional model to generate a voxel block matrix; (3.3) Use nested while loops to traverse the rows and columns of the voxel block matrix to obtain updated combinations of building blocks in the building block dictionary.

9. The user interaction method of a customized toy combination according to claim 8, characterized in that: The specific steps of traversing the rows and columns of the three-dimensional model in step (3.3) are: (3.3.1) Check if the current voxel is colored and has not been processed before; if the voxel is colored, increment the block counter; otherwise, start traversing the next voxel in the next row or column; (3.3.2) Sort the blocks in the block dictionary in descending order by volume, traverse the sorted block dictionary, and for each block size, create a block with the color of the current voxel; (3.3.3) Extract the sub-matrices in the horizontal and vertical voxel block directions from the voxel block matrix. If the sub-matrix matches the building block in the building block dictionary, install the building block in the current position in the horizontal or vertical direction and update the voxel block matrix; otherwise, continue to traverse the building block dictionary. (3.3.4) When the voxel block matrix is ​​completely updated by the building blocks in the building block dictionary, exit the loop and obtain the updated combination composed of the building blocks in the building block dictionary.

10. The user interaction method of a customized toy combination according to claim 1, characterized in that: The segmentation method in step (3) is based on the text prompt input by the user, and the three-dimensional model is segmented into a set of triangular facets corresponding to the category mark through the guidance of the text prompt.

11. The user interaction method of a customized toy combination according to claim 10, characterized in that: The method for generating the category mark is: The three-dimensional model is rendered from a random viewpoint by a renderer, and the rendered image is processed by a pre-trained image segmentation model or an image detection model to detect the part to be segmented or the bounding box of the part to be segmented in the image, each bounding box corresponds to a prompt, and the bounding box or detection area of ​​each perspective is fused or reprojected onto the three-dimensional model from multiple perspectives; the fusion process is to use a voting mechanism for each face, or to optimize the color of each model surface vertex or face by using a reconstruction loss method to make it closer to the result of each rendering perspective, or for each bounding box, use the area-weighted average of all vertices of all faces in the bounding box, project the average point onto the bounding box, and take the face where the projection is located as a main face, and use the geodesic distance of the path length from one face to another along the surface to calculate the distance from other faces in the bounding box to the face. The geodesic distance of Next, fit a Gaussian distribution to the geodesic distance and calculate the probability density value of each face as a weight; Finally, the weights in multiple views are aggregated into a geodesic score vector, and the maximum value of the geodesic score vector is used as a marker of whether it belongs to the category.

12. The user interaction method of a customized toy combination according to claim 11, characterized in that: For each face visible in the bounding box, compute its local neighborhood, where neighbors are determined by mesh connectivity: face g is a neighbor of face f if two faces f and g share at least one vertex; to this end, construct a q-ranked neighborhood as follows: For a face f in the bounding box n , we consider a face g∈bounding box region as a neighbor if there exists a path on the graph with no more than q vertices between them; Then, for each face f n ∈ bounding box area, calculate the neighborhood visibility score vector by calculating the ratio of the number of visible faces in its neighborhood to the overall neighborhood size; Next, the weights in multiple views are aggregated into a visibility score vector; the visibility score vector is multiplied by the above-mentioned geodesic score vector, and the maximum value obtained is used as a mark of whether it belongs to the category.

13. A customized toy, characterized in that: The customized toy includes a plurality of building blocks, wherein the building blocks are a combination selected from a building block dictionary according to a user's description of the toy, and the building blocks include a building block body and building block splicing parts. The building blocks are output according to any splicing method described in claims 1-12, and are connected to each other using the splicing parts to obtain a three-dimensional customized toy.

14. A customized toy according to claim 13, characterized in that: The user's description of the toy is in at least one of the following forms: text information, audio information, video information, picture information, image information, two-dimensional sketch information, three-dimensional sketch information, three-dimensional model information, emotion vector, smell information, EEG information, point cloud information, key pair, code book, and numerical value.

15. A customized toy according to claim 13, characterized in that: The building block is provided with an expandable portion.

16. A customized toy according to claim 13, characterized in that: The customized toy further comprises a building block bottom plate, on which a second protrusion for the building blocks to be placed and attached is provided.

17. A customized toy according to claim 16, characterized in that: The building blocks and the building block base are processed layer by layer by a 3D printer. The 3D printing layer pattern has regular interlayer diameter changes, and the vertical roughness Ra is 5-50um.

18. A printing method for customized toys, characterized in that: include: The first step is to use the sub-model body generated by the splicing method described in any one of claims 1 to 12 according to user needs; The second step is to print the sub-models of the same color in a mosaic.

19. A printing method for customized toys according to claim 18, characterized in that: When the sub-model body performs platter printing, after completing N layers of platter, it searches for building blocks whose sizes are less than or equal to the Nth layer of platter and have not yet been arranged for printing. After finding suitable building blocks, a detachable support layer is generated on the top of the corresponding position of the building blocks of the Nth layer. After the support layer is printed, the N+1th layer of building blocks continues to be printed until the Kth layer, and the total height is less than or equal to the total processing height of the processing device, where N is greater than or equal to 1.

20. A 3D printer, characterized in that: It includes motion drive components, print head, heatable printing plate, and camera for monitoring the printing process; The motion drive assembly drives the print head to print building blocks according to a preset trajectory and the printing method described in any one of claims 18-19, and the camera monitors the building block printing process.