Color prediction method and system for full-color 3D printing

Abstract

The present invention discloses a color prediction method and system for full-color 3D printing, the method comprising: obtaining a printed object image and 3D model data based on which printing is performed during the printing process of a 3D printer; identifying multiple color-deficient parts and color-difference parts in the printed object image based on a color-deficient recognition algorithm and a color-difference recognition algorithm; predicting the color of each of the color-deficient parts and the color-difference parts based on the 3D model data and a preset color comparison prediction algorithm to obtain a predicted color result; and controlling the 3D printer to perform color correction spraying on the color-deficient parts and the color-difference parts of the printed object based on the predicted color result. It can be seen that the present invention can effectively improve the color complementation function and color printing effect of 3D printing, and realize smarter and better 3D full-color printing.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a color prediction method and system for full-color 3D printing. Background Art

[0002] The so-called 3D printing technology is the process of using additive manufacturing technology to print three-dimensional objects with any shape and geometric features, and 3D printers, in layman's terms, are a type of industrial robot. 3D printers have a wide range of applications, and can be used in aerospace, construction, automobiles, defense, dentistry, and so on. The target market for more affordable desktop home 3D printers is mainly DIY, 3D printing enthusiasts, lighthouse customers, and academic research and computer fields. When processing full-color 3D printing, existing 3D printers generally use three-dimensional spraying technology to spray three-dimensional colors on the surface of crafts to form full-color prints. However, the models printed and sprayed by 3D printing in the prior art may have different colors from the original models or even lack colors, and the prior art cannot recognize the color difference and lack of the printed objects, so the printing color effect is poor. It can be seen that the prior art has defects that need to be solved urgently. It can be seen that the prior art has defects that need to be solved urgently. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a color prediction method and system for full-color 3D printing, which can effectively improve the color completion function and color printing effect of 3D printing, and realize smarter and better 3D full-color printing.

[0004] In order to solve the above technical problems, the first aspect of the present invention discloses a color prediction method for full-color 3D printing, the method comprising:

[0005] Acquire a printed object image and 3D model data based on the printed object during the printing process of the 3D printer;

[0006] Based on a color deficiency recognition algorithm and a color difference recognition algorithm, a plurality of color deficiency parts and color difference parts in the printed image are recognized;

[0007] According to the 3D model data, based on a preset color comparison prediction algorithm, predict the color of each of the color-deficient part and the color-difference part to obtain a predicted color result;

[0008] According to the predicted color result, the 3D printer is controlled to perform color correction spraying on the color-deficient part and the color-difference part of the printed object.

[0009] As an optional implementation, in the first aspect of the present invention, the printed matter image includes a plurality of printed matter images obtained by photographing the printed matter from a plurality of positions and in a plurality of directions.

[0010] As an optional embodiment, in the first aspect of the present invention, the identifying multiple color-deficient parts and color-difference parts in the printed image based on the color-deficient recognition algorithm and the color-difference recognition algorithm includes:

[0011] Grouping all the printed matter images based on viewpoints and angles to obtain a plurality of image groups;

[0012] Inputting each of the image groups into a preset low-saturation color recognition model to obtain a color-deficient portion corresponding to each of the image groups;

[0013] Based on a two-dimensional and three-dimensional mapping algorithm, the color-deficient portion and the 3D model data are compared in color difference to screen out a plurality of color-difference portions.

[0014] As an optional implementation, in the first aspect of the present invention, all the printed matter images are grouped based on viewpoints and angles to obtain a plurality of image groups, including:

[0015] For any two of the printed matter images, obtaining the device position of the image acquisition device corresponding to the two printed matter images and the lens direction during acquisition;

[0016] Calculating the position distance between the device positions of the two printed matter images;

[0017] Calculating the direction difference between the lens directions when the two printed matter images were acquired;

[0018] Calculating a weighted average of the position distance and the direction difference to obtain an image difference between the two printed matter images;

[0019] According to the image difference, all the printed matter images are grouped based on a clustering algorithm to obtain a plurality of image groups; the image difference between any two printed matter images in each of the image groups is less than a preset difference threshold.

[0020] As an optional embodiment, in the first aspect of the present invention, the low-saturation color recognition model includes an image-corresponding component position recognition model and a color saturation recognition model; the image-corresponding component position recognition model is used to identify the majority item in the component position type corresponding to all the printed images in the image group; the component position type is a partial position with a high possibility of missing or a partial position with a low possibility of missing; the partial position with a high possibility of missing is a component connection part, an invisible display surface part or a low-visibility display surface part; the color saturation recognition model is an RNN neural network model, which is trained by a training data set including multiple training images and corresponding component position annotations and color-missing part annotations, and is used to identify the color-missing part in the image group according to the image group and the majority item.

[0021] As an optional implementation, in the first aspect of the present invention, the color-deficient part and the 3D model data are compared based on the two-dimensional and three-dimensional mapping algorithm to screen out multiple color-difference parts, including:

[0022] For each color-deficient portion, determining a framing viewpoint and a framing direction according to the device position of the image group corresponding to the color-deficient portion and the lens direction during acquisition;

[0023] Framing the 3D model data according to the framing viewpoint and the framing direction to obtain model part data corresponding to the color-deficient part;

[0024] Calculating the color difference between the color-deficient part and the model part data;

[0025] When the color difference is greater than a preset color threshold, the color-deficient portion is determined to be a color difference portion.

[0026] As an optional implementation, in the first aspect of the present invention, predicting the color of each of the color-deficient part and the color-difference part based on the 3D model data and a preset color comparison prediction algorithm to obtain a predicted color result includes:

[0027] For each color-deficient portion that is not the color-difference portion, determining the color of the corresponding model portion data in the 3D model data as a predicted color result of the color-deficient portion;

[0028] For each color difference portion, determining the color of the corresponding model portion data in the 3D model data as the reference color of the color difference portion;

[0029] Inputting the reference colors of multiple nearby parts near the color difference part into a color prediction neural network to obtain a predicted color corresponding to the color difference part; the color prediction neural network is an RNN neural network, which is trained by a training data set including multiple training nearby part colors and corresponding target part color annotations;

[0030] The original color of the color difference part, the average value of the reference color and the predicted color are calculated to obtain the predicted color result of the color difference part.

[0031] As an optional embodiment, in the first aspect of the present invention, controlling the 3D printer to perform color correction spraying on the color-deficient part and the color-difference part of the printed object according to the predicted color result includes:

[0032] Based on a preset position conversion mathematical relationship, according to the device position corresponding to each color-deficient part or the color-difference part and the lens direction during acquisition, the print object position corresponding to each color-deficient part or the color-difference part is calculated;

[0033] Generate a color spraying instruction corresponding to the print object position and the predicted color result;

[0034] The color spraying instruction is sent to the color spraying device of the 3D printer to perform color correction spraying on the color-deficient part and the color-difference part of the printed object.

[0035] A second aspect of an embodiment of the present invention discloses a color prediction system for full-color 3D printing, the system comprising:

[0036] An acquisition module, used for acquiring a printed object image and 3D model data based on which printing is performed during the printing process of the 3D printer;

[0037] A recognition module, used for recognizing a plurality of color-deficient parts and color-difference parts in the printed image based on a color-deficient recognition algorithm and a color-difference recognition algorithm;

[0038] A prediction module, configured to predict the color of each of the color-deficient part and the color-difference part according to the 3D model data and based on a preset color comparison prediction algorithm to obtain a predicted color result;

[0039] A control module is used to control the 3D printer to perform color correction spraying on the color-deficient part and the color-difference part of the printed object according to the predicted color result.

[0040] As an optional implementation, in the second aspect of the present invention, the printed matter image includes a plurality of printed matter images obtained by photographing the printed matter from a plurality of positions and in a plurality of directions.

[0041] As an optional implementation, in the second aspect of the present invention, the specific manner in which the recognition module recognizes the multiple color-deficient parts and color-difference parts in the printed image based on the color-deficient recognition algorithm and the color-difference recognition algorithm includes:

[0042] Grouping all the printed matter images based on viewpoints and angles to obtain a plurality of image groups;

[0043] Inputting each of the image groups into a preset low-saturation color recognition model to obtain a color-deficient portion corresponding to each of the image groups;

[0044] Based on a two-dimensional and three-dimensional mapping algorithm, the color-deficient portion and the 3D model data are compared in color difference to screen out a plurality of color-difference portions.

[0045] As an optional implementation, in the second aspect of the present invention, the recognition module groups all the printed matter images based on viewpoints and angles to obtain a plurality of image groups, including:

[0046] For any two of the printed matter images, obtaining the device position of the image acquisition device corresponding to the two printed matter images and the lens direction during acquisition;

[0047] Calculating the position distance between the device positions of the two printed matter images;

[0048] Calculating the direction difference between the lens directions when the two printed matter images were acquired;

[0049] Calculating a weighted average of the position distance and the direction difference to obtain an image difference between the two printed matter images;

[0050] According to the image difference, all the printed matter images are grouped based on a clustering algorithm to obtain a plurality of image groups; the image difference between any two printed matter images in each of the image groups is less than a preset difference threshold.

[0051] As an optional embodiment, in the second aspect of the present invention, the low-saturation color recognition model includes an image-corresponding component position recognition model and a color saturation recognition model; the image-corresponding component position recognition model is used to identify the majority item in the component position type corresponding to all the printed images in the image group; the component position type is a partial position with a high possibility of missing or a partial position with a low possibility of missing; the partial position with a high possibility of missing is a component connection part, an invisible display surface part or a low-visibility display surface part; the color saturation recognition model is an RNN neural network model, which is trained by a training data set including multiple training images and corresponding component position annotations and color-missing part annotations, and is used to identify the color-missing part in the image group according to the image group and the majority item.

[0052] As an optional implementation, in the second aspect of the present invention, the recognition module compares the color-deficient part with the 3D model data based on a two-dimensional and three-dimensional mapping algorithm to screen out a plurality of color-difference parts in a specific manner, including:

[0053] For each color-deficient portion, determining a framing viewpoint and a framing direction according to the device position of the image group corresponding to the color-deficient portion and the lens direction during acquisition;

[0054] Framing the 3D model data according to the framing viewpoint and the framing direction to obtain model part data corresponding to the color-deficient part;

[0055] Calculating the color difference between the color-deficient part and the model part data;

[0056] When the color difference is greater than a preset color threshold, the color-deficient portion is determined to be a color difference portion.

[0057] As an optional implementation, in the second aspect of the present invention, the prediction module predicts the color of each of the color-deficient part and the color-difference part according to the 3D model data based on a preset color comparison prediction algorithm, and obtains a specific method of predicting the color result, including:

[0058] For each color-deficient portion that is not the color-difference portion, determining the color of the corresponding model portion data in the 3D model data as a predicted color result of the color-deficient portion;

[0059] For each color difference portion, determining the color of the corresponding model portion data in the 3D model data as the reference color of the color difference portion;

[0060] Inputting the reference colors of multiple nearby parts near the color difference part into a color prediction neural network to obtain a predicted color corresponding to the color difference part; the color prediction neural network is an RNN neural network, which is trained by a training data set including multiple training nearby part colors and corresponding target part color annotations;

[0061] The original color of the color difference part, the average value of the reference color and the predicted color are calculated to obtain the predicted color result of the color difference part.

[0062] As an optional embodiment, in the second aspect of the present invention, the control module controls the 3D printer to perform color correction spraying on the color-deficient part and the color-difference part of the printed object according to the predicted color result, including:

[0063] Based on a preset position conversion mathematical relationship, according to the device position corresponding to each color-deficient part or the color-difference part and the lens direction during acquisition, the print object position corresponding to each color-deficient part or the color-difference part is calculated;

[0064] Generate a color spraying instruction corresponding to the print object position and the predicted color result;

[0065] The color spraying instruction is sent to the color spraying device of the 3D printer to perform color correction spraying on the color-deficient part and the color-difference part of the printed object.

[0066] A third aspect of the present invention discloses another color prediction system for full-color 3D printing, the system comprising:

[0067] A memory storing executable program code;

[0068] a processor coupled to the memory;

[0069] The processor calls the executable program code stored in the memory to execute part or all of the steps in the color prediction method for full-color 3D printing disclosed in the first aspect of the present invention.

[0070] The fourth aspect of the present invention discloses a computer storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute some or all of the steps in the color prediction method for full-color 3D printing disclosed in the first aspect of the present invention.

[0071] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0072] The present invention can identify the color missing and color difference parts of the printed object through a recognition algorithm, and then perform color prediction based on a color comparison prediction algorithm to control the 3D printer to perform color correction spraying on the printed object, thereby effectively improving the color completion function and color printing effect of 3D printing, and achieving smarter and better 3D full-color printing. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0074] Figure 1 It is a flow chart of a color prediction method for full-color 3D printing disclosed in an embodiment of the present invention.

[0075] Figure 2 It is a structural schematic diagram of a color prediction system for full-color 3D printing disclosed in an embodiment of the present invention.

[0076] Figure 3 It is a structural schematic diagram of another color prediction system for full-color 3D printing disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0077] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0078] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or equipment that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or equipment.

[0079] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0080] The present invention discloses a color prediction method and system for full-color 3D printing, which can identify the color missing part and color difference part of the printed object through a recognition algorithm, and then perform color prediction based on a color comparison prediction algorithm to control the 3D printer to perform color correction spraying on the printed object, thereby effectively improving the color complement function and color printing effect of 3D printing, and realizing smarter and better 3D full-color printing. The following are detailed descriptions.

[0081] Embodiment 1

[0082] See also Figure 1 , Figure 1 : is a flow chart of a color prediction method for full-color 3D printing disclosed in an embodiment of the present invention. Figure 1 The color prediction method for full-color 3D printing described can be applied to a data processing system / data processing device / data processing server (wherein the server includes a local processing server or a cloud processing server). Figure 1 As shown, the color prediction method for full-color 3D printing may include the following operations:

[0083] 101. During the printing process of the 3D printer, a printed object image and 3D model data based on which the printing is performed are obtained.

[0084] 102. Based on a color deficiency recognition algorithm and a color difference recognition algorithm, a plurality of color deficiency parts and color difference parts in the printed image are recognized.

[0085] 103. According to the 3D model data, based on a preset color comparison prediction algorithm, the color of each color-deficient part and color-difference part is predicted to obtain a predicted color result.

[0086] 104. According to the predicted color result, the 3D printer is controlled to perform color correction spraying on the color-deficient part and the color-difference part of the printed object.

[0087] It can be seen that the above-mentioned embodiments of the invention can identify the color missing parts and color difference parts of the printed object through the recognition algorithm, and then perform color prediction based on the color comparison prediction algorithm to control the 3D printer to perform color correction spraying on the printed object, thereby effectively improving the color completion function and color printing effect of 3D printing, and realizing smarter and better 3D full-color printing.

[0088] As an optional embodiment, in the above steps, the printed object image includes a plurality of printed object images obtained by photographing the printed object from a plurality of positions and in a plurality of directions.

[0089] It can be seen that through the above optional embodiments, the content of the printed image is clarified to better characterize the surface condition of the printed object, assist in improving the color complement function and color printing effect of 3D printing, and achieve smarter and better 3D full-color printing.

[0090] As an optional embodiment, in the above steps, based on the color deficiency recognition algorithm and the color difference recognition algorithm, identifying multiple color deficiency parts and color difference parts in the printed image includes:

[0091] Grouping all printed matter images based on viewpoints and angles to obtain a plurality of image groups;

[0092] Input each image group into a preset low-saturation color recognition model to obtain the color-deficient part corresponding to each image group;

[0093] Based on the two-dimensional and three-dimensional mapping algorithm, the color difference between the color-missing part and the 3D model data is compared to screen out multiple color difference parts.

[0094] It can be seen that through the above-mentioned optional embodiments, all printed images can be grouped based on viewpoints and angles to obtain multiple image groups, and then the color missing parts and color difference parts can be determined based on the low-saturation color recognition model and the two- and three-dimensional mapping algorithm, so as to perform more accurate color correction spraying later, thereby assisting in improving the color completion function and color printing effect of 3D printing, and realizing smarter and better 3D full-color printing.

[0095] As an optional embodiment, in the above steps, all printed matter images are grouped based on viewpoints and angles to obtain multiple image groups, including:

[0096] For any two printed matter images, obtaining the device position of the image acquisition device corresponding to the two printed matter images and the lens direction during acquisition;

[0097] Calculating the position distance between the device positions of the two printed matter images;

[0098] Calculating the direction difference between the lens directions when the two printed object images are acquired;

[0099] Calculate the weighted sum average of the position distance and the direction difference to obtain the image difference between the two printed images;

[0100] According to the image difference, all printed matter images are grouped based on a clustering algorithm to obtain a plurality of image groups; the image difference between any two printed matter images in each image group is less than a preset difference threshold.

[0101] It can be seen that through the above optional embodiments, it is possible to group all printed images to obtain multiple image groups based on the difference calculation between the image acquisition device position and the lens direction during acquisition, so as to facilitate more accurate color correction and spraying later, thereby assisting in improving the color completion function and color printing effect of 3D printing, and realizing smarter and better 3D full-color printing.

[0102] As an optional embodiment, in the above steps, the low-saturation color recognition model includes an image-corresponding component position recognition model and a color saturation recognition model; the image-corresponding component position recognition model is used to identify the majority item in the component position type corresponding to all printed images in the image group; the component position type is a partial position with a high possibility of missing or a partial position with a low possibility of missing; the partial position with a high possibility of missing is a component connection part, an invisible display surface part or a low-visibility display surface part; the color saturation recognition model is an RNN neural network model, which is trained by a training data set including multiple training images and corresponding component position annotations and color-missing part annotations, and is used to identify the color-missing part in the image group according to the image group and the majority item.

[0103] It can be seen that through the above-mentioned optional embodiments, the model details and workflow of the low-saturation color recognition model are clarified, so as to more accurately realize the identification of the color-deficient parts of the image group, so as to perform more accurate color correction spraying later, thereby assisting in improving the color completion function and color printing effect of 3D printing, and realizing smarter and better 3D full-color printing.

[0104] As an optional embodiment, in the above steps, based on the two-dimensional and three-dimensional mapping algorithm, the color-deficient part and the 3D model data are compared in color difference to screen out multiple color difference parts, including:

[0105] For each color-deficient portion, determine the framing viewpoint and framing direction according to the device position of the image group corresponding to the color-deficient portion and the lens direction when acquiring the image;

[0106] Framing the 3D model data according to the framing viewpoint and the framing direction to obtain the model part data corresponding to the color-deficient part;

[0107] Calculate the color difference between the color-deficient part and the model part data;

[0108] When the color difference is greater than a preset color threshold, the color-deficient portion is determined to be a color difference portion.

[0109] It can be seen that through the above-mentioned optional embodiments, the framing parameters can be determined based on the image acquisition parameters of the color-missing part, so as to determine the model part data corresponding to the color-missing part for color difference comparison, so as to more accurately realize the identification of the color difference part, so as to carry out more accurate color correction spraying later, thereby assisting in improving the color completion function and color printing effect of 3D printing, and realizing smarter and better 3D full-color printing.

[0110] As an optional embodiment, in the above steps, according to the 3D model data, based on a preset color comparison prediction algorithm, the color of each color-deficient part and the color-difference part is predicted to obtain a predicted color result, including:

[0111] For each color-deficient part that is not a color difference part, the color of the corresponding model part data in the 3D model data is determined as a predicted color result of the color-deficient part;

[0112] For each color difference part, determining the color of the corresponding model part data in the 3D model data as the reference color of the color difference part;

[0113] Inputting the reference colors of multiple nearby parts near the color difference part into a color prediction neural network to obtain a predicted color corresponding to the color difference part; optionally, the color prediction neural network is an RNN neural network, which is trained by a training data set including multiple training nearby part colors and corresponding target part color annotations;

[0114] The average values ​​of the original color, the reference color and the predicted color of the color difference part are calculated to obtain the predicted color result of the color difference part.

[0115] It can be seen that through the above-mentioned optional embodiments, the color of each color-missing part and color-difference part can be predicted based on the color of the corresponding model part data in the 3D model data and the color prediction neural network to obtain a predicted color result, so as to perform more accurate color correction spraying later, thereby assisting in improving the color completion function and color printing effect of 3D printing, and realizing smarter and better 3D full-color printing.

[0116] As an optional embodiment, in the above steps, according to the predicted color result, controlling the 3D printer to perform color correction spraying on the color-deficient part and the color-difference part of the printed object includes:

[0117] Based on a preset mathematical relationship of position conversion, the print position corresponding to each color missing part or color difference part is calculated according to the device position corresponding to each color missing part or color difference part and the lens direction during acquisition;

[0118] Generate a color spraying instruction corresponding to the print position and the predicted color result;

[0119] The color spraying instruction is sent to the color spraying device of the 3D printer to perform color correction spraying on the color missing parts and color difference parts of the printed object.

[0120] It can be seen that through the above optional embodiments, it is possible to generate a color spraying instruction corresponding to the print object position and the predicted color result, so as to perform color correction spraying on the print object, thereby improving the color printing effect of 3D printing and achieving smarter and better 3D full-color printing.

[0121] Embodiment 2

[0122] See also Figure 2 , Figure 2 : is a schematic diagram of a color prediction system for full-color 3D printing disclosed in an embodiment of the present invention. Figure 2 The color prediction system for full-color 3D printing described can be applied to a data processing system / data processing device / data processing server (wherein the server includes a local processing server or a cloud processing server). Figure 2 As shown, the color prediction system for full-color 3D printing may include:

[0123] The acquisition module 201 is used to acquire the printed object image and the 3D model data based on which the printing is performed during the printing process of the 3D printer.

[0124] The recognition module 202 is used to recognize multiple color-deficient parts and color-difference parts in the printed image based on a color-deficient recognition algorithm and a color-difference recognition algorithm.

[0125] The prediction module 203 is used to predict the color of each color-deficient part and color-difference part according to the 3D model data based on a preset color comparison prediction algorithm to obtain a predicted color result.

[0126] The control module 204 is used to control the 3D printer to perform color correction spraying on the color-deficient and color-difference parts of the printed object according to the predicted color result.

[0127] It can be seen that the above-mentioned embodiments of the invention can identify the color missing parts and color difference parts of the printed object through the recognition algorithm, and then perform color prediction based on the color comparison prediction algorithm to control the 3D printer to perform color correction spraying on the printed object, thereby effectively improving the color completion function and color printing effect of 3D printing, and realizing smarter and better 3D full-color printing.

[0128] As an optional embodiment, the printed matter image includes a plurality of printed matter images obtained by photographing the printed matter from a plurality of positions and in a plurality of directions.

[0129] It can be seen that through the above optional embodiments, the content of the printed image is clarified to better characterize the surface condition of the printed object, assist in improving the color complement function and color printing effect of 3D printing, and achieve smarter and better 3D full-color printing.

[0130] As an optional embodiment, the specific manner in which the recognition module recognizes multiple color-deficient parts and color-difference parts in the printed image based on the color-deficient recognition algorithm and the color-difference recognition algorithm includes:

[0131] Grouping all printed matter images based on viewpoints and angles to obtain a plurality of image groups;

[0132] Input each image group into a preset low-saturation color recognition model to obtain the color-deficient part corresponding to each image group;

[0133] Based on the two-dimensional and three-dimensional mapping algorithm, the color difference between the color-missing part and the 3D model data is compared to screen out multiple color difference parts.

[0134] It can be seen that through the above-mentioned optional embodiments, all printed images can be grouped based on viewpoints and angles to obtain multiple image groups, and then the color missing parts and color difference parts can be determined based on the low-saturation color recognition model and the two- and three-dimensional mapping algorithm, so as to perform more accurate color correction spraying later, thereby assisting in improving the color completion function and color printing effect of 3D printing, and realizing smarter and better 3D full-color printing.

[0135] As an optional embodiment, the recognition module groups all printed matter images based on viewpoints and angles to obtain a plurality of image groups, including:

[0136] For any two printed matter images, obtaining the device position of the image acquisition device corresponding to the two printed matter images and the lens direction during acquisition;

[0137] Calculating the position distance between the device positions of the two printed matter images;

[0138] Calculating the direction difference between the lens directions when the two printed object images are acquired;

[0139] Calculate the weighted sum average of the position distance and the direction difference to obtain the image difference between the two printed images;

[0140] According to the image difference, all printed matter images are grouped based on a clustering algorithm to obtain a plurality of image groups; the image difference between any two printed matter images in each image group is less than a preset difference threshold.

[0141] It can be seen that through the above optional embodiments, it is possible to group all printed images to obtain multiple image groups based on the difference calculation between the image acquisition device position and the lens direction during acquisition, so as to facilitate more accurate color correction and spraying later, thereby assisting in improving the color completion function and color printing effect of 3D printing, and realizing smarter and better 3D full-color printing.

[0142] As an optional embodiment, the low-saturation color recognition model includes an image-corresponding component position recognition model and a color saturation recognition model; the image-corresponding component position recognition model is used to identify the majority item in the component position type corresponding to all printed images in the image group; the component position type is a partial position with a high possibility of missing or a partial position with a low possibility of missing; the partial position with a high possibility of missing is a component connection part, an invisible display surface part or a low-visibility display surface part; the color saturation recognition model is an RNN neural network model, which is trained by a training data set including multiple training images and corresponding component position annotations and color-missing part annotations, and is used to identify the color-missing part in the image group according to the image group and the majority item.

[0143] It can be seen that through the above-mentioned optional embodiments, the model details and workflow of the low-saturation color recognition model are clarified, so as to more accurately realize the identification of the color-deficient parts of the image group, so as to perform more accurate color correction spraying later, thereby assisting in improving the color completion function and color printing effect of 3D printing, and realizing smarter and better 3D full-color printing.

[0144] As an optional embodiment, the recognition module compares the color difference between the color-deficient part and the 3D model data based on the two-dimensional and three-dimensional mapping algorithm to screen out multiple color difference parts, including:

[0145] For each color-deficient portion, determine the framing viewpoint and framing direction according to the device position of the image group corresponding to the color-deficient portion and the lens direction when acquiring the image;

[0146] Framing the 3D model data according to the framing viewpoint and the framing direction to obtain the model part data corresponding to the color-deficient part;

[0147] Calculate the color difference between the color-deficient part and the model part data;

[0148] When the color difference is greater than a preset color threshold, the color-deficient portion is determined to be a color difference portion.

[0149] It can be seen that through the above-mentioned optional embodiments, the framing parameters can be determined based on the image acquisition parameters of the color-missing part, so as to determine the model part data corresponding to the color-missing part for color difference comparison, so as to more accurately realize the identification of the color difference part, so as to carry out more accurate color correction spraying later, thereby assisting in improving the color completion function and color printing effect of 3D printing, and realizing smarter and better 3D full-color printing.

[0150] As an optional embodiment, the prediction module predicts the color of each color-deficient part and color-difference part according to the 3D model data based on a preset color comparison prediction algorithm, and obtains a specific method of predicting the color result, including:

[0151] For each color-deficient part that is not a color difference part, the color of the corresponding model part data in the 3D model data is determined as a predicted color result of the color-deficient part;

[0152] For each color difference part, determining the color of the corresponding model part data in the 3D model data as the reference color of the color difference part;

[0153] Inputting the reference colors of multiple nearby parts near the color difference part into a color prediction neural network to obtain a predicted color corresponding to the color difference part; optionally, the color prediction neural network is an RNN neural network, which is trained by a training data set including multiple training nearby part colors and corresponding target part color annotations;

[0154] The average values ​​of the original color, the reference color and the predicted color of the color difference part are calculated to obtain the predicted color result of the color difference part.

[0155] It can be seen that through the above-mentioned optional embodiments, the color of each color-missing part and color-difference part can be predicted based on the color of the corresponding model part data in the 3D model data and the color prediction neural network to obtain a predicted color result, so as to perform more accurate color correction spraying later, thereby assisting in improving the color completion function and color printing effect of 3D printing, and realizing smarter and better 3D full-color printing.

[0156] As an optional embodiment, the control module controls the 3D printer to perform color correction spraying on the color-deficient part and the color-difference part of the printed object according to the predicted color result, including:

[0157] Based on a preset mathematical relationship of position conversion, the print position corresponding to each color missing part or color difference part is calculated according to the device position corresponding to each color missing part or color difference part and the lens direction during acquisition;

[0158] Generate a color spraying instruction corresponding to the print position and the predicted color result;

[0159] The color spraying instruction is sent to the color spraying device of the 3D printer to perform color correction spraying on the color missing parts and color difference parts of the printed object.

[0160] It can be seen that through the above optional embodiments, it is possible to generate a color spraying instruction corresponding to the print object position and the predicted color result, so as to perform color correction spraying on the print object, thereby improving the color printing effect of 3D printing and achieving smarter and better 3D full-color printing.

[0161] Embodiment 3

[0162] See also Figure 3 , Figure 3 It is another color prediction system for full-color 3D printing disclosed in an embodiment of the present invention. Figure 3 The color prediction system for full-color 3D printing described is applied in a data processing system / data processing device / data processing server (wherein the server includes a local processing server or a cloud processing server). Figure 3 As shown, the color prediction system for full-color 3D printing may include:

[0163] A memory 301 storing executable program codes;

[0164] a processor 302 coupled to the memory 301;

[0165] The processor 302 calls the executable program code stored in the memory 301 to execute the steps of the color prediction method for full-color 3D printing described in the first embodiment.

[0166] Embodiment 4

[0167] An embodiment of the present invention discloses a computer-readable storage medium storing a computer program for electronic data exchange, wherein the computer program enables a computer to execute the steps of the color prediction method for full-color 3D printing described in the first embodiment.

[0168] Embodiment 5

[0169] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute the steps of the color prediction method for full-color 3D printing described in the first embodiment.

[0170] The above describes specific embodiments of the present specification, and other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily have to be performed in the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0171] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0172] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0173] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification may be in the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the embodiments of this specification may be in the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0174] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0175] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0176] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0177] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0178] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0179] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0180] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0181] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0182] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0183] Finally, it should be noted that the color prediction method and system for full-color 3D printing disclosed in the embodiments of the present invention only disclose the preferred embodiments of the present invention, which are only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A color prediction method for full-color 3D printing, characterized in that: The method comprises: Acquire a printed object image and 3D model data based on which the printing is performed during the printing process of the 3D printer; the printed object image includes a plurality of printed object images obtained by photographing the printed object from a plurality of positions and in a plurality of directions; Based on the color deficiency recognition algorithm and the color difference recognition algorithm, a plurality of color deficiency parts and color difference parts in the printed image are recognized, including: For any two of the printed matter images, obtaining the device position of the image acquisition device corresponding to the two printed matter images and the lens direction during acquisition; Calculating the position distance between the device positions of the two printed matter images; Calculating the direction difference between the lens directions when the two printed matter images were acquired; Calculating a weighted average of the position distance and the direction difference to obtain an image difference between the two printed matter images; According to the image difference, all the printed matter images are grouped based on a clustering algorithm to obtain a plurality of image groups; the image difference between any two printed matter images in each of the image groups is less than a preset difference threshold; Inputting each of the image groups into a preset low-saturation color recognition model to obtain a color-deficient portion corresponding to each of the image groups; Based on a two-dimensional and three-dimensional mapping algorithm, the color-deficient part and the 3D model data are compared in color difference to screen out a plurality of color-difference parts; According to the 3D model data, based on a preset color comparison prediction algorithm, predict the color of each of the color-deficient part and the color-difference part to obtain a predicted color result; According to the predicted color result, controlling the 3D printer to perform color correction spraying on the color-deficient part and the color-difference part of the printed object includes: Based on a preset position conversion mathematical relationship, according to the device position corresponding to each color-deficient part or the color-difference part and the lens direction during acquisition, the print object position corresponding to each color-deficient part or the color-difference part is calculated; Generate a color spraying instruction corresponding to the print object position and the predicted color result; The color spraying instruction is sent to the color spraying device of the 3D printer to perform color correction spraying on the color-deficient part and the color-difference part of the printed object.

2. The color prediction method for full-color 3D printing according to claim 1, characterized in that: The low-saturation color recognition model includes an image-corresponding component position recognition model and a color saturation recognition model; the image-corresponding component position recognition model is used to identify the majority item in the component position type corresponding to all the printed material images in the image group; the component position type is a partial position with a high possibility of missing or a partial position with a low possibility of missing; the partial position with a high possibility of missing is a component connection part, an invisible display surface part or a low-visibility display surface part; the color saturation recognition model is an RNN neural network model, which is trained by a training data set including multiple training images and corresponding component position annotations and color-missing part annotations, and is used to identify the color-missing part in the image group according to the image group and the majority item.

3. The color prediction method for full-color 3D printing according to claim 1, characterized in that: The color difference comparison between the color-deficient part and the 3D model data based on the two-dimensional and three-dimensional mapping algorithm to screen out multiple color difference parts includes: For each color-deficient portion, determining a framing viewpoint and a framing direction according to the device position of the image group corresponding to the color-deficient portion and the lens direction during acquisition; Framing the 3D model data according to the framing viewpoint and the framing direction to obtain model part data corresponding to the color-deficient part; Calculating the color difference between the color-deficient part and the model part data; When the color difference is greater than a preset color threshold, the color-deficient portion is determined to be a color difference portion.

4. The color prediction method for full-color 3D printing according to claim 3, characterized in that: The step of predicting the color of each of the color-deficient part and the color-difference part based on the 3D model data and a preset color comparison prediction algorithm to obtain a predicted color result includes: For each color-deficient portion that is not the color-difference portion, determining the color of the corresponding model portion data in the 3D model data as a predicted color result of the color-deficient portion; For each color difference portion, determining the color of the corresponding model portion data in the 3D model data as the reference color of the color difference portion; Inputting the reference colors of multiple nearby parts near the color difference part into a color prediction neural network to obtain a predicted color corresponding to the color difference part; the color prediction neural network is an RNN neural network, which is trained by a training data set including multiple training nearby part colors and corresponding target part color annotations; The original color of the color difference part, the average value of the reference color and the predicted color are calculated to obtain the predicted color result of the color difference part.

5. A color prediction system for full-color 3D printing, characterized in that: The system comprises: An acquisition module, used for acquiring a print image of a print object and 3D model data based on which printing is performed during the printing process of the 3D printer; the print image includes a plurality of print images obtained by photographing the print object from a plurality of positions and in a plurality of directions; The recognition module is used to recognize multiple color-deficient parts and color-difference parts in the printed image based on a color-deficient recognition algorithm and a color-difference recognition algorithm, and includes: For any two of the printed matter images, obtaining the device position of the image acquisition device corresponding to the two printed matter images and the lens direction during acquisition; Calculating the position distance between the device positions of the two printed matter images; Calculating the direction difference between the lens directions when the two printed matter images were acquired; Calculating a weighted average of the position distance and the direction difference to obtain an image difference between the two printed matter images; According to the image difference, all the printed matter images are grouped based on a clustering algorithm to obtain a plurality of image groups; the image difference between any two printed matter images in each of the image groups is less than a preset difference threshold; Inputting each of the image groups into a preset low-saturation color recognition model to obtain a color-deficient portion corresponding to each of the image groups; Based on a two-dimensional and three-dimensional mapping algorithm, the color-deficient part and the 3D model data are compared in color difference to screen out a plurality of color-difference parts; A prediction module, configured to predict the color of each of the color-deficient part and the color-difference part according to the 3D model data and based on a preset color comparison prediction algorithm to obtain a predicted color result; A control module, for controlling the 3D printer to perform color correction spraying on the color-deficient part and the color-difference part of the printed object according to the predicted color result, comprising: Based on a preset position conversion mathematical relationship, according to the device position corresponding to each color-deficient part or the color-difference part and the lens direction during acquisition, the print object position corresponding to each color-deficient part or the color-difference part is calculated; Generate a color spraying instruction corresponding to the print object position and the predicted color result; The color spraying instruction is sent to the color spraying device of the 3D printer to perform color correction spraying on the color-deficient part and the color-difference part of the printed object.

6. A color prediction system for full-color 3D printing, characterized in that: The system comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the color prediction method for full-color 3D printing according to any one of claims 1 to 4.

Images