Photographing device capable of automatically splicing pixel blocks based on mechanical arm AI control

Abstract

The invention discloses a photographing device capable of automatically splicing pixel blocks based on mechanical arm AI control, which is characterized in that a jigsaw mechanical arm is mounted on an operation table, and the splicing of the pixel blocks is automatically realized based on AI algorithm control; a camera and a displayer are arranged above the operation table, a sound control device is arranged in the camera, and after a photographing instruction is received, the camera is started to execute photographing; the camera collects an image containing a head portrait and transmits the image to the operation table, the operation table carries out pixelation processing on the image, outputs a plurality of pixel blocks, drives the puzzle mechanical arm to splice the pixel blocks into a head portrait pattern according to a pixel splicing rule, and transmits the head portrait pattern to the display to be displayed; and the 3D printer is in communication connection with the operation table, receives the head portrait pattern sent by the operation table, and calls a preset 3D printing model to print and output the photo sticker of the head portrait pattern. According to the device, automatic pixel splicing controlled by the mechanical arm AI is realized, so that the application of an intelligent robot is more flexible, and the generation of a large-head sticker is more scientific and technological.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of photographing, and particularly relates to a photographing device for automatically splicing pixel blocks based on AI control of a mechanical arm. BACKGROUND

[0002] A traditional sticker is generated by capturing a head portrait of a sticker photographer by using an image capturing device and then superimposing the head portrait with a background image.

[0003] With the progress of science and technology, artificial intelligence technology has developed rapidly, and in today's era, such high-tech is also used in more and more occasions, especially in combination with robots, so that the application range of intelligent robots is also very wide. Through the mechanical arm, the puzzle gradually enters the public vision, so that the generation of the sticker based on the AI control of the mechanical arm becomes possible. SUMMARY

[0004] The purpose of the application is to provide a photographing device for automatically splicing pixel blocks based on AI control of a mechanical arm, and to realize 3D printing of a pixelized head portrait pattern.

[0005] To solve the above problems, the technical scheme of the application is as follows:

[0006] A photographing device for automatically splicing pixel blocks based on AI control of a mechanical arm, comprising:

[0007] An operation table, on which a puzzle mechanical arm is installed, the puzzle mechanical arm automatically realizes splicing of pixel blocks based on AI algorithm control;

[0008] A camera and a display are arranged above the operation table, a sound control device is arranged in the camera, after the sound control device receives a photographing instruction, the camera is started to execute photographing; the image collected by the camera includes a head portrait, the image is transmitted to the operation table, the operation table performs pixelization processing on the image, outputs a plurality of pixel blocks, and drives the puzzle mechanical arm to splice the pixel blocks into a head portrait pattern according to a pixel splicing rule, and the head portrait pattern is transmitted to the display for display;

[0009] A 3D printer, which is in communication connection with the operation table, receives the head portrait pattern sent by the operation table, and calls a preset 3D printing model to print and output the head portrait pattern.

[0010] According to an embodiment of the application, the operation table comprises a head portrait extraction module, a pixelized image module and a display screen;

[0011] The head portrait extraction module is configured to acquire the image transmitted by the camera, extract a head portrait from the image based on an image extraction algorithm, and transmit the obtained head portrait to the pixelized image module;

[0012] The pixelization image module performs parallel pixelization processing on each pixel in the avatar, and outputs a plurality of pixel blocks to the display screen.

[0013] According to an embodiment of the present application, the pixelization image module performing pixelization processing on the avatar includes:

[0014] The avatar is segmented into a plurality of sub-pictures according to avatar features, and the sub-pictures are subjected to parallel pixelization processing.

[0015] According to an embodiment of the present application, the parallel pixelization processing on the sub-pictures includes:

[0016] The color value and the transparency value of each pixel are determined according to the pixel information of the sub-picture;

[0017] A pixel block of each pixel is created, and the pixels of the pixel block are set according to the color value and the transparency value of the each pixel; wherein the resolution of the pixel block is N*N, and N is an integer greater than or equal to 2;

[0018] After all the pixels in the sub-picture are pixelized, the pixelized picture is scaled to 1 / N 2 .

[0019] According to an embodiment of the present application, the parallel pixelization processing on the sub-pictures includes:

[0020] The row pixels and the column pixels with an interval of M pixels in the sub-picture are determined, wherein M is an integer greater than or equal to 2;

[0021] The transparency values of the row pixels and the column pixels are adjusted to a preset transparency value to complete the pixelization processing.

[0022] According to an embodiment of the present application, the picture assembling robot assembles the pixel blocks into an avatar pattern according to a pixel assembly rule includes:

[0023] The pixel blocks corresponding to each avatar feature are identified to obtain a face pixel block, an eye pixel block, a mouth pixel block, a nose pixel block, and a hair pixel block;

[0024] The face pixel block, the eye pixel block, the mouth pixel block, the nose pixel block, and the hair pixel block are assembled based on an image assembly template to obtain an avatar pattern.

[0025] According to an embodiment of the present application, the identification of the pixel blocks corresponding to each avatar feature includes:

[0026] A database of each avatar feature template is established, and the database of each avatar feature template includes a face shape template database, an eye shape template database, a mouth shape template database, a nose shape template database, and a hair shape template database;

[0027] Based on the various head portrait feature template database, a head portrait feature classification model is created and trained;

[0028] Each head portrait feature is identified by the head portrait feature classification model.

[0029] According to an embodiment of the present application, two puzzle mechanical arms are installed on the operation platform, and the two puzzle mechanical arms are oppositely arranged to jointly complete the splicing of the head portrait pattern.

[0030] Compared with the prior art, the present application has the following advantages and positive effects:

[0031] In an embodiment of the present application, an automatic splicing pixel block photographing device based on mechanical arm AI control is provided. The puzzle mechanical arm is installed on the operation platform, and the splicing of the pixel block is automatically realized based on the AI algorithm control. A camera and a display are arranged above the operation platform, and a voice control device is arranged in the camera. After receiving the photographing instruction, the camera is started to perform photographing. The image containing the head portrait collected by the camera is transmitted to the operation platform. The operation platform performs pixelization processing on the image, outputs a plurality of pixel blocks, and drives the puzzle mechanical arm to splice the pixel blocks into a head portrait pattern according to the pixel splicing rule, and transmits the head portrait pattern to the display for display. The 3D printer is in communication connection with the operation platform, receives the head portrait pattern sent by the operation platform, and calls the preset 3D printing model to print and output the head portrait pattern sticker. The device realizes the automatic pixel splicing of the mechanical arm AI control, makes the application of the intelligent robot more flexible, and makes the generation of the sticker more scientific and technological. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 FIG. 1 is a schematic diagram of an automatic splicing pixel block photographing device based on mechanical arm AI control in an embodiment of the present application;

[0033] Figure 2 FIG. 2 is a schematic diagram of image pixelization in an embodiment of the present application;

[0034] Figure 3 FIG. 3 is another schematic diagram of image pixelization in an embodiment of the present application;

[0035] Figure 4 FIG. 4 is a schematic diagram of the overall automatic splicing pixel block photographing device based on mechanical arm AI control in an embodiment of the present application. DETAILED DESCRIPTION

[0036] The present application will be further described in detail below in combination with the drawings and specific embodiments. According to the following description and claims, the advantages and features of the present application will be more apparent.

[0037] Please refer toFigure 1 The embodiment provides a photographing device for automatically splicing pixel blocks based on AI control of a mechanical arm, comprising:

[0038] An operation table 1 is provided with a puzzle mechanical arm 2 installed thereon, and the puzzle mechanical arm 2 automatically realizes splicing of pixel blocks based on AI algorithm control.

[0039] A camera and a display 3 are arranged above the operation table 1, a sound control device is arranged in the camera, and the sound control device starts the camera to perform photographing after receiving a photographing instruction; the image collected by the camera includes a portrait, the image is transmitted to the operation table, the operation table performs pixelization processing on the image, outputs a plurality of pixel blocks, and drives the puzzle mechanical arm to splice the pixel blocks into a portrait pattern according to a pixel splicing rule, and the portrait pattern is transmitted to the display 3 for display.

[0040] A 3D printer 4 is in communication connection with the operation table 1, receives the portrait pattern sent by the operation table 1, and calls a preset 3D printing model 5 to print and output the portrait pattern.

[0041] Through the device, a customer can stand in front of the camera and say “take a photo”, which is captured by the sound control device in the camera, and then the camera is started to perform photographing. The image (photo) collected by the camera is transmitted to the operation table, and after being processed by the puzzle mechanical arm on the operation table, a portrait pattern is generated and displayed on the display for the customer to view, and the portrait pattern can also be 3D printed and output as a sticker, thereby providing the customer with a novel experience.

[0042] The implementation of the puzzle mechanical arm based on AI algorithm control to automatically realize splicing of pixel blocks is as follows:

[0043] First, the operation table needs to process the image transmitted by the camera as follows:

[0044] The operation table includes a portrait extraction module, a pixelized image module and a display screen, the portrait extraction module is configured to obtain the image transmitted by the camera, perform portrait extraction on the image based on an image extraction algorithm, and transmit the obtained portrait to the pixelized image module.

[0045] The pixelized image module performs parallel pixelization processing on each pixel point in the portrait, and outputs a plurality of pixel blocks to the display screen, so that the puzzle mechanical arm performs pixel block splicing.

[0046] Specifically, the portrait extraction module extracts the portrait from the image transmitted by the camera by using a matting method. The matting method can be implemented by the following steps:

[0047] Feature point detection is performed on the image. The feature point is a point where the gray value of the image changes sharply or a point with a large curvature on the edge of the image (i.e., the intersection of two edges), which has distinctive characteristics, can effectively reflect the essential characteristics of the image, and can identify the target object in the image. Specifically, a feature point detection algorithm can be used to obtain the feature points in the image. For the portrait in the image, a template can be used to detect the feature points of the image. The image region is circled according to the feature points, and the image region is cut out to obtain the portrait.

[0048] The following steps can also be used to achieve the above method:

[0049] The portrait in the image is detected, and a rectangular frame containing the portrait is output. The pixels in the portrait rectangular frame are classified to separate the foreground and background of the portrait, and the segmentation result is subjected to erosion and expansion morphological processing to generate a trimap image. The portrait in the portrait rectangular frame is finely cut out according to the trimap image. The portrait detection herein uses a trained full convolutional deep neural network; the pixel classification and cutout are both full convolutional deep neural networks with a trained Encoder / Decoder structure.

[0050] In addition to the above cutout method, other existing cutout methods can also be used as long as the portrait in the image can be cut out. After the portrait extraction module obtains the portrait, the portrait is transmitted to the pixelated image module.

[0051] The pixelated image module performs pixelization processing on the portrait, including:

[0052] The portrait is segmented into multiple subgraphs according to the portrait features, and the subgraphs are subjected to pixelization parallel processing. The portrait features in the segmentation of the portrait into multiple subgraphs include face, eye, nose, mouth, ear, and hair. When segmenting the portrait, the portrait can be finely segmented according to the specific shape of the portrait features, or the portrait can be segmented into multiple subgraphs of the same size with location information. Then, the subgraphs are subjected to pixelization parallel processing, that is, the pixelization of multiple subgraphs is processed simultaneously.

[0053] In this embodiment, the pixelization processing on the subgraph includes:

[0054] The color value and transparency value of each pixel point are determined according to the pixel point information of the subgraph, and a pixel block of each pixel point is created. The pixel points of the pixel block are set according to the color value and transparency value of each pixel point. The resolution of the pixel block is N x N, and N is an integer greater than or equal to 2. After all the pixel points in the subgraph are pixelized, the pixelized image is scaled to 1 / N 2 .

[0055] By creating a pixel block in the subgraph, that is, taking the original pixel point in the subgraph as the center to expand to a pixel block of N x N pixel points, and setting the pixel block according to the color value and transparency value of the original pixel point, N 2 -1 pixel points are added, and after the creation of the pixel block is completed, the image is scaled to 1 / N 2 , so as to ensure that the size of the pixelated image remains unchanged.

[0056] As shown in Figure 2 , when N = 3, the ARGB value (Alpha-Red-Green-Blue) of the original pixel point 2 is (128, 128, 128, 128), the ARGB value (Alpha-Red-Green-Blue) of the other pixel point 1 in the pixel block is set to (128, 128, 128, 128), and after the creation of the pixel block is completed, the pixelated image is scaled to 1 / 3 2 , so as to ensure that the size of the image remains unchanged.

[0057] In an embodiment, when it is determined that N is an odd number greater than or equal to 2, it is determined that each pixel point is a center pixel point of a pixel block; and other pixel points in the pixel block other than the center pixel point are set according to the color value and transparency value of each pixel point. By setting the other pixel points in the pixel block other than the center pixel point according to the color value and transparency value of each pixel point when it is determined that N is an odd number greater than or equal to 2, the reliability and efficiency of the pixel block creation process are ensured.

[0058] In addition to the above-mentioned subgraph pixelization processing method, the embodiment also provides the following method:

[0059] Determine the row pixel points and column pixel points in the subgraph that are spaced by M pixel points, wherein M is an integer greater than or equal to 2; and adjust the transparency values of the row pixel points and the column pixel points to a preset transparency value to complete the pixelization processing.

[0060] By adjusting the transparency values of the row pixel points and the column pixel points to a preset transparency value, the operation steps are simplified, and the user's use experience is improved.

[0061] As shown in Figure 3 , when M is 3, the spacing between adjacent row pixel points is 3 pixel points, the spacing between adjacent column pixel points is 3 pixel points, and the pixel point 4 is retained, and the pixel point 3 is set to be transparent white to realize the pixelization of the image.

[0062] After the pixelization image module completes the pixelization processing of the subgraph, the image that has completed the pixelization is transmitted to a display screen on an operation table within the operating range of the jigsaw mechanical arm, so that the jigsaw mechanical arm performs pixel block splicing.

[0063] In this embodiment, the puzzle robot arm assembles the pixel blocks into the portrait pattern according to the pixel assembly rule, which includes the following cases:

[0064] In the case where the pixelized image module performs fine segmentation according to the specific shape of the portrait feature to obtain complete sub-images of face, eye, nose, mouth, ear, and hair, the following steps are adopted when the puzzle robot arm assembles the pixel blocks:

[0065] The pixel blocks corresponding to each portrait feature are identified to obtain face pixel blocks, eye pixel blocks, mouth pixel blocks, nose pixel blocks, ear pixel blocks, and hair pixel blocks; and the face pixel blocks, eye pixel blocks, mouth pixel blocks, nose pixel blocks, ear pixel blocks, and hair pixel blocks are assembled based on the image assembly template to obtain the portrait pattern.

[0066] In the identification of the pixel blocks corresponding to each portrait feature, a pre-trained classification model can be used. Specifically, a database of each portrait feature template is established, which includes a face shape template database, an eye shape template database, a mouth shape template database, a nose shape template database, an ear template database, and a hair shape template database; based on the database of each portrait feature template, a portrait feature classification model is created and trained; and the pixel blocks corresponding to each portrait feature are identified through the portrait feature classification model.

[0067] Common and typical portrait features are saved in the database of each portrait feature template. The face shape template database can include various face shape templates such as round face, square face, and so on; the eye shape template database can include various eye shape templates such as Danfeng eye, peach blossom eye, and apricot eye; the nose shape template database can include various nose shape templates such as hawk nose, wide nose, and garlic nose, and so on, which are not listed here.

[0068] A portrait feature classification model is established through a classification learning method. After the model is established, the portrait feature template vector is automatically learned through the existing samples in the portrait feature template database by inputting the portrait, and the specific portrait feature template in the portrait feature database corresponding to the input portrait can be determined through the template vector, thereby realizing the identification of the portrait feature.

[0069] After the recognition of the portrait feature is completed, the face pixel block, the eye pixel block, the mouth pixel block, the nose pixel block, the ear pixel block and the hair pixel block are spliced based on the image splicing template to obtain the portrait pattern. The image splicing template can be determined as follows: first, the face pixel block is determined, and then the face pixel block, the eye pixel block, the mouth pixel block, the nose pixel block, the ear pixel block and the hair pixel block are spliced according to the distribution principle that the eyes, the nose and the mouth are in the face and the ears and the hair are outside the face, and the seamless connection of the pixel blocks is ensured. Since the face pixel block, the eye pixel block, the mouth pixel block, the nose pixel block, the ear pixel block and the hair pixel block are cut from the same portrait, the splicing can be easily realized after the positional relationship between them is established.

[0070] In the case that the pixelized image module divides the portrait into a plurality of sub-pictures of the same size and configures the position information, the portrait will be divided into a plurality of sub-pictures of the same size according to the preset division ratio. For example, a portrait of 30*30 size is divided into 9 sub-pictures of 10*10 size according to the division ratio. At the same time of dividing the portrait, the corresponding position identification information needs to be marked, which is 1 to 9 respectively. Therefore, when the jigsaw mechanical arm performs pixel block splicing, only the position identification information carried by each pixel block needs to be obtained, and then the splicing can be performed in sequence. Of course, the sequence here is not simply arranging the pixel blocks of 1 to 9 in a row, but according to the trained image splicing template. The image splicing template is the splicing according to the positional relationship of the sub-pictures during the previous image division.

[0071] The jigsaw mechanical arm in this embodiment can be one or two. If there are two, the two jigsaw mechanical arms are arranged oppositely and jointly complete the splicing of the pixel portrait pattern. For example, the jigsaw can be completed through the mode of alternating actions.

[0072] After the jigsaw mechanical arm completes the splicing of the pixel blocks, the pixel portrait pattern is transmitted to the display for display. If the customer is satisfied, the pixel portrait pattern can be printed out by the 3D printer.

[0073] Please refer to Figure 1 The side of the operation table 1 is provided with a maintenance opening 6 for facilitating the maintenance of the operation table. A protective fence 8 is arranged around the operation table 1 to prevent the operation table from being damaged due to collision. A plurality of columns 7 are further arranged on the operation table 1, which are used to support the ceiling 9, as shown in Figure 4 The ceiling 9 is embedded with white linear lamps on the surface facing the operation table 1 for illumination. In actual application, the entity of the operation table 1 is made of 1.2 cm thick satin stainless steel, the display 3 is a 75-inch liquid crystal display, the column 7 is a 100 mm diameter frosted black baked paint cylinder, and the protective fence 8 is a super-white tempered hot-bent laminated glass. This is a kind of scheme actually adopted, and of course other materials can be used to make the operation table, the column and the protective fence.

[0074] The embodiments of the present application are described in detail above with reference to the accompanying drawings, but the present application is not limited to the above-described embodiments. Even if various changes are made to the present application, if the changes fall within the scope of the present claims and their equivalents, they are still within the protective scope of the present application.

Claims

1. A photographing device for automatically splicing pixel blocks based on AI control of a mechanical arm, characterized in that, The application relates to a head portrait generation device. The device comprises: an operation table, on which a puzzle mechanical arm is installed, the puzzle mechanical arm automatically realizes pixel block splicing based on an AI algorithm control; a camera and a display are arranged above the operation table, a sound control device is arranged in the camera, after receiving a photographing instruction, the sound control device starts the camera to execute photographing; the image collected by the camera comprises a head portrait, the image is transmitted to the operation table, the operation table performs pixelization processing on the image, outputs a plurality of pixel blocks, drives the puzzle mechanical arm to splice the pixel blocks into a head portrait pattern according to a pixel splicing rule, and transmits the head portrait pattern to the display for display; 2.The photographing device based on the AI control of the mechanical arm and the automatic splicing pixel block according to claim 1, wherein a 3D printer is in communication connection with the operation table, receives the head portrait pattern sent by the operation table, and calls a preset 3D printing model to print and output the head portrait pattern. The operation table comprises a head portrait extraction module, a pixelized image module and a display screen; the head portrait extraction module is configured to acquire an image transmitted by the camera, extract a head portrait from the image based on an image extraction algorithm, and transmit the head portrait to the pixelized image module; 3.The photographing device based on the AI control of the mechanical arm for automatically splicing pixel blocks according to claim 2, wherein, the pixelized image module performs parallel pixelization processing on each pixel point in the head portrait and outputs a plurality of pixel blocks to the display screen. The pixelization processing of the head portrait by the pixelized image module comprises: 4.The photographing device based on the AI control of the mechanical arm and the automatic splicing pixel block according to claim 3, wherein, segmenting the head portrait into a plurality of subgraphs according to head portrait features, and performing parallel pixelization processing on the subgraphs. The parallel pixelization processing on the subgraphs comprises: determining the color value and the transparency value of each pixel point according to the pixel point information of the subgraph; After all the pixel points in the subgraph complete pixelization, the pixelized image is scaled to 1 / N 2 . 5.The photographing device based on the AI control of the mechanical arm for automatically splicing pixel blocks according to claim 3, wherein creating a pixel block of each pixel point, and setting the pixel points of the pixel block according to the color value and the transparency value of each pixel point; wherein the resolution of the pixel block is N*N, and N is an integer greater than or equal to 2; The parallel pixelization processing on the subgraphs comprises: determining the row pixel points and the column pixel points in the subgraph which are spaced by M pixel points, wherein M is an integer greater than or equal to 2; 6.The photographing device based on the AI control of the mechanical arm for automatically splicing pixel blocks, according to claim 1, wherein, adjusting the transparency values of the row pixel points and the column pixel points to preset transparency values to complete the pixelization processing. The puzzle mechanical arm splices the pixel blocks into a head portrait pattern according to a pixel splicing rule comprises: identifying the pixel blocks corresponding to each head portrait feature to obtain face pixel blocks, eye pixel blocks, mouth pixel blocks, nose pixel blocks and hair pixel blocks; 7.The photographing device based on the AI control of the mechanical arm for automatically splicing the pixel block, according to claim 6, wherein, splicing the face pixel blocks, the eye pixel blocks, the mouth pixel blocks, the nose pixel blocks and the hair pixel blocks based on an image splicing template to obtain a head portrait pattern. The identification of the pixel blocks corresponding to each head portrait feature comprises: establishing a plurality of head portrait feature template databases, wherein the head portrait feature template databases comprise a face shape template database, an eye shape template database, a mouth shape template database, a nose shape template database and a hair shape template database; creating and training a head portrait feature classification model based on the plurality of head portrait feature template databases; 8.The photographing device based on the AI control of the mechanical arm for automatically splicing pixel blocks according to claim 1, wherein, identifying the pixel blocks corresponding to each head portrait feature through the head portrait feature classification model. Two puzzle mechanical arms are installed on the operation table, the two puzzle mechanical arms are oppositely arranged, and the two puzzle mechanical arms jointly complete the splicing of the head portrait pattern.

Images