A robot painting path generation method and system based on image recognition

Abstract

The application discloses a kind of robot drawing path generation method and system based on image recognition, and drawing path generation method includes the following steps: S1, original bitmap image is preprocessed;S2, line drawing bitmap image is generated according to the original bitmap image after preprocessing;S3, vectorization processing is carried out to line drawing bitmap image, and stroke path is extracted;S4, the content of stroke path is perceived and simplified processing is carried out;S5, stroke path after simplification processing is converted into instruction set that robot can directly execute.Drawing path generation system includes image preprocessing module, artificial intelligence interface module, vectorization engine module and path optimizer module.The application combines artificial intelligence art creation, computer vision processing and robot path planning together, forms a complete, automated processing flow, not only focuses on processing flow in one isolated link, solves more fundamental, more challenging whole-process through problem.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and in particular relates to a method and system for generating robot painting paths based on image recognition. Background Technology

[0002] Converting digital images into paths that robots or plotters can execute is a long-standing challenge in the field of automation. Traditional solutions primarily rely on classic computer vision algorithms, such as edge detection algorithms like the Canny operator to extract image contours, and then converting these pixel contours into vector paths. The limitation of these methods is that they can only mechanically "trace" the physical edges of objects, lacking artistry and creativity, and cannot transform a rich photograph (such as a portrait) into an aesthetically pleasing, stylized line drawing.

[0003] With the rapid development of generative AI, numerous image processing techniques have emerged capable of converting photographs into different artistic styles (such as sketches, oil paintings, and cartoons); these techniques are often referred to as style transfer. However, the ultimate goal of these techniques is to generate a new bitmap image for human viewing. These AI-generated images often contain artistic brushstrokes, shadows, and impure lines, making them difficult to process effectively by traditional edge detection algorithms and therefore unsuitable for direct use in robot painting path generation. How to combine AI's artistic creation capabilities with the precise physical execution capabilities of robots to form an automated workflow is a gap in the current technological field.

[0004] The existing patent document, "A method and system for optimizing painting paths based on AI drawing" (publication number CN116630481A), is based on an iterative path generation method that approaches the target by repeatedly generating and comparing paths. However, it is limited to the AI ​​generation stage and does not provide a complete solution that starts from any image.

[0005] Existing patent document 2, "An AI-based drawing path optimization system and method" (publication number CN119205951A), is based on a post-path optimization method. While it improves AI drawing efficiency by optimizing and filtering path information, it cannot preserve the artistic details of the artwork. Summary of the Invention

[0006] The purpose of this invention is to solve the above-mentioned technical problems existing in the prior art, and to provide a robot painting path generation method and system based on image recognition. This method combines artificial intelligence art creation, computer vision processing and robot path planning to form a complete and automated processing flow. It does not only focus on a single isolated link in the processing flow, but also solves the more fundamental and challenging problem of the entire process being connected.

[0007] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0008] A method for generating robot painting paths based on image recognition, characterized by the following steps:

[0009] S1. Preprocess the original bitmap image;

[0010] S2. Generate a line drawing bitmap image based on the preprocessed original bitmap image in S1;

[0011] S3. Vectorize the line drawing bitmap image in S2 and extract the stroke paths;

[0012] S4. Perform content-aware simplification on the stroke paths in S3:

[0013] (4.1) Locate the key feature regions in the line drawing bitmap image in S2;

[0014] (4.2) Set a first simplification intensity for stroke paths that fall within the key feature area in (4.1) and set a second simplification intensity for stroke paths that fall outside the key feature area in (4.1). The first simplification intensity is less than the second simplification intensity.

[0015] S5. Convert the simplified stroke path in S4 into an instruction set that the robot can directly execute.

[0016] A robot painting path generation system based on image recognition, characterized in that it includes: an image preprocessing module, an artificial intelligence interface module, a vectorization engine module, and a path optimizer module;

[0017] The image preprocessing module is used to adjust the size of the bitmap image and output the resized bitmap image to the artificial intelligence interface module.

[0018] The artificial intelligence interface module is used to call the AI-based image generation model. This model generates a line drawing bitmap image based on the bitmap image output by the image preprocessing module, and outputs the line drawing bitmap image to the vectorization engine module and the path optimizer module.

[0019] The vectorization engine module is used to convert the line drawing bitmap image output by the artificial intelligence interface module into a set of stroke paths;

[0020] The path optimizer module is used to simplify the stroke paths and output the simplified stroke paths.

[0021] The present invention, by adopting the above-described technical solution, has the following beneficial effects:

[0022] 1. High degree of automation: It realizes the fully automatic conversion from any image to the final robot-executable drawing path without human intervention.

[0023] 2. Combining artistry and feasibility: It ensures the artistic beauty of the generated work, while ensuring the feasibility and efficiency of the path through subsequent processing.

[0024] 3. Improved the quality of finished paintings: By identifying and protecting key facial feature areas during the path optimization stage, a differentiated and focused simplification strategy was implemented, avoiding oversimplification that would lose the charm of the portrait. This significantly reduced the amount of data while ensuring the artistic effect of the finished painting.

[0025] 4. This invention combines artificial intelligence art creation, computer vision processing, and robot path planning to form a complete and automated processing flow. It does not only focus on a single isolated link in the processing flow (such as "how to generate a path" or "how to optimize a path"), but also solves the more fundamental and challenging problem of connecting the entire process. Attached Figure Description

[0026] The present invention will be further described below with reference to the accompanying drawings:

[0027] Figure 1 This is a flowchart of the robot drawing path generation method in this invention;

[0028] Figure 2 This is a schematic diagram of the robot drawing path generation system in this invention;

[0029] Figure 3 This is a detailed flowchart of the robot drawing path generation method in this invention;

[0030] Figure 4 This is a flowchart of stroke path optimization in this invention. Detailed Implementation

[0031] like Figure 1 and Figure 3 As shown, this invention provides a method for generating robot painting paths based on image recognition, comprising the following steps:

[0032] S1. Receive an original bitmap image uploaded by the user, such as a portrait photo. The image preprocessing module scales the original bitmap image to achieve preprocessing and obtain a bitmap image of standardized size, such as adjusting the original bitmap image to 1024×1024 pixels to meet the size requirements of the subsequent image generation AI model.

[0033] The image preprocessing module outputs the preprocessed bitmap image to the artificial intelligence interface module.

[0034] S2, the artificial intelligence interface module sends the preprocessed original bitmap image from S1 and the pre-designed Prompt text prompt to an external or locally deployed image generation AI model. The image generation AI model uses the multimodal understanding capabilities of generative AI to generate a line drawing bitmap image focusing on the portrait and with a specified artistic style, and outputs the line drawing bitmap image to the vectorization engine module and the path optimizer module.

[0035] Among them, the image generation AI model is an image generation model based on the Transformer architecture, and the Prompt text prompt can be "a clean single-line black and white sketch, minimalist style".

[0036] The S3 vectorization engine module performs vectorization processing on the line art bitmap image in S2, as follows:

[0037] (3.1) Binarization: Convert the line drawing bitmap image into a black and white image;

[0038] (3.2) Skeletonization: The Zhang-Suen thinning algorithm is used to thin the lines in the line drawing bitmap image into single-pixel width lines, forming a clear skeleton. This step is a key prerequisite for ensuring the uniqueness and accuracy of subsequent stroke path extraction.

[0039] (3.3) Path tracing: Traverse all black pixels and extract several independent stroke paths composed of coordinate point sequences by connecting adjacent pixels, forming a stroke path set.

[0040] Although the line art bitmap generated by the image generation AI model meets artistic requirements, it is still a bitmap, and the lines in it are uneven in thickness or contain noise, so it cannot be directly used by robots that require precise vector coordinates. The vectorization engine module converts the lines in the line art bitmap image into vector paths.

[0041] The S4 path optimizer module receives the line drawing bitmap image from S2 and the set of stroke paths from S3. It uses a content-aware simplification algorithm to simplify the stroke paths in S3, thereby reducing the number of coordinate points in the stroke paths and solving the problem that traditional simplification algorithms cannot distinguish the importance of paths.

[0042] The content-aware simplification algorithm includes a key region detection algorithm and a path simplification algorithm. The key region detection algorithm includes at least one of face recognition algorithm, object detection algorithm, and image saliency detection algorithm. In this embodiment, the key region detection algorithm adopts a face recognition algorithm. The face recognition algorithm is a facial feature point detection algorithm based on machine learning, which is used to locate key regions of the face such as eyes, eyebrows, nose, lips, and contours. It belongs to the prior art and will not be described in detail here.

[0043] Simplified processing steps as follows Figure 4 As shown:

[0044] (4.1) Run the facial feature point detection algorithm once using the face recognition algorithm to detect the coordinates of 68 feature points of the face in the line drawing bitmap image in S2. Based on the detected feature point coordinates, calculate and locate the key facial features such as eyes, eyebrows, nose, and lips, and define these key facial features as protected areas.

[0045] (4.2) When simplifying stroke paths, the path simplification algorithm is used to set the first simplification intensity for stroke paths that fall within the key facial features area in (4.1) and the second simplification intensity for stroke paths that fall outside the key facial features area in (4.1). The first simplification intensity is less than the second simplification intensity.

[0046] Stroke paths falling within key feature regions are simplified with low intensity to preserve details to the maximum extent; stroke paths falling outside key feature regions are simplified with high intensity to significantly compress the data. Since the number of stroke path points after vectorization is still too large, further optimization is needed to improve the efficiency of robot drawing.

[0047] The path simplification algorithm is an algorithm that reduces the number of points constituting the curve while preserving the original shape features as much as possible. It is an existing technology and will not be elaborated here.

[0048] (4.3) The stroke paths that fall outside the key facial features area in (4.1) are classified into linear paths and curved paths. A third simplification intensity is set for linear paths and a fourth simplification intensity is set for curved paths. The third simplification intensity is greater than the fourth simplification intensity, so as to achieve more refined optimization of stroke paths outside the protected area.

[0049] The classification method for stroke paths includes a combination of one or more of the following calculation steps:

[0050] a. Calculate the total curvature: Traverse each non-endpoint coordinate point in the stroke path, calculate the angle between the two vectors formed by the non-endpoint coordinate point and the two coordinate points before and after it, and sum or average all the calculated angles; if the calculation result exceeds the first preset threshold, it indicates that the stroke path has significant curvature characteristics and classifies the stroke path as a curved path; otherwise, the stroke path is classified as a linear path.

[0051] b. Calculate the direction change: Convert the stroke path into several pairs of vector segments connected end to end, and calculate the change in direction angle between each pair of adjacent vector segments; if the average or standard deviation of all calculated direction angle changes exceeds the second preset threshold, it indicates that the stroke path has rich direction changes and the stroke path is classified as a curved path; otherwise, the stroke path is classified as a linear path.

[0052] c. Calculate the line fitting error: Perform a least squares linear fitting on all coordinate points on the stroke path to obtain the best-fitting line. Then calculate the vertical distance from each coordinate point on the stroke path to the best-fitting line, and calculate the average of all vertical distances as the fitting error. If the fitting error exceeds the third preset threshold, it indicates that the stroke path deviates from the straight line and the stroke path is classified as a curved path. Otherwise, the stroke path is classified as a linear path.

[0053] This step, through a differentiated simplification strategy, effectively preserves the spirit of the portrait while reducing the number of data points by an average of 40%-60%.

[0054] Steps S3 and S4 bridge the technological gap between AI-generated artistic creation and precise robotic execution, achieving a better balance between preserving artistic details and improving work efficiency compared to traditional image processing methods.

[0055] S5. For each stroke path after simplification in S4: before the stroke, add the first coordinate point as the pen lifting instruction; then the entire stroke path is used as the pen ending instruction; after the stroke is completed, add the last coordinate point as the pen lifting instruction.

[0056] The process of generating control commands is as follows:

[0057] Traverse each simplified stroke path in S4 to generate a set of instructions that the robot can directly execute: "lift pen - move - place pen - draw - lift pen". The steps are as follows:

[0058] (5.1) Obtain the first coordinate point of the current stroke path and combine it with the Z-axis coordinate of the preset pen lifting height to generate a starting three-dimensional path point as the starting position of the pen and obtain the pen lifting start command.

[0059] The pen-lifting start command is used to control the robot to move the lifted pen to the starting position.

[0060] (5.2) Traverse all coordinate points in the current stroke path, combine each coordinate point with the Z-axis coordinate of the preset stroke height to generate continuous three-dimensional path points, which serve as the drawing path of the brush, and obtain the stroke drawing instruction.

[0061] The "Place Pen" drawing command controls the robot to place the pen and move it along the drawing path to complete the drawing action.

[0062] (5.3) Obtain the last coordinate point of the current stroke path and combine it with the Z-axis coordinate of the preset pen lifting height to generate a final three-dimensional path point as the final position of the pen and obtain the final pen lifting command.

[0063] The final pen-lifting command controls the robot to lift the pen after completing the drawing action, preparing it to move to the starting position of the next stroke path.

[0064] By adding path points indicating the lifting of the pen before and after each independent stroke path, and maintaining the pen-falling state in the middle of the drawing path, the robot is able to complete complex paintings discretely and discontinuously, thus accurately replicating the human behavior of painting segment by segment.

[0065] The system performs proportional scaling and coordinate system transformation on the coordinate points of all stroke paths. This process uses a linear interpolation algorithm or a range mapping algorithm. For each two-dimensional pixel (px, py), the system obtains the following parameters according to a preset configuration file:

[0066] a. Source coordinate system range: the width and height of the image, such as [0, 1024] pixels;

[0067] b. Target coordinate system range: The physical drawing area boundary of the robot workbench, such as X-axis [-150, 150] mm, Y-axis [100, 400] mm.

[0068] First, calculate the relative position of each 2D pixel on the X and Y axes of the source coordinate system, such as (px / image_width, ranging from 0 to 1) and (py / image_height, ranging from 0 to 1). Then, multiply the calculated relative position by the physical width of the target coordinate system and add the origin offset of the target coordinate system to calculate the precise coordinates (mx, my) of the 2D pixel in the robot's physical space coordinate system. This process applies to all pixels, ensuring proportional scaling and accurate positioning of the overall graphic.

[0069] Ultimately, an efficient and aesthetically pleasing drawing path data containing precise coordinates and robot motion instructions is generated, which can be sent to the robot control system to perform physical drawing.

[0070] The purpose of coordinate system transformation is to accurately map the two-dimensional coordinate points extracted from the image, which are in "pixels", to a physical space coordinate system that the robot can understand, which is in "millimeters" or "meters". This ensures that the size, scale and position of the final drawn graphic in the physical world are as expected.

[0071] like Figure 2 As shown, a robot painting path generation system based on image recognition includes: an image preprocessing module, an artificial intelligence interface module, a vectorization engine module, and a path optimizer module.

[0072] The image preprocessing module is used to adjust the size of the bitmap image and output the resized bitmap image to the artificial intelligence interface module.

[0073] The artificial intelligence interface module is used to call the AI-based image generation model. This model generates a line drawing bitmap image based on the bitmap image output by the image preprocessing module, and outputs the line drawing bitmap image to the vectorization engine module and the path optimizer module.

[0074] The vectorization engine module is used to convert the line drawing bitmap image output by the artificial intelligence interface module into a set of stroke paths.

[0075] The path optimizer module integrates a feature region recognizer. The feature region recognizer uses a key region detection algorithm to locate key feature regions in the line drawing bitmap image. It sets different simplification intensities for stroke paths within and outside the key feature regions, and outputs the simplified stroke paths.

[0076] The robot control system communicates with the drawing path generation system of the present invention via the WebSocket protocol. The robot control system receives and parses the control command data sent by the drawing path generation system and drives the robot to perform actions.

[0077] The drawing path generation system of this invention can be implemented on one or more servers. The above-mentioned functional modules can be independent software services or different functional components within a unified application. They work together to complete the path generation method of this invention.

[0078] The above are merely specific embodiments of the present invention, but the technical features of the present invention are not limited thereto. Any simple changes, equivalent substitutions, or modifications made based on the present invention to solve essentially the same technical problems and achieve essentially the same technical effects are all covered within the protection scope of the present invention.

Claims

1. A method for generating robot painting paths based on image recognition, characterized in that, Includes the following steps: S1. Preprocess the original bitmap image; S2. Generate a line drawing bitmap image based on the preprocessed original bitmap image in S1; S3. Vectorize the line drawing bitmap image in S2 and extract the stroke paths; S4. Perform content-aware simplification on the stroke paths in S3: (4.1) Locate the key feature regions in the line drawing bitmap image in S2; (4.2) Set a first simplification intensity for stroke paths that fall within the key feature area in (4.1) and set a second simplification intensity for stroke paths that fall outside the key feature area in (4.1). The first simplification intensity is less than the second simplification intensity. (4.3) Classify stroke paths that fall outside the key feature region in (4.1) into linear paths and curved paths. Set a third simplification intensity for linear paths and a fourth simplification intensity for curved paths. The third simplification intensity is greater than the fourth simplification intensity. S5. Convert the simplified stroke path in S4 into an instruction set that the robot can directly execute; For each stroke path after simplification: before the stroke, add the first coordinate point as the pen lifting instruction; then the entire stroke path is used as the pen ending instruction; after the stroke is completed, add the last coordinate point as the pen lifting instruction. The process of generating control commands is as follows: Traverse each simplified stroke path in S4 to generate a set of instructions that the robot can directly execute. The steps are as follows: (5.1) Obtain the first coordinate point of the current stroke path and combine it with the Z-axis coordinate of the preset pen lifting height to generate a starting three-dimensional path point as the starting position of the pen and obtain the pen lifting start command. The pen-lifting start command is used to control the robot to move the lifted pen to the starting position; (5.2) Traverse all coordinate points in the current stroke path, combine each coordinate point with the Z-axis coordinate of the preset stroke height to generate continuous three-dimensional path points, which serve as the drawing path of the brush, and obtain the stroke drawing instruction. The pen-dropping drawing command is used to control the robot to put down the pen and move along the drawing path to complete the drawing action; (5.3) Obtain the last coordinate point of the current stroke path and combine it with the Z-axis coordinate of the preset pen lifting height to generate a final three-dimensional path point as the final position of the pen and obtain the final pen lifting command. The final pen-lifting command is used to control the robot to lift the pen after completing the drawing action.

2. The method for generating robot painting paths based on image recognition according to claim 1, characterized in that: Step S3 specifically involves: performing skeletonization processing on the line art bitmap image in S2, refining the lines in the line art bitmap image into single-pixel width lines, and then performing path tracing to extract the stroke paths.

3. The method for generating robot painting paths based on image recognition according to claim 1, characterized in that: In step S4, a content-aware simplification algorithm is used to simplify the stroke paths in S3. The content-aware simplification algorithm includes a key region detection algorithm and a path simplification algorithm. The simplification steps are as follows: (4.1) Use the key region detection algorithm to locate the key feature regions in the line drawing bitmap image in S2; (4.2) Use the path simplification algorithm to set a first simplification intensity for stroke paths that fall within the key feature area in (4.1) and a second simplification intensity for stroke paths that fall outside the key feature area in (4.1). The first simplification intensity is less than the second simplification intensity.

4. The method for generating robot painting paths based on image recognition according to claim 3, characterized in that: The key region detection algorithm includes at least one of the following: face recognition algorithm, object detection algorithm, and image saliency detection algorithm.

5. The method for generating robot painting paths based on image recognition according to claim 1, characterized in that: Step S1 specifically involves: receiving the original bitmap image uploaded by the user, scaling the original bitmap image to obtain a bitmap image of standardized size.

6. The method for generating robot painting paths based on image recognition according to claim 1, characterized in that: Step S2 specifically involves calling an AI-based image generation model, which generates a line drawing bitmap image based on the preprocessed original bitmap image from S1.

7. A painting path generation system for the robot painting path generation method based on image recognition as described in claim 1, characterized in that, include: Image preprocessing module, artificial intelligence interface module, vectorization engine module, and path optimizer module; The image preprocessing module is used to adjust the size of the bitmap image and output the resized bitmap image to the artificial intelligence interface module; The artificial intelligence interface module is used to call an artificial intelligence-based image generation model, which generates a line drawing bitmap image based on the bitmap image output by the image preprocessing module, and outputs the line drawing bitmap image to the vectorization engine module and the path optimizer module. The vectorization engine module is used to convert the line drawing bitmap image output by the artificial intelligence interface module into a set of stroke paths; The path optimizer module is used to simplify the stroke path and output the simplified stroke path.

8. The painting path generation system according to claim 7, characterized in that: The path optimizer module locates key feature regions in the line drawing bitmap image through a feature region recognizer, and sets different simplification intensities for stroke paths within and outside the key feature regions.

9. The painting path generation system according to claim 8, characterized in that: The feature region recognizer uses a key region detection algorithm to locate key feature regions in the line drawing bitmap image.

Images