Intelligent paint spraying method and device based on visual recognition and electronic equipment

Abstract

The invention discloses an intelligent paint spraying method and device based on visual recognition and electronic equipment, and relates to the field of image recognition. The method comprises the following steps: acquiring an image of a to-be-painted conduit; the image is segmented, and the contour and position coordinates of the guide pipe to be painted are recognized; a key point detection algorithm is adopted to determine the space direction of the to-be-painted guide pipe; according to the contour, the position coordinates and the space direction, a three-dimensional parameterized model of the to-be-painted guide pipe is generated; and according to the three-dimensional parameterized model, a nozzle movement track is generated, and a mechanical arm is controlled to spray paint according to the nozzle movement track. By implementing the technical scheme provided by the invention, the paint spraying efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of image recognition, and specifically to an intelligent painting method, device and electronic equipment based on visual recognition. Background Art

[0002] With the rapid development of industrial automation and precision manufacturing technologies, the requirements for production efficiency and product quality in industrial production lines continue to increase. In the automotive, aviation, and heavy industry sectors, intricate parts such as catheters require precise and uniform painting. These workpieces often have complex geometries and diverse materials, placing higher demands on the precision and adaptability of the painting process. Traditional painting technology has struggled to meet these high-efficiency and high-quality production demands, especially when processing large quantities and a wide variety of workpieces.

[0003] At present, existing painting technologies mainly rely on preset programs and manual settings, which are feasible when dealing with standardized and mass-produced products, but these methods are powerless when faced with personalized and variable-shaped workpieces. First, traditional painting technology requires manual positioning and trajectory planning of the workpiece, which is not only time-consuming and labor-intensive, but also easily leads to low painting efficiency. Secondly, fixed painting procedures lack flexibility. For workpieces with complex shapes or different sizes, it is difficult to achieve precise coverage. Different programs are often required according to the different types of workpieces, which greatly reduces production efficiency. Therefore, the traditional method has the problem of low painting efficiency.

[0004] Therefore, there is an urgent need for an intelligent painting method, device and electronic equipment based on visual recognition. Summary of the Invention

[0005] The present application provides an intelligent painting method, device and electronic equipment based on visual recognition, which improves the painting efficiency.

[0006] In a first aspect of the present application, an intelligent painting method based on visual recognition is provided, which includes: acquiring an image of a conduit to be painted; segmenting the image to identify the contour and position coordinates of the conduit to be painted; determining the spatial direction of the conduit to be painted using a key point detection algorithm; generating a three-dimensional parametric model of the conduit to be painted based on the contour, the position coordinates, and the spatial direction; generating a nozzle motion trajectory based on the three-dimensional parametric model, and controlling a robotic arm to spray paint according to the nozzle motion trajectory.

[0007] By employing the above technical solution, an image of the catheter to be painted is captured, segmented to identify the catheter's outline and position coordinates, and then a key point detection algorithm is used to determine the catheter's spatial orientation. This allows accurate acquisition of the catheter's geometric information and spatial posture. Based on this, a three-dimensional parametric model of the catheter is generated, and the nozzle's motion trajectory is planned accordingly, controlling the robotic arm for automated painting. This method uses visual recognition technology to obtain precise information about the catheter, ensuring a targeted and adaptable spraying process. The integration of the catheter's geometry and posture information into a three-dimensional parametric model provides a unified data foundation for trajectory planning and process control. Automatically generating the nozzle's motion path based on the three-dimensional parametric model shortens production preparation time, improves spraying consistency, and helps ensure efficient mass production.

[0008] Optionally, generating the nozzle motion trajectory according to the three-dimensional parametric model specifically includes: constructing a grid model of the conduit to be painted based on the three-dimensional parametric model; evenly arranging multiple paint path planning seed points on the grid model; taking each of the paint path planning seed points as a starting point, using the Dijkstra algorithm to search for the shortest path in the grid model to generate the nozzle motion trajectory.

[0009] By adopting the above technical solution, a mesh model of the catheter is constructed based on a three-dimensional parametric model, and the complex curved surface is converted into regular grid units. By evenly arranging multiple path planning seed points on the grid model, full coverage and uniform distribution of the spray path can be achieved, avoiding omissions and duplications. The shortest path is searched using the Dijkstra algorithm with the spray path planning seed point as the starting point, thereby obtaining the globally optimal motion trajectory and improving the spraying efficiency. Grid modeling simplifies the path planning problem, and the application of the Dijkstra algorithm ensures the efficiency and smoothness of the motion trajectory. This path planning method achieves the optimal spraying trajectory within the range of motion and constraints of the robot arm by optimizing seed point selection and path search, reducing energy consumption and time consumption during the spraying process, and improving production efficiency and product quality.

[0010] Optionally, the evenly arranging multiple paint path planning seed points on the grid model specifically includes: dividing the grid model into multiple grid units of equal size; calculating the center point coordinates of each of the grid units; and using the center point coordinates as the paint path planning seed point to obtain multiple evenly distributed paint path planning seed points.

[0011] By adopting the above technical solution, the seed points for paint path planning are evenly arranged on the surface of the catheter by means of grid division. By dividing the grid model into grid units of equal size, the regularity and uniformity of the seed point distribution are ensured. Using the center point coordinates of the grid unit as the seed point avoids the seed points being too dense or sparse, which not only reduces the amount of calculation but also ensures the integrity of the path coverage. This method of grid division and seed point selection is simple and efficient. The density and distribution of the seed points can be flexibly controlled by adjusting the size of the grid unit. Evenly distributed seed points are conducive to generating balanced and regular spray trajectories, improving the efficiency of trajectory planning and the consistency of the spraying effect. At the same time, grid division simplifies the coordinate calculation of the seed point to the calculation of the grid center point, reducing the computational complexity.

[0012] Optionally, the image is segmented to identify the outline and position coordinates of the conduit to be painted, specifically including: extracting edge pixels in the image; using a region growing algorithm to divide the image into multiple regions based on the connectivity of the edge pixels; calculating the geometric parameters of each of the regions; comparing the geometric parameters with a preset parameter threshold to obtain the target region where the conduit to be painted is located; fitting the boundary pixels of the target region to extract the outline of the conduit to be painted; calculating multiple minimum circumscribed rectangles of the outline, and determining the position coordinates of the conduit to be painted in the image based on the minimum circumscribed rectangles.

[0013] By employing the above technical solution, the catheter's outline and position coordinates can be accurately segmented and extracted from the image. By extracting the image's edge pixels, the catheter's boundaries can be preliminarily identified. A region growing algorithm is used to segment the image based on the connectivity of the edge pixels, distinguishing between the foreground catheter and the background. By calculating the geometric parameters of each segmented region and comparing them with a preset parameter threshold, the target region where the catheter is located can be automatically identified, avoiding the errors and interference of manual positioning. Fitting the target region's boundary pixels yields a smooth, regular catheter outline. Calculating the minimum bounding rectangle of the outline directly determines the catheter's position coordinates in the image, providing a precise reference for subsequent processing.

[0014] Optionally, the use of a key point detection algorithm to determine the spatial direction of the conduit to be painted specifically includes: obtaining multi-perspective images of the conduit to be painted, the multi-perspective images including a front view, a side view and a top view; extracting key points from the multi-perspective images, and calculating the size-invariant feature vectors of each of the key points; determining matching key points from the key points based on the size-invariant feature vectors; calculating the spatial coordinate transformation relationship of the matching key points, and establishing the three-dimensional coordinates of the surface points of the conduit to be painted based on the spatial coordinate transformation relationship and the camera imaging model; fitting the three-dimensional coordinates to obtain a straight line equation, which represents the direction of the central axis of the conduit to be painted; and using the direction of the central axis as the spatial direction of the conduit to be painted.

[0015] By adopting the above technical solution, the spatial posture of the catheter is determined through key point detection and matching. By collecting catheter images from multiple angles, such as front, side, and top views, complete and three-dimensional visual information can be obtained. Extracting key points from multi-view images and calculating their scale-invariant features can achieve matching between images. By matching feature vectors, the corresponding key points of images from different perspectives are found, and then the three-dimensional spatial coordinates of these key points are calculated through coordinate transformation and camera model. The straight line obtained by fitting the spatial coordinate points is the central axis of the catheter, reflecting the direction of the catheter in space. Using the axis direction as the spatial direction of the catheter can accurately characterize the posture of the catheter. This method cleverly utilizes multi-view information to resolve the uncertainty of posture estimation under a single perspective. By triangulating key points, high-precision calculation of three-dimensional direction is achieved, avoiding the complex calculations caused by feature recognition and fitting, and providing a reliable direction reference for catheter modeling.

[0016] Optionally, generating a three-dimensional parametric model of the conduit to be painted based on the contour, the position coordinates and the spatial direction specifically includes: performing equally spaced sampling on the contour to obtain multiple contour sampling points; constructing three-dimensional point cloud data based on the multiple contour sampling points, and generating a three-dimensional parametric model of the conduit to be painted based on the three-dimensional point cloud data.

[0017] By employing this technical solution, evenly spaced contour sampling can reduce data volume and improve modeling efficiency while preserving shape characteristics. The 3D point cloud data constructed from the sampling points accurately reflects the spatial distribution and shape characteristics of the catheter. The generated 3D parametric model, based on the 3D point cloud data, has a regular topological structure and continuous surface shape, making it easy to store, edit, and render. The 3D parametric model describes the 3D morphology of the catheter through mathematical equations, facilitating subsequent operations.

[0018] Optionally, after generating the nozzle motion trajectory according to the three-dimensional parametric model and controlling the robotic arm to spray paint according to the nozzle motion trajectory, the method further includes: during the painting process, obtaining an image of the spraying area captured by a visual sensor; performing similarity calculation between the spraying area image and a preset spraying image to determine whether the spraying quality is qualified; if it is determined that the spraying quality is unqualified, sending a prompt message, wherein the prompt message is used to prompt the staff that the spraying quality is unqualified.

[0019] By employing the above-mentioned technical solution, the integrity, uniformity, and accuracy of the spraying process are assessed by acquiring an image of the sprayed area and comparing it to a pre-set spray image. Similarity calculations employ image processing and pattern recognition algorithms. By extracting features and matching them to measure the differences between the sprayed image and the standard image, the degree of spraying quality can be comprehensively and objectively determined. Once an unqualified spraying defect is detected, the system automatically issues an alert, prompting the operator to address it promptly and prevent the release of defective products. This method utilizes machine vision to achieve online inspection and quality control of the spraying process, overcoming the lag and limitations of manual spot checks and enabling early detection, early warning, and early resolution of defects. By moving product quality supervision forward into the production process, it reduces rework and scrap, and improves production efficiency and yield rates. Real-time collection and analysis of spraying data also provides a basis for process optimization and equipment maintenance, promoting intelligent and refined management of spraying production.

[0020] In a second aspect of the present application, an intelligent paint spraying device based on visual recognition is provided, which includes: an acquisition module and a processing module, wherein: the acquisition module is used to acquire an image of a conduit to be painted; the processing module is used to segment the image and identify the contour and position coordinates of the conduit to be painted; the processing module is also used to determine the spatial direction of the conduit to be painted using a key point detection algorithm; the processing module is used to generate a three-dimensional parametric model of the conduit to be painted based on the contour, the position coordinates and the spatial direction; the processing module is used to generate a nozzle motion trajectory based on the three-dimensional parametric model, and control the robotic arm to spray paint according to the nozzle motion trajectory.

[0021] In the third aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device performs any of the methods described above.

[0022] In a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores instructions. When the instructions are executed, any one of the methods described above is executed.

[0023] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. By capturing an image of the catheter to be painted, the image is segmented to identify the catheter's outline and position coordinates. A key point detection algorithm is then used to determine the catheter's spatial orientation, accurately capturing the catheter's geometric information and spatial posture. This model is then used to generate a 3D parametric model of the catheter, which is then used to plan the nozzle's motion trajectory and control the robotic arm for automated painting. This method uses visual recognition technology to obtain precise information about the catheter, ensuring a targeted and adaptable spraying process. The integration of the catheter's geometry and posture information into a 3D parametric model provides a unified data foundation for trajectory planning and process control. Automatically generating the nozzle's motion path based on the 3D parametric model shortens production preparation time, improves spraying consistency, and helps ensure efficient mass production.

[0024] 2. Construct a mesh model of the catheter based on a three-dimensional parametric model, and convert complex curved surfaces into regular grid units. Evenly arranging multiple path planning seed points on the grid model can achieve full coverage and uniform distribution of the spray path, avoiding omissions and duplications. Using the spray path planning seed point as the starting point, the Dijkstra algorithm is used to search for the shortest path, thereby obtaining the globally optimal motion trajectory and improving spraying efficiency. Grid modeling simplifies the path planning problem, and the use of the Dijkstra algorithm ensures the efficiency and smoothness of the motion trajectory. This path planning method achieves the optimal spraying trajectory within the range of motion and constraints of the robotic arm by optimizing seed point selection and path search, reducing energy and time consumption during the spraying process, and improving production efficiency and product quality.

[0025] 3. Use grid division to evenly arrange the seed points for spray path planning on the surface of the catheter. By dividing the grid model into grid units of equal size, the regularity and uniformity of the seed point distribution are ensured. Using the center point coordinates of the grid unit as the seed point avoids the seed points being too dense or sparse, which not only reduces the amount of calculation but also ensures the integrity of the path coverage. This method of grid division and seed point selection is simple and efficient. By adjusting the size of the grid unit, the density and distribution of the seed points can be flexibly controlled. Evenly distributed seed points are conducive to generating balanced and regular spray trajectories, improving the efficiency of trajectory planning and the consistency of the spraying effect. At the same time, grid division simplifies the coordinate calculation of the seed point to the calculation of the grid center point, reducing the computational complexity. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a flow chart of an intelligent painting method based on visual recognition disclosed in an embodiment of the present application; Figure 2 This is a module diagram of an intelligent paint spraying device based on visual recognition disclosed in an embodiment of the present application; Figure 3 This is a structural diagram of an electronic device disclosed in an embodiment of the present application.

[0027] Description of the accompanying drawings: 201, acquisition module; 202, processing module; 300, electronic device; 301, processor; 302, communication bus; 303, user interface; 304, network interface; 305, memory. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0029] In the description of the embodiments of this application, words such as "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "for example" or "for instance" in the embodiments of this application should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "for example" or "for instance" is intended to present the relevant concepts in a concrete manner.

[0030] In the description of the embodiments of the present application, the term "multiple" means two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0031] This application provides an intelligent painting method based on visual recognition, referring to Figure 1 , Figure 1This is a flow chart of a method for intelligent painting based on visual recognition provided by an embodiment of the present application. The method is applied to a server, which is a server that executes an intelligent painting program based on visual recognition. The server can be a single server, a server cluster consisting of multiple servers, or a cloud computing service center. The method includes steps S101 to S105, which are as follows: Step S101: Acquire an image of a catheter to be painted.

[0032] In step S101, the tube to be painted is suspended at a workstation and secured with magnets. The server communicates with a visual sensor at the workstation to acquire images of the tube. The visual sensor can be an industrial camera, a video camera, or other imaging device, installed at various locations and angles on the workstation to capture comprehensive image information of the tube. First, the server sends an image acquisition command to the workstation, activating the visual sensor. The visual sensor captures multi-view images of the tube to be painted in real time and uploads the images to the server. After receiving the raw image data, the server performs preprocessing. Preprocessing operations include image denoising, enhancement, and correction to improve image quality and eliminate the effects of ambient lighting, lens distortion, and other factors. Image denoising methods can include median filtering and wavelet transform; image enhancement methods can include histogram equalization and the Retinex algorithm; and image correction methods can include perspective transformation and radial correction. The preprocessed images are temporarily stored in the server's memory or hard disk for subsequent analysis and processing.

[0033] For example, assume four industrial cameras are installed above a workstation, one in front of, one behind, one to the left, and one to the right of a duct. Each camera has a resolution of 12 megapixels and is connected to a server via a gigabit network. When the server issues a capture command, the four cameras are triggered synchronously, capturing four high-definition images of the duct being painted. These images are then transmitted to the server via the network. The server performs Gaussian filtering and histogram equalization on each of the four images. The server then renames the images to front.jpg, right.jpg, back.jpg, and left.jpg in a clockwise direction from the viewing angle, stores them in folders, and writes the image metadata to a database.

[0034] Step S102: Segment the image to identify the outline and position coordinates of the conduit to be painted.

[0035] In step S102, the image is segmented to identify the outline and position coordinates of the conduit to be painted, specifically including: extracting edge pixels in the image; using a region growing algorithm to divide the image into multiple regions based on the connectivity of the edge pixels; calculating the geometric parameters of each region; comparing the geometric parameters with preset parameter thresholds to obtain the target region where the conduit to be painted is located; fitting the boundary pixels of the target region to extract the outline of the conduit to be painted; calculating multiple minimum bounding rectangles of the outline, and determining the position coordinates of the conduit to be painted in the image based on the minimum bounding rectangles.

[0036] Specifically, the server uses edge detection algorithms, such as the Canny algorithm and the Sobel algorithm, to extract edge pixels from the image. Edge pixels are pixels with large changes in image grayscale values, and often correspond to the outlines and details of objects. By setting a preset pixel threshold, the server can filter out most of the background noise and retain only the edge outline of the catheter to be painted. Next, the server uses a region growing algorithm to divide the image into multiple regions based on the connectivity of the edge pixels. Starting from a seed pixel, the server uses the region growing algorithm to continuously merge adjacent similar pixels into the same area until it can no longer expand. In this way, the image is divided into several connected pixel blocks, each of which represents an independent object or background area.

[0037] To determine which area is the catheter to be painted, the server calculates the geometric parameters of each area, such as area, perimeter, circularity, and aspect ratio. These geometric parameters reflect the size and shape characteristics of the object. The server compares the calculated geometric parameters with preset parameter thresholds to select the target area most likely to be the catheter. The preset parameter thresholds are empirically determined based on the standard size and shape of the catheter and actual processing errors. After determining the target area, the server fits the boundary pixels to extract the precise outline of the catheter to be painted. Contour fitting methods include Hough transform and least squares method. The fitted outline consists of a series of connected pixel points or line segments, which can effectively describe the catheter's shape.

[0038] Finally, the server calculates the outline's circumscribed rectangle—the smallest rectangle that precisely encloses the outline. The coordinates of the rectangle's four vertices represent the location of the catheter to be painted in the image. For example, suppose the server extracts a closed circular edge contour from the front view with an area of ​​10,000 square pixels, a perimeter of 400 pixels, a circularity of 0.8, and an aspect ratio of 2.5. These parameters are very close to the preset thresholds, so the server determines this area as the target catheter area for painting. The server then uses the least squares method to fit a smooth curve as the catheter's outline. The curve consists of 50 pixels connected in sequence to form a closed loop. Next, the server calculates the minimum circumscribed rectangle of the outline and obtains the coordinates of the four vertices (100, 200), (700, 200), (100, 600), and (700, 600), indicating that the horizontal position of the tube to be painted in the image is between 100 and 700 pixels, the vertical position is between 200 and 600 pixels, the direction is parallel to the edge of the image, the width is approximately 600 pixels, and the height is approximately 400 pixels.

[0039] Step S103: using a key point detection algorithm to determine the spatial orientation of the conduit to be painted.

[0040] In step S103, a key point detection algorithm is used to determine the spatial direction of the conduit to be painted, specifically including: obtaining multi-view images of the conduit to be painted, the multi-view images including a front view, a side view, and a top view; extracting key points from the multi-view images, and calculating the size-invariant feature vector of each key point; determining matching key points from the key points based on the size-invariant feature vector; calculating the spatial coordinate transformation relationship of the matching key points, and establishing the three-dimensional coordinates of the surface points of the conduit to be painted based on the spatial coordinate transformation relationship and the camera imaging model; fitting the three-dimensional coordinates to obtain a straight line equation, which represents the direction of the central axis of the conduit to be painted; and using the direction of the central axis as the spatial direction of the conduit to be painted.

[0041] Specifically, the server acquires multi-view images of the catheter to be painted, including front, side, and top views. These images capture the catheter from different angles, providing projection information of the catheter in three-dimensional space. Multi-view images can be obtained by placing multiple cameras around the workstation and triggering them synchronously. Next, the server extracts key points from each view image. Key points are salient points in an image that are easy to track and match, such as corners and spots. Key point detection algorithms include SIFT, SURF, and ORB. Key point detection algorithms automatically locate key points by analyzing the gradient, scale, and directional characteristics of the image.

[0042] For each key point, the server calculates its size-invariant feature vector. The size-invariant feature vector is a mathematical representation that can describe the local texture of the key point and remains invariant to changes in the image size, rotation, brightness, etc. Size-invariant features include SIFT descriptors and SURF descriptors. Then, based on the similarity of the size-invariant feature vectors, the server searches for matching key points among the key points of images with different perspectives to obtain matching point pairs. A matching point pair refers to a pair of key points that correspond to the same physical point in different images. The more similar the feature vectors of a pair of key points, the greater the possibility that they belong to a matching point pair. The feature matching algorithm can be Euclidean distance matching, FLANN matching, etc. Through key point matching, the server establishes a correspondence between images with different perspectives.

[0043] Based on the matching point pairs, the server calculates the spatial coordinate transformation relationship between images from different perspectives. The spatial coordinate transformation relationship is described by a rotation matrix and a translation vector, which represents the relative position of one perspective relative to another. The server uses the RANSAC algorithm to estimate the values ​​of the transformation matrix and translation vector from the matching point pairs. After determining the transformation relationship between the perspectives, the server combines the camera imaging model (such as the pinhole imaging model) to convert the two-dimensional image coordinates into three-dimensional space coordinates. For each pixel point on the surface of the catheter, the server calculates its coordinates in three-dimensional space through the camera imaging model to obtain a three-dimensional point cloud. Finally, the server fits the three-dimensional point cloud to obtain the equation of a three-dimensional straight line. This straight line approximately represents the central axis of the catheter, and its direction vector is the direction of the catheter in space. Three-dimensional straight line fitting can be implemented using algorithms such as the least squares method and RANSAC.

[0044] For example, suppose the server is viewed from the front, left, and top. Figure 3 The catheter was photographed from different angles, and 50, 40, and 60 SIFT keypoints were extracted, respectively. Through feature vector matching, 20, 30, and 25 pairs of matching points were found between the front and left views, the front and top views, and the left and top views, respectively. Using the RANSAC algorithm, the server estimated the rotation matrix and translation vector between the viewpoints from the matching point pairs. Then, based on the pinhole imaging model, the 3D coordinates of 1,000 catheter surface points were calculated. Least squares fitting of these 3D points yielded the equation of a line with a direction vector of (0.8, 0.2, 0.6), indicating that the catheter is tilted 40 degrees along the x-axis, 10 degrees along the y-axis, and 30 degrees along the z-axis.

[0045] Step S104: generating a three-dimensional parametric model of the conduit to be painted according to the outline, position coordinates and spatial direction.

[0046] In step S104, a three-dimensional parametric model of the conduit to be painted is generated based on the contour, position coordinates, and spatial direction, specifically including: performing equally spaced sampling on the contour to obtain multiple contour sampling points; constructing three-dimensional point cloud data based on the multiple contour sampling points, and generating a three-dimensional parametric model of the conduit to be painted based on the three-dimensional point cloud data.

[0047] Specifically, the server generates a 3D parametric model of the pipe to be painted based on its contour, position coordinates, and spatial orientation. The server then performs equally spaced sampling on the contour of the pipe. Contour sampling involves selecting a series of equally spaced points on a contour curve, using these discrete points to approximate a continuous contour. The server uses equal arc length sampling and equal parameter sampling methods to automatically generate evenly distributed sampling points along the contour.

[0048] After obtaining the contour sampling points, the server combines their coordinates with the position and orientation information to construct 3D point cloud data. Specifically, the server first converts the pixel coordinates of the contour sampling points into spatial coordinates based on the position coordinates. Then, using the spatial orientation of the catheter as the normal vector, the server translates the contour sampling points along the normal vector to generate 3D point cloud data of the catheter surface. By rotating and stacking the contour sampling points, the server generates a 3D point cloud covering the entire catheter surface. Finally, the server uses this 3D point cloud data to generate a 3D parametric model of the catheter.

[0049] Step S105: Generate a nozzle motion trajectory according to the three-dimensional parameterized model, and control the robotic arm to spray paint according to the nozzle motion trajectory.

[0050] In step S105, a nozzle motion trajectory is generated according to the three-dimensional parametric model, specifically including: constructing a grid model of the duct to be painted based on the three-dimensional parametric model; evenly arranging multiple paint path planning seed points on the grid model; using each paint path planning seed point as a starting point, using the Dijkstra algorithm to search for the shortest path in the grid model to generate the nozzle motion trajectory.

[0051] Specifically, the server constructs a mesh model of the catheter based on a three-dimensional parametric model. The mesh model is a discretized surface composed of a series of interconnected polygonal patches, capable of approximating the actual shape of the catheter with arbitrary precision. Methods for constructing the mesh model include triangulation and quadrilateralization. The server selects the appropriate triangulation method and parameters based on the catheter's geometric characteristics. For example, a regular quadrilateral mesh can be used for a straight cylindrical tube, while an adaptive triangular mesh can be used for a curved tube. The resulting mesh model consists of vertices, edges, and faces. Next, the server evenly distributes multiple painting path planning seed points on the mesh model. Using each painting path planning seed point as a starting point, the server uses the Dijkstra algorithm to search for the shortest path on the mesh model. The Dijkstra algorithm is a single-source shortest path algorithm that finds the shortest path from a starting point to all other vertices in a weighted graph. When applied to painting path planning, the vertices of the mesh model are the nodes of the graph, and the mesh edges are the edges of the graph. Starting from each paint path planning seed point, the server runs the Dijkstra algorithm to find the shortest path to all other vertices. These shortest paths are then stitched together to form a complete printhead trajectory. Finally, the server sends the generated printhead trajectory to the robotic arm control system, which controls the printhead's movement and spraying according to the trajectory.

[0052] In one possible implementation, multiple paint path planning seed points are evenly arranged on a grid model, specifically including: dividing the grid model into multiple grid units of equal size; calculating the center point coordinates of each grid unit; and using the center point coordinates as the paint path planning seed point to obtain multiple evenly distributed paint path planning seed points.

[0053] Specifically, the server performs meshing on the catheter's mesh model. Meshing involves dividing the surface of the mesh model into multiple mesh cells of equal size and similar shape, according to specific rules and scales. The server uses a uniform meshing method, generating a series of parallel dividing lines within the mesh model's parameter domain at a fixed step size, cutting the surface into rectangular or triangular mesh cells of equal size. For example, a cylindrical catheter with a length of 1000 mm and a diameter of 200 mm can be evenly divided axially into 100 mesh cells, each 10 mm long.

[0054] Next, the server calculates the coordinates of the center point of each grid cell. Since the shape of the grid cell is regular, the position of the center point is also fixed, and it can be directly calculated analytically. For example, for a rectangular grid cell, the center point is the intersection of the diagonals; for a triangular grid cell, the center point is the intersection of the three medians. During calculation, the server first obtains the vertex coordinates of the grid cell, which can be read from the vertex list of the grid model by indexing. Then, according to the vertex coordinates and the type of grid cell, the coordinates of the center point are calculated using the corresponding formula. For example, for a rectangular grid cell composed of vertices (0, 0), (0, 1), (1, 0), and (1, 1), the coordinates of its center point are (0.5, 0.5).

[0055] Finally, the server uses the calculated center point coordinates as the painting path planning seed points. Because the grid cells are evenly divided, their center points are also evenly distributed on the grid model, meeting the requirements for seed point arrangement. The server can directly place the center point coordinates of all grid cells into an array or list as input to the Dijkstra algorithm. The number of seed points obtained in this way is the same as the number of grid cells, and the distribution is also consistent with the grid cells. For example, for 100 divided grid cells, the server will generate 100 seed points, which appear in a regular 10×10 dot matrix distribution on the catheter surface.

[0056] In a possible embodiment, after step S105, the method further includes: during the painting process, obtaining an image of the spraying area captured by a visual sensor; performing similarity calculation between the image of the spraying area and a preset spraying image to determine whether the spraying quality is qualified; if it is determined that the spraying quality is unqualified, sending a prompt message, wherein the prompt message is used to remind the staff that the spraying quality is unqualified.

[0057] Specifically, during the spraying process, the server acquires images of the spray area from a visual sensor in real time. The timing and frequency of these acquisitions can be set based on the spraying speed and quality requirements, typically every few seconds or at a few path points. The captured images are transmitted to the server via a communication interface for subsequent processing. For example, the visual sensor captures a color image with a resolution of 2560×1920 pixels every second and transmits it to the server via the GigE interface. The server then calculates the similarity between the captured spray area image and the preset spray image. The preset spray image represents the ideal spraying effect generated based on the 3D model of the catheter and the spraying process parameters, reflecting the target spraying state. The server calculates the similarity between the two images using image comparison algorithms, such as feature matching. The similarity can be expressed as a value between 0 and 1, with 1 indicating exact matches and 0 indicating complete differences. For example, the server uses the SSIM algorithm to perform a multi-scale, multi-channel comparison between the captured spray area image and the preset spray image, generating a similarity score.

[0058] Based on the calculated similarity, the server determines whether the current spraying quality is acceptable. A similarity threshold is pre-set within the server; results above or equal to this threshold are considered acceptable, while those below this threshold are considered unacceptable. If the server determines that the current spraying quality is unacceptable, a prompt message is sent to alert staff. This prompt message can be sent in a variety of ways, such as a pop-up warning box on the monitoring interface, push notifications to networked devices, and triggering audio and visual alarms, ensuring that information is conveyed to relevant personnel in a timely and accurate manner.

[0059] Reference Figure 2 The present application also provides an intelligent painting device based on visual recognition, which is a server. The server includes an acquisition module 201 and a processing module 202, wherein: the acquisition module 201 is used to acquire an image of the conduit to be painted; the processing module 202 is used to segment the image and identify the contour and position coordinates of the conduit to be painted; the processing module 202 is also used to determine the spatial direction of the conduit to be painted using a key point detection algorithm; the processing module 202 is used to generate a three-dimensional parametric model of the conduit to be painted based on the contour, position coordinates and spatial direction; the processing module 202 is used to generate a nozzle motion trajectory based on the three-dimensional parametric model, and control the robotic arm to spray paint according to the nozzle motion trajectory.

[0060] In one possible implementation, the processing module 202 generates a nozzle motion trajectory based on a three-dimensional parametric model, specifically including: the processing module 202 constructs a grid model of the conduit to be painted based on the three-dimensional parametric model; the processing module 202 evenly arranges multiple paint path planning seed points on the grid model; the processing module 202 uses each paint path planning seed point as a starting point, and uses the Dijkstra algorithm to search for the shortest path in the grid model to generate the nozzle motion trajectory.

[0061] In one possible implementation, the processing module 202 evenly arranges multiple paint path planning seed points on the grid model, specifically including: the processing module 202 divides the grid model into multiple grid units of equal size; calculates the center point coordinates of each grid unit; the processing module 202 uses the center point coordinates as the paint path planning seed point to obtain multiple evenly distributed paint path planning seed points.

[0062] In one possible embodiment, the processing module 202 segments the image and identifies the outline and position coordinates of the conduit to be painted, specifically including: the processing module 202 extracts edge pixels in the image; uses a region growing algorithm to divide the image into multiple regions based on the connectivity of the edge pixels; the processing module 202 calculates geometric parameters of each region; the processing module 202 compares the geometric parameters with a preset parameter threshold to obtain a target region where the conduit to be painted is located; the processing module 202 fits the boundary pixels of the target region to extract the outline of the conduit to be painted; the processing module 202 calculates multiple minimum circumscribed rectangles of the outline, and determines the position coordinates of the conduit to be painted in the image based on the minimum circumscribed rectangles.

[0063] In one possible embodiment, the processing module 202 uses a key point detection algorithm to determine the spatial direction of the conduit to be painted, specifically including: the acquisition module 201 acquires multi-perspective images of the conduit to be painted, the multi-perspective images including a front view, a side view, and a top view; the processing module 202 extracts key points from the multi-perspective images and calculates the size-invariant feature vector of each key point; based on the size-invariant feature vector, the matching key points are determined from the key points; the processing module 202 calculates the spatial coordinate transformation relationship of the matching key points, and establishes the three-dimensional coordinates of the surface points of the conduit to be painted based on the spatial coordinate transformation relationship and the camera imaging model; the processing module 202 fits the three-dimensional coordinates to obtain a straight line equation, which represents the direction of the central axis of the conduit to be painted; the processing module 202 uses the direction of the central axis as the spatial direction of the conduit to be painted.

[0064] In one possible implementation, the processing module 202 generates a three-dimensional parametric model of the conduit to be painted based on the contour, position coordinates, and spatial direction. Specifically, the processing module 202 performs equally spaced sampling on the contour to obtain multiple contour sampling points; the processing module 202 constructs three-dimensional point cloud data based on the multiple contour sampling points, and generates a three-dimensional parametric model of the conduit to be painted based on the three-dimensional point cloud data.

[0065] In a possible embodiment, after the processing module 202 generates a nozzle motion trajectory according to the three-dimensional parametric model and controls the robotic arm to spray paint according to the nozzle motion trajectory, the method also includes: the acquisition module 201 acquires the spray area image captured by the visual sensor during the painting process; the processing module 202 calculates the similarity between the spray area image and the preset spray image to determine whether the spraying quality is qualified; if the processing module 202 determines that the spraying quality is unqualified, it sends a prompt message, and the prompt message is used to remind the staff that the spraying quality is unqualified.

[0066] It should be noted that the above embodiments provide devices that implement their functions using only the division of the above functional modules as examples. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiments are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0067] This application also provides an electronic device. Figure 3 , Figure 3 3. This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. The electronic device 300 may include: at least one processor 301, at least one network interface 304, a user interface 303, a memory 305, and at least one communication bus 302.

[0068] The communication bus 302 is used to implement the connection and communication between these components.

[0069] The user interface 303 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 303 may also include a standard wired interface and a wireless interface.

[0070] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0071] The processor 301 may include one or more processing cores. Using various interfaces and circuits, the processor 301 connects to various components within the server. It executes instructions, programs, code sets, or instruction sets stored in the memory 305, as well as accesses data stored in the memory 305, to perform various server functions and process data. Optionally, the processor 301 may be implemented using at least one of the following hardware forms: a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 301 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing content displayed on the display screen; and the modem handles wireless communications. It is understood that the modem may not be integrated into the processor 301 but implemented as a separate chip.

[0072] Among them, the memory 305 may include a random access memory (RAM) or a read-only memory (Read-Only Memory). Optionally, the memory 305 includes a non-transitory computer-readable storage medium. The memory 305 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 305 may also optionally be at least one storage device located away from the aforementioned processor 301. Refer to Figure 3 , as a computer storage medium, the memory 305 may include an operating system, a network communication module, a user interface module, and an application program of an intelligent painting method based on visual recognition.

[0073] exist Figure 3In the electronic device 300 shown, the user interface 303 is mainly used to provide an input interface for the user and obtain the data input by the user; and the processor 301 can be used to call the memory 305 to store an application program for an intelligent painting method based on visual recognition. When executed by one or more processors 301, the electronic device 300 executes one or more of the methods described in the above embodiments. It should be noted that for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should know that this application is not limited to the described order of actions, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for this application.

[0074] The present application further provides a computer-readable storage medium storing instructions, which, when executed by one or more processors 301 , enable the electronic device 300 to perform one or more of the methods described in the above embodiments.

[0075] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0076] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are merely schematic, such as the division of units, which is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interface, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0077] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0078] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0079] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of this application, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the method of this application. The aforementioned memory includes various media that can store program code, such as USB flash drives, mobile hard drives, magnetic disks, or optical disks.

[0080] The foregoing is merely an exemplary embodiment of the present disclosure and is not intended to limit the scope of the present disclosure. In other words, any equivalent variations and modifications made in accordance with the teachings of the present disclosure are still within the scope of the present disclosure. Those skilled in the art will readily conceive of other embodiments of the present disclosure after considering the disclosure and the practical implications thereof.

[0081] This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not described herein. The description and examples are to be considered as exemplary only, and the scope and spirit of the present disclosure are to be defined by the claims.

Claims

1. An intelligent painting method based on visual recognition, characterized in that: The method comprises: Acquire an image of a conduit to be painted; Segmenting the image to identify the outline and position coordinates of the conduit to be painted; Using a key point detection algorithm to determine the spatial orientation of the conduit to be painted; Generating a three-dimensional parameterized model of the conduit to be painted according to the contour, the position coordinates and the spatial direction; A movement trajectory of the nozzle is generated according to the three-dimensional parameterized model, and the robot arm is controlled to spray paint according to the movement trajectory of the nozzle.

2. The method according to claim 1, characterized in that Generating the nozzle motion trajectory according to the three-dimensional parameterized model specifically includes: Based on the three-dimensional parameterized model, construct a grid model of the conduit to be painted; Evenly arranging a plurality of painting path planning seed points on the grid model; Taking each of the spraying path planning seed points as a starting point, the Dijkstra algorithm is used to search for the shortest path in the grid model to generate the movement trajectory of the spray head.

3. The method according to claim 2, characterized in that The step of evenly arranging a plurality of painting path planning seed points on the grid model specifically includes: Dividing the grid model into a plurality of grid units of equal size; Calculate the center point coordinates of each of the grid cells; The center point coordinates are used as the painting path planning seed points to obtain a plurality of evenly distributed painting path planning seed points.

4. The method according to claim 1, characterized in that The segmenting of the image to identify the outline and position coordinates of the conduit to be painted specifically includes: Extracting edge pixels in the image; Using a region growing algorithm, the image is divided into a plurality of regions according to the connectivity of the edge pixels; Calculating geometric parameters of each of the regions; Comparing the geometric parameters with a preset parameter threshold to obtain a target area where the conduit to be painted is located; Fitting the boundary pixels of the target area to extract the outline of the conduit to be painted; A plurality of minimum circumscribed rectangles of the contour are calculated, and the position coordinates of the conduit to be painted in the image are determined according to the minimum circumscribed rectangles.

5. The method according to claim 1, characterized in that The method of using a key point detection algorithm to determine the spatial direction of the conduit to be painted specifically includes: Acquire a multi-view image of the conduit to be painted, wherein the multi-view image includes a front view, a side view, and a top view; Extracting key points from the multi-view images and calculating size-invariant feature vectors of each of the key points; Determining matching key points from the key points according to the size-invariant feature vector; Calculating the spatial coordinate transformation relationship of the matching key points, and establishing the three-dimensional coordinates of the surface points of the conduit to be painted according to the spatial coordinate transformation relationship and the camera imaging model; Fitting the three-dimensional coordinates to obtain a straight line equation, wherein the straight line equation represents the direction of the central axis of the conduit to be painted; The central axis direction is used as the spatial direction of the conduit to be painted.

6. The method according to claim 1, characterized in that Generating a three-dimensional parameterized model of the conduit to be painted according to the contour, the position coordinates and the spatial direction specifically includes: Performing equal-interval sampling on the contour to obtain a plurality of contour sampling points; Three-dimensional point cloud data is constructed according to the plurality of contour sampling points, and a three-dimensional parameterized model of the conduit to be painted is generated according to the three-dimensional point cloud data.

7. The method according to claim 1, characterized in that After generating a nozzle motion trajectory according to the three-dimensional parameterized model and controlling the robot arm to spray paint according to the nozzle motion trajectory, the method further includes: During the painting process, the image of the spraying area collected by the visual sensor is obtained; Calculate the similarity between the spraying area image and the preset spraying image to determine whether the spraying quality is qualified; If it is determined that the spraying quality is unqualified, a prompt message is sent, and the prompt message is used to remind the staff that the spraying quality is unqualified.

8. An intelligent painting device based on visual recognition, characterized in that: The device comprises an acquisition module (201) and a processing module (202), wherein: The acquisition module (201) is used to acquire an image of the conduit to be painted; The processing module (202) is used to segment the image and identify the contour and position coordinates of the conduit to be painted; The processing module (202) is further used to determine the spatial direction of the conduit to be painted using a key point detection algorithm; The processing module (202) is used to generate a three-dimensional parameterized model of the conduit to be painted according to the contour, the position coordinates and the spatial direction; The processing module (202) is used to generate a nozzle motion trajectory according to the three-dimensional parameterized model, and control the robot arm to spray paint according to the nozzle motion trajectory.

9. An electronic device, characterized in that: The electronic device (300) comprises a processor (301), a memory (305), a user interface (303) and a network interface (304), wherein the memory (305) is used to store instructions, the user interface (303) and the network interface (304) are used to communicate with other devices, and the processor (301) is used to execute the instructions stored in the memory (305) so that the electronic device (300) executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1 to 7 is executed.

Images