Intelligent Label Printing Optimization System, Method and Device Based on Image Recognition

Abstract

The present application relates to the technical fields of intelligent printing and industrial automation control, and discloses an intelligent label printing optimization system, method and device based on image recognition. The method includes: constructing a surface feature matrix of the object according to the surface three-dimensional point cloud data and texture information of the object to be labeled; performing regional adaptability evaluation on the surface feature matrix of the object to obtain a heat map to determine the target labeling area; obtaining the semantic information corresponding to the preset label content to generate the layout constraint conditions corresponding to the label content; generating an optimized layout plan corresponding to the label content according to the layout constraint conditions, and controlling a preset printing device to print according to the optimized layout plan to form a target label; generating a motion trajectory corresponding to a preset robotic arm according to the first pose information corresponding to the object to be labeled, the second pose information of the preset printing device, and the area range corresponding to the target area, and controlling the robotic arm to paste the target label to the target area according to the motion trajectory.

Description

Technical Field

[0001] The present application relates to the field of intelligent printing and industrial automation control technology, and in particular to an intelligent label printing optimization system, method and device based on image recognition. Background Art

[0002] With the rapid development of industrial automation technology, intelligent label printing systems have been widely used in intelligent manufacturing, logistics management and other fields. Traditional label printing technology mainly relies on manual experience to select labeling locations, and has the following technical bottlenecks:

[0003] Poor adaptability to complex surfaces: Conventional monocular vision systems cannot accurately capture changes in surface curvature and material reflectivity, resulting in problems such as label warping and falling off on concave and convex surfaces or reflective materials;

[0004] Static layout rules: Existing label generation systems mostly use fixed template libraries, without considering the importance differences of semantic information and the dynamic matching of physical constraints of the labelable area, resulting in the obstruction of key information or imbalance of character proportions;

[0005] Insufficient mechanical control accuracy: Traditional robot arm path planning algorithms have difficulty maintaining the normal fit of the end effector on uneven surfaces, which can easily cause bubbles or wrinkles.

[0006] Therefore, there is an urgent need for an intelligent label printing optimization system, method and device based on image recognition to solve at least one of the above problems. Summary of the invention

[0007] The present application provides an intelligent label printing optimization system, method and device based on image recognition, aiming to solve the problems that traditional label printing technology mainly relies on manual experience to select labeling positions, has poor adaptability to complex surfaces, static solidification of typesetting rules and insufficient mechanical control accuracy.

[0008] In a first aspect, the present application provides a smart label printing optimization method based on image recognition, comprising:

[0009] The surface three-dimensional point cloud data and texture information of the object to be labeled are obtained according to the preset multispectral image acquisition device, so as to construct the surface feature matrix of the object including the spatial curvature features and material properties;

[0010] Performing a regional adaptability evaluation on the surface feature matrix of the object to obtain a thermal map including a probability distribution of labelable regions, so as to determine a target labeling region according to a maximum response value corresponding to the thermal map;

[0011] Obtain the semantic information corresponding to the preset label content, calculate the semantic element weights corresponding to the semantic information, and generate the layout constraint conditions corresponding to the label content according to the area range, semantic element weights, and semantic information corresponding to the target labeling area; the layout constraint conditions at least include a character scaling ratio threshold, an information hierarchy visualization rule, label image size parameters, and material adhesion stability parameters;

[0012] Generate an optimized layout plan corresponding to the label content according to the layout constraint conditions, and control a preset printing device to print according to the optimized layout plan to form a target label;

[0013] Based on the non-uniform rational B-spline curve fitting algorithm, generate the motion trajectory of a preset robotic arm according to the first pose information corresponding to the object to be labeled, the second pose information of the preset printing device, and the area range corresponding to the target area, and control the robotic arm to paste the target label to the target area according to the motion trajectory.

[0014] In some embodiments, the obtaining of the surface three-dimensional point cloud data and texture information of the object to be labeled by a preset multi-spectral image acquisition device for constructing an object surface feature matrix including spatial curvature features and material attributes includes: inputting the texture information into a preset parallel convolutional neural network to extract texture feature maps of multiple spectral channels; inputting the surface three-dimensional point cloud data into a preset graph convolutional network to perform spatial curvature modeling on the three-dimensional point cloud data and output spatial curvature features including spectral reflectance and material attributes; splicing the texture feature maps and the spatial curvature features into a feature tensor, and performing dimensionality reduction processing on the spliced feature tensor through a differentiable pooling layer to generate an object surface feature matrix with local geometric preservation.

[0015] In some embodiments, the performing of a regional adaptability evaluation on the object surface feature matrix to obtain a heat map including the probability distribution of labelable areas includes: constructing a deep regional segmentation network; the deep regional segmentation network adopts an encoder-decoder structure of the U-Net architecture, introducing a channel attention mechanism in the encoding stage for importance weighting of the object surface feature matrix; embedding a deformable convolution module in the decoding stage to dynamically adjust the receptive field to adapt to the morphological features of different curvature regions; inputting the object surface feature matrix into the deep regional segmentation network and outputting a probability distribution map including the label suitability of each pixel point; performing morphological closing operation on the probability distribution map to eliminate discrete noise points and generate a heat map with regional connectivity.

[0016] In some embodiments, determining the target labeling area according to the maximum response value corresponding to the thermal map includes: constructing an objective function according to the maximum area, minimum principal curvature variance, and maximum edge sharpness index corresponding to the target labeling area; optimizing the thermal map according to the preset objective function and multi-objective optimization algorithm to obtain multiple candidate labeling areas; performing non-dominated sorting on the multiple candidate labeling areas according to the NSGA-II algorithm, and calculating the weight coefficient corresponding to each sorted candidate labeling area according to the entropy weight method; in the multiple candidate labeling areas, obtaining the continuous area with the weight coefficient greater than the preset threshold as the target labeling area.

[0017] In some embodiments, calculating the semantic element weights corresponding to the semantic information to generate the layout constraint conditions corresponding to the label content according to the area range, semantic element weights, and semantic information corresponding to the target labeling area includes: extracting the semantic dependency tree corresponding to the semantic information according to the preset BERT model; calculating the importance score of each semantic node of the semantic dependency tree as the semantic element weight based on the preset graph attention network; converting the area range parameters into two-dimensional binning constraints and mapping the semantic element weights to the weighted coefficients of the information hierarchy visualization rules; the area range parameters include one or more of geometric dimension parameters, curvature feature parameters, edge feature parameters, material property parameters, spatial position and orientation parameters, topological parameters, and stability parameters; obtaining the layout constraint conditions corresponding to the label content according to the two-dimensional binning constraints and weighted coefficients based on the Lagrange multiplier method.

[0018] In some embodiments, generating the optimized layout scheme corresponding to the label content according to the layout constraint conditions includes: constructing an intelligent layout architecture based on a generative adversarial network; the generator of the generative adversarial network adopts a multi-scale feature fusion network with self-attention mechanism, and the discriminator of the generative adversarial network introduces a differentiable rendering module to simulate the actual printing effect; optimizing the generator parameters of the generative adversarial network based on reinforcement learning, and generating the reward function corresponding to the reinforcement learning according to information entropy, aesthetic evaluation score, and material matching degree; generating a style transfer vector according to the preset visual specification and layout constraint conditions to embed the style transfer vector into the latent space corresponding to the intelligent layout architecture; inputting the label content into the intelligent layout architecture to generate the optimized layout scheme.

[0019] In some embodiments, based on the non-uniform rational B-spline curve fitting algorithm, a motion trajectory corresponding to a preset robotic arm is generated according to the first pose information corresponding to the object to be labeled, the second pose information of the preset printing device, and the area range corresponding to the target area, including: constructing a kinematic model based on the Lie group space for converting the first pose information and the second pose information into a rigid body transformation matrix in a preset space; performing path planning according to the kinematic model and the rigid body transformation matrix to generate an initial motion trajectory; wherein, a curvature continuity constraint of the non-uniform rational B-spline curve is introduced into the configuration space corresponding to the initial motion trajectory; simulating and calculating the dynamic obstacle avoidance information of the initial motion trajectory according to a preset physical engine to optimize the initial motion trajectory according to the dynamic obstacle avoidance information to generate the motion trajectory; wherein, the dynamic obstacle avoidance information is calculated according to the collision probability, the kinetic energy of the obstacle, and the path energy consumption corresponding to the initial motion trajectory.

[0020] In some embodiments, the step of controlling the robotic arm to paste the target label onto the target area according to the motion trajectory includes: obtaining the torque feedback information of the end effector corresponding to the robotic arm, and adjusting the dynamic parameters of the robotic arm according to the torque feedback information; monitoring the labeling process of the robotic arm in real time according to a preset online vision feedback device, and when it is detected that the object to be labeled has a pose offset and / or surface abnormal deformation, performing real-time optimization on the motion trajectory to control the robotic arm to complete the labeling process according to the dynamically optimized dynamic parameters and motion trajectory.

[0021] In a second aspect, the present application provides an intelligent label printing optimization system based on image recognition, including:

[0022] A multi-spectral image acquisition device for obtaining the surface three-dimensional point cloud data and texture information of the object to be labeled;

[0023] A printing device for printing the target label;

[0024] A robotic arm for pasting the target label on the object to be labeled;

[0025] A control device is configured to construct an object surface feature matrix including spatial curvature features and material properties based on the surface three-dimensional point cloud data and texture information; perform regional adaptability evaluation on the object surface feature matrix to obtain a heat map including the probability distribution of labelable regions, and determine a target labelable region according to the maximum response value corresponding to the heat map; obtain the semantic information corresponding to the preset label content, calculate the semantic element weights corresponding to the semantic information, and generate layout constraint conditions corresponding to the label content according to the regional range, semantic element weights, and semantic information corresponding to the target labelable region; the layout constraint conditions at least include a character scaling ratio threshold, an information hierarchy visualization rule, label image size parameters, and material attachment stability parameters; generate an optimized layout plan corresponding to the label content according to the layout constraint conditions, and control a preset printing device to print according to the optimized layout plan to form a target label; based on the non-uniform rational B-spline curve fitting algorithm, generate a motion trajectory of a preset robotic arm according to the first pose information corresponding to the object to be labeled, the second pose information of the preset printing device, and the regional range corresponding to the target region, and control the robotic arm to paste the target label to the target region according to the motion trajectory.

[0026] In a third aspect, the present application provides an intelligent label printing optimization device based on image recognition, including:

[0027] A matrix construction module is configured to obtain the surface three-dimensional point cloud data and texture information of the object to be labeled according to a preset multi-spectral image acquisition device, and construct an object surface feature matrix including spatial curvature features and material properties;

[0028] A region determination module is configured to perform regional adaptability evaluation on the object surface feature matrix to obtain a heat map including the probability distribution of labelable regions, and determine a target labelable region according to the maximum response value corresponding to the heat map;

[0029] An information acquisition module is configured to obtain the semantic information corresponding to the preset label content, calculate the semantic element weights corresponding to the semantic information, and generate layout constraint conditions corresponding to the label content according to the regional range, semantic element weights, and semantic information corresponding to the target labelable region; the layout constraint conditions at least include a character scaling ratio threshold, an information hierarchy visualization rule, label image size parameters, and material attachment stability parameters;

[0030] A scheme generation module is configured to generate an optimized layout plan corresponding to the label content according to the layout constraint conditions, and control a preset printing device to print according to the optimized layout plan to form a target label;

[0031] The labeling completion module is used to generate the motion trajectory of a preset robotic arm based on the non-uniform rational B-spline curve fitting algorithm according to the first pose information corresponding to the object to be labeled, the second pose information of a preset printing device, and the area range corresponding to the target area, so as to control the robotic arm to paste the target label onto the target area according to the motion trajectory.

[0032] This application provides an intelligent label printing optimization system, method, and device based on image recognition. The three-dimensional point cloud data and texture information of the object to be labeled are obtained through a multi-spectral image acquisition device, and a surface feature matrix including spatial curvature features and material properties is constructed. Based on this matrix, regional adaptability evaluation is carried out to generate a probability distribution heat map of the labelable area, and the optimal target labeling area is dynamically determined through the maximum response value. Multidimensional data (curvature, material) are fused to improve the recognition accuracy of complex surfaces (such as curved surfaces and multi-material splicing surfaces), avoiding the subjective errors of manual experience.

[0033] By analyzing the semantic information of the label content, calculating the semantic element weights (such as the priority of key words and graphics), and combining the size, shape, and material stability parameters (such as friction coefficient and adhesion) of the target area, a typesetting scheme including character scaling, information hierarchy, and size constraints is generated. The dynamic adaptation of the label content to the labeling area is realized to ensure the readability of information and the reliability of label adhesion.

[0034] The non-uniform rational B-spline (NURBS) curve fitting algorithm is adopted, combined with the object pose, the printing device pose, and the target area range, to generate a smooth motion trajectory of the robotic arm. It solves the positioning deviation problem caused by path mutation in traditional mechanical control, especially suitable for high-precision fitting of curved surfaces or irregular surfaces.

[0035] Through three-dimensional feature modeling and heat map analysis, the optimal labeling position of irregular surfaces (such as concave and convex curved surfaces and multi-material mixed surfaces) can be automatically identified, overcoming the poor adaptability problem caused by traditional methods relying on manual experience. By fusing semantic weights and physical constraints (such as material adhesion stability), the adaptive scaling and layout optimization of the label content are realized, avoiding the missing or deformation of label information caused by inconsistent regional size or shape under static rules. The whole process from data acquisition, area selection, typesetting generation to robotic arm labeling is automated, reducing manual intervention; through NURBS trajectory planning and dual-pose collaborative control, the labeling position error is controlled within millimeters, meeting the requirements of precision manufacturing scenarios. It can be applied to scenarios difficult to handle by traditional methods such as curved surface packaging, electronic component labeling, and medical device identification, especially suitable for flexible materials, micro-devices, or complex working conditions requiring multi-label collaborative typesetting.

[0036] Compared with traditional label printing technologies, this method solves the three core problems of strong dependence on manual experience, poor generalization of static rules, and insufficient mechanical positioning accuracy through the full-link optimization of "three-dimensional feature modeling - semantic dynamic layout - high-precision motion control", significantly improving the efficiency, quality, and scene coverage ability of label printing.

[0037] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit this application. Brief Description of the Drawings

[0038] To more clearly illustrate the technical solutions of the embodiments of this application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are some embodiments of this application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0039] Figure 1 It is a schematic block diagram of the structure of an intelligent label printing optimization system provided by an embodiment of this application;

[0040] Figure 2 It is a schematic flow chart of the steps of an intelligent label printing optimization method based on image recognition provided by an embodiment of this application;

[0041] Figure 3 It is a schematic block diagram of the structure of a control device provided by an embodiment of this application.

[0042] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit this application. Detailed Description of the Specific Embodiments

[0043] The following will clearly and completely describe the technical solutions in the embodiments of this application with reference to the drawings in the embodiments of this application. Obviously, the described embodiments are some, but not all, of the embodiments of this application. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of this application.

[0044] The flow charts shown in the drawings are only illustrative examples, and do not necessarily include all the contents and operations / steps, nor do they necessarily need to be executed in the described order. For example, some operations / steps can also be decomposed, combined, or partially merged. Therefore, the actual execution order may change according to the actual situation.

[0045] It should be understood that, in order to facilitate the clear description of the technical solutions of the embodiments of the present invention, in the embodiments of the present invention, the words "first", "second", etc. are used to distinguish the same items or similar items with substantially the same functions and effects. Those skilled in the art can understand that the words "first", "second", etc. do not limit the quantity and execution order, and the words "first", "second", etc. do not necessarily limit the difference.

[0046] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0047] It should also be understood that the term “and / or” used in the specification and appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0048] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0049] With the rapid development of industrial automation technology, intelligent label printing systems have been widely used in intelligent manufacturing, logistics management and other fields. Traditional label printing technology mainly relies on manual experience to select labeling locations, and has the following technical bottlenecks:

[0050] Poor adaptability to complex surfaces: Conventional monocular vision systems cannot accurately capture changes in surface curvature and material reflectivity, resulting in problems such as label warping and falling off on concave and convex surfaces or reflective materials;

[0051] Static layout rules: Existing label generation systems mostly use fixed template libraries, without considering the importance differences of semantic information and the dynamic matching of physical constraints of the labelable area, resulting in the obstruction of key information or imbalance of character proportions;

[0052] Insufficient mechanical control accuracy: Traditional robot arm path planning algorithms have difficulty maintaining the normal fit of the end effector on uneven surfaces, which can easily cause bubbles or wrinkles.

[0053] To solve the above problems, please refer to Figure 1 , the present application provides an intelligent label printing optimization system based on image recognition, comprising: a multispectral image acquisition device for acquiring surface three-dimensional point cloud data and texture information of an object to be labeled;

[0054] A printing device, used for printing a target label;

[0055] A robotic arm for pasting the target label onto the object to be labeled;

[0056] A control device for constructing an object surface feature matrix containing spatial curvature features and material properties based on the surface three-dimensional point cloud data and texture information; performing regional adaptability evaluation on the object surface feature matrix to obtain a heat map including the probability distribution of the labelable area, so as to determine the target labeling area according to the maximum response value corresponding to the heat map; obtaining the semantic information corresponding to the preset label content, calculating the semantic element weights corresponding to the semantic information, so as to generate the typesetting constraint conditions corresponding to the label content according to the regional range, semantic element weights and semantic information corresponding to the target labeling area; the typesetting constraint conditions at least include a character scaling ratio threshold, an information hierarchy visualization rule, a label image size parameter and a material adhesion stability parameter; generating an optimized typesetting scheme corresponding to the label content according to the typesetting constraint conditions, so as to control a preset printing device to print according to the optimized typesetting scheme to form a target label; based on the non-uniform rational B-spline curve fitting algorithm, generating a motion trajectory corresponding to a preset robotic arm according to the first pose information corresponding to the object to be labeled, the second pose information of the preset printing device and the regional range corresponding to the target area, so as to control the robotic arm to paste the target label onto the target area according to the motion trajectory.

[0057] Specifically, the multi-spectral image acquisition device obtains the three-dimensional point cloud data (including curvature and concavity-convex features) and texture information (such as material reflectivity and roughness) of the object surface through multi-spectral imaging technology, providing high-precision input for subsequent analysis.

[0058] The control device, as a computing center, completes the full-process processing from data parsing to motion planning.

[0059] The printing device prints a highly adaptable label (such as a warping-resistant material) according to the optimized typesetting scheme.

[0060] The robotic arm is equipped with a flexible end effector to achieve precise labeling of complex curved surfaces.

[0061] Construct an object surface feature matrix from point cloud data, fuse spatial curvature (such as Gaussian curvature) and material properties (such as friction coefficient, reflectivity), and quantitatively evaluate the labeling feasibility of different regions. Use a convolutional neural network (CNN) to evaluate the regional adaptability of the feature matrix and output a probabilistic heat map. The region with the maximum response value is the optimal labeling position (such as avoiding high-curvature or reflective regions). Analyze the semantic information of the label content (such as product name, barcode), and extract the key element weights through natural language processing (NLP) technology (such as barcode weight > explanatory text). Generate typesetting rules in combination with the physical constraints (area, curvature) of the target region: character scaling ratio threshold: ensure the minimum readable size; information hierarchy visualization rule: high-weight content is centered and enlarged first; material attachment parameter: adjust the label adhesive type according to the surface roughness. Fit the robotic arm motion trajectory based on the NURBS (Non-Uniform Rational B-Spline) algorithm, and dynamically adjust the normal angle and pressure of the end effector to ensure bubble-free adhesion of the label on the uneven surface.

[0062] For example, scan the object surface with a multispectral device to generate point cloud data with a precision of 0.1 mm, and synchronously collect RGB and near-infrared textures. For example, when scanning a metal tank, the near-infrared band can penetrate the surface oil stain to accurately identify the actual material. Through the collaboration of multimodal data fusion and intelligent algorithms, the system has achieved the leap from "empirical labeling" to "adaptive labeling".

[0063] Please refer to Figure 2 , Figure 2 is a schematic flowchart of an intelligent label printing optimization method based on image recognition provided by an embodiment of the present application. The execution device of the method is the control device of the intelligent label printing optimization system provided by any embodiment of the present application.

[0064] As Figure 2 shown, the provided method includes steps S101 to S105. Among them, the control device can be a handheld terminal, a laptop computer, a wearable device, or a robot, etc., and is used to implement steps S101 to S105 and their corresponding embodiments.

[0065] Step S101. Obtain the surface three-dimensional point cloud data and texture information of the object to be labeled according to a preset multispectral image acquisition device, and use it to construct an object surface feature matrix containing spatial curvature features and material properties.

[0066] Specifically, in this step, a multi-spectral image acquisition device is used to obtain the three-dimensional point cloud data and texture information of the object to be labeled, and an object surface feature matrix is constructed by combining spatial curvature and material properties. The three-dimensional point cloud data reflects the geometric shape of the object surface (such as concavity, convexity, curvature), and the texture information includes color, reflectivity, etc. The multi-spectral data enhances the ability to identify different materials (such as metal, plastic). The feature matrix integrates the physical space and material properties, providing a data basis for subsequent region evaluation.

[0067] For example, a lidar or structured light scanner is used to obtain the three-dimensional point cloud data, and at the same time, a multi-spectral camera is used to collect multi-band texture information such as RGB and infrared. The curvature features (calculated by local surface fitting) in the point cloud data are fused with the texture features (such as texture statistics extracted by the gray-level co-occurrence matrix) to form a multi-dimensional matrix. For example, each point cloud coordinate corresponds to parameters such as curvature value and material reflectivity. A tensor data structure is used to store the feature matrix, and the dimensions include spatial coordinates (X, Y, Z), curvature (C), material type (M), etc.

[0068] Combining geometric and material information can avoid misjudgment of the labeling area caused by single data (such as areas with high curvature but suitable materials being excluded). The multi-spectral data overcomes the problem of texture distortion under a single light source. For example, the overexposure of reflective materials under visible light can be compensated by infrared data.

[0069] Step S102. Perform a regional adaptability evaluation on the object surface feature matrix to obtain a heat map including the probability distribution of the labelable areas, and determine the target labeling area according to the maximum response value corresponding to the heat map.

[0070] Specifically, a regional adaptability evaluation is performed based on the feature matrix to generate a heat map to quantify the probability distribution of the labeling area, and the optimal labeling position is determined through the maximum response value. The adaptability evaluation includes indicators such as surface flatness, material adhesion, and visibility. The high-probability areas in the heat map meet the preset physical and visual conditions.

[0071] For example, a superpixel segmentation algorithm based on graph theory (such as the SLIC algorithm) is used to divide the feature matrix into candidate areas. A convolutional neural network (CNN) classifier is trained, and the sub-blocks of the feature matrix of the candidate areas are input, and the labeling adaptability score (0-1 probability value) is output. The training data annotation includes samples of "suitable / unsuitable" areas manually labeled. The scores of each area are mapped to color intensities to generate a heat map, and the maximum response area is selected as the target area through non-maximum suppression (NMS).

[0072] The evaluation criteria are dynamically optimized through machine learning to adapt to different object types (such as curved containers, flat packages). The heat map intuitively locates the optimal area, reducing the time cost of traditional manual trial and error.

[0073] Step S103. Obtain the semantic information corresponding to the preset label content, calculate the semantic element weights corresponding to the semantic information, and generate the layout constraint conditions corresponding to the label content according to the regional range, semantic element weights, and semantic information corresponding to the target labeling area; the layout constraint conditions at least include a character scaling ratio threshold, an information hierarchy visualization rule, label image size parameters, and material adhesion stability parameters.

[0074] Specifically, parse the semantic information of the label content (such as text, icons), calculate the semantic element weights, and generate layout rules in combination with the geometric constraints of the target area. For example, important information (such as the shelf life) needs to be enlarged for display, and icons need to avoid deforming across high-curvature areas.

[0075] For example, use NLP technology to extract text keywords (such as "dangerous", "important"), and identify the categories of icons through a pre-trained object detection model (such as YOLO). Calculate the text importance weights based on the TF-IDF algorithm, and the icon weights are based on preset rules (such as the safety sign weight = 0.9).

[0076] The scaling ratio is used to calculate the font size range based on the size of the minimum bounding rectangle of the area and the weights. The adhesion stability is used to adjust the upper limit of the label size through material properties (such as the affinity coefficient between the adhesive and the material).

[0077] The weights drive the highlighting of key information, avoiding information loss caused by scaling or deformation of the label content. Combine the material stability parameters (such as the viscosity threshold) to prevent the label from falling off.

[0078] Step S104. Generate an optimized layout plan corresponding to the label content according to the layout constraint conditions, and control a preset printing device to print according to the optimized layout plan to form a target label.

[0079] Specifically, generate an optimized plan according to the layout constraints and control the printing device to output the label. The optimization goals include information integrity, aesthetics, and physical feasibility, and a multi-objective optimization algorithm is used to generate the Pareto optimal solution.

[0080] For example, use a genetic algorithm or a reinforcement learning model, use the constraint conditions as the boundary, and use the information density, alignment, and aesthetics score (through a pre-trained aesthetics evaluation model) as the optimization goals to generate a layout plan. Convert the layout plan into printing instructions (such as PDF / vector graphics), drive an inkjet or laser printer to output, and adjust the printing parameters (such as ink concentration, heating temperature) according to the material properties.

[0081] Automated layout is achieved by replacing manual design, especially suitable for large quantities of differentiated labels (such as different language versions). The printing accuracy is adjusted based on the material parameters to avoid printing blurring or penetration problems.

[0082] Step S105. Based on the non-uniform rational B-spline (NURBS) curve fitting algorithm, generate the motion trajectory of a preset robotic arm according to the first pose information corresponding to the object to be labeled, the second pose information of a preset printing device, and the regional range corresponding to the target area, so as to control the robotic arm to paste the target label onto the target area according to the motion trajectory.

[0083] Specifically, generate the robotic arm motion trajectory based on NURBS curve fitting to ensure accurate label fitting on complex surfaces. Trajectory planning needs to coordinate the relative poses of the robotic arm and the object to avoid collisions and optimize the path length.

[0084] For example, obtain the object pose (the first pose) and the printing device outlet pose (the second pose) through a vision sensor (such as a binocular camera), and establish a coordinate system transformation matrix. Use the boundary points of the target area as control points, and generate a smooth trajectory using the NURBS algorithm, satisfying the robotic arm joint acceleration constraints. Discretize the trajectory into a time-position-velocity sequence, drive the robotic arm motion through a PID controller, and adjust the path in real-time feedback.

[0085] The high-degree-of-freedom curves of NURBS adapt to irregular surfaces, avoiding label warping caused by traditional straight paths. Trajectory optimization reduces the idle travel of the robotic arm and improves the labeling speed (such as reducing the cycle time by 30%).

[0086] This method realizes the full-process automation from data acquisition to label attachment through multi-spectral data fusion, dynamic area evaluation, semantics-driven layout, and high-precision motion control, solving problems such as inaccurate position selection, easy label detachment, and difficult curved surface labeling in traditional methods, and is particularly suitable for high-demand labeling tasks for complex materials and curved surfaces in industrial scenarios.

[0087] In some embodiments, obtaining the surface three-dimensional point cloud data and texture information of the object to be labeled according to a preset multi-spectral image acquisition device, for constructing an object surface feature matrix including spatial curvature features and material attributes, includes: inputting the texture information into a preset parallel convolutional neural network to extract texture feature maps of multiple spectral channels; inputting the surface three-dimensional point cloud data into a preset graph convolutional network to perform spatial curvature modeling on the three-dimensional point cloud data, and output spatial curvature features including spectral reflectance and material attributes; performing feature tensor splicing on the texture feature maps and the spatial curvature features, and performing dimensionality reduction processing on the spliced feature tensor through a differentiable pooling layer to generate an object surface feature matrix with local geometric preservation.

[0088] Multi-band texture images such as RGB, near-infrared (NIR), and short-wave infrared (SWIR) collected by a multi-spectral camera. Independent convolutional branches are designed for each spectral channel (such as RGB, NIR). The network structure is a ResNet-18 variant, with each branch containing 5 residual blocks and an output channel number of 256. The feature maps output by each branch are weighted and fused through a channel attention mechanism (SE module) to generate a multi-spectral texture feature map (size H×W×256).

[0089] For example, for a metal surface, the NIR channel can effectively capture the texture of the oxide layer, while the RGB channel may fail due to specular reflection interference. The channel attention automatically enhances the weight of the NIR features. Three-dimensional point cloud data (N points, each point containing XYZ coordinates).

[0090] The graph convolutional network (GCN) constructs a K-nearest neighbor graph (K = 20) from the point cloud, with each point as a graph node and the edge weights calculated by the Euclidean distance. Through the hierarchical aggregation of the GCN, the principal curvatures (maximum curvature C_max, minimum curvature C_min) and mean curvature (C_avg) of the local surface are extracted. A fully connected layer is added at the end of the GCN to classify the material type (such as metal, plastic, glass) according to the point cloud reflectivity (mapped from multi-spectral data). The spatial curvature feature vector (C_max, C_min, C_avg) and material category encoding (One-hot vector) of each point are output. The texture feature map (H×W×256) is tensor-aligned with the spatial curvature features (N×6, including curvature values and material encoding), including using the extrinsic parameter matrix of the multi-spectral camera and the 3D scanner to map the texture image pixels and point cloud coordinates to the same coordinate system. For each three-dimensional point, its corresponding texture feature (256-dimensional) and curvature / material feature (6-dimensional) are associated to generate a 262-dimensional local feature vector. A max-mean pooling layer is used to reduce the dimensionality of the local feature vector to 64 dimensions, and the reduced object surface feature matrix (N×64) is output, retaining key geometric and material information.

[0091] The parallel CNN branches extract complementary texture information for different spectral characteristics (such as NIR suppressing specular reflection and SWIR penetrating surface stains). Combined with the curvature modeling of the GCN, it can still stably generate a feature matrix on the surface of complex materials (such as matte metal, transparent plastic).

[0092] For example, for a transparent bottle, the RGB channel cannot effectively capture the texture due to transmitted light interference, but the SWIR channel can penetrate the bottle surface and accurately divide the labelable area in combination with the curvature data (high-curvature area at the bottle mouth).

[0093] GCN aggregates the curvature of neighboring points through the graph structure, avoiding the local fitting error in the traditional ICP algorithm, and reducing the curvature calculation error to ±0.02 mm⁻¹ (experimental data). Differentiable pooling dynamically balances the maximum response (highlighting key features) and the average response (preserving global information). Compared with traditional PCA dimensionality reduction, the feature reconstruction error is reduced by 18%.

[0094] In some embodiments, the regional adaptability evaluation of the object surface feature matrix to obtain a heat map including the probability distribution of the labelable region includes: constructing a deep region segmentation network; the deep region segmentation network adopts an encoder-decoder structure of the U-Net architecture, and introduces a channel attention mechanism in the encoding stage to perform importance weighting on the object surface feature matrix; embedding a deformable convolution module in the decoding stage to dynamically adjust the receptive field to adapt to the morphological features of different curvature regions; inputting the object surface feature matrix into the deep region segmentation network to output a probability distribution map including the label suitability of each pixel point; and generating a heat map with regional connectivity by eliminating discrete noise points in the probability distribution map according to morphological closing operation.

[0095] This embodiment proposes an improved deep region segmentation network for the regional adaptability evaluation in step S102, enhances the adaptability to complex surface features through the channel attention mechanism and the deformable convolution module, and optimizes the connectivity of the heat map by combining morphological post-processing, so as to solve the problems of fixed receptive field and noise sensitivity of the traditional U-Net in the segmentation of curved surface objects.

[0096] The encoder is based on the encoding structure of U-Net and includes 4 downsampling stages. Each stage consists of 2 convolutional layers + a channel attention module (SE Block), and the output channel numbers are 64, 128, 256, and 512 respectively. The decoder corresponds to 4 upsampling stages. Each stage uses a deformable convolution module (Deformable Convolution v2) instead of a conventional convolution, with a convolution kernel size of 3×3, and the offset is predicted by an additional convolutional layer. The feature maps of the encoder and the decoder are fused by channel attention weighting. The loss function uses the combination of Dice loss + Focal Loss for joint optimization to balance positive and negative samples (the proportion of labelable regions is usually less than 10%). A random affine transformation (rotation ±15°, scaling 0.8 - 1.2 times) and Gaussian noise (σ = 0.05) are applied to the feature matrix.

[0097] The morphological post-processing includes: Binarization: Binarize the probability distribution map (0 - 1) with a threshold of 0.5 to generate an initial mask. Closing operation: Use a 3×3 elliptical structural kernel, first dilate and then erode to fill holes and smooth edges. Connected component screening: Retain the connected regions with an area greater than 50 pixels to generate the final heat map.

[0098] By adaptively adjusting the sampling position of the convolution kernel, the convolution kernel in the decoder stage fits the geometric shape of high-curvature regions (such as the side surface of a cylinder), and the intersection over union (IoU) of segmentation is increased by 12% (experimental data: from 0.78 to 0.87).

[0099] For example, for automotive curved surface parts, the traditional U-Net missegments the continuous curved surface into multiple discrete regions due to the fixed convolution kernel. In this embodiment, deformable convolution is used to maintain the regional continuity.

[0100] In some embodiments, determining the target labeling region according to the maximum response value corresponding to the heat map includes: constructing an objective function according to the maximum region area, minimum principal curvature variance, and maximum edge sharpness index corresponding to the target labeling region; optimizing the heat map according to the preset objective function and multi-objective optimization algorithm to obtain multiple candidate labeling regions; performing non-dominated sorting on the multiple candidate labeling regions according to the NSGA-II algorithm, and calculating the weight coefficient corresponding to each sorted candidate labeling region according to the entropy weight method; in the multiple candidate labeling regions, obtaining the continuous region with the weight coefficient greater than the preset threshold as the target labeling region.

[0101] In this embodiment, for the selection process of the target labeling region in step S102, a candidate region screening method based on multi-objective optimization and NSGA-II algorithm is proposed. By balancing indicators such as area, curvature uniformity, and edge clarity, the problem of local optimality caused by the traditional single-threshold method is solved.

[0102] By maximizing the candidate region area, it is ensured that the label size is sufficient. By minimizing the principal curvature variance within the region, it is ensured that the labeling surface is flat. By maximizing the average edge gradient, it is ensured that the label boundary is clear. The candidate region needs to be a connected domain in the heat map, and the area ≥ the preset minimum label size.

[0103] The NSGA-II optimization process includes: population initialization: randomly select 100 connected domains from the heat map as the initial population. Non-dominated sorting: perform Pareto front stratification on the population according to the objective function value. Crowding degree calculation: calculate the crowding distance of individuals in the same front layer to retain diversity. Selection, crossover, mutation: select parent individuals by tournament. Simulated binary crossover (SBX) generates offspring. Polynomial mutation (mutation rate 0.1) increases diversity. Iterative optimization: run for 50 generations and output the Pareto optimal solution set (about 20 candidate regions).

[0104] Weight calculation includes: normalizing the objective values, performing min-max normalization on the region area, principal curvature variance, and average edge gradient of each candidate region. Information entropy calculation and weight assignment, for example, the weights of the region area, principal curvature variance, and average edge gradient are 0.4, 0.35, and 0.25 respectively.

[0105] Consider the area, curvature, and edge sharpness simultaneously to avoid deviation of a single index (e.g., the largest area may select a high-curvature area).

[0106] In some embodiments, calculating the semantic element weights corresponding to the semantic information, and generating the layout constraint conditions corresponding to the label content according to the region range, semantic element weights, and semantic information corresponding to the target labeling area, includes: extracting the semantic dependency tree corresponding to the semantic information according to a preset BERT model; calculating the importance score of each semantic node of the semantic dependency tree as the semantic element weight based on a preset graph attention network; converting the region range parameters into two-dimensional bin-packing constraints, and mapping the semantic element weights to the weighted coefficients of the information level visualization rules; the region range parameters include one or more of geometric dimension parameters, curvature feature parameters, edge feature parameters, material property parameters, spatial position and orientation parameters, topological parameters, and stability parameters; obtaining the layout constraint conditions corresponding to the label content according to the two-dimensional bin-packing constraints and the weighted coefficients based on the Lagrange multiplier method.

[0107] This embodiment proposes an automated weight assignment method based on semantic dependency analysis and graph attention network (GAT) for the calculation of semantic element weights and the generation process of layout constraints in step S103, and combines the Lagrange multiplier method to fuse multi-dimensional constraints into a solvable optimization problem, solving the problems that traditional layout rules rely on manual experience and it is difficult to quantify semantic importance.

[0108] The input data is like label text (such as "Dangerous goods: flammable liquid, storage temperature ≤ 30°C"). Use the pre-trained BERT-base model to fine-tune on the label text dataset (100,000 industrial labels), and output the semantic dependency tree. By parsing the sentence backbone (such as "Dangerous goods" as the root node, "flammable liquid" as the attribute sub-node, and "temperature ≤ 30°C" as the condition sub-node). Example output: ROOT("Dangerous goods") → ATTRIBUTE("flammable liquid") → CONDITION("temperature ≤ 30°C").

[0109] The graph attention network (GAT) takes the dependency tree nodes as graph nodes and the dependency relationships as edges, designs 2 layers of GAT, with 4 heads in each layer, and outputs the weights of each node (such as the root node weight 0.6, the attribute node weight 0.3, and the condition node weight 0.1).

[0110] The region range parameters mainly include the following key parameters, which are used to quantify the physical characteristics and spatial limitations of the target labeling area to guide the label layout design:

[0111] 1. Geometric dimension parameters: Area: The surface area of the target region, which determines the maximum expandable size of the label. Boundary dimensions: The length and width of the minimum bounding rectangle (or ellipse) of the region, or the major axis length of an irregular shape, used for two-dimensional bin packing constraints. Shape factor: Such as aspect ratio and compactness, which affect the layout adaptability of the label.

[0112] 2. Curvature feature parameters: Principal curvature: The maximum and minimum curvature values of the region surface, used to evaluate the flatness of label attachment. Curvature variance: The degree of change in surface curvature (as described in the above embodiments), and regions with high variance may need to be avoided. Mean curvature: The overall degree of bending, which affects the visual readability of the label.

[0113] 3. Edge feature parameters: Edge sharpness index: The sharpness of the edge, and insufficient sharpness may cause label positioning deviation. Edge continuity: Whether the edge is closed or has breaks, which affects the paste integrity of the label.

[0114] 4. Material property parameters: Surface roughness: Affects the adhesion of the printing material and the durability of the label. Adhesion coefficient: The affinity of the material for glue, which determines whether an additional adhesive layer is required.

[0115] 5. Spatial position and orientation parameters: Region center coordinates: The three-dimensional position on the object surface, used for robotic arm path planning. Normal direction: The orientation of the region surface, which determines the rotation angle of the label (such as parallel / perpendicular to the viewing angle).

[0116] 6. Topological parameters: Connectivity: Whether it is a simply connected region or contains multiple discrete sub-regions. Hole detection: Whether there are non-labelable holes or depressions within the region.

[0117] 7. Stability parameters: Material attachment stability: A prediction parameter that combines surface properties and environmental factors (such as humidity, temperature) to ensure label persistence. In industrial labeling scenarios, converting the region range parameters into two-dimensional bin packing constraints requires combining multi-dimensional requirements such as geometric adaptation, physical properties, and visual readability. The specific steps are as follows:

[0118] 1: Geometric parameter mapping. Define the basic layout boundary: Determine the maximum placeable range of the label through the geometric parameters of the target region: Minimum bounding rectangle (MBR): Input: The boundary dimensions of the target region (length L, width W). Constraint: Label size l ≤ L, w ≤ W; Optimization goal: Maximize the label area utilization rate l×w. Shape factor adaptation: Input: Aspect ratio R = L / W; Constraint: The label aspect ratio r = l / w needs to satisfy |r - R| ≤ ΔR (ΔR is the tolerance threshold). Adjust the label ratio through affine transformation, or divide the content into a multi-column layout.

[0119] 2: Curvature Compensation - Dynamically Adjust Label Deformation. For curved surface areas, introduce curvature constraints to solve the problem of label fitting deformation: Principal Curvature Compensation: Input: Maximum principal curvature kmax and curvature direction. Constraint: Character scaling ratio threshold s ≤ 1 / (1 + α × kmax) (α is the material elastic coefficient). Use NURBS curve to fit the label edge and dynamically adjust the character spacing. Curvature Variance Limit: Input: Curvature variance σ k2 . Constraint: The label needs to avoid high variance areas (when σ k2 > threshold, reduce the label size). Algorithm: Generate a mask based on the curvature field to limit the label layout in low variance areas.

[0120] 3. Edge Alignment and Sharpness Enhancement: Combine edge sharpness parameters to optimize label alignment accuracy: Edge Sharpness Constraint Input: Edge gradient magnitude: G. Constraint: The offset d between the label edge and the region edge ≤ β / G (β is the alignment tolerance coefficient). Align the label contour through Canny edge detection + ICP registration algorithm. Connectivity Verification: Input: Region connectivity marker (whether it is a single connected domain). Constraint: If the region is not connected, the label needs to be split into multiple sub - labels, and the distance between sub - labels > γ (to prevent overlap). Generate sub - label layout points through region skeleton extraction (Medial Axis).

[0121] 4: Material Stability Fusion: Convert material properties into physical constraints to ensure label adhesion reliability: Roughness Compensation: Input: Surface roughness Ra. Constraint: Minimum label contact area A min = λ × Ra (λ is the adhesion safety factor). Increase label edge anchor points through Delaunay triangulation. Adhesion Enhancement: Input: Adhesion coefficient μ. Constraint: The blank width b reserved at the label edge ≥ μ -1 × safety margin. Generate an annular buffer around the label to avoid edge warping.

[0122] 5. Multi-objective optimization modeling: Considering the above constraints, a two-dimensional bin-packing optimization model is constructed. The objective function is: max(k1×l×w / (L×W) + k2×sustainability score - k3×deformation penalty term); where k1×l×w / (L×W) is the area utilization rate; the weights k1, k2, and k3 are dynamically allocated by semantic element weights (for example, for safety warning labels, k2 needs to be increased, while for decorative labels, k1 may need to be increased). k1 reflects the user's emphasis on space utilization, k3 reflects the tolerance for label deformation caused by surface fitting, and l and w are the actual length and width of the label. The readability score quantifies the visual clarity of label content in the target area and is calculated by weighting the font contrast (the color difference between the label and the background (calculated by formulas such as the CIELAB color difference formula)), the information hierarchy (the layout priority of key information (such as barcodes, warning symbols) defined by semantic element weights), and the character recognizability (the ratio of character height to the viewing distance (such as the minimum viewing angle requirement in ISO standards)). The deformation penalty term is used to penalize label deformation caused by surface fitting. If the label is attached to a high-curvature area (such as the side of a cylinder), the characters may be distorted, and the label size needs to be restricted or forced to wrap by the penalty term. If the material has poor elasticity (such as hard plastic), slight deformation will trigger a high penalty.

[0123] The solution process uses mixed-integer programming (MIP) to handle discrete constraints (such as whether the characters wrap) and combines simulated annealing (SA) to optimize continuous variables (such as the label rotation angle).

[0124] The range of the target area (length L×width W) can also be transformed into a rectangular arrangement space. Define variables xi and yi as the upper-left coordinates of the i-th semantic element, and xj and yj as the upper-left coordinates of the j-th semantic element. The constraints are: xi + wi ≤ L, yi + hi ≤ W( ); where wi and hi are the element sizes, which are mapped by weights (for example, an element with a weight of 0.6 occupies 60% of the area).

[0125] Construct an objective function to maximize the information density and aesthetics: L = ∑wi×log(Ai) + λ∑(xi - xj) 2 s.t. two-dimensional bin-packing constraints; where Ai is the element area, λ is the aesthetic penalty coefficient, and the optimal layout is solved through the KKT conditions. xj represents the x-axis coordinate component of the j-th element.

[0126] GAT dynamically allocates weights according to the syntactic structure (for example, "dangerous goods" has the highest weight as the core noun). Compared with statistical methods such as TF-IDF, the recognition accuracy of important elements is increased by 25% (the F1 value increases from 0.72 to 0.90). For example, in a drug label, the "taboo" node in "Taboo: Pregnant women are prohibited" obtains a higher weight through GAT, ensuring that the font is enlarged and prominent.

[0127] In some embodiments, generating an optimized typesetting scheme corresponding to the label content according to the typesetting constraint conditions includes: constructing an intelligent typesetting architecture based on a generative adversarial network; the generator of the generative adversarial network adopts a multi-scale feature fusion network with a self-attention mechanism, and the discriminator of the generative adversarial network introduces a differentiable rendering module to simulate the actual printing effect; optimizing the generator parameters of the generative adversarial network based on reinforcement learning, and generating a reward function corresponding to the reinforcement learning according to information entropy, aesthetic evaluation score, and material matching degree; generating a style transfer vector according to preset visual specifications and typesetting constraint conditions, and embedding the style transfer vector into the latent space corresponding to the intelligent typesetting architecture; inputting the label content into the intelligent typesetting architecture to generate the optimized typesetting scheme.

[0128] In this embodiment, for the generation of the optimized typesetting scheme in step S104, an intelligent typesetting architecture based on a generative adversarial network (GAN) and reinforcement learning (RL) is proposed. The actual printing effect is simulated through differentiable rendering, and diverse visual specifications are adapted through style transfer, solving the problems of poor flexibility in traditional templated typesetting and strong subjectivity in aesthetic evaluation.

[0129] The multi-scale self-attention network of the generator contains 4 resolution branches (from 256×256 to 32×32), and each branch integrates a Transformer self-attention module to generate layout parameters (element positions, sizes, fonts) through cross-scale feature fusion. The semantic vector of the input label text (the weight output in Embodiment 4) is added to the style transfer vector (from the visual specification library).

[0130] The differentiable rendering module of the discriminator is used to render the layout parameters output by the generator into a bitmap, simulating effects such as halftone dithering and ink diffusion of the printer. The discriminator distinguishes between real label images (sampled from the industrial image library) and generated images, and the loss function is the Wasserstein GAN (WGAN) loss.

[0131] By maximizing the information entropy between elements, a balanced arrangement of content is encouraged. Based on a pre-trained ResNet-50 aesthetic scoring model (fine-tuned on the AVA dataset), a score from 0 to 1 is output. Calculate the attachment stability of the generated layout (for example, small fonts score low on rough materials). Use the PPO (Proximal Policy Optimization) algorithm to update the generator parameters, extract the typesetting style (font family, color scheme, icon style) from the enterprise visual specification library (such as ISO standards, brand VI manuals), and encode it into a 128-dimensional latent vector through VAE. Adjust the generator output according to different specification requirements (such as "medical serious style" and "consumer product fashionable style").

[0132] The discriminator adapts the layout output by the generator to the actual printing device by simulating printing effects (such as the toner particle texture of laser printing). Tests show that the PSNR value between the printed manuscript and the design manuscript reaches 38 dB (only 32 dB for traditional methods).

[0133] In some embodiments, based on the non-uniform rational B-spline curve fitting algorithm, a motion trajectory corresponding to a preset robotic arm is generated according to the first pose information corresponding to the object to be labeled, the second pose information of the preset printing device, and the region range corresponding to the target region, including: constructing a kinematic model based on the Lie group space for converting the first pose information and the second pose information into a rigid body transformation matrix in a preset space; performing path planning according to the kinematic model and the rigid body transformation matrix to generate an initial motion trajectory; wherein, a curvature continuity constraint of the non-uniform rational B-spline curve is introduced into the configuration space corresponding to the initial motion trajectory; simulating and calculating the dynamic obstacle avoidance information of the initial motion trajectory according to a preset physical engine, so as to optimize the initial motion trajectory according to the dynamic obstacle avoidance information to generate the motion trajectory; wherein, the dynamic obstacle avoidance information is calculated according to the collision probability, the kinetic energy of the obstacle, and the path energy consumption corresponding to the initial motion trajectory.

[0134] In this embodiment, aiming at the problem of generating the motion trajectory of the robotic arm in step S105, a path planning method based on kinematic modeling in the Lie group space and optimization of non-uniform rational B-spline (NURBS) curves is proposed, which combines the improved RRT* algorithm with physical engine simulation for dynamic obstacle avoidance to solve the problems of non-smooth trajectories and high collision risks of traditional planning algorithms under complex pose constraints.

[0135] By representing the poses of the robotic arm base (first pose) and the printing device (second pose) as Lie group elements \(T_1, T_2\in SE(3)\) respectively, that is:

[0136] ;

[0137] where \(R\) is the rotation matrix and \(t\) is the translation vector. The rigid body transformation matrix from the base to the printing device is obtained through Lie group operations , for subsequent path planning.

[0138] The improved RRT* extension includes introducing the curvature continuity constraint of the NURBS curve when sampling nodes in the configuration space (C-space) to ensure that the path between nodes meets the joint acceleration limit of the robotic arm. The curvature constraint includes that the maximum curvature of the path segment ≤ the square of the maximum joint acceleration / the commanded velocity.

[0139] Perform NURBS interpolation on the discrete path points generated by RRT* to generate a parameterized trajectory. Input the trajectory into the PyBullet physics engine to calculate the collision probability Pc, the kinetic energy of the obstacle Eobs (related to the speed of the moving obstacle), and the path energy consumption Epath. Construct a loss function λ1Pc + λ2Eobs + λ3Epath; λ1, λ2, and λ3 are the corresponding weights. Optimize the NURBS control points through gradient descent to minimize the value of the loss function.

[0140] In some embodiments, the controlling the robotic arm to paste the target label onto the target area according to the motion trajectory includes: obtaining the torque feedback information of the end effector corresponding to the robotic arm, and adjusting the dynamic parameters of the robotic arm according to the torque feedback information; monitoring the labeling process of the robotic arm in real time according to a preset on-line vision feedback device, and when it is detected that the pose of the object to be labeled has a deviation and / or the surface has an abnormal deformation, optimizing the motion trajectory in real time, so as to control the robotic arm to complete the labeling process according to the dynamically optimized dynamic parameters and motion trajectory.

[0141] This embodiment proposes a closed-loop control strategy based on torque feedback and on-line vision correction for the dynamic control of the labeling process of the robotic arm in step S105. By adjusting the dynamic parameters and trajectory parameters in real time, it solves the labeling deviation problem caused by the pose deviation and surface deformation of the object. According to the data of the six-dimensional force / torque sensor of the end effector, the joint impedance parameters (mass-damping-stiffness matrix M, D, K) are adjusted in real time.

[0142] A binocular RGB-D camera (frame rate 30Hz) and a laser profiler are used to reconstruct the 3D point cloud of the object surface in real time. The pose offset ΔT ∈ SE(3) and the deformation amount ΔS are detected by matching the point cloud with the CAD model through the ICP algorithm. If ΔS > Sth (deformation threshold), predict the deformation trend of the labeling area according to the finite element analysis (FEA) model and adjust the end contact force. If ΔT causes a collision risk for the original trajectory, trigger local NURBS trajectory optimization with a response delay < 50ms.

[0143] The torque feedback controls the fluctuation of the end contact force within ±1N (±5N for traditional PID control), and the label bubble rate is reduced by 90%. In the labeling of soft packaging bags, surface depressions cause label warping in traditional methods, and this embodiment achieves perfect fitting through force control adaptability.

[0144] The embodiments of the present application also provide an intelligent label printing optimization device. The intelligent label printing optimization device is used to execute the steps of the intelligent label printing optimization method based on image recognition shown in the above embodiments. The intelligent label printing optimization device can be a single server or a server cluster, or the intelligent label printing optimization device can be a terminal, and the terminal can be a handheld terminal, a laptop computer, a wearable device, a robot, etc.

[0145] The intelligent label printing optimization device includes:

[0146] A matrix construction unit, configured to obtain the surface three-dimensional point cloud data and texture information of the object to be labeled according to a preset multispectral image acquisition device, and to construct an object surface feature matrix including spatial curvature features and material attributes;

[0147] A target determination unit, configured to perform a regional adaptability evaluation on the object surface feature matrix, obtain a heat map including the probability distribution of the labelable area, and determine the target labelable area according to the maximum response value corresponding to the heat map;

[0148] An information acquisition unit, configured to obtain the semantic information corresponding to the preset label content, calculate the semantic element weights corresponding to the semantic information, and generate the layout constraint conditions corresponding to the label content according to the regional range, semantic element weights, and semantic information corresponding to the target labelable area; the layout constraint conditions at least include a character scaling ratio threshold, an information level visualization rule, a label image size parameter, and a material attachment stability parameter;

[0149] A solution generation unit, configured to generate an optimized layout solution corresponding to the label content according to the layout constraint conditions, and control a preset printing device to print according to the optimized layout solution to form a target label;

[0150] A labeling completion unit, configured to generate a motion trajectory corresponding to a preset robotic arm based on the non-uniform rational B-spline curve fitting algorithm according to the first pose information corresponding to the object to be labeled, the second pose information of the preset printing device, and the regional range corresponding to the target area, and control the robotic arm to paste the target label to the target area according to the motion trajectory.

[0151] It should be noted that those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the above-described intelligent label printing optimization device and each unit can refer to the corresponding processes in the embodiments of the intelligent label printing optimization method based on image recognition described in the above embodiments, and will not be repeated here.

[0152] The above intelligent label printing optimization method is implemented in the form of a computer program, and the computer program can run on the above device.

[0153] Please refer to Figure 3 , Figure 3 which is a schematic block diagram of the control device provided by an embodiment of the present application. The control device includes a processor, a memory, and a network interface connected through a device bus. Among them, the memory may include a storage medium and an internal memory.

[0154] The storage medium can store an operating device and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can be made to execute an embodiment of any intelligent label printing optimization method based on image recognition.

[0155] The processor is used to provide computing and control capabilities to support the operation of the entire control device.

[0156] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can be made to execute any intelligent label printing optimization system method.

[0157] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art can understand that Figure 3 the structure shown in

[0158] is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the terminal to which the solution of the present application is applied. The specific control device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0159] Among them, in one embodiment, the processor is used to run the computer program stored in the memory to implement the following steps:

[0160] Obtain the surface three-dimensional point cloud data and texture information of the object to be labeled according to a preset multispectral image acquisition device, and use it to construct an object surface feature matrix including spatial curvature features and material attributes;

[0161] Perform a regional adaptability assessment on the surface feature matrix of the object to obtain a heat map including the probability distribution of the labelable area, and determine the target labeling area according to the maximum response value corresponding to the heat map;

[0162] Obtain the semantic information corresponding to the preset label content, calculate the semantic element weights corresponding to the semantic information, and generate the layout constraint conditions corresponding to the label content according to the area range, semantic element weights, and semantic information corresponding to the target labeling area; the layout constraint conditions at least include the character scaling ratio threshold, the information level visualization rule, the label image size parameter, and the material attachment stability parameter;

[0163] Generate an optimized layout plan corresponding to the label content according to the layout constraint conditions, and control a preset printing device to print according to the optimized layout plan to form a target label;

[0164] Based on the non-uniform rational B-spline curve fitting algorithm, generate the motion trajectory of a preset robotic arm according to the first pose information corresponding to the object to be labeled, the second pose information of the preset printing device, and the area range corresponding to the target area, and control the robotic arm to paste the target label to the target area according to the motion trajectory.

[0165] It should be noted that those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described processor can refer to the corresponding process in the method embodiments described in the above-mentioned embodiments, and will not be repeated here.

[0166] An embodiment of the present application further provides a computer-readable storage medium, the computer-readable storage medium stores a computer program, the computer program includes program instructions, and the processor executes the program instructions to implement the steps of the intelligent label printing optimization method based on image recognition provided in the above embodiments of the present application.

[0167] Among them, the computer-readable storage medium may be an internal storage unit of the control device described in the foregoing embodiment, such as the hard disk or memory of the control device. The computer-readable storage medium may also be an external storage device of the control device, such as a plug-in hard disk equipped on the control device, a smart media card (SMC), a secure digital (SD) card, a flash card, etc.

[0168] As described above, it is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should all be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An intelligent label printing optimization method based on image recognition, characterized in that, Including: Obtain the surface three-dimensional point cloud data and texture information of the object to be labeled according to a preset multispectral image acquisition device, for constructing an object surface feature matrix including spatial curvature features and material properties; Conduct a regional adaptability evaluation on the object surface feature matrix to obtain a heat map including the probability distribution of labelable regions, so as to determine the target labeling region according to the maximum response value corresponding to the heat map; Obtain the semantic information corresponding to the preset label content, calculate the semantic element weights corresponding to the semantic information, so as to generate the typesetting constraint conditions corresponding to the label content according to the regional range, semantic element weights and semantic information corresponding to the target labeling region; the typesetting constraint conditions at least include character scaling ratio thresholds, information hierarchy visualization rules, label image size parameters and material adhesion stability parameters; Generate an optimized typesetting scheme corresponding to the label content according to the typesetting constraint conditions, so as to control a preset printing device to print according to the optimized typesetting scheme to form a target label; Based on the non-uniform rational B-spline curve fitting algorithm, generate the motion trajectory of a preset robotic arm according to the first pose information corresponding to the object to be labeled, the second pose information of the preset printing device and the regional range corresponding to the target region, so as to control the robotic arm to paste the target label to the target region according to the motion trajectory.

2. The method according to claim 1, characterized in that, The obtaining the surface three-dimensional point cloud data and texture information of the object to be labeled according to a preset multispectral image acquisition device, for constructing an object surface feature matrix including spatial curvature features and material properties, includes: Input the texture information into a preset parallel convolutional neural network to extract texture feature maps of multiple spectral channels; Input the surface three-dimensional point cloud data into a preset graph convolutional network to perform spatial curvature modeling on the three-dimensional point cloud data and output spatial curvature features including spectral reflectance and material properties; Perform feature tensor splicing on the texture feature maps and the spatial curvature features, and perform dimensionality reduction processing on the spliced feature tensor through a differentiable pooling layer to generate an object surface feature matrix with local geometric preservation.

3. The method according to claim 1, characterized in that The conducting a regional adaptability evaluation on the object surface feature matrix to obtain a heat map including the probability distribution of labelable regions, includes: Construct a deep region segmentation network; the deep region segmentation network adopts an encoder-decoder structure of the U-Net architecture, introduces a channel attention mechanism in the encoding stage for importance weighting of the object surface feature matrix; embeds a deformable convolution module in the decoding stage to dynamically adjust the receptive field to adapt to the morphological features of different curvature regions; Input the object surface feature matrix into the deep region segmentation network and output a probability distribution map including the label suitability of each pixel point; Process the probability distribution map according to morphological closing operation to eliminate discrete noise points and generate a heat map with regional connectivity.

4. The method according to claim 1, characterized in that, The determining the target labeling region according to the maximum response value corresponding to the heat map, includes: Construct an objective function according to the maximum regional area, minimum principal curvature variance and maximum edge sharpness index corresponding to the target labeling region; Optimize the thermal map according to a preset objective function and a multi-objective optimization algorithm to obtain multiple candidate labeling regions; Perform non-dominated sorting on the multiple candidate labeling regions according to the NSGA-II algorithm, and calculate the weight coefficient corresponding to each candidate labeling region after sorting according to the entropy weight method; In the multiple candidate labeling regions, obtain the continuous region with the weight coefficient greater than the preset threshold as the target labeling region.

5. The method according to claim 1, characterized in that, Calculating the semantic element weights corresponding to the semantic information, so as to generate the layout constraint conditions corresponding to the label content according to the region range, semantic element weights and semantic information corresponding to the target labeling region, including: Extract the semantic dependency tree corresponding to the semantic information according to a preset BERT model; Calculate the importance score of each semantic node of the semantic dependency tree based on a preset graph attention network as the semantic element weight; Convert the region range parameters corresponding to the region range into two-dimensional bin-packing constraints, and map the semantic element weights to the weighted coefficients of the information-level visualization rules; the region range parameters include one or more of geometric dimension parameters, curvature feature parameters, edge feature parameters, material property parameters, spatial position and orientation parameters, topological parameters, and stability parameters; Based on the Lagrange multiplier method, obtain the layout constraint conditions corresponding to the label content according to the two-dimensional bin-packing constraints and the weighted coefficients.

6. The method according to claim 1, wherein Generating the optimized layout plan corresponding to the label content according to the layout constraint conditions, including: Construct an intelligent layout architecture based on a generative adversarial network; the generator of the generative adversarial network adopts a multi-scale feature fusion network with self-attention mechanism, and the discriminator of the generative adversarial network introduces a differentiable rendering module to simulate the actual printing effect; optimize the generator parameters of the generative adversarial network based on reinforcement learning, and generate the reward function corresponding to the reinforcement learning according to the information entropy, aesthetic evaluation score, and material matching degree; Generate a style transfer vector according to the preset visual specification and layout constraint conditions, so as to embed the style transfer vector into the latent space corresponding to the intelligent layout architecture; Input the label content into the intelligent layout architecture to generate the optimized layout plan.

7. The method according to claim 1, wherein Based on the non-uniform rational B-spline curve fitting algorithm, generate the motion trajectory of a preset robotic arm according to the first pose information corresponding to the object to be labeled, the second pose information of the preset printing device, and the region range corresponding to the target region, including: Construct a kinematic model based on the Lie group space for converting the first pose information and the second pose information into a rigid body transformation matrix in a preset space; Perform path planning according to the kinematic model and the rigid body transformation matrix to generate an initial motion trajectory; wherein, a curvature continuity constraint of a non-uniform rational B-spline curve is introduced into the configuration space corresponding to the initial motion trajectory; Simulate and calculate the dynamic obstacle avoidance information of the initial motion trajectory according to a preset physical engine, so as to optimize the initial motion trajectory according to the dynamic obstacle avoidance information to generate the motion trajectory; wherein, the dynamic obstacle avoidance information is calculated according to the collision probability, obstacle kinetic energy, and path energy consumption corresponding to the initial motion trajectory.

8. The method according to claim 1, characterized in that Controlling the robotic arm to paste the target label onto the target area according to the motion trajectory includes: Obtaining the torque feedback information of the end effector corresponding to the robotic arm, and adjusting the dynamic parameters of the robotic arm according to the torque feedback information; Real-time monitoring the labeling process of the robotic arm by a preset online vision feedback device. When it is detected that the pose of the object to be labeled has a deviation and / or the surface has abnormal deformation, the motion trajectory is optimized in real time, so as to control the robotic arm to complete the labeling process according to the dynamically optimized parameters and the motion trajectory.

9. An intelligent label printing optimization device based on image recognition, characterized in that, It includes: A matrix construction module, configured to obtain the surface three-dimensional point cloud data and texture information of the object to be labeled according to a preset multi-spectral image acquisition device, and construct an object surface feature matrix including spatial curvature features and material attributes; A region determination module, configured to perform a region adaptability evaluation on the object surface feature matrix, obtain a heat map including the probability distribution of the labelable region, and determine the target labeling region according to the maximum response value corresponding to the heat map; An information acquisition module, configured to obtain the semantic information corresponding to the preset label content, calculate the semantic element weights corresponding to the semantic information, and generate the layout constraint conditions corresponding to the label content according to the region range, semantic element weights, and semantic information corresponding to the target labeling region; the layout constraint conditions at least include a character scaling ratio threshold, an information level visualization rule, a label image size parameter, and a material adhesion stability parameter; A scheme generation module, configured to generate an optimized layout scheme corresponding to the label content according to the layout constraint conditions, and control a preset printing device to print according to the optimized layout scheme to form a target label; A labeling completion module, configured to generate a motion trajectory corresponding to a preset robotic arm based on the non-uniform rational B-spline curve fitting algorithm according to the first pose information corresponding to the object to be labeled, the second pose information of the preset printing device, and the region range corresponding to the target region, so as to control the robotic arm to paste the target label onto the target region according to the motion trajectory.

10. An intelligent label printing optimization system based on image recognition, characterized in that, It includes: A multi-spectral image acquisition device, configured to obtain the surface three-dimensional point cloud data and texture information of the object to be labeled; A printing device, configured to print a target label; A robotic arm, configured to paste the target label on the object to be labeled; A control device, configured to construct an object surface feature matrix including spatial curvature features and material attributes according to the surface three-dimensional point cloud data and texture information; Perform regional adaptability evaluation on the surface feature matrix of the object to obtain a heat map including the probability distribution of the labelable area, and determine the target labeling area according to the maximum response value corresponding to the heat map; obtain the semantic information corresponding to the preset label content, calculate the semantic element weights corresponding to the semantic information, and generate the layout constraint conditions corresponding to the label content according to the regional range, semantic element weights and semantic information corresponding to the target labeling area; the layout constraint conditions at least include the character scaling ratio threshold, the information level visualization rule, the label image size parameter and the material adhesion stability parameter; generate an optimized layout plan corresponding to the label content according to the layout constraint conditions, and control a preset printing device to print according to the optimized layout plan to form a target label; Based on the non-uniform rational B-spline curve fitting algorithm, generate the motion trajectory of a preset robotic arm according to the first pose information corresponding to the object to be labeled, the second pose information of the preset printing device and the regional range corresponding to the target area, and control the robotic arm to paste the target label to the target area according to the motion trajectory.

Images