A mechanical arm body polishing method and system based on double-domain semantic verification

Abstract

The application discloses a kind of mechanical arm body polishing methods and systems based on double-domain semantic verification in the technical field of industrial robot, and the method comprises the following steps: obtaining operation instruction and collecting three-dimensional information;Operation instruction is analyzed, and is mapped into semantic feature vector;Seven-dimensional geometric feature vector of target area is extracted, and is mapped into geometric semantic feature vector;The cosine similarity of and is calculated, and the exponential Euclidean distance penalty term between geometric feature vector and normalized vector of standard template is calculated;Double-domain joint matching score is obtained by weighted summation, when the score is not lower than preset threshold, allow to execute processing, otherwise reject execution and request to reconfirm.The application carries out joint quantization verification in processing semantic domain and workpiece geometric feature domain two information domains, prevents misprocessing caused by instruction ambiguity or spatial positioning error from mechanism, significantly improves the reliability and safety of man-machine interaction in industrial polishing operation.

Description

Technical Field

[0001] This invention relates to the field of industrial robot technology, and in particular to a robotic arm-based polishing method and system based on dual-domain semantic verification. It is applicable to automated robotic polishing of parts made of metal, plastic and other materials in industrial settings, and is especially suitable for industrial scenarios with high requirements for human-machine interaction reliability and error prevention capabilities. Background Technology

[0002] With the rapid development of intelligent manufacturing technology, industrial robot polishing systems have been widely used in precision manufacturing. The need for natural interaction between operators and robot systems is becoming increasingly prominent. Operators expect to convey their processing intentions to the robot system through intuitive methods such as natural language commands, allowing the system to autonomously complete target area identification, trajectory planning, and processing execution. However, while this human-machine interaction method improves operational convenience, it also introduces new safety hazards.

[0003] In existing industrial polishing systems, operators typically specify the processing area using a teach pendant or offline programming software, and the system directly executes the processing actions based on the operating instructions. This type of system has the following technical drawbacks: First, traditional systems typically rely solely on a single semantic judgment to confirm the machining target. That is, they only check whether the semantics of the operation instruction itself are recognizable, without verifying whether the geometric properties of the workpiece area pointed to by the instruction physically match the machining semantics. When the operation instruction is ambiguous or the operator inputs an instruction that does not match the intention due to fatigue, negligence, or other reasons, the system cannot distinguish between the two fundamentally different states of "semantic resolvability" and "target area consistent with machining semantics," resulting in a significant risk of erroneous machining.

[0004] Second, existing industrial polishing systems lack a joint quantitative verification mechanism for the semantics of operating instructions and the physical geometric characteristics of the workpiece, and there is no mathematical framework to comprehensively quantify semantic similarity and geometric fit into a unified confidence score. This results in the system being unable to identify potential mismatches between instructions and targets through cross-validation in abnormal situations such as ambiguous operating instructions, human error, or spatial positioning deviations, leading to a severe deficiency in the ability to prevent misoperation in industrial settings.

[0005] Third, the existing system does not fully verify the legality of the target area before processing. It mainly relies on manual review by operators before execution, which has a limited degree of automation and lacks quantitative verification result records, making it difficult to trace and analyze the processing decision-making process. Summary of the Invention

[0006] This application provides a robotic arm-based polishing method and system based on dual-domain semantic verification, which solves the problem of erroneous processing caused by incorrect or ambiguous instructions in existing industrial polishing systems. By using a mechanism to jointly quantify and verify operation instructions in two independent dimensions—processing semantics and workpiece geometric features—it provides reliable safety protection for industrial polishing operations.

[0007] This application provides a robotic arm-based polishing method based on dual-domain semantic verification, comprising the following steps: S1, Command Acquisition and 3D Information Acquisition: Receives operation commands input by the operator, which include semantic descriptions of the processing target and spatial positioning information of the target area, and acquires 3D point cloud data of the workpiece surface through a 3D vision sensor; S2, Semantic Feature Vector Extraction: Semantic parsing of the operation instructions to extract a set of semantic tags consisting of processing part type, processing action semantics, and target area identifier. Through semantic encoding mapping function Will Mapped to a unified verification embedding space In 3D semantic feature vector ; S3, Geometric Feature Vector Construction: Based on the spatial positioning information, a subset of the target region point cloud is extracted from the 3D point cloud data, and a seven-dimensional geometric feature vector is extracted. ,in The mean value of curvature amplitude, The standard deviation of the curvature amplitude, and For the maximum and minimum principal curvature, The average deflection angle of the normal vector. Let be the surface area. The mean curvature of the point-by-point region; for the seven-dimensional geometric feature vector Dimensional normalization yields normalized geometric eigenvectors. And through geometric semantic mapping function Will Mapped to the unified verification embedding space In 3D geometric semantic feature vector ; S4, Dual-domain joint matching score calculation: Query and obtain the semantic label set from the geometric feature knowledge base. Corresponding standard geometric feature template vector and its normalized vector ; Simultaneously based on the semantic feature vector With the geometric semantic feature vector Calculate normalized cosine similarity in the semantic domain of processing. And based on the normalized geometric feature vector The normalized vector of the standard geometric feature template Calculate the exponential geometric distance penalty term in the geometric feature domain of the workpiece. Then, the joint matching score of the two domains is obtained by weighted summation. ,in This is the distance attenuation coefficient. w1 and w2 are weighting coefficients. , and ; S5, Processing legality judgment: When the dual-domain joint matching score Not lower than the preset threshold If the processing target is deemed legitimate and credible, and processing is permitted, then trajectory planning and polishing processing will be performed; when... Below the preset threshold When this occurs, the rejection mechanism is triggered, prohibiting the processing and sending a reconfirmation request to the operator.

[0008] The beneficial effects of the above embodiments are as follows: This method overcomes the shortcomings of traditional single-domain verification methods that cannot cross-verify whether the semantics of the instruction and the geometric and physical features of the target area are consistent by jointly quantifying and verifying the two information domains of the processing semantic domain and the workpiece geometric feature domain. It effectively prevents erroneous processing caused by ambiguity of operation instructions or spatial positioning errors from a mechanism perspective, and significantly improves the reliability and safety of human-machine interaction in industrial polishing operations.

[0009] Based on the above embodiments, this application can be further improved as follows: In one embodiment of this application, in step S3, the seven-dimensional geometric feature vector include: Reflecting the overall curvature of the region, the local principal curvature is calculated for each point in the point cloud of the target region. and Further statistics show that... ; To synthesize the curvature amplitude point by point The standard deviation of curvature amplitude within the target area reflects the uniformity of curvature variation in the area. , Used to identify extreme bending features; It reflects the degree of dispersion of the region's normal vectors and is used to distinguish between planar regions and complex curved surface regions. It is obtained by calculating the mean angle between the normal vector of each point and the average normal vector of the region. It is used to constrain the size of the region and is obtained by summing the areas of the triangular mesh faces of the target region. The mean curvature of each point within the target region is used to describe the overall signed bending trend of the target region, and is combined with... and The numerical and sign relationships help distinguish different surface types such as convex, concave, and saddle-shaped surfaces; the seven-dimensional geometric feature vectors according to Perform dimensional normalization, where and For the first The calibration mean and calibration scale parameter of each geometric feature component .

[0010] Technical effects: The specific calculation methods for each component in the seven-dimensional geometric feature vector are defined to ensure that each feature component has a clear physical meaning and a reproducible calculation process, making the geometric features extracted under different working conditions comparable, and providing a standardized input basis for the quantitative verification of the geometric feature domain of the workpiece.

[0011] In one embodiment of this application, in step S4, the normalized vector of the standard geometric feature template vector... The following methods are used to pre-establish and store the data in the geometric feature knowledge base: A classification system is established according to the type of machining part, including sharp edge regions, rounded corner regions, planar regions, and hole feature regions; geometric measurements are performed on typical workpiece standard samples for each type of machining part, and standard seven-dimensional geometric feature vectors are extracted. According to the same normalization parameters as in step S3 and Dimensional normalization is performed to obtain .

[0012] Technical benefits: By introducing a geometric feature knowledge base, the systematized management of processing part types and standard geometric templates is realized, enabling dual-domain verification to quickly obtain standard templates based on historical calibration data, improving verification efficiency and ensuring the consistency of verification standards between different operations.

[0013] In one embodiment of this application, in step S4, the normalized cosine similarity... middle, , For vector dot product, It is the L2 norm. , When the semantic encoding or geometric semantic mapping result is a zero vector, the current feature encoding is deemed invalid, cosine similarity calculation is not performed, and a re-confirmation or re-extraction process is triggered. The exponential geometric distance penalty term middle, The Euclidean distance between the normalized vector of the standard geometric feature template vector and the normalized geometric feature vector; the dual-domain joint matching score ∈ [0,1].

[0014] Technical effect: It ensures the physical interpretability and dimensional consistency of the scoring function, giving the verification results a clear confidence level meaning.

[0015] In one embodiment of this application, step S5 is followed by step S6, trajectory planning and machining execution: based on the geometric features of the target area confirmed by dual-domain semantic verification and the corresponding type of process reference information obtained from the geometric feature knowledge base, a machining trajectory adapted to the workpiece surface is generated, and the robotic arm is driven to perform polishing operation according to the planned trajectory; the process reference information includes at least one of machining trajectory line spacing, trajectory step length, end contact force range, tool rotation speed range, and feed speed range.

[0016] Technical effect: By linking the dual-domain verification results with the process reference information in the geometric feature knowledge base, an automated chain is realized from "verification passed" to "automatic matching of process parameters" and then to "processing execution", reducing the step of manually setting process parameters and improving processing efficiency.

[0017] In one embodiment of this application, step S6 is followed by step S7, which is a processing quality verification: After polishing, the geometric features of the processing area are re-measured, and the re-measured geometric features are compared with the standard geometric features of the target area before processing. The processing quality is evaluated according to the process specifications. If the processing quality does not meet the process requirements, the quality defect area is located, the local trajectory is re-planned, and the robotic arm is driven to perform re-polishing on the defect area. After the re-polishing is completed, the quality verification is performed again. The process is repeated until the quality meets the standard or the preset maximum number of re-polishing times is reached. If the maximum number of re-polishing times is reached but the standard is still not met, manual intervention is requested.

[0018] Technical effect: The closed-loop verification mechanism for processing quality achieves automatic closed-loop management of processing quality through the cyclical control of retesting-comparison-re-grinding, effectively reducing the risk of workpiece scrap due to substandard polishing in a single polishing. At the same time, the maximum number of re-grinding attempts is limited to avoid infinite cycles, thus balancing processing quality and production efficiency.

[0019] In one embodiment of this application, step S1 further includes an instruction conflict pre-check step: performing a consistency pre-check on the processing part type described by the natural language instruction in the operation instruction and the preliminary geometric attributes of the area pointed to by the spatial positioning instruction. If the consistency pre-check fails, a conflict warning is triggered and the operator is requested to reconfirm.

[0020] Technical benefits: The addition of a pre-instruction conflict detection step creates a two-tiered protection system of "coarse detection + fine detection," which can intercept obvious instruction conflicts at an early stage, avoiding unnecessary geometric feature extraction and semantic encoding calculations, thereby improving both system efficiency and security.

[0021] This application also provides a robotic arm-based polishing system based on dual-domain semantic verification, employing the aforementioned robotic arm-based polishing method, including: The perception layer includes: an industrial-grade 3D vision sensor for acquiring 3D point cloud data of the workpiece surface; an operation command acquisition module for receiving natural language commands and spatial positioning commands input by the operator; a force feedback module for acquiring contact force information when the robotic arm end effector contacts the workpiece; and an environmental monitoring module for monitoring the working environment. The cognitive layer includes: an instruction semantic parsing module, used to perform semantic analysis and semantic feature extraction on operation instructions; a workpiece geometric feature analysis module, used to extract the seven-dimensional geometric feature vector of the target workpiece region from point cloud data; and a dual-domain semantic verification unit, used to perform joint quantitative verification of the machining semantic domain and the workpiece geometric feature domain, calculate the dual-domain joint score, and determine the threshold. The decision layer is used to generate a machining trajectory sequence adapted to the workpiece surface based on the confirmed geometric features of the target area and existing trajectory planning methods in the field after the dual-domain semantic verification unit has passed the determination, and to complete the matching of machining process reference information. The execution layer is used to drive the robotic arm to complete the polishing operation according to the trajectory planned by the decision layer, and to monitor the execution status and processing quality in real time.

[0022] Technical Effects: This system adopts a four-layer architecture design consisting of a perception layer, a cognition layer, a decision-making layer, and an execution layer. The dual-domain semantic verification unit is deployed in the core cognition layer, forming a complete system link from data acquisition to semantic understanding, dual-domain verification, trajectory planning, processing execution, and quality monitoring, providing a systematic dual-domain protection solution for industrial polishing sites. Attached Figure Description

[0023] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. The elements or parts in the drawings are not necessarily drawn to scale.

[0024] Figure 1 This is a flowchart illustrating the steps of a robotic arm-based polishing method based on dual-domain semantic verification in an embodiment of this application. Detailed Implementation

[0025] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. After reading the present invention, any modifications of the present invention in various equivalent forms by those skilled in the art will fall within the scope defined by the appended claims.

[0026] Example 1: like Figure 1 As shown, a robotic arm-based polishing method based on dual-domain semantic verification includes the following steps: S1, Command Acquisition and 3D Information Acquisition: Receives operation commands input by the operator, which include semantic descriptions of the processing target and spatial positioning information of the target area, and acquires 3D point cloud data of the workpiece surface through a 3D vision sensor.

[0027] Specifically: Operators input operation instructions through a human-machine interface. The operation instructions include a semantic description of the processing target and spatial positioning information of the target area. The system receives the instructions and stores them in the current operation instruction record.

[0028] The workpiece is scanned using an industrial-grade 3D vision sensor to collect 3D point cloud data of its surface. A subset of the point cloud data of the target region is then extracted based on spatial positioning commands for subsequent geometric feature extraction. Preprocessing of the point cloud data employs conventional and mature methods in this field.

[0029] S2, Semantic Feature Vector Extraction: Semantic parsing of operation instructions to extract a set of semantic tags consisting of processing part type, processing action semantics, and target area identifier. Through semantic encoding mapping function Will Mapped to a unified verification embedding space In 3D semantic feature vector .

[0030] S3, Geometric Feature Vector Construction: Based on spatial positioning information, a subset of the target region's point cloud is extracted from the 3D point cloud data, and a seven-dimensional geometric feature vector is extracted. ,in The average curvature amplitude reflects the overall curvature of the region. Local principal curvature is calculated for each point in the point cloud of the target region. and Then, the statistics were obtained; To synthesize the curvature amplitude point by point The standard deviation of curvature amplitude within the target area reflects the uniformity of curvature variation in the area. and For the maximum and minimum principal curvature, , Used to identify extreme bending features; The mean angle of the normal vector reflects the degree of dispersion of the normal vector in the region. It is used to distinguish between planar regions and complex curved surface regions. It is obtained by calculating the mean angle between the normal vector at each point and the average normal vector of the region. The surface area is used to constrain the size of the region and is obtained by summing the areas of the triangular meshes in the target region. The mean curvature of each point within the target region is used to describe the overall signed bending trend of the target region, and is combined with... and The numerical and sign relationships help distinguish different surface types such as convex, concave, and saddle-shaped surfaces; and the seven-dimensional geometric feature vectors according to Dimensional normalization yields normalized geometric eigenvectors. And through geometric semantic mapping function Will Mapped to a unified verification embedding space In 3D geometric semantic feature vector , .in and For the first The calibration mean and calibration scale parameter of each geometric feature component Before calculating the geometric features of the target region, the normal vectors of the point cloud in the target region are subjected to consistent orientation processing, so that the measured point cloud of the target region and the standard geometric feature template adopt the same normal direction convention; wherein, the unit normal vector of each point in the point cloud undergoes consistent orientation processing, and the principal curvature and The signed principal curvatures are all calculated according to the same normal direction.

[0031] In steps S2-S3, the natural language portion of the operation instructions is first semantically parsed to extract the semantic tag set S, and then... Mapped to semantic feature vectors Simultaneously, normalize the geometric feature vector of the target region. pass Mapped to geometric semantic feature vectors ,in and All are located in the unified verification embedding space E, providing input for processing semantic domain verification.

[0032] S4, Dual-domain joint matching score calculation: querying and retrieving the semantic label set from the geometric feature knowledge base. Corresponding standard geometric feature template vector and its normalized vector ; Simultaneously based on semantic feature vectors Geometric semantic feature vectors Calculate normalized cosine similarity in the semantic domain of processing. , , For vector dot product, It is the L2 norm. , When the semantic encoding or geometric semantic mapping result is a zero vector, the current feature encoding is deemed invalid, cosine similarity calculation is not performed, and a re-confirmation or re-extraction process is triggered. ; and based on normalized geometric eigenvectors Normalized vector of standard geometric feature template Calculate the exponential geometric distance penalty term in the workpiece's geometric feature domain. ; Then, the joint matching score of the two domains is obtained by weighted summation. ,in This is the distance attenuation coefficient. w1 and w2 are weighting coefficients. , and .

[0033] S5, Processing Legality Judgment: When the dual-domain joint matching score is... Not lower than the preset threshold When the processing target is deemed legitimate and credible, processing is permitted; when... Below the preset threshold When this occurs, the rejection mechanism is triggered, prohibiting the processing and sending a reconfirmation request to the operator.

[0034] Specifically, when score ≥ τ, the processing target is determined to be legal, reliable, and executable, and trajectory planning and grinding processing are performed; when score < τ, the system refuses to execute, sends a reconfirmation request to the operator, and records the current score value and the sub-values ​​of the two domains for the operator's reference, and the process returns to step S1.

[0035] Furthermore, the trajectory planning and polishing steps in the above method include: S6, Trajectory Planning: Based on the geometric features of the target area confirmed by dual-domain semantic verification, existing trajectory planning methods in this field are used, combined with corresponding type process reference information from the workpiece surface geometry and geometric feature knowledge base, to generate a machining trajectory sequence adapted to the workpiece surface. Specific trajectory planning algorithms and parameters are set according to actual process requirements, and conventional trajectory generation methods in the field of industrial polishing are adopted.

[0036] Processing Execution and Monitoring: The robotic arm is driven to perform polishing operations along a planned trajectory. Real-time contact force information is collected at the end effector via a force feedback module. Based on process requirements, the contact force is monitored in real-time, and process reference information is dynamically adjusted. Force feedback control employs conventional control methods in this field, with control parameters adjusted according to industrial site requirements.

[0037] S7, Processing Quality Verification: After polishing, the geometric information of the processed area is re-measured. The processing quality monitoring module compares the re-measured geometric features with the standard geometric features of the target area before processing and evaluates the processing quality according to the process specifications. If the processing quality meets the process requirements, the operation is deemed qualified and the process ends. If the processing quality does not meet the process requirements, the system locates the defective area, resubmits the geometric information of the defective area to the decision-making layer for local trajectory replanning, drives the robotic arm to perform re-polishing of the defective area, and performs quality verification again after re-polishing. This process is repeated until the quality meets the standard or the preset maximum number of re-polishing times is reached. If the maximum number of re-polishing times is reached and the standard is still not met, manual intervention is requested.

[0038] The core improvement of this invention lies in constructing a dual-domain semantic verification mechanism for industrial polishing scenarios, and using a quantitative scoring function to accurately measure the confidence level of the matching between the processing intention and the physical features of the workpiece. Dual-domain semantic verification refers to the system simultaneously performing joint quantitative verification of the correspondence between the processing intention specified by the operation command and the target workpiece region in two independent and complementary information domains: the processing semantic domain and the workpiece geometric feature domain.

[0039] The processing semantic domain is further explained as follows: Perform semantic parsing on the operation instructions and extract the set of semantic tags related to the processing parts in the instructions. A semantic coding mapping function is constructed using conventional semantic coding methods. Map S to a d-dimensional semantic feature vector in the unified verification embedding space E. , Simultaneously, normalize the geometric feature vector of the target region. Through geometric semantic mapping function Mapped to the same unified verification embedding space E, representing d-dimensional geometric semantic feature vectors. , Then, the normalized cosine similarity will be calculated. , as a verification metric for processing semantic domains.

[0040] The geometric feature domain of the workpiece is further explained as follows: Calculate the principal curvatures κ1(pi) and κ2(pi) at each point within the target workpiece region, and construct a seven-dimensional geometric feature vector. ,in The mean value of curvature amplitude, The standard deviation of the curvature amplitude, and For the maximum and minimum principal curvatures, The average deflection angle of the normal vector. Let be the surface area. This represents the mean curvature of the region; since the dimensions of each component are different, g is first normalized according to the calibration parameters. and the standard geometric feature template vector Normalization Then calculate the exponential Euclidean distance penalty between the two. , as a verification index for the geometric feature domain of the workpiece.

[0041] The following is a further explanation of the two-domain semantic verification: Dual-domain joint quantization scoring: The validation results from the two domains are combined into a unified dual-domain joint matching score through weighted summation. ; in , w1 and w2 are weighting coefficients. , and λ is the distance attenuation coefficient, λ>0; score∈[0,1]. When the score is not lower than the preset threshold τ, the system determines that the processing target is legal, reliable, and executable, and allows the subsequent processing flow to be triggered; when the score is lower than the preset threshold τ, the rejection mechanism is triggered, execution is prohibited, and reconfirmation is requested.

[0042] The dual-domain joint scoring function adopts the following form, ensuring its reasonableness as a confidence score: (1) Range analysis: cosine similarity Therefore, normalized cosine similarity Exponential penalty items Since λ>0 and the Euclidean distance Therefore, the exponent is not positive. When the two normalized geometric eigenvectors are completely identical, the penalty term takes the maximum value of 1; as the Euclidean distance increases, the penalty term monotonically decreases and approaches 0. Therefore, under the constraint that w1+w2=1 and w1>0, w2>0, score∈[0,1], the theoretical maximum value score=1 is only obtained when both domains are completely matched, which meets the range requirements of the confidence score.

[0043] (2) Monotonicity analysis: Let , ,but The score monotonically increases with respect to the cosine similarity c. In other words, the greater the semantic similarity, the higher the score; the score is monotonically non-increasing with respect to the Euclidean distance d. When d > 0, In other words, the smaller the Euclidean distance between the normalized geometric feature vectors, the higher the geometric matching degree and the higher the score. Monotonicity ensures that the scoring function reflects the degree of matching between the two domains consistently and is physically interpretable.

[0044] (3) Heterogeneous complementarity: The normalized cosine similarity term measures the similarity between semantic and geometric meanings in the unified verification embedding space, and is sensitive to the embedding direction; the exponential Euclidean distance term measures the degree of agreement of physical geometric attributes in the normalized geometric feature space, avoiding distance distortion caused by the direct addition of different units. The two terms originate from different information links, forming a heterogeneous complementary verification mechanism. A decrease in the matching degree of any domain will lead to a decrease in the overall score; when the reduced score is lower than the preset threshold τ, a rejection mechanism is triggered, which is superior to the robustness of single-domain verification when the information is incomplete.

[0045] (4) Parameter adjustability: The weight coefficients w1 and w2 can be adjusted according to the relative emphasis on semantic confidence and geometric confidence in the industrial site; the attenuation coefficient λ controls the sensitivity of geometric distance penalty, and the larger λ is, the more sensitive it is to geometric deviation; the threshold τ is calibrated according to the safety requirements and false alarm rate requirements of the industrial site; all of the above parameters have clear engineering meanings and can be determined by those skilled in the art according to conventional engineering calibration methods.

[0046] (5) Degeneracy verification: When w2=0, the score degenerates to pure processing semantic domain similarity verification; when w1=0, the score degenerates to pure geometric feature domain verification; the present invention adopts a dual-domain joint scheme with w1>0 and w2>0, which has more complete verification capabilities than any single-domain degeneracy scheme.

[0047] The seven-dimensional geometric eigenvectors are further explained below: Seven-dimensional geometric eigenvectors The geometric properties of the target workpiece region are comprehensively characterized from multiple dimensions. The physical meaning and calculation method of each component are as follows: (Mean curvature amplitude): This is the average value of the comprehensive curvature amplitude within the target area, reflecting the overall curvature degree of the area. The κmean in the sharp-edge region is significantly higher than that in the planar region, and is one of the most important feature components for distinguishing the type of processing part.

[0048] (Standard deviation of curvature amplitude): The degree of dispersion of curvature distribution, reflecting the uniformity of curvature variation in a region. κstd is usually higher in transitional regions than in homogeneous regions.

[0049] , (Maximum and minimum principal curvature): Extreme values ​​of principal curvature, used to identify extreme curvature features, such as sharp edges. (maximum) and saddle-shaped surface ( and (different sign).

[0050] (Average deflection angle of normal vector): The average angle between the normal vectors of all points within the target area and the average normal vector of the area. It reflects the degree of dispersion of the area's normal vectors and is used to distinguish planar areas. (Approaching 0) and complex curved surface regions.

[0051] (Surface area): The geometric area of ​​the target region, used to constrain the region size and prevent matching errors caused by deviations in the region's extent.

[0052] (Mean mean curvature of the region): Mean curvature , Preserving the sign information of the principal curvature is used to describe the overall signed bending trend of the target region; combined with and The numerical and sign relationships can help distinguish different surface types such as convex, concave, and saddle surfaces, providing a more complete geometric description than a single curvature amplitude index.

[0053] The aforementioned seven-dimensional feature vector design covers the main geometric dimensions for identifying the processing part type in industrial polishing scenarios, enabling sufficient discriminative power in verifying the workpiece's geometric feature domain. The calculation of each feature component employs conventional geometric feature calculation methods in this field, and the calculation parameters are calibrated and determined by those skilled in the art based on workpiece characteristics and process requirements.

[0054] The following three typical operating conditions illustrate the working process of the dual-domain semantic verification mechanism to verify the physical rationality of the scoring function.

[0055] Condition 1: The instruction and target are a perfect match (verification passed); The operator inputs a voice command to grind the sharp right edge of the workpiece, while simultaneously using a spatial positioning command to point to the sharp right edge area of ​​the workpiece (the point cloud geometric feature extraction results for this area are as follows:). High, Larger Much larger It exhibits typical sharp-edge geometric features.

[0056] Processing semantic domain: Semantic parsing extracts the semantic label set S containing sharp edges and polishing, and semantic feature vectors. Geometric semantic feature vector of the target region They are oriented close to each other in the unified verification embedding space E. High; Workpiece geometric feature domain: Normalized seven-dimensional geometric feature vector of the target region Standardized normalized geometric feature template vectors for acute edge classes in the geometric feature knowledge base The Euclidean distance is small, and the exponential penalty term is small. Close to 1; Overall rating: The value is high enough that the score ≥ τ is satisfied, the double-domain check passes, and the processing is allowed.

[0057] Condition 2: The semantics are correct, but the geometric features of the target area do not match (the workpiece geometric feature domain fails to pass). The operator input a voice command to grind the sharp right edge of the workpiece, but due to an operational error, the spatial positioning command was directed to the flat area on the side of the workpiece (this area...). (Approaching zero, exhibiting typical planar geometric characteristics).

[0058] Processing semantic domain: Semantic parsing is correct. High; Workpiece geometric feature domain: Normalized seven-dimensional geometric feature vector of the target region (Plane features with near-zero curvature) and the standard normalized template vector of the sharp-edge class The Euclidean distance is large, and the exponential penalty term is large. Significantly small, approaching 0; Overall score: Under the condition that the weight coefficients and thresholds have been safely calibrated, a significantly smaller geometric matching degree can cause the score to drop below the threshold τ, resulting in the failure of the dual-domain verification. The system will then trigger the rejection mechanism and prompt the operator to reconfirm the target area.

[0059] This test case verified the effective interception capability of the workpiece geometric feature domain for operational errors that are semantically correct but have incorrect targets.

[0060] Condition 3: The geometric features of the target region are reasonable, but the semantic description is ambiguous (processing semantic domain fails). The operator pointed to an area that was actually rounded ( The region exhibits medium-sized, rounded corner geometric features, but due to ambiguity in the language expression, the semantic parsing module interprets the instruction as a grinding planar semantic, leading to... Geometric and semantic features of the target region Directional deviation in the unified verification embedding space E Too low.

[0061] Processing semantic domain: Significantly lower due to semantic bias; Workpiece geometric feature domain: Calculate the principal curvatures κ1(pi) and κ2(pi) at each point within the target workpiece region, and construct a seven-dimensional geometric feature vector g, where... The mean value of curvature amplitude, The standard deviation of the curvature amplitude, and For the maximum principal curvature and the minimum principal curvature, The average deflection angle of the normal vector. Let be the surface area. The mean curvature of the region is given. Since the dimensions of each component are different, g is first normalized to a normalized geometric feature vector according to the calibration parameters, and the standard geometric feature template vector is normalized to a standard normalized geometric feature template vector. Then, the exponential Euclidean distance penalty between the two is calculated as a verification index of the workpiece geometric feature domain.

[0062] Overall score: The score is low overall, below the threshold τ, and the dual-domain validation fails. The system requests the operator to reconfirm the intent of the instruction.

[0063] This test case verifies the effective detection capability of the processing semantic domain for semantic ambiguity, and also shows that the geometric feature domain can cross-constrain the inconsistency between the invoked semantic category and the actual geometric features of the target region. This mechanism does not require the geometric feature domain to be completely independent of the semantic parsing results, but rather forms a complementary verification between the scoring results of different information links.

[0064] The industrial security significance of the dual-domain semantic verification mechanism of this invention is reflected in the following aspects.

[0065] First, by using a dual-domain joint quantification scoring system, a quantitative threshold for execution is established at the control logic level, preventing processing actions from being directly triggered due to errors in a single information domain. In abnormal situations such as operator fatigue or inaccurate command input in industrial settings, the dual-domain rejection mechanism effectively prevents erroneous processing, protecting the safety of the workpiece and equipment.

[0066] Secondly, when the score falls below the threshold τ, the system forcibly refuses to execute the operation and actively requests the operator to reconfirm, providing the operator with an opportunity to verify the processing intention and preventing the operator from directly entering a dangerous processing state without realizing the error. This control logic conforms to the design principles of fail-safe industrial safety equipment.

[0067] Third, by recording the dual-domain score and its sub-values ​​(cosine similarity term and geometric distance penalty term) in a structured manner, the system has the ability to quantitatively trace the verification process of each processing operation, providing a reliable basis for quality management and accident analysis.

[0068] In summary, this invention constructs a dual-domain joint scoring system by jointly quantifying and verifying the processing semantic domain and the workpiece geometric feature domain. A reliable dual-layer quantization protection mechanism has been introduced into the human-machine interaction link of the industrial polishing system. Unlike the traditional single-domain verification method, the dual-domain joint quantization verification significantly improves the reliability and safety of human-machine interaction in industrial polishing operations, and has clear industrial application value.

[0069] Furthermore, in this embodiment, the geometric feature knowledge base is the fundamental support for the normal operation of the dual-domain semantic verification mechanism of this invention, and mainly includes the following: (1) Classification system of processing parts: Classify according to the geometric attributes of workpiece areas commonly used in industrial polishing operations. Typical categories include sharp edge area (category label: sharp_edge), rounded corner area (category label: fillet), flat area (category label: flat), hole feature area (category label: hole), etc. In actual deployment, the classification system is expanded according to the type of workpiece in the industrial site.

[0070] (2) Standard geometric feature template vector Normalization parameters: For each type of processing part, a corresponding seven-dimensional standard geometric feature template vector is established. Record the typical geometric feature reference values ​​for this type of part; at the same time, record the normalized parameters of each geometric component. and , used to obtain and Template vector data and normalized parameters are obtained by geometric measurement and calibration of standard samples of typical workpieces. The specific calibration method adopts conventional engineering calibration methods.

[0071] in: , , 。

[0072] (3) Standard semantic feature description: For each type of processing part, record the semantic description words and synonyms commonly used by operators in the industrial site, for use in semantic encoding and comparison in the processing semantic domain verification; the standard semantic feature description is established based on the actual language habits in the industrial site.

[0073] (4) Process Reference Information Configuration: Process reference information: For each type of machining part, record the process reference information related to the polishing operation. The process reference information includes, but is not limited to, machining trajectory line spacing, trajectory step length, end contact force range, tool speed range, feed rate range, limit on the number of re-polishing times, and machining quality judgment threshold. The above process reference information is used to provide a reference for trajectory planning, machining execution, and quality verification after the dual-domain semantic verification is passed. Its specific values ​​can be determined by trial machining verification or conventional engineering calibration methods according to the workpiece material, surface quality requirements, tool type, and on-site process requirements.

[0074] Example 2: A robotic arm-based polishing system based on dual-domain semantic verification is used to execute the method described in Embodiment 1, comprising a four-layer architecture of perception layer, cognition layer, decision layer and execution layer.

[0075] The perception layer is responsible for collecting the raw information required for system operation, mainly including: an industrial-grade 3D vision sensor for acquiring 3D point cloud data of the workpiece surface; an operation command acquisition module for receiving natural language commands and spatial positioning commands input by the operator; a force feedback module for acquiring contact force information when the robotic arm's end effector contacts the workpiece; and an environmental monitoring module for monitoring the working environment. Each module in the perception layer uses a conventional industrial field sensor configuration, with specific sensor selection determined based on process requirements.

[0076] The cognitive layer is the core layer of the system of this invention, mainly including: an instruction semantic parsing module, used to perform semantic analysis and semantic feature extraction on operation instructions; a workpiece geometric feature analysis module, used to extract the seven-dimensional geometric feature vector of the target workpiece region from point cloud data; a dual-domain semantic verification unit, used to perform joint quantization verification of the machining semantic domain and the workpiece geometric feature domain, calculate the dual-domain joint score and make threshold determination; and an instruction conflict detection unit, used to perform consistency pre-check on operation instruction information from different sources before entering the dual-domain verification process.

[0077] After the dual-domain semantic verification is passed, the decision-making layer generates a machining trajectory sequence adapted to the workpiece surface based on the confirmed geometric features of the target area and uses existing trajectory planning methods in the field, and completes the automatic matching of machining process reference information.

[0078] The execution layer is responsible for driving the robotic arm to complete the polishing operation according to the trajectory instructions output by the decision layer. It monitors the processing process in real time through force feedback and performs processing quality verification after processing is completed, forming a quality-driven processing quality closed loop.

[0079] The workflow of the dual-domain semantic verification unit includes the following steps: Step 1: Command conflict pre-check; Operators describe their processing intentions using natural language commands (e.g., grinding the sharp edge on the right side of the workpiece) and simultaneously determine the target workpiece area using spatial positioning commands. The command conflict detection unit performs a pre-check on these two types of information: if there is a significant inconsistency between the processing part type described by the natural language command and the geometric attributes of the area pointed to by the spatial positioning command (e.g., the language command describes a sharp edge, but the area pointed to by the spatial positioning command is determined by the perception layer to have planar geometric features), the consistency pre-check fails, and an alert is issued prompting the operator to reconfirm; the process does not proceed to the dual-domain verification process. After the pre-check passes, the process proceeds to dual-domain semantic verification.

[0080] Step 2: Semantic Feature Extraction (Processing Semantic Domain Input Preparation); The instruction semantic parsing module uses existing industrial semantic parsing methods to analyze natural language instructions, extracting semantic tags such as processing part type (e.g., sharp edge, rounded corner, flat surface), processing action semantics (e.g., grinding, polishing), and target area identifiers, thus forming a semantic tag set. .

[0081] Construct a semantic coding mapping function using conventional semantic coding methods (such as bag-of-words model, TF-IDF features, or industrial semantic dictionaries, which are mature methods in this field). Map S to a d-dimensional semantic feature vector in the unified verification embedding space E. The specific encoding method and the selection of dimension d are determined based on the actual language habits and geometric feature knowledge base design in the industrial field.

[0082] Step 3: Extraction of workpiece geometric features (Preparation of workpiece geometric feature domain input); Based on spatial positioning commands, the workpiece geometric feature analysis module extracts a subset of the target region's point cloud from the 3D point cloud data acquired by the sensing layer. The point cloud is then preprocessed (using conventional and mature methods in the field, including noise reduction and coordinate transformation). Subsequently, a seven-dimensional geometric feature vector is extracted using conventional geometric feature calculation methods in the field. ; in, The mean value of the curvature amplitude of the target region; The standard deviation of the curvature amplitude; and These are the maximum and minimum values ​​of the principal curvature, respectively. This is the mean of the angles between the normal vectors at each point and the average normal vector of the region; The area of ​​the target region's surface; This represents the mean curvature of the region.

[0083] The calculation parameters for each characteristic component are determined by those skilled in the art based on the workpiece characteristics and process requirements.

[0084] Since the curvature, angle, and area components in g have different dimensions, the calibrated mean is used first. With calibration scale parameters Normalize each component to obtain ,in Then, conventional geometric semantic mapping methods (such as normalized concatenation, lookup table mapping, or feature transformation, etc., which are mature methods in this field) are used to construct... ,Will Mapped to d-dimensional geometric semantic feature vectors in the unified verification embedding space E This is used for calculating the cosine similarity of the semantic domain.

[0085] Simultaneously, the standard geometric feature template vector corresponding to the processing part type determined by the semantic label set S is queried from the geometric feature knowledge base. And obtained using the same normalization parameter It is used for calculating the Euclidean distance in the geometric feature domain of a workpiece.

[0086] Step 4: Joint quantitative scoring across two domains; The dual-domain semantic verification unit performs joint quantization verification in two functionally complementary heterogeneous information domains and calculates the dual-domain joint matching score: ; The physical meanings and calculation methods of each parameter are as follows: Normalized cosine similarity term : ; in Let be the inner product of two vectors. , When the semantic encoding or geometric semantic mapping result is a zero vector, the current feature encoding is determined to be invalid, cosine similarity calculation is not performed, and reconfirmation or re-extraction processing is triggered. It is an L2 norm. and They are all located in the unified verification embedding space E, so cosine similarity can be calculated. The closer the value is to 1, the higher the degree of matching in the semantic domain of the processing.

[0087] Exponential geometric distance penalty term : ; Euclidean distance d metric normalized geometric feature template vector Normalized seven-dimensional geometric feature vectors of the target region Geometric deviation between; exponential mapping The unbounded Euclidean distance is compressed to the interval (0,1], and exhibits a monotonically non-increasing response to increasing distance. When the Euclidean distance is zero, the penalty term is 1; as the Euclidean distance increases, the penalty term approaches zero. λ>0 is the attenuation coefficient; the larger the value of λ, the more sensitive the penalty term is to geometric deviation.

[0088] Overall rating: ; The weighting coefficients w1>0, w2>0, and w1+w2=1, ensuring that score∈[0,1]. The specific values ​​of w1 and w2 are determined by those skilled in the art based on the relative emphasis on semantic confidence and geometric confidence in industrial settings. The value of λ is determined by those skilled in the art based on the scale and accuracy requirements of the normalized geometric features.

[0089] The mathematical properties of the scoring function guarantee that: the score monotonically increases with respect to cosine similarity; the score increases with respect to normalized Euclidean distance. Monotonically non-increasing; the two domains are heterogeneous and complementary. Under the condition that the threshold τ is calibrated for safety, a low value in either domain can reduce the overall score and trigger the rejection mechanism.

[0090] Step 5: Threshold determination and execution decision; The calculated score is compared with the preset threshold τ: When score ≥ τ: the verification is passed, the system determines that the processing target is legal, reliable and executable, sends an execution permission signal to the decision layer, and transmits the confirmed target area's geometric features and semantic information to the decision layer for trajectory planning.

[0091] When score < τ: the verification fails, the system triggers the rejection mechanism, prohibits the sending of execution signals to the decision-making level, and issues a warning to the operator's interface, indicating that there is an inconsistency between the processing semantics and the geometric features of the target region, requesting the operator to review and confirm the processing intention, and the process returns to the instruction acquisition step.

[0092] The threshold τ should be set by comprehensively considering the tolerance for the risk of erroneous processing in the industrial environment and the operational efficiency requirements, and should be determined by a person skilled in the art based on the characteristics of the workpiece and the safety requirements of the industrial environment. The higher the τ, the more stringent the verification and the lower the risk of erroneous processing, but the operator needs more precise command input; the lower the τ, the greater the operational flexibility, but the risk of misjudgment increases accordingly.

[0093] The technical solutions described in the embodiments of this application have at least the following technical effects or advantages: (1) Improve processing reliability through dual-domain joint quantization verification: Unlike the traditional single-domain verification method, this invention performs quantization verification of the processing target from both the semantic and geometric layers before processing. The dual-domain information corroborates each other, effectively reducing the risk of misprocessing caused by judgment errors or information ambiguity in a single information domain.

[0094] (2) Provide a unified confidence quantification score: score∈[0,1] provides a clear quantitative indicator for the verification result, making the verification process numerically recordable and traceable, and improving the standardization of industrial site quality management.

[0095] (3) The scoring function has clear mathematical characteristics and can be physically interpreted: the function range, monotonicity and heterogeneous complementarity all have clear mathematical guarantees and industrial physical meanings, and the parameters all have clear engineering adjustment semantics, which facilitates engineering calibration and industrial application.

[0096] (4) Dual-domain rejection mechanism enhances industrial security: When the matching degree of either domain is significantly low, the score can be lowered to trigger rejection, providing operators with an opportunity to verify the processing intention and avoiding the direct execution of dangerous operations.

[0097] (5) Good engineering adaptability: The dual-domain verification mechanism is implemented using engineering control logic and conventional feature calculation methods. It does not depend on a specific algorithm framework and can be adapted to various robotic arm platforms and industrial sensors, thus possessing good engineering versatility.

[0098] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A robotic arm-based polishing method based on dual-domain semantic verification, characterized in that, Includes the following steps: S1: Receives operation instructions from the operator, which include a semantic description of the processing target and spatial positioning information of the target area, and collects three-dimensional point cloud data of the workpiece surface through a three-dimensional vision sensor; S2: Perform semantic parsing on the operation instructions to extract a set of semantic tags consisting of the processing part type, processing action semantics, and target area identifier. Through semantic encoding mapping function Mapped to a unified verification embedding space In 3D semantic feature vector ; S3, Geometric Feature Vector Construction: Based on the spatial positioning information, a subset of the target region point cloud is extracted from the 3D point cloud data, and a seven-dimensional geometric feature vector is extracted. ,in The mean value of curvature amplitude, The standard deviation of the curvature amplitude, and For the maximum and minimum principal curvature, The average deflection angle of the normal vector. Let be the surface area. The mean curvature of the region at each point; For the seven-dimensional geometric feature vector Dimensional normalization yields normalized geometric eigenvectors. And through the geometric semantic mapping function Mapped to the unified verification embedding space In 3D geometric semantic feature vector ; S4, Dual-domain joint matching score calculation: Query the geometric feature knowledge base to obtain the semantic label set. Corresponding standard geometric feature template vector and its normalized vector ; Based on the semantic feature vector With the geometric semantic feature vector Calculate normalized cosine similarity in the semantic domain of processing. And based on the normalized geometric feature vector The normalized vector of the standard geometric feature template Calculate the exponential geometric distance penalty term in the workpiece's geometric feature domain. Then, the joint matching score of the two domains is obtained by weighted summation. ,in This is the distance attenuation coefficient. w1 and w2 are weighting coefficients. , and ; S5, Processing legality judgment: When the dual-domain joint matching score Not lower than the preset threshold When the processing target is deemed legitimate and credible, processing is permitted, and trajectory planning and polishing are performed; when Below the preset threshold At this time, processing is prohibited and a reconfirmation request is sent to the operator.

2. The robotic arm body polishing method according to claim 1, characterized in that: In step S3, the seven-dimensional geometric feature vector middle: Reflecting the overall curvature of the region, the local principal curvature is calculated for each point in the point cloud of the target region. and Further statistics show that... ; To synthesize the curvature amplitude point by point The standard deviation of curvature amplitude within the target area reflects the uniformity of curvature variation in the area. , Used to identify extreme bending features; It reflects the degree of dispersion of the region's normal vectors and is used to distinguish between planar regions and complex curved surface regions. It is obtained by calculating the mean angle between the normal vector of each point and the average normal vector of the region. It is used to constrain the size of the region and is obtained by summing the areas of the triangular mesh faces of the target region. Used to describe the overall signed curvature trend of the target area, and combined with and The numerical and sign relationships help distinguish different surface types; For the seven-dimensional geometric feature vector according to Perform dimensional normalization, where and For the first The calibration mean and calibration scale parameter of each geometric feature component .

3. The robotic arm body polishing method according to claim 2, characterized in that: In step S4, the normalized vector of the standard geometric feature template vector The following methods are used to pre-establish and store the data in the geometric feature knowledge base: A classification system is established according to the type of machining part, including sharp edge regions, rounded corner regions, planar regions, and hole feature regions; geometric measurements are performed on typical workpiece standard samples for each type of machining part, and standard seven-dimensional geometric feature vectors are extracted. According to the same normalization parameters as in step S3 and Dimensional normalization is performed to obtain .

4. The robotic arm body polishing method according to claim 1, characterized in that: In step S4, the normalized cosine similarity middle, , For vector dot product, It is the L2 norm. , , The exponential geometric distance penalty term middle, The Euclidean distance between the normalized vector of the standard geometric feature template vector and the normalized geometric feature vector; the dual-domain joint matching score ∈ [0,1].

5. The robotic arm body polishing method according to claim 1, characterized in that: It also includes step S6, trajectory planning and processing execution: based on the geometric features of the target area confirmed by dual-domain semantic verification and the corresponding type of process reference information obtained from the geometric feature knowledge base, a processing trajectory adapted to the workpiece surface is generated, and the robotic arm is driven to perform polishing operation according to the planned trajectory. The process reference information includes at least one of the following: machining trajectory line spacing, trajectory step length, end contact force range, tool rotation speed range, and feed speed range.

6. The robotic arm body polishing method according to claim 5, characterized in that: Step S6 is followed by step S7, which is a processing quality verification: After polishing, the geometric features of the processing area are re-measured, and the re-measured geometric features are compared with the standard geometric features of the target area before processing. The processing quality is evaluated according to the process specifications. If the processing quality does not meet the process requirements, the quality defect area is located, the local trajectory is re-planned, and the robotic arm is driven to perform re-polishing on the defect area. After the re-polishing is completed, the quality verification is performed again. This process is repeated until the quality meets the standard or the preset maximum number of re-polishing times is reached. If the maximum number of re-polishing times is reached but the standard is still not met, manual intervention is requested.

7. The robotic arm body polishing method according to claim 1, characterized in that: Step S1 also includes an instruction conflict pre-check step: performing a consistency pre-check on the processing part type described by the natural language instruction in the operation instruction and the preliminary geometric attributes of the area pointed to by the spatial positioning instruction. If the consistency pre-check fails, a conflict warning is triggered and the operator is requested to reconfirm.

8. A robotic arm-based polishing system based on dual-domain semantic verification, characterized in that, The robotic arm body polishing method as described in any one of claims 1-7 includes: The perception layer includes: an industrial-grade 3D vision sensor for acquiring 3D point cloud data of the workpiece surface; an operation command acquisition module for receiving natural language commands and spatial positioning commands input by the operator; a force feedback module for acquiring contact force information when the robotic arm end effector contacts the workpiece; and an environmental monitoring module for monitoring the working environment. The cognitive layer includes: an instruction semantic parsing module, used to perform semantic analysis and semantic feature extraction on operation instructions; a workpiece geometric feature analysis module, used to extract the seven-dimensional geometric feature vector of the target workpiece region from point cloud data; and a dual-domain semantic verification unit, used to perform joint quantitative verification of the machining semantic domain and the workpiece geometric feature domain, calculate the dual-domain joint score, and determine the threshold. The decision layer is used to generate a machining trajectory sequence adapted to the workpiece surface based on the confirmed geometric features of the target area and existing trajectory planning methods in the field after the dual-domain semantic verification unit has passed the determination, and to complete the matching of machining process reference information. The execution layer is used to drive the robotic arm to complete the polishing operation according to the trajectory planned by the decision layer, and to monitor the execution status and processing quality in real time.

Images