A machine sorting method, system, device, and medium based on vector paper semantic parsing.

Abstract

This specification provides a machine sorting method, system, device, and medium based on vector drawing semantic parsing. The method includes: acquiring geometric and non-structural information based on vector drawings; determining part attribute relationships based on the geometric and non-structural information; determining execution attributes based on the multiple part attribute relationships of multiple vector drawings and material sorting rules; generating a sorting task list based on the execution attributes; and controlling a robot to execute sorting tasks based on the sorting task list.

Description

Technical Field

[0001] This specification relates to the field of machine sorting, and in particular to a machine sorting method, system, device and medium based on vector paper semantic parsing. Background Technology

[0002] With the deepening of industrial and intelligent manufacturing, automated and flexible production lines have become standard configurations in modern factories. In industries such as sheet metal processing, aerospace, and automobile manufacturing, CNC cutting centers (such as laser and plasma cutting machines) have been widely adopted, capable of efficiently cutting entire sheets of sheet metal into dozens or even hundreds of parts of various shapes. However, the sorting process after cutting—that is, how to quickly and accurately sort the cut parts from the scrap rack and classify and stack them according to material, batch, order, etc.—still relies heavily on manual labor. This work is labor-intensive, inefficient, and prone to errors, omissions, or even scratches due to human fatigue or negligence, becoming a key bottleneck restricting the overall efficiency and quality improvement of the production line. Industrial robots have significant advantages in high precision, high reliability, and continuous operation, making them an ideal choice to replace manual labor in sorting tasks. To achieve automated sorting by robots, the core lies in providing them with a precise "brain" and "eyes," namely, a command system that tells the robot "where to pick up which parts" and "where to place the parts." On the one hand, the main technical approach currently used to guide robot material sorting is machine vision. This typically involves deploying cameras above the sorting station to scan the cut sheet and using image recognition algorithms to locate each part in real time. However, this method is highly dependent on the environment, requires real-time processing of the entire complex scene, and cannot identify the categorization of parts. On the other hand, the source of all geometric and process information for parts comes from the DXF format nesting diagram generated during the design phase. This diagram not only contains the precise coordinates and outline shape of each part on the sheet metal but also embeds attributes such as part number and material through layers and text annotations. However, current technology typically only uses the DXF diagram to generate cutting paths (NC code). Once cutting is complete, the diagram's purpose is fulfilled, and the rich information it contains is not used to guide the downstream material sorting process, causing an interruption in the information flow. The material sorting robot becomes "blind" because it cannot directly utilize this "original blueprint."

[0003] Therefore, it is desirable to provide a machine sorting method, system, device, and medium based on vector graphics semantic parsing, which can perform in-depth analysis and intelligent processing of DXF nesting diagrams, extracting a complete information chain that can be used to guide cutting and directly drive the sorting robot to perform sorting operations, thereby realizing the full-process digitalization and seamless connection from "cutting" to "sorting". Summary of the Invention

[0004] One embodiment of this specification provides a machine sorting method based on semantic parsing of vector graphics. The method includes: acquiring geometric and unstructured information based on vector graphics; determining part attribute relationships based on the geometric and unstructured information; determining execution attributes based on multiple part attribute relationships from multiple vector graphics and through sorting rules, wherein the execution attributes include sorting target location, execution sequence, and target material frame; generating a sorting task list based on the execution attributes; and controlling a robot to execute sorting tasks based on the sorting task list.

[0005] One embodiment of this specification provides a machine sorting system based on vector drawing semantic parsing. The system includes an acquisition module, a determination module, and a control module. The acquisition module is configured to acquire geometric and non-structural information based on vector drawings. The determination module is configured to determine part attribute relationships based on the geometric and non-structural information. Based on multiple part attribute relationships from multiple vector drawings, and through sorting rules, it determines execution attributes, including sorting target location, execution sequence, and target material frame. The control module is configured to generate a sorting task list based on the execution attributes and control the robot to execute sorting tasks based on the sorting task list.

[0006] This specification provides one or more embodiments of a machine sorting device based on vector paper semantic parsing, including a processor for executing the aforementioned machine sorting method based on vector paper semantic parsing.

[0007] This specification provides one or more embodiments of a computer-readable storage medium that stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes a machine sorting method based on vector paper semantic parsing. Attached Figure Description

[0008] This specification will be further described by way of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting; in these embodiments, the same reference numerals denote the same structures, wherein:

[0009] Figure 1 These are exemplary block diagrams of a machine sorting system based on vector paper semantic parsing, as shown in some embodiments of this specification.

[0010] Figure 2 This is an exemplary flowchart of a machine sorting method based on vector paper semantic parsing, as shown in some embodiments of this specification.

[0011] Figure 3 These are exemplary schematic diagrams of the association model shown according to some embodiments of this specification;

[0012] Figure 4 This is an exemplary schematic diagram illustrating the generation of a sorting task list according to some embodiments of this specification. Detailed Implementation

[0013] To more clearly illustrate the technical solutions of the embodiments in this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely some examples or embodiments of this specification. For those skilled in the art, these drawings can be applied to other similar scenarios without creative effort. Unless obvious from the context or otherwise specified, the same reference numerals in the drawings represent the same structures or operations.

[0014] It should be understood that the terms “system,” “device,” “unit,” and / or “module” used herein are one way to distinguish different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.

[0015] As indicated in this specification and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of expressly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0016] Flowcharts are used in this specification to illustrate the operations performed by the system according to embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.

[0017] Figure 1 This is an exemplary block diagram of a machine sorting system based on vector paper semantic parsing, according to some embodiments of this specification. In some embodiments, the machine sorting system 100 may include an acquisition module 110, a relationship determination module 120, an attribute determination module 130, and a control module 140.

[0018] The acquisition module 110 refers to the module used to acquire information related to the part. The information related to the part may include geometric information and non-structural information.

[0019] In some embodiments, the acquisition module 110 is configured to acquire geometric and non-structural information based on vector graphics.

[0020] The relationship determination module 120 refers to the module used to determine the relationship between part attributes.

[0021] In some embodiments, the relationship determination module is configured to determine part attribute relationships based on geometric information and non-structural information.

[0022] In some embodiments, the relationship determination module 120 is configured to: construct an attribute association graph, the nodes of which include geometric information, sequence numbers of non-structural information, and text entities; and determine the attribute relationships of parts based on the attribute association graph through an association model, wherein the association model is a graph neural network model, and the attribute relationships of parts include the size, part number, material, and process of the parts.

[0023] The attribute determination module 130 refers to the module used to determine the execution attributes.

[0024] In some embodiments, the attribute determination module 130 is configured to determine the execution attribute based on the attribute relationships of multiple parts on multiple vector drawings and through material allocation rules.

[0025] Control module 140 refers to the module used to control the robot to perform sorting tasks.

[0026] In some embodiments, the control module 140 is configured to generate a sorting task list based on execution attributes, and control the robot to perform sorting tasks based on the sorting task list.

[0027] In some embodiments, the control module 140 may be further configured to: determine the shape complexity and deformation tendency of the part based on geometric information; determine a target grasping rule from multiple preset grasping rules based on the shape complexity and deformation tendency, and generate a grasping strategy for the part based on the target grasping rule, the grasping strategy including grasping direction, grasping point position and suction cup distribution; and generate a sorting task list based on the grasping strategy.

[0028] For more information about the above modules, please see Figures 2-4 And its explanation.

[0029] It should be noted that the above description of the machine sorting system and its modules based on vector paper semantic parsing is for ease of description only and should not be construed as limiting this specification to the scope of the embodiments described. It is understood that those skilled in the art, after understanding the principles of this system, may arbitrarily combine the various modules or construct subsystems connected to other modules without departing from these principles. In some embodiments, Figure 1The acquisition module, determination module, and control module disclosed herein can be different modules within a single system, or a single module can implement the functions of the two aforementioned modules. For example, the modules can share a single storage module, or each module can have its own separate storage module. Such variations are all within the scope of protection of this specification.

[0030] Figure 2 This is an exemplary flowchart of a machine sorting method based on vector paper semantic parsing, according to some embodiments of this specification. In some embodiments, process 200 may be executed by a processor, and process 200 may include steps 210-240. In some embodiments, the processor may be integrated into one or more modules of acquisition module 110, relationship determination module 120, attribute determination module 130, and control module 140.

[0031] In some embodiments, the processor may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), an application-specific instruction set processor (ASIP), a physical arithmetic processing unit (PPU), a digital signal processor (DSP), a processor, a microprocessor unit, a reduced instruction set computer (RISC), a microprocessor, or any combination thereof. In some embodiments, the processor may be local or remote. In some embodiments, the processor may be implemented on a cloud platform. By way of example only, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an internal cloud, a multi-tiered cloud, or any combination thereof. The processor may be local or remote.

[0032] Step 210: Obtain geometric and non-structural information based on vector graphics.

[0033] Vector graphics are digital graphic files that use geometric primitives such as points, lines, curves, and polygons to represent images. For example, vector graphics can be DXF (Drawing Exchange Format) files, DWG (AutoCAD Drawing) files, or other similar engineering drawing files.

[0034] In some embodiments, vector graphics drawings contain block instances. A block instance is a geometric combination of entities such as parts and standard parts in a vector graphics drawing. Block attributes refer to attribute information in a block instance, such as part number, material, and specifications.

[0035] In some embodiments, the processor can retrieve vector graphics directly from the storage device, or a technician can upload the vector graphics to the machine sorting system.

[0036] Geometric information refers to data that describes the spatial shape, size, position, and other characteristics of an object. For example, geometric information may include the outer contour, inner contour, location and size of holes, geometric center, and area of ​​a part.

[0037] In some embodiments, the processor can reconstruct discrete straight lines and arcs into complete boundaries with inner and outer loop relationships through various methods, thereby obtaining geometric information.

[0038] For example, the processor can read and parse vector graphics using a DXF parsing library, extracting raw data such as primitives (e.g., lines, arcs), layers, and text. The raw data is preprocessed by filtering out redundant primitives (annotations, dimension lines, etc.) irrelevant to the part's contour, retaining only the geometric primitives describing the part's shape (discrete lines, arcs, etc.). A contour tracing / loo detection algorithm based on primitive clustering (adjacency / connectivity search) is then used to cluster and trace the geometric primitives, reconstructing them into complete closed contours representing individual parts. After contour reconstruction, the acquisition module can further calculate the geometric center and area of ​​each part's contour based on the complete closed contours.

[0039] Unstructured information refers to information used to describe an object's attributes, specifications, or identifiers, but which does not directly define the object's geometry. For example, unstructured information may include a part's material, thickness, part number, customer name, and manufacturing process.

[0040] In some embodiments, the processor may acquire unstructured information through a variety of methods.

[0041] For example, the processor can extract specific attribute values ​​such as material, thickness, part number, and process from vector graphics by using regular expressions or keyword matching (such as matching "material", "thickness", etc.).

[0042] For example, the processor can also obtain unstructured information by associating with external databases. External databases refer to databases outside the system that store unstructured information about multiple components, including Manufacturing Execution Systems (MES), Product Lifecycle Management (PLM) systems, etc., and can be predetermined.

[0043] Step 220: Determine the relationship between part attributes based on geometric and non-structural information.

[0044] Part attribute relationships refer to the correspondence or association between geometric information and non-structural information. For example, a part attribute relationship can be a binding relationship established between the geometric information of a part and the corresponding non-structural information of that part.

[0045] In some embodiments, the processor can determine the component attribute relationships in a variety of ways.

[0046] In some embodiments, the processor may employ a spatial proximity-based matching method to determine part attribute relationships.

[0047] As an example, the processor can use the Point-in-Polygon (PIP) algorithm to determine whether the insertion point or geometric center of each text entity falls inside a recognized part contour. If it does, the processor can bind the attribute information parsed from the text entity to the part contour, thus establishing part attribute relationships. If it does not, the processor can use a nearest neighbor search algorithm to determine the part attribute relationships, calculating the shortest distance between the text entity and all part contours. If the shortest distance is less than a preset distance threshold, the processor associates the corresponding unstructured information of the text entity with the nearest part contour to obtain the part attribute relationships. For a description of text entities, see [link to documentation]. Figure 3 And its contents.

[0048] In some embodiments, the processor can also parse text entities or layer names to determine part attribute relationships. As an example only, the processor can place parts of a specific material or thickness on a predefined layer and set the layer name (e.g., set the layer name to "STEEL_3MM"). It can directly parse non-structural information from the layer name where the part is located and match the non-structural information to the corresponding part through geometric relationships to establish part attribute relationships.

[0049] In some embodiments, part attributes can be obtained from block attributes, layer names, and associated text. That is, a part can obtain multiple part attributes through multiple channels. In response to the processor obtaining multiple part attributes of a part through multiple methods, the processor can bind the part with the highest priority part attribute. The priority can be block attribute > external database > layer name > text entity. For more information on determining part attribute relationships, see [link to documentation]. Figure 3 And its contents.

[0050] Step 230: Based on the attribute relationships of multiple parts in multiple vector drawings, determine the execution attributes through material allocation rules.

[0051] Material sorting rules refer to the conditions or criteria used to guide how to classify and handle parts with different properties. For example, material sorting rules can be set as follows: all parts made of 'stainless steel' with a thickness of less than 5 mm are placed in material frame 1; all parts made of 'carbon steel' are placed in material frame 2. In some embodiments, material sorting rules can be set by the processor based on default presets.

[0052] Execution attributes are specific parameters used to guide sorting actions. In some embodiments, execution attributes include the sorting target location, execution sequence, and target material box.

[0053] The sorting target location refers to the target spatial coordinates where the part should be placed during the sorting process. For example, the sorting target location can be a specific coordinate point within the target material box that the robot's end effector needs to reach.

[0054] An execution sequence refers to the order or priority in which multiple sorting actions are performed. For example, an execution sequence can be a number that instructs the robot to pick up the part with sequence number 1 first, and then pick up the part with sequence number 2.

[0055] A target crate is a container or designated area used to receive and store a specific category of sorted parts. For example, a target crate can be a physical cart, pallet, or specific storage location on a shelf in the production area.

[0056] In some embodiments, the processor can traverse each part and its attribute relationships, and match the part attribute relationships with the material allocation rules in the material allocation rule library to determine the material allocation rules. The material allocation rule library is a database that stores material allocation rule vectors, where each vector represents the relationship between part attribute relationships and material allocation rules. In some embodiments, the material allocation rule library can be constructed by technicians based on historical data.

[0057] For example, for a part with non-structural information of {material: carbon steel, thickness: 8 mm}, the processor matches the classification rule as "parts made of 'carbon steel' should be placed in frame 2", thus determining that the target frame for this part is frame 2. The sorting target position can be determined as the coordinate of the next available empty space in frame 2, and the execution sequence is assigned incrementally according to the matching order.

[0058] Step 240: Generate a sorting task list based on the execution attributes, and control the robot to execute sorting tasks based on the sorting task list.

[0059] A sorting task list refers to a collection of one or more sorting tasks. For example, a sorting task list may include location information and all the action logic required to ensure successful sorting.

[0060] A sorting task refers to a single, complete pick-and-place operation instruction for a single part or a group of parts. For example, a sorting task can be a single, complete operation that "grabs part A from coordinates (x1, y1) on the cutting table and places it at coordinates (x2, y2) on target hull number 2".

[0061] In some embodiments, the processor can parse and determine the gripping pose of the part, assign a part ID to the part, determine the sorting order, and package and serialize the gripping pose, part ID, and sorting order into a sorting task list.

[0062] For example, the processor can parse parts using a DXF parsing library to obtain the closed contour of the part; calculate the centroid of the part based on the closed contour; calculate the grasping pose of the part based on the position of the centroid; assign a part ID to each part to be sorted; determine the sorting order of the parts through an optimization algorithm; encapsulate the grasping pose, part ID, and sorting order into a data packet; and serialize the data packet into a sorting task list. The grasping pose refers to the robot's posture when grasping the part, its source position coordinates, and the coordinates of the placement point. The source position coordinates can be obtained from geometric information. The part ID is a unique identifier for the part. The sorting order refers to the order in which the robot performs the sorting tasks. For explanations of the optimization algorithm and data packet, see [link to documentation]. Figure 4 And its contents.

[0063] In some embodiments, the processor can control the robot to perform sorting tasks via industrial network communication based on a sorting task list. The processor can send the generated JSON-formatted sorting task list to the robot's programmable logic controller (PLC) via TCP / IP protocol. The PLC parses the sorting task list, converts each sorting task into a specific motion instruction, and drives the robot to complete the gripping and placement operations of the parts one by one according to the execution sequence and motion instructions. The processor can also receive real-time feedback from the robot on the execution status (such as successful gripping, placement completed) and dynamically adjust subsequent sorting tasks based on the feedback.

[0064] The methods provided in some embodiments of this specification automatically acquire the geometric and non-structural information of parts and establish accurate attribute relationships by directly performing semantic parsing on vector drawings. This replaces traditional manual drawing interpretation and data entry, significantly reducing sorting errors caused by human mistakes. By automatically generating a structured sorting task list through preset sorting rules, a seamless connection is achieved from design drawings to robot execution, greatly improving the efficiency and accuracy of automated steel plate blanking and sorting scenarios and reducing reliance on human experience.

[0065] It should be noted that the above description of process 200 is for illustrative purposes only and does not limit the scope of this specification. Those skilled in the art can make various modifications and changes to process 200 under the guidance of this specification. However, these modifications and changes remain within the scope of this specification.

[0066] Figure 3 This is an exemplary schematic diagram of an association model shown according to some embodiments of this specification.

[0067] In some embodiments, the processor can construct an attribute association graph 310, the nodes of which include geometric information, sequence numbers of unstructured information, and text entities; based on the attribute association graph 310, the component attribute relationship 330 is determined through an association model 320, the association model 320 being a graph neural network model, and the component attribute relationship 330 including the component's corresponding size, part number, material, and process.

[0068] An attribute association graph is a graph that represents the relationships between geometric information, sequence lines, and text entities.

[0069] In some embodiments, nodes in an attribute association graph may include geometric information, sequence numbers, and text entities. Association edges exist between nodes to represent their physical location relationships. The attribute of an association edge is its weight. For a description of geometric information and text entities, see [link to documentation]. Figure 2 And its contents.

[0070] A serial number leader is a graphic element in a drawing used to connect or point to specific information. For example, a serial number leader can be used to connect the serial number of a part with the corresponding geometric information of that part.

[0071] Text entities refer to text entities in vector graphics, such as dimension entities, serial numbers, and other text.

[0072] In some embodiments, the processor can construct an attribute association graph based on geometric information, unstructured information, and text entities. In some embodiments, for nodes of the same type, there are no associated edges between sequence number leaders; associated edges between geometric information are identified through existing primitive topological relationships; and associated edges between text entities are determined according to the blocks of the text entities. As an example only, the processor can determine whether text entities belong to the same block through primitive clustering and contour tracing, and determine associated edges between multiple text entities belonging to the same block.

[0073] In some embodiments, after determining the associated edges between nodes of the same type, for nodes of different types, the processor can calculate the distance between nodes of different types and generate an associated edge between the two closest nodes. In some embodiments, in response to the distance between two nodes being less than a preset distance threshold, the processor can establish an associated edge between these two nodes. The preset distance threshold can be determined based on empirical presets. In some embodiments, the preset distance thresholds are different for different types of nodes.

[0074] In some embodiments, the processor can determine the attribute relationships of parts based on the constructed attribute association graph and an association model. An association model is a model used to determine the attribute relationships of parts. In some embodiments, the association model can be a graph neural network (GNN) model, with the attribute association graph as input and the attribute relationships of parts as output.

[0075] In some embodiments, the processor can obtain an association model by training multiple first training samples with first labels using methods such as gradient descent. The first training samples include attribute association maps corresponding to historical vector drawings, and the first labels include the attribute relationships of parts corresponding to historical vector drawings.

[0076] In some embodiments, the first training sample can be obtained from historical data. The first label can be obtained based on manual annotation.

[0077] In some embodiments, the processor can perform multiple rounds of iterative training on the initial association model based on multiple sets of training samples with training labels, until the iteration termination condition is met, thus obtaining a trained association model. At least one round of iterative training includes: inputting one or more sets of training samples into the initial association model to obtain model outputs corresponding to one or more sets of training samples; substituting the model outputs corresponding to one or more sets of training samples, and the training labels corresponding to one or more sets of training samples, into a predefined loss function formula to calculate the value of the loss function; iteratively updating the model parameters in the initial association model based on the value of the loss function, until the iteration termination condition is met, thus ending the iteration and obtaining a trained association model. The iterative updating of the initial association model parameters can be performed using various methods, for example, based on gradient descent. The iteration termination condition can include loss function convergence or the number of iterations reaching an iteration threshold. The iteration termination condition can be loss function convergence, the number of iterations reaching a preset threshold, or the loss function value being less than a preset threshold.

[0078] Some embodiments in this specification structure discrete geometric and textual information in drawings by constructing attribute association graphs and using graph neural network models to learn deep relationships between them. Compared with traditional rule-based or simple distance matching methods, this method can more accurately handle complex and irregular drawing layouts and effectively identify multiple attributes of parts, such as dimensions, part numbers, materials, and processes. This significantly improves the accuracy and automation level of part attribute relationship identification, providing a reliable data foundation for subsequent machine sorting.

[0079] In some embodiments, the processor can determine candidate association edges based on the geometric positional relationship between any two nodes in the attribute association graph, wherein the candidate association edges exist between the two nodes; and determine the candidate association edges that satisfy a preset weight threshold as association edges.

[0080] Geometric positional relationship refers to the relationship between two nodes in an attribute association graph, representing their relative position or proximity in space. For example, geometric positional relationship can include the overlapping or containing bounding boxes of two nodes, or the distance between two nodes being less than a preset distance threshold.

[0081] Candidate association edges are edges determined based on the geometric positional relationship between two nodes and used to initially connect these two nodes in the attribute association graph. In some embodiments, candidate association edges may exist between geometric information and sequence number leaders, between geometric information and text entities, or between sequence number leaders and text entities.

[0082] In some embodiments, the processor can determine the Euclidean distance between any two nodes in the association graph and identify the edges between two nodes whose Euclidean distance is less than a distance threshold as candidate association edges. The distance threshold can be preset by a technician based on experience, and the distance threshold varies for different nodes.

[0083] For example, for geometric information in the association graph, the processor can use a spatial index structure (such as an R-tree or a Kd-tree) to retrieve other nodes near the geometric information. If the other nodes are indexed leads, the processor can calculate the Euclidean distance between the geometric information and the indexed leads. If the Euclidean distance is less than the distance threshold, the edge between the geometric information and the indexed leads is a candidate association edge.

[0084] In some embodiments, the processor may determine candidate associated edges that meet a preset weight threshold as associated edges.

[0085] The preset weight threshold refers to a pre-set numerical standard used to filter candidate related edges. For example, the preset weight threshold can be preset to 0.8, and the determination module can determine the candidate related edges with a weight value greater than 0.8 as the final related edges.

[0086] In some embodiments, the determining module can calculate a weight value for each candidate association edge, and determine the candidate association edges with weight values ​​greater than or equal to a preset weight threshold as association edges in the attribute association graph. In some embodiments, the determining module can calculate the weight value based on preset rules.

[0087] For example, the serial number, which is usually directly associated with geometric information, is partially or entirely located inside its geometric contour. Therefore, the preset rules may include: for candidate associated edges that connect geometric information and serial number leaders, if the candidate associated edge is established based on bounding box overlap, its weight can be set to a higher value (such as 1); if the bounding boxes do not overlap, but the distance between the two nodes is less than a preset distance threshold, its weight can be set to a lower value (such as 0).

[0088] In some embodiments, the weight value can also be negatively correlated with the distance between nodes.

[0089] For example, for candidate association edges connecting sequence number leader nodes and text entity nodes, their weights can be calculated based on the distance between the nodes. Therefore, preset rules can include: the determining module can calculate the ratio of the distance between nodes to the maximum association distance, and use the difference between 1 and this ratio as the weight. The maximum association distance can be determined based on a preset. The closer the node pair, the higher the weight of the candidate association edge between them.

[0090] In some embodiments of this specification, all possible candidate association edges are initially established through geometric positional relationships, ensuring a high recall rate; by setting differentiated weights for different node relationships and performing threshold filtering, false associations are accurately removed, effectively avoiding erroneous connections between nodes, thus balancing recall and accuracy, and significantly improving the quality and reliability of the final generated attribute association graph.

[0091] In some embodiments, the preset weight threshold is related to the map size and overall shape complexity of the unstructured information.

[0092] Drawing size refers to a parameter used to characterize the dimensions of a drawing sheet. For example, drawing size can be a standard drawing size such as A1, A2, or A3.

[0093] Overall shape complexity refers to a parameter used to quantify the complexity of all the geometric information contained in a single drawing. For example, a drawing containing a large number of dense and detailed geometric shapes has a higher overall shape complexity than a drawing containing only a few simple shapes.

[0094] In some embodiments, the processor can identify the title bar area in unstructured information through template matching or coordinate positioning. Then, optical character recognition (OCR) is performed on the text information in the title bar area, and preset map size keywords (such as "A1", "A2", "A3", etc.) are matched from the recognition results to determine the map size of the current unstructured information.

[0095] In some embodiments, the processor can iterate through all geometric information, sum the shape complexities of all geometric information, and use the sum as the overall shape complexity. For an explanation of shape complexity, see [link to documentation]. Figure 4 And its contents.

[0096] In some embodiments, the preset weight threshold is positively correlated with the map size and negatively correlated with the overall shape complexity.

[0097] Some embodiments in this specification achieve dynamic adaptive adjustment of the preset weight threshold by associating it with the size of the drawing and the overall shape complexity. For large drawings with sparse content, a larger threshold can effectively associate distant elements; for dense drawings, a smaller threshold can avoid erroneous associations. By adaptively adjusting the preset weight threshold, the accuracy and robustness of information association are significantly improved, and the method's universality for drawings of different sizes is enhanced.

[0098] Figure 4 This is an exemplary schematic diagram illustrating the generation of a sorting task list according to some embodiments of this specification.

[0099] In some embodiments, the processor can determine the shape complexity 420 and deformation tendency 430 of a part based on geometric information 410; based on the shape complexity 420 and deformation tendency 430, determine a target grasping rule 450 from a plurality of preset grasping rules 440, and generate a grasping strategy 460 for the part based on the target grasping rule 450; and generate the sorting task list 470 based on the grasping strategy 460. For a description of geometric information 410, see [link to documentation]. Figure 2 And its contents.

[0100] Shape complexity refers to an indicator used to characterize the irregularity or complexity of a part's geometry. In some embodiments, the more bends and corners a part's contour has, the higher its shape complexity.

[0101] In some embodiments, the processor can extract two-dimensional images or three-dimensional model information of the part's contour, calculate the number of inflection points and the sum of curvature changes on the part's contour using image processing or geometric analysis algorithms, normalize and weight the calculation results to obtain the shape complexity. The weights for the weighted summation can be determined based on empirical presets. Higher shape complexity indicates a more complex shape for the part.

[0102] Deformation tendency refers to an indicator used to characterize the likelihood of a part deforming when subjected to external forces. In some embodiments, the softer the material of the part, or the more dispersed the distribution of its pores and the larger the pore area, the higher the deformation tendency of the part.

[0103] In some embodiments, the processor can obtain the material hardness parameters from the unstructured information of a part; analyze its geometric information to identify the location and area of ​​all pores; and perform a comprehensive weighted calculation on multiple factors such as the material hardness parameters, the dispersion of the pore distribution, and the ratio of the total pore area to the total part area to obtain the deformation tendency. The weights for the weighted summation can be determined based on empirical presets. See the specification for unstructured information. Figure 2 And its contents.

[0104] Preset gripping rules refer to a set of rules that are pre-defined to establish the correspondence between part attributes and gripping strategies. For example, when the shape complexity and deformation tendency of a part are both low (e.g., shape complexity is below the complexity threshold, deformation tendency is below the deformation threshold), the preset gripping rules could be: adopting a strategy of uniformly distributing gripping points centered on the part's center of gravity, i.e., preset gripping rules for "parts with low complexity and low deformation tendency". As another example, when the shape complexity and deformation tendency of a part are both high (e.g., shape complexity is above or equal to the complexity threshold, deformation tendency is above or equal to the deformation threshold), the preset gripping rules could be: calling a physics simulation engine to simulate various different gripping point locations and suction cup distribution schemes, i.e., preset gripping rules for "parts with high complexity and high deformation tendency". The complexity threshold and deformation threshold can be determined by technical personnel based on historical data.

[0105] A target grasping rule refers to a grasping rule selected from multiple preset grasping rules based on the attributes of a specific part, and applicable to that specific part. In some embodiments, for a part with a complex shape but low deformation tendency, the processor can match and select the rule that best suits the combination of attributes from the preset grasping rules; this rule is the target grasping rule.

[0106] A grasping strategy refers to a set of specific parameters or instructions used to guide a robot's end effector to perform a grasping action. In some embodiments, the grasping strategy includes grasping direction, grasping point location, and suction cup distribution.

[0107] The gripping direction refers to the direction or posture along which the robot's end effector approaches and grips the part. For example, the gripping direction could be vertically downward from directly above the part.

[0108] A gripping point refers to the specific location where the robot's end effector contacts the surface of a part to perform a gripping action. For example, a gripping point can be one or more two-dimensional or three-dimensional coordinate points on the surface of the part.

[0109] Suction cup distribution refers to the spatial arrangement or activation pattern of multiple suction cups on the robot's end effector during grasping. For example, suction cup distribution can be a uniform array of all suction cups, or it can be that only a portion of the suction cups located in specific positions are activated.

[0110] In some embodiments, the processor can combine the quantized scores of shape complexity and deformation tendency into a feature vector; in a preset vector database, by calculating the similarity between this feature vector and the corresponding vectors of each rule in the database, the preset crawling rule with the highest similarity is retrieved and used as the target crawling rule. The preset vector database refers to a preset vector database that stores multiple preset crawling rules. Similarity may include cosine similarity, etc.

[0111] In some embodiments, the process of generating a grasping strategy based on target grasping rules is directly related to the content of the rules. For example, if the matching target grasping rule is "for parts with low complexity and low deformation tendency," the processor can calculate the geometric center of the part; set the grasping direction to vertically downward; uniformly generate multiple grasping points on the surface of the part based on the geometric center; and set the suction cup distribution to full array activation. As another example, if the matching target grasping rule is "for parts with high complexity and high deformation tendency," the processor can call a physical simulation engine to simulate various different grasping point and suction cup distribution schemes on the part model; evaluate the grasping stability and part deformation degree under each suction cup distribution scheme; select the suction cup distribution scheme with the best overall score, and use its corresponding grasping direction, grasping points, and suction cup distribution as the final grasping strategy. The suction cup distribution scheme refers to the method of setting the suction cup distribution.

[0112] In some embodiments, the processor can convert information such as gripping direction and gripping point in the gripping strategy into pose data in the robot coordinate system; convert the suction cup distribution strategy into on / off control commands for specific suction cups on the robot end effector; combine the current position of the part and the position of the target hopper to plan a complete motion trajectory, including a series of action commands such as approach, gripping, lifting, moving and placing; and integrate the action commands into a sorting task list for the robot to read and execute sequentially.

[0113] Some embodiments in this specification, by determining the shape complexity and deformation tendency of the part and matching it with preset gripping rules to generate a customized gripping strategy, overcome the limitations of traditional single gripping methods, and can provide the optimal gripping scheme for complex and easily deformable parts, thereby significantly improving the sorting success rate and stability, reducing the risk of part damage, and enhancing the efficiency and applicability of automated sorting systems.

[0114] In some embodiments, the processor may assign a unique identifier and sorting order to each part, the sorting order being determined based on an optimization algorithm; each task unit may be encapsulated into a structured data packet, the data packet including a unique identifier, geometric information, gripping point location, gripping tool parameters, placement point coordinates, and sorting order.

[0115] A unique identifier is data used to uniquely identify and distinguish a specific object among multiple parts. For example, a unique identifier can be a serial number, a string represented by a barcode or QR code, or a Universally Unique Identifier (UUID).

[0116] Sorting order refers to determining the sequence in which multiple items to be sorted are processed. For example, for parts A, B, and C, the sorting order could be B→C→A. In some embodiments, the sorting order is determined based on an optimization algorithm.

[0117] An optimization algorithm is a computational method or program used to find the optimal or near-optimal solution among all possible solutions based on one or more preset evaluation criteria. For example, an optimization algorithm could be used to determine the sorting sequence that minimizes the robot's movement path, total time, or avoids collisions with other equipment.

[0118] A task unit is a logical set of relevant information required to complete a specific sorting action. In some embodiments, a task unit may correspond to the complete operation of a robot picking up a specific part and placing it in a designated location.

[0119] A data packet is a unit of data organized in a specific format for data transmission or storage. For example, a data packet can be a JSON or XML file containing all the information needed to perform a sorting task. In some embodiments, a data packet includes a unique identifier, geometric information, gripping points, gripping tool parameters, placement point coordinates, and sorting order.

[0120] Gripping tool parameters refer to the operational parameters that need to be set when a robot's end effector (such as a gripper) performs a gripping action. For example, gripping tool parameters may include the gripper's opening and closing width, gripping force, gripping speed, etc.

[0121] In some embodiments, the processor can calculate the geometric centroid and gripping pose of the part in real time based on the closed contour of the part, and combine the part material, thickness and hole distribution obtained from the attribute association map to determine the optimal suction cup array layout and gripping pressure parameters by comparing with a preset gripping rule library, and construct the suction cup array layout and gripping pressure parameters as gripping tool parameters.

[0122] Placement point coordinates refer to the spatial coordinate information used to define the target position where a part is placed. For example, placement point coordinates can be three-dimensional coordinate values ​​(X, Y, Z) representing the target position in a specific coordinate system, or they can include attitude information (such as rotation angle).

[0123] In some embodiments, the processor can automatically match the determined part attributes with the sorting rule base to allocate the target placement area for the part and calculate the placement point coordinates of the part.

[0124] In some embodiments, the processor may assign an auto-incrementing sequence number (e.g., Part_001, Part_002, ...) to each part in a sorting task, or generate a universally unique identifier (UUID) for each part as a unique identifier to ensure the uniqueness of the part in a wider system or multiple tasks.

[0125] In some embodiments, if the optimization objective is to minimize the path, the processor can use algorithms for solving the Traveling Salesman Problem (TSP), such as genetic algorithms or ant colony algorithms, as the optimization algorithm. The optimization algorithm takes the coordinates of all parts to be picked up and the coordinates of the target placement point as input, calculates and compares the total movement distance of the robot's end effector under different sorting sequences, and finally selects the sequence with the shortest total path as the sorting sequence.

[0126] In some embodiments, if collision avoidance and efficiency improvement are the optimization objectives, the processor can use a cost-based greedy algorithm as the optimization algorithm. The optimization algorithm comprehensively evaluates the "cost" of grasping each part. Factors affecting the cost evaluation may include: whether the part is obscured by other parts, whether there are obstacles in the grasping path, and the estimated time required for the robot to perform the grasping action. The optimization algorithm selects the part with the lowest current cost for grasping, thereby dynamically generating an efficient and safe sorting sequence.

[0127] In some embodiments, the processor can encapsulate a task unit as a JSON (JavaScript Object Notation) object. This JSON object contains multiple key-value pairs, such as "part_id", "grasp_point", and "tool_params", corresponding to unique identifiers, grab points, grab tool parameters, and other information, respectively. The control module can serialize this JSON object into a string and send it to the robot via the TCP / IP protocol.

[0128] In some embodiments, the processor may also use XML (eXtensible Markup Language) format to encapsulate data packets. Each information item of the task unit is contained within a corresponding tag, forming a hierarchical document structure that facilitates parsing and execution by the robot control system.

[0129] Some embodiments in this specification achieve orderly, efficient, and safe sorting by assigning unique identifiers to parts and using a sorting order determined by an optimized algorithm. The optimized algorithm significantly shortens robot paths, reduces time consumption, and avoids collisions. Furthermore, encapsulating task information into structured data packets ensures the accuracy and integrity of instructions, simplifies communication and data parsing in the robot control system, and thus improves the stability and scalability of the entire sorting system.

[0130] This specification provides an embodiment of a machine sorting device based on vector paper semantic parsing, including a processor for executing the above-described machine sorting method based on vector paper semantic parsing.

[0131] This specification provides a computer-readable storage medium that stores computer instructions. When a computer reads the computer instructions from the storage medium, the computer executes the aforementioned machine sorting method based on vector paper semantic parsing.

[0132] The basic concepts have been described above. Obviously, for those skilled in the art, the detailed disclosure above is merely illustrative and does not constitute a limitation of this specification. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and corrections to this specification. Such modifications, improvements, and corrections are suggested in this specification and therefore remain within the spirit and scope of the exemplary embodiments described herein.

[0133] Furthermore, this specification uses specific terms to describe embodiments thereof. For example, "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations throughout this specification do not necessarily refer to the same embodiment. Moreover, certain features, structures, or characteristics in one or more embodiments of this specification can be appropriately combined.

[0134] Furthermore, unless expressly stated in the claims, the order of processing elements and sequences, the use of numbers and letters, or other names described in this specification are not intended to limit the order of the processes and methods described herein. Although various examples have been discussed in the foregoing disclosure of some embodiments of the invention that are currently considered useful, it should be understood that such details are for illustrative purposes only, and the appended claims are not limited to the disclosed embodiments; rather, the claims are intended to cover all modifications and equivalent combinations that conform to the spirit and scope of the embodiments described herein. For example, while the system components described above can be implemented using hardware devices, they can also be implemented solely using software solutions, such as installing the described system on existing servers or mobile devices.

[0135] Similarly, it should be noted that, in order to simplify the description disclosed herein and thus aid in the understanding of one or more embodiments of the invention, the foregoing description of embodiments in this specification may sometimes combine multiple features into a single embodiment, drawing, or description thereof. However, this method of disclosure does not imply that the subject matter of this specification requires more features than those mentioned in the claims. In fact, the embodiments contain fewer features than all the features of a single embodiment disclosed above.

[0136] In some embodiments, numbers describing the quantity of components and attributes are used. It should be understood that such numbers used in the description of embodiments are modified in some examples with the terms "approximately," "approximately," or "generally." Unless otherwise stated, "approximately," "approximately," or "generally" indicates that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may be changed depending on the characteristics required by individual embodiments. In some embodiments, numerical parameters should take into account specified significant digits and employ a general method of digit reservation. Although the numerical ranges and parameters used to confirm their breadth of range in some embodiments of this specification are approximate values, in specific embodiments, such values ​​are set as precisely as feasible.

[0137] For each patent, patent application, patent application publication, and other material such as articles, books, specifications, publications, and documents referenced in this specification, the entire contents of that material are incorporated herein by reference. This excludes historical application documents that are inconsistent with or conflict with the content of this specification. It should be noted that if there are any inconsistencies or conflicts between the descriptions, definitions, and / or terminology used in the supplementary materials to this specification and the content of this specification, the descriptions, definitions, and / or terminology used in this specification shall prevail.

[0138] Finally, it should be understood that the embodiments described in this specification are merely illustrative of the principles of the embodiments described herein. Other variations may also fall within the scope of this specification. Therefore, alternative configurations of the embodiments described herein are intended to be illustrative rather than limiting, and should be considered consistent with the teachings of this specification. Accordingly, the embodiments described herein are not limited to those explicitly introduced and described herein.

Claims

1. A machine sorting method based on vector paper semantic parsing, characterized in that, include: Obtaining geometric and unstructured information based on vector graphics; An attribute association graph is constructed, wherein the nodes of the attribute association graph include the geometric information, the sequence number leaders of the unstructured information, and the text entities. Based on the attribute association graph, the attribute relationships of the parts are determined through an association model, wherein the association model is a graph neural network model, and the attribute relationships of the parts include the size, part number, material, and process of the parts. The construction of the attribute association graph further includes: determining candidate association edges based on the geometric positional relationship between any two nodes in the attribute association graph, wherein the candidate association edges exist between the two nodes; and determining the candidate association edges that satisfy a preset weight threshold as association edges. The preset weight threshold is related to the size of the unstructured information graph and the overall shape complexity, wherein the preset weight threshold is positively correlated with the size of the graph and negatively correlated with the overall shape complexity. Based on the relationship between the attributes of multiple parts in the multiple vector drawings, the execution attributes are determined by the material sorting rules. The execution attributes include the sorting target location, the execution sequence, and the target material frame. A sorting task list is generated based on the execution attributes, and the robot is controlled to execute sorting tasks based on the sorting task list.

2. The machine sorting method as described in claim 1, characterized in that, The process of generating a sorting task list based on the execution attributes includes: The shape complexity and deformation tendency of the part are determined based on the geometric information; Based on the shape complexity and the deformation tendency, a target grasping rule is determined from multiple preset grasping rules, and a grasping strategy for the part is generated based on the target grasping rule. The grasping strategy includes grasping direction, grasping point position and suction cup distribution. The sorting task list is generated based on the described crawling strategy.

3. The machine sorting method as described in claim 2, characterized in that, The step of generating the sorting task list based on the grasping strategy further includes: Each part is assigned a unique identifier and a sorting order, which is determined based on an optimization algorithm; Each task unit is encapsulated into a structured data packet, which includes the unique identifier, the geometric information, the grab point location, the grab tool parameters, the placement point coordinates, and the sorting order.

4. A machine sorting system based on vector paper semantic parsing, characterized in that, It includes an acquisition module, a relationship determination module, an attribute determination module, and a control module: The acquisition module is configured as follows: Obtaining geometric and unstructured information based on vector graphics; The relationship determination module is configured as follows: An attribute association graph is constructed, wherein the nodes of the attribute association graph include the geometric information, the sequence number leaders of the unstructured information, and the text entities. Based on the attribute association graph, the attribute relationships of the parts are determined through an association model, wherein the association model is a graph neural network model, and the attribute relationships of the parts include the size, part number, material, and process of the parts. The construction of the attribute association graph further includes: determining candidate association edges based on the geometric positional relationship between any two nodes in the attribute association graph, wherein the candidate association edges exist between the two nodes; and determining the candidate association edges that satisfy a preset weight threshold as association edges. The preset weight threshold is related to the size of the unstructured information graph and the overall shape complexity, wherein the preset weight threshold is positively correlated with the size of the graph and negatively correlated with the overall shape complexity. The attribute determination module is configured as follows: Based on the relationship between the attributes of multiple parts in the multiple vector drawings, the execution attributes are determined by the material sorting rules. The execution attributes include the sorting target location, the execution sequence, and the target material frame. The control module is configured as follows: A sorting task list is generated based on the execution attributes, and the robot is controlled to execute sorting tasks based on the sorting task list.

5. The machine sorting system as described in claim 4, characterized in that, The control module is further configured to: The shape complexity and deformation tendency of the part are determined based on the geometric information; Based on the shape complexity and the deformation tendency, a target grasping rule is determined from multiple preset grasping rules, and a grasping strategy for the part is generated based on the target grasping rule. The grasping strategy includes grasping direction, grasping point position and suction cup distribution. The sorting task list is generated based on the described crawling strategy.

6. The machine sorting system as described in claim 5, characterized in that, The control module is further configured to: Each part is assigned a unique identifier and a sorting order, which is determined based on an optimization algorithm; Each task unit is encapsulated into a structured data packet, which includes the unique identifier, the geometric information, the grab point location, the grab tool parameters, the placement point coordinates, and the sorting order.

7. A machine sorting device based on vector paper semantic parsing, comprising a processor, the processor being configured to execute the machine sorting method based on vector paper semantic parsing as described in any one of claims 1 to 3.

8. A computer-readable storage medium storing computer instructions, wherein when a computer reads the computer instructions in the storage medium, the computer executes the machine sorting method based on vector paper semantic parsing as described in any one of claims 1 to 3.

Images