Rebar drawing recognition method and system based on multi-modal large model and storage medium

Abstract

The application discloses a steel bar drawing recognition method and system based on a multimodal large model and a storage medium, the method comprising: obtaining a steel bar drawing image to be processed, carrying out standardized preprocessing, eliminating image noise and unifying input resolution to obtain a steel bar image to be recognized. A multimodal prompt word containing role setting, task description and output format constraint is constructed, which is input into a pre-trained multimodal large model together with the steel bar image to be recognized, image features are extracted through a visual encoder, the corresponding relationship between geometric topological structure and size annotation is recognized relying on a semantic alignment mechanism, and a steel bar geometric parameter structured text description is output. The text is analyzed in a structured manner, the side length and bending angle numerical values are extracted, and non-negativity and geometric logical integrity are checked. After the check, the geometric parameters are converted into a numerical control steel bar processing equipment executable control instruction sequence to drive the equipment to complete automatic processing.

Description

Technical Field

[0001] This invention belongs to the field of intelligent processing technology, and more specifically, relates to a method, system and storage medium for rebar diagram recognition based on a multimodal large model. Background Technology

[0002] In the construction and rebar processing industry, CNC machining of rebar is a core process. Traditional rebar processing typically relies on manual interpretation of paper or electronic rebar detail drawings, followed by manual input of parameters such as rebar shape, side length, and angles into a CNC rebar bending machine or bending center. With the development of the Industrial Internet, automatically extracting rebar parameters through image recognition technology is becoming a trend.

[0003] Current automation solutions mainly rely on traditional computer vision (CV) algorithms, such as line detection based on Hough Transform or text extraction based on OCR (Optical Character Recognition). The working principle is usually as follows: first, the outline of the line is extracted through image processing, then the number is recognized through OCR, and finally, the number is matched with the line based on the spatial distance (for example, matching the number "5800" to the longest line segment).

[0004] Although existing technologies perform reasonably well under standard drafting methods, they have significant shortcomings in actual engineering scenarios: Semantic understanding is lacking: Traditional algorithms rely on geometric rules for matching. When the label position is offset (for example, "5800" is not close to the long side) or there are leader labels, the rules are very easy to fail, resulting in parameter matching errors.

[0005] Poor robustness: Traditional edge detection algorithms are prone to producing breaks or artifacts in hand-drawn drawings, blurry images, or images containing noise (such as uneven line thickness or numbers sticking together with lines), leading to failure in topological structure recognition.

[0006] Weak generalization ability: It cannot flexibly adapt to the drafting standards of different design institutes, and the rule code often needs to be rewritten for each new drawing format. Summary of the Invention

[0007] This invention aims to solve the problems of low efficiency and error-proneness in traditional rebar drawing recognition, as well as the poor adaptability of existing technologies to non-standard drawings and the difficulty in directly connecting with CNC machining. It improves the accuracy and stability of rebar drawing recognition, realizes efficient connection from drawings to automated processing, and promotes the intelligent development of the building rebar processing process.

[0008] To address the aforementioned deficiencies or improvement needs of existing technologies, as a first aspect of this invention, the present invention provides a method for rebar diagram recognition based on a multimodal large model, comprising: S1. Obtain a large-scale image of the steel bar to be processed, and perform standardized preprocessing on the large-scale image to eliminate image noise and unify the input resolution to obtain an image of the steel bar to be identified; S2. Construct multimodal prompt words that include role settings, task descriptions and output format constraints. Input the image of the rebar to be identified and the multimodal prompt words into a pre-trained multimodal large model. Use the visual encoder of the multimodal large model to extract image features and identify the correspondence between the geometric topology and dimension annotations in the image based on the semantic alignment mechanism. Output a structured text description containing the geometric parameters of the rebar. S3. Perform structured parsing on the structured text description to extract the side length and bending angle values ​​of the reinforcing bars; and perform geometric logic verification on the extracted values, the geometric logic verification including at least non-negativity check and format integrity check. S4. After the verification is passed, the parsed steel bar geometric parameters are mapped and converted into a sequence of control instructions that can be executed by the CNC steel bar processing equipment to drive the equipment to perform automated processing.

[0009] Furthermore, the detailed drawing image in S1 specifically refers to: an original image containing the geometric shape and dimension annotations of the reinforcing bars, obtained through an industrial camera, scanner, or exported from CAD software; the original image includes any one of the following: photographs of paper drawings, electronic drawing files, and hand-drawn drawings.

[0010] Furthermore, the multimodal prompt words in S2 also include a few-sample example module; The few-sample example module contains at least one set of input-output pairs for demonstrating to the multimodal large model the mapping rules from the rebar detail image to a specific format text description. The specific format text description adopts the sequence form of "length, angle; length, angle", where positive and negative angle values ​​represent clockwise and counterclockwise bending, respectively.

[0011] The multimodal large model is a generative pre-trained model with visual-language cross-modal alignment capability; Furthermore, the multimodal large model in S2 is configured to bind dimension annotation numbers with corresponding geometric line segments based on semantic understanding under non-standard drawing conditions such as offset annotation positions, presence of leader annotations, or image blurring, so as to output the correct rebar geometric parameters.

[0012] Furthermore, the specific process of identifying the correspondence between geometric topological structures and dimension annotations in the image based on the semantic alignment mechanism in S2 is as follows: First, topological feature extraction and vectorization are performed. The set of geometric line segments in the rebar image is abstracted into point cloud data, a Vietoris-Rips complex is constructed, and the birth and death process of topological features is observed by dynamically adjusting the scale parameter, thereby calculating the Betti number and generating a persistent graph. Subsequently, the discrete topological features in the persistent graph are transformed into vector representations in a continuous function space using a persistent landscape function, thereby quantifying the topological stability of closed regions and connectivity in the rebar drawing. Secondly, a semantic alignment energy functional based on Riemannian manifolds is constructed. The dimension annotation text is encoded into semantic vectors and mapped onto the geometric manifold. An energy functional containing Dirichlet energy terms and data fidelity terms is constructed. The aim is to find the optimal mapping function that minimizes the mapping error between the geometric topology and the semantic vector while maintaining the geometric smoothness of the geometric manifold, thereby achieving alignment between geometric topological distance and semantic distance. Next, an optimization solution based on spectral analysis is performed; the Laplace-Beltrami operator is introduced to construct a generalized eigenvalue problem, and the orthogonal spectral basis of the geometric manifold is obtained by solving the generalized eigenvalue problem, thus decomposing the complex geometric structure of the manifold into a computable combination of basis functions; Finally, the correspondence between the geometric topology and the dimension annotation text is determined; by calculating the projection coefficients of the mapping function under the orthogonal spectral basis, a bijective relationship between the geometric topology points and the dimension semantic vector is established, thereby completing the automatic identification and matching of geometric structure and dimension information in the reinforcement drawings.

[0013] Furthermore, the specific process of transforming the discrete topological features in the persistent graph into a vector representation in a continuous function space using persistent landscape functions is as follows: First, construct the triangular waveform basis functions; for the persistent graph... Each topological feature point in the data, at its birth scale and death scale As parameters, construct a function defined in the real number field. Piecewise linear functions on Generate the corresponding triangular waveform basis function, whose mathematical expression is: Wherein, the peak position of the triangular waveform basis function is located at The bottom width corresponds to the length of the persistent interval of the topological feature point. , This represents the total number of topological feature points in the persistent graph. It is an index variable used to iterate through each topological feature point in the set; Secondly, generate a hierarchical sorting sequence; for each predefined scale variable within the domain. Calculate the set of function values ​​for all the aforementioned triangular waveform basis functions at this scale variable. The obtained function values ​​are then sorted in descending order of their numerical values ​​to form an ordered sequence of function values. ; Next, extract the top-level envelope curve; then, in the ordered sequence of function values, extract the first... The maximum function value at the i-th position is defined as the i-th Layered persistent landscape function The value at this scale, i.e. By sliding the scale variable Traverse the entire domain to generate a set of continuous function curves describing the hierarchical distribution of topological feature intensity. Finally, vectorized discrete sampling is performed on the continuous persistent landscape function curves. In the sampling point set The function values ​​are solved by sampling the sequence of function values. As a numerical vector representation of the topology of steel reinforcement drawings.

[0014] Furthermore, the energy functional comprising the Dirichlet energy term and the data fidelity term is specifically as follows: in, The representation is defined on the geometric manifold The total energy functional on, The optimal mapping function to be solved is used to map the geometry on the manifold to the scale semantic space; The Dirichlet energy term Used for constraint mapping functions Smoothness on the manifold, where manifold gradient operator on, Represents the Riemannian metric tensor norm, For volume elements on the manifold, this term preserves the connectivity of the geometric topology by minimizing the rate of change of the function on the manifold; The data fidelity item Used to constrain the consistency between mapping results and observed data, where Let the coordinates of the geometric nodes on the manifold be... This is the semantic vector of the dimension annotation corresponding to this node. This is a non-negative regularization parameter used to balance the weight relationship between manifold smoothness constraints and data fidelity constraints.

[0015] Furthermore, S3 also includes a feedback retry mechanism: if the geometric logic verification fails, the system automatically adjusts the parameters or prompting strategy of the multimodal prompt words and returns to step S2 to re-perform the reasoning until the verification passes or the preset retry limit is reached.

[0016] As a second aspect of the present invention, a rebar drawing recognition system based on a multimodal large model is also provided, comprising: The image acquisition and preprocessing unit is used to acquire a large-scale image of the steel bar to be processed, and to perform standardized preprocessing on the large-scale image to eliminate image noise and unify the input resolution, so as to obtain an image of the steel bar to be identified. A multimodal feature recognition unit is used to construct multimodal prompt words containing role settings, task descriptions and output format constraints. The image of the steel bar to be recognized and the multimodal prompt words are input together into a pre-trained multimodal large model. The visual encoder of the multimodal large model is used to extract image features, and the correspondence between the geometric topology and dimension annotations in the image is identified based on the semantic alignment mechanism. The output is a structured text description containing the geometric parameters of the steel bar. The parameter parsing and verification unit is used to perform structured parsing on the structured text description, extract the side length and bending angle values ​​of the reinforcing bars, and perform geometric logic verification on the extracted values, which includes at least non-negativity checks and format integrity checks. The instruction generation and drive unit is used to map the parsed steel bar geometric parameters into a sequence of control instructions that can be executed by the CNC steel bar processing equipment after the verification is passed, so as to drive the equipment to perform automated processing.

[0017] As a third aspect of the present invention, a computer-readable storage medium is also provided, on which a computer program is stored, the computer program being executed by a processor as described in any one of claims 1-8: a method for identifying rebar diagrams based on a multimodal large model.

[0018] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects: 1. The present invention provides a method for recognizing rebar drawings based on a multimodal large model. By performing standardized preprocessing on the large-scale rebar drawing images to be processed, image noise can be effectively eliminated and the input resolution can be unified, providing stable and standardized input data for subsequent model recognition. This avoids recognition deviations caused by differences in image quality, ensures the stability and consistency of image feature extraction by the multimodal large model, and enables drawings from different sources and with different resolutions to be converted into standard images that meet the model input requirements, laying the foundation for accurate recognition of geometric topology and dimension annotations.

[0019] 2. The present invention provides a rebar drawing recognition method based on a multimodal large model. By constructing multimodal prompt words that include role settings, task descriptions, and output format constraints, and combining a visual encoder and semantic alignment mechanism of a pre-trained multimodal large model, it can accurately identify the correspondence between geometric topology and dimension annotations in the image, and output a structured text description containing rebar geometric parameters. It can still achieve effective recognition under non-standard drawing conditions such as annotation offset, leader line annotation, or image blurring, thereby improving the adaptability and accuracy of rebar drawing information extraction.

[0020] 3. The present invention provides a rebar drawing recognition method based on a multimodal large model. By performing structured parsing and geometric logic verification on the structured text description, it can accurately extract the side length and bending angle values ​​of the rebar, complete the non-negativity and format integrity checks to ensure the compliance and validity of the extracted parameters, and after the verification is passed, convert the geometric parameters into a sequence of control instructions that can be executed by CNC rebar processing equipment, directly driving the equipment to complete automated processing, simplifying the information conversion process of rebar processing, and improving the efficiency and reliability of processing execution. Attached Figure Description

[0021] Figure 1 This is a flowchart of the rebar diagram recognition method based on a multimodal large model according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the original image of the reinforcing steel bar diagram according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the system units in an embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0023] Example 1 Please refer to Figure 1 This embodiment 1 provides a method for rebar drawing recognition based on a multimodal large model, including: S1. Obtain a large-scale image of the steel bar to be processed, and perform standardized preprocessing on the large-scale image to eliminate image noise and unify the input resolution to obtain an image of the steel bar to be identified; S2. Construct multimodal prompt words that include role settings, task descriptions and output format constraints. Input the image of the rebar to be identified and the multimodal prompt words into a pre-trained multimodal large model. Use the visual encoder of the multimodal large model to extract image features and identify the correspondence between the geometric topology and dimension annotations in the image based on the semantic alignment mechanism. Output a structured text description containing the geometric parameters of the rebar. S3. Perform structured parsing on the structured text description to extract the side length and bending angle values ​​of the reinforcing bars; and perform geometric logic verification on the extracted values, the geometric logic verification including at least non-negativity check and format integrity check. S4. After the verification is passed, the parsed steel bar geometric parameters are mapped and converted into a sequence of control instructions that can be executed by the CNC steel bar processing equipment to drive the equipment to perform automated processing.

[0024] This embodiment further elaborates on the above steps.

[0025] (1) Image acquisition and preprocessing In the process of steel bar processing, the large-scale steel bar drawing is the core carrier for conveying the geometric shape and dimensional parameters of the steel bar. Its image quality directly affects the accuracy of subsequent recognition. However, the large-scale steel bar images obtained in industrial sites often have a variety of problems. It is necessary to complete the acquisition and standardization preprocessing of the images first to lay the foundation for subsequent recognition work.

[0026] Please refer to Figure 2 The first step is to acquire the detailed images of the steel bars to be processed. This can be done by taking pictures or scanning the paper steel bar detailed images using image acquisition devices such as industrial cameras or scanners, or by directly exporting electronic steel bar detailed images from CAD drawing software. All acquired original images must contain the complete geometric shape of the steel bars and the corresponding dimension annotations. Specifically, these images can be divided into photographs of paper drawings, electronic drawing files, and hand-drawn drawings, to ensure that all the core information required can be obtained in the subsequent recognition process.

[0027] Due to the complex environment of industrial sites, the acquired raw images may contain noise and inconsistent resolution, which can interfere with subsequent feature extraction and recognition. Therefore, standardized preprocessing of the raw images is necessary. During preprocessing, an adaptive thresholding algorithm is used to convert the raw images into binary images, effectively removing background noise and enhancing the contrast between the rebar lines and dimension markings, making the geometric contours and dimensions of the rebar clearer. Simultaneously, the processed images are uniformly scaled to the optimal input resolution of the multimodal large model to avoid deviations in feature extraction due to resolution differences, ensuring the accuracy and stability of subsequent recognition work.

[0028] (2) Multimodal feature recognition After image acquisition and preprocessing, the process moves to multimodal cue construction and intelligent recognition. First, multimodal cue words are constructed. These cue words contain complete task instructions and consist of four core parts: 1) Role setting: explicitly designating the model as a "CNC rebar processing engineer," enabling it to perform rebar drawing recognition tasks from a professional perspective; 2) Task description: requiring the model to identify U-shaped, L-shaped, and other geometric topological structures in the image, along with their corresponding side lengths, to extract key information from the drawing; 3) Output format constraints: forcing the model to output in the format of "side length, bending angle," specifically prompting the model to identify the rebar parameters in the image, with clockwise bending being positive and counterclockwise being negative, and the output format being "length 1, angle 1; length 2, angle 2...", with the last segment having an angle of 0; 4) A few-shot example module: this module contains at least one set of input-output pairs to demonstrate the mapping rules from the rebar detail drawing image to a specific format text description, allowing the model to master the sequential output requirements.

[0029] The multimodal large model employed is a generative pre-trained model with visual-linguistic cross-modal alignment capabilities. After inputting the pre-processed image to be recognized along with the constructed multimodal prompts, the model's internal visual encoder first extracts line and character features from the image, such as the U-shaped line features composed of three lines and size character features like 5800 and 375. Then, through the attention mechanism of the large language model, the character features are semantically aligned with the corresponding geometric line segments. 5800 is bound to the longest line segment in the middle of the image, and 375 is matched to the two shorter sides. Based on the direction of the U-shaped opening, the bending angle is inferred to be -90 degrees, i.e., a counter-clockwise right-angle bend. This model is specially configured to stably bind dimension annotations to corresponding geometric line segments and output accurate rebar geometric parameters even under non-standard drawing conditions such as offset annotation positions, the presence of leader lines, or blurred images, relying on semantic understanding capabilities.

[0030] Throughout the process, the model relies on a semantic alignment mechanism to identify the correspondence between geometric topology and dimension annotations, transforming visual information from drawings into standardized and interpretable parametric information. This eliminates the need for manual drawing reading and data entry. First, it abstracts the set of geometric line segments in the rebar image into point cloud data in Euclidean space, constructs a Vietoris-Rips complex, observes the birth and death process of topological features by adjusting scale parameters, calculates the Betti numbers of the point cloud in zero and one dimensions, and generates a persistent graph. ,in Indicates the first The scale of occurrence of a topological feature Indicates the first The vanishing scale of a topological feature This represents the total number of topological feature points in the persistent graph. This is the index variable for iterating through each topological feature point in the set; For each topological feature point in the persistent graph, a piecewise linear function in the real domain is constructed using birth and death scales as parameters, generating the corresponding triangular waveform basis function. The peak position of the triangular waveform basis function is located at The bottom width corresponds to the length of the persistent interval of the topological feature point. ; For each predefined scale variable within the domain, calculate the set of function values ​​for all triangular waveform basis functions at that scale variable. Arrange the function values ​​in descending order of magnitude to form an ordered sequence of function values. Then, select the first function value from the ordered sequence... The maximum function value at the i-th position is defined as the i-th Layered persistent landscape function For continuous persistent landscape function curves at a preset sampling point set... The function values ​​are solved to obtain the sequence of sampled function values. This sequence is the vector representation of discrete topological features in the persistent graph in the continuous function space. The transformation achieved using the persistent landscape function can be expressed as follows: ,in The scaling variable is used to traverse the intervals in which topological features persist. For the number of landscape layers, This is an indicator function; it takes a value of 1 when the condition is met, and a value of 0 otherwise. The purpose of this step is to transform the abstract, non-numerical geometric topology in the reinforcement drawings into a stable, computable vector form, providing a unified quantitative carrier for subsequent alignment with dimensional semantic information. Secondly, a semantic alignment energy functional based on a Riemannian manifold is constructed. The dimension-annotated text is encoded into semantic vectors by a pre-trained model and mapped to a geometric manifold. A total energy functional containing a Dirichlet energy term and a data fidelity term is then constructed. ; The representation is defined on the geometric manifold The total energy functional on, Let be the optimal mapping function to be solved, used to map the geometry on the manifold to the scale semantic space. manifold gradient operator on, Represents the Riemannian metric tensor norm, d Let the volume element be on the manifold. Let the coordinates be the geometric nodes on the manifold. This is the semantic vector for the dimension annotation of this node. Here, is the non-negative regularization parameter; where, the Dirichlet energy term Used to constrain the smoothness of the mapping function on the manifold, preserving the connectivity of the geometric topology by minimizing the rate of change of the function on the manifold; data fidelity term. The regularization parameter is used to balance the weighting relationship between manifold smoothness constraints and data fidelity constraints, constraining the consistency between the mapping results and the observed data. In this step, the vector representation of the continuous function space serves as the quantization input of the geometric topology, and together with the dimension annotation semantic vector, it participates in the construction of the optimization objective of the energy functional, providing constraints for the subsequent solution of the mapping relationship. The optimization solution based on spectral analysis is performed again, and the generalized eigenvalue problem is constructed by introducing the Laplace-Beltrami operator. , Let be a second-order differential operator defined on a Riemannian manifold. For eigenvalues, For the Riemannian metric tensor, For the corresponding characteristic functions, solve the generalized eigenvalue problem to obtain the orthogonal spectral basis of the geometric manifold. The purpose of this step is to solve the energy functional of the previous step by spectral decomposition, to complete the projection decomposition of the vector representation of the continuous function space under the orthogonal spectral basis, to transform the complex topological feature vectors into a set of computable linear combinations of basis functions, and to decompose the complex manifold geometry into computable combinations of orthogonal basis functions by using the spectral properties of the operators, thereby reducing the complexity of subsequent matching calculations. Finally, the correspondence between the geometric topology and the dimension annotation text is determined. The expansion coefficients of the mapping function under the orthogonal spectral basis are calculated. The topological features and dimension semantic vectors corresponding to the vector representation in the continuous function space are associated and matched through these expansion coefficients: the projection coefficients of the vector under the spectral basis reflect the distribution pattern of the topological features on different basis functions, and the geometric topological point whose projection pattern of the dimension semantic vector under the same spectral basis is closest to it is the target to be matched. Based on the expansion coefficients, the matching relationship between geometric nodes and dimension annotations is determined, and a bijective relationship is established between geometric topological points and dimension semantic vectors, completing the automatic identification and matching of geometric structure and dimension information in the reinforcement drawings. This step utilizes all the previous processing results and achieves a one-to-one correspondence from geometric topological vectors to dimension semantic vectors through coefficient matching of spectral basis projection, completing the entire semantic alignment process.

[0031] In this way, the geometric structure and dimensional information in the steel reinforcement drawings can be automatically identified and matched. The entire process from drawing feature extraction to dimensional parameter confirmation can be completed without manual intervention, effectively solving the problems of low efficiency and large error in traditional identification methods. It realizes the efficient conversion from drawings to processing instructions, adapts to the needs of automated processing, and promotes the intelligent upgrading of the construction field.

[0032] (3) Parameter parsing and verification After intelligently recognizing the reinforcing steel drawings and obtaining the corresponding parameter text descriptions, to ensure the accuracy and usability of subsequent CNC machining instructions, the recognized output content needs to be standardized and rigorously verified. This step effectively filters out abnormal data caused by recognition deviations, preventing parameter errors from affecting the quality of reinforcing steel processing. The system first receives the text results output by the multimodal large model, performs structured processing on the text content based on preset parsing rules, and extracts the side length values ​​and corresponding bending angle values ​​of each segment of the reinforcing steel from the output content through string matching and sequence extraction. This transforms the discrete text information into a regular parameter sequence that can be directly called, providing a clear data foundation for subsequent verification and conversion work.

[0033] After extraction, the system performs a comprehensive geometric logic check on all parameters. The check primarily includes two core dimensions: first, a non-negativity check, confirming that all side length values ​​are positive, eliminating zero or negative values ​​due to recognition anomalies, and ensuring the parameters conform to the physical rules of actual rebar processing; second, a format integrity check, verifying that the number of extracted parameter groups and the data arrangement conform to the preset sequence specifications, ensuring there are no missing parameters, incorrect formats, or other issues. Simultaneously, the system has a built-in feedback retry mechanism. If the verification result does not meet the requirements, it automatically determines the recognition result as invalid and, based on the type of verification failure, adjusts the relevant parameters or prompting strategies of the multimodal prompt words, re-triggering the model's inference and recognition process until the verification passes or the preset retry limit is reached, thereby maximizing the reliability of the final parameters.

[0034] The qualified parameters, after complete analysis and multiple verifications, have a standardized format and numerical characteristics that meet the actual processing requirements. They can be directly used as a reliable basis for subsequent processing instruction conversion, eliminating the tedious steps of manual verification and correction. This further improves the stability and automation of the entire process from drawing recognition to equipment processing, effectively solving the problems of low efficiency and easy omission of errors in traditional manual verification, and providing a solid data guarantee for the automated processing of steel bars.

[0035] (4) Instruction generation and driving After the geometric parameters of the reinforcing bars are verified and confirmed to be qualified by geometric logic, the processing instruction conversion stage begins. This is a key step connecting drawing recognition and actual automated processing. Its core purpose is to convert the parsed regular parameters into control instructions that the CNC reinforcing bar processing equipment can directly recognize and execute, so as to achieve seamless connection from drawing information to processing actions, completely replacing the traditional mode of manual programming and manual operation of equipment, and improving processing efficiency and accuracy.

[0036] The validated parameter sequence includes the side length values ​​of each segment of the rebar and the corresponding bending angle values. The system converts each set of parameters into executable control code for CNC rebar processing equipment (such as rebar bending machines or bending centers) according to preset parameter and instruction mapping rules. Common control codes include PLC signals or G-codes, ensuring precise compatibility between the instructions and the equipment's control system. Specifically, for each set of parameters, the system generates corresponding equipment action instructions. For example, for parameter combinations, it generates instructions for the feeding mechanism to advance the corresponding side length and for the bending head to rotate according to the required angle. When the angle is negative, the bending head rotates counterclockwise; when the angle is positive, it rotates clockwise. For the last set of parameters with an angle of 0, the system generates instructions for the feeding mechanism to advance the corresponding side length and then activate the cutting mechanism to cut the rebar, ensuring the integrity of the rebar processing.

[0037] After all the control commands corresponding to the parameters are generated, the system integrates these commands into a complete control command sequence according to the order of rebar processing, avoiding processing errors caused by disordered command sequence. Subsequently, this control command sequence is sent to the control terminal of the CNC rebar processing equipment. Upon receiving the commands, the equipment executes feeding, bending, and cutting actions sequentially according to the command sequence, completing the automated processing of the entire rebar without manual intervention. This process not only ensures a high degree of consistency between the processing actions and the drawing parameters, reducing errors caused by manual operation, but also significantly shortens the time from parameter confirmation to equipment startup, adapting to the high-efficiency needs of large-scale rebar processing, promoting the intelligent and automated realization of the entire rebar processing process, further reducing labor costs, and ensuring the stability of processing quality.

[0038] Example 2 Please refer to Figure 3 This embodiment 2 provides a rebar drawing recognition system based on a multimodal large model, including: The image acquisition and preprocessing unit is used to acquire a large-scale image of the steel bar to be processed, and to perform standardized preprocessing on the large-scale image to eliminate image noise and unify the input resolution, so as to obtain an image of the steel bar to be identified. A multimodal feature recognition unit is used to construct multimodal prompt words containing role settings, task descriptions and output format constraints. The image of the steel bar to be recognized and the multimodal prompt words are input together into a pre-trained multimodal large model. The visual encoder of the multimodal large model is used to extract image features, and the correspondence between the geometric topology and dimension annotations in the image is identified based on the semantic alignment mechanism. The output is a structured text description containing the geometric parameters of the steel bar. The parameter parsing and verification unit is used to perform structured parsing on the structured text description, extract the side length and bending angle values ​​of the reinforcing bars, and perform geometric logic verification on the extracted values, which includes at least non-negativity checks and format integrity checks. The instruction generation and drive unit is used to map the parsed steel bar geometric parameters into a sequence of control instructions that can be executed by the CNC steel bar processing equipment after the verification is passed, so as to drive the equipment to perform automated processing.

[0039] Example 3 This embodiment 3 also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement any step of a rebar diagram recognition method based on a multimodal large model.

[0040] The computer-readable storage medium may include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0041] For a description of the computer-readable storage medium provided in this application, please refer to the above method embodiments; further details will not be repeated here.

[0042] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for steel bar map recognition based on a multi-modal large model, characterized in that, include: S1. Obtain a large-scale image of the steel bar to be processed, and perform standardized preprocessing on the large-scale image to eliminate image noise and unify the input resolution to obtain the image of the steel bar to be identified; S2. Construct multimodal prompt words that include role settings, task descriptions and output format constraints. Input the image of the rebar to be identified and the multimodal prompt words into a pre-trained multimodal large model. Use the visual encoder of the multimodal large model to extract image features and identify the correspondence between the geometric topology and dimension annotations in the image based on the semantic alignment mechanism. Output a structured text description containing the geometric parameters of the rebar. S3. Perform structured parsing on the structured text description to extract the side length and bending angle values ​​of the reinforcing bars; and perform geometric logic verification on the extracted values, the geometric logic verification including at least non-negativity check and format integrity check. S4. After the verification is passed, the parsed rebar geometric parameters are mapped and converted into a sequence of control instructions that can be executed by the CNC rebar processing equipment to drive the equipment to perform automated processing.

2. The steel bar map recognition method based on a multi-modal large model according to claim 1, characterized in that, The detailed image in S1 specifically refers to the original image containing the geometric shape and dimension annotations of the reinforcing bars, obtained through an industrial camera, scanner, or exported from CAD software; the original image includes any one of the following: photographs of paper drawings, electronic drawing files, and hand-drawn drawings.

3. The steel bar map recognition method based on a multi-modal large model according to claim 1, characterized in that, The multimodal prompt words in S2 also include a few-sample example module; The few-sample example module contains at least one set of input-output pairs for demonstrating to the multimodal large model the mapping rules from rebar detail images to specific format text descriptions. The specific format text descriptions adopt the sequence form of "length, angle; length, angle", where positive and negative angle values ​​represent clockwise and counterclockwise bending, respectively.

4. The multimodal large model is a generative pre-trained model with visual-language cross-modal alignment capability; The steel bar diagram recognition method based on a multi-modal large model according to claim 1 is characterized in that, The multimodal large model in S2 is configured to bind dimension annotation numbers with corresponding geometric line segments based on semantic understanding under non-standard drawing conditions such as offset annotation positions, presence of leader annotations, or image blurring, so as to output the correct rebar geometric parameters.

5. The steel bar map recognition method based on a multi-modal large model according to claim 4, characterized in that, The specific process of identifying the correspondence between geometric topology and dimension annotations in the image based on semantic alignment mechanism in S2 is as follows: First, topological feature extraction and vectorization are performed. The set of geometric line segments in the rebar image is abstracted into point cloud data, a Vietoris-Rips complex is constructed, and the birth and death process of topological features is observed by dynamically adjusting the scale parameter, thereby calculating the Betti number and generating a persistent graph. Subsequently, the discrete topological features in the persistent graph are transformed into vector representations in a continuous function space using a persistent landscape function, thereby quantifying the topological stability of closed regions and connectivity in the rebar drawing. Secondly, a semantic alignment energy functional based on Riemannian manifolds is constructed; The dimension annotation text is encoded into semantic vectors and mapped onto a geometric manifold. An energy functional containing Dirichlet energy terms and data fidelity terms is constructed. The aim is to find the optimal mapping function that minimizes the mapping error between the geometric topology and the semantic vector while maintaining the geometric smoothness of the geometric manifold, thereby achieving alignment between geometric topological distance and semantic distance. Next, perform an optimization solution based on spectral analysis; The Laplace-Beltrami operator is introduced to construct a generalized eigenvalue problem. By solving the generalized eigenvalue problem, the orthogonal spectral basis of the geometric manifold is obtained, and the complex geometric structure of the manifold is decomposed into a computable combination of basis functions. Finally, the correspondence between the geometric topology and the dimension annotation text is determined; by calculating the projection coefficients of the mapping function under the orthogonal spectral basis, a bijective relationship between the geometric topology points and the dimension semantic vector is established, thereby completing the automatic identification and matching of geometric structure and dimension information in the reinforcement drawings.

6. The method for rebar drawing recognition based on a multimodal large model according to claim 5, characterized in that, The specific process of transforming the discrete topological features in the persistent graph into a vector representation in a continuous function space using persistent landscape functions is as follows: First, construct the triangular waveform basis functions; for the persistent graph... Each topological feature point in the data, at its birth scale and death scale As parameters, construct a function defined in the real number field. Piecewise linear functions on Generate the corresponding triangular waveform basis function, whose mathematical expression is: Wherein, the peak position of the triangular waveform basis function is located at The bottom width corresponds to the length of the persistent interval of the topological feature point. , This represents the total number of topological feature points in the persistent graph. It is an index variable used to iterate through each topological feature point in the set; Secondly, generate a hierarchical sorting sequence; for each predefined scale variable within the domain. Calculate the set of function values ​​for all the aforementioned triangular waveform basis functions at this scale variable. The obtained function values ​​are then sorted in descending order of their numerical values ​​to form an ordered sequence of function values. ; Next, extract the top-level envelope curve; then, in the ordered sequence of function values, extract the first... The maximum function value at the i-th position is defined as the i-th Layered persistent landscape function The value at this scale, i.e. By sliding the scale variable Traverse the entire domain to generate a set of continuous function curves describing the hierarchical distribution of topological feature intensity. Finally, vectorized discrete sampling is performed on the continuous persistent landscape function curves. In the sampling point set The function values ​​are solved by sampling the sequence of function values. As a numerical vector representation of the topology of steel reinforcement drawings.

7. The method for rebar diagram recognition based on a multimodal large model according to claim 5, characterized in that, The energy functional containing the Dirichlet energy term and the data fidelity term is specifically as follows: in, The representation is defined on the geometric manifold The total energy functional on, The optimal mapping function to be solved is used to map the geometry on the manifold to the scale semantic space; The Dirichlet energy term Used for constraint mapping functions Smoothness on the manifold, where manifold gradient operator on, Represents the Riemannian metric tensor norm, For volume elements on the manifold, this term preserves the connectivity of the geometric topology by minimizing the rate of change of the function on the manifold; The data fidelity item Used to constrain the consistency between mapping results and observed data, where Let the coordinates be the geometric nodes on the manifold. This is the semantic vector of the dimension annotation corresponding to this node. This is a non-negative regularization parameter used to balance the weight relationship between manifold smoothness constraints and data fidelity constraints.

8. The method for rebar drawing recognition based on a multimodal large model according to claim 1, characterized in that, S3 also includes a feedback retry mechanism: if the geometric logic verification fails, the system automatically adjusts the parameters or prompting strategy of the multimodal prompt words and returns to step S2 to re-perform the reasoning until the verification passes or the preset retry limit is reached.

9. A reinforcement drawing recognition system based on a multimodal large model, characterized in that, include: The image acquisition and preprocessing unit is used to acquire a large-scale image of the steel bar to be processed, and to perform standardized preprocessing on the large-scale image to eliminate image noise and unify the input resolution, so as to obtain an image of the steel bar to be identified. A multimodal feature recognition unit is used to construct multimodal prompt words containing role settings, task descriptions and output format constraints. The image of the steel bar to be recognized and the multimodal prompt words are input together into a pre-trained multimodal large model. The visual encoder of the multimodal large model is used to extract image features, and the correspondence between the geometric topology and dimension annotations in the image is identified based on the semantic alignment mechanism. The output is a structured text description containing the geometric parameters of the steel bar. The parameter parsing and verification unit is used to perform structured parsing on the structured text description, extract the side length and bending angle values ​​of the reinforcing bars, and perform geometric logic verification on the extracted values, which includes at least non-negativity checks and format integrity checks. The instruction generation and drive unit is used to map the parsed steel bar geometric parameters into a sequence of control instructions that can be executed by the CNC steel bar processing equipment after the verification is passed, so as to drive the equipment to perform automated processing.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program is executed by a processor as described in any one of claims 1-8: a method for identifying reinforcement drawings based on a multimodal large model.

Images