A method for detecting cross-sectional dimensions of structural components based on visual model and local element method

Abstract

The present invention discloses a method for detecting the cross-sectional dimensions of a structural component based on a visual model and a local unit method, which belongs to the technical field of intelligent detection of engineering structures, and comprises: photographing the measured structural component with a camera to obtain an image of the measured structural component; calibrating the camera to obtain camera calibration parameters; inputting the image of the measured structural component into a preset image segmentation model, obtaining an effective mask of the image of the measured structural component using the image segmentation model, and extracting a background mask from the effective mask; fusing the background mask with the image of the measured structural component to generate a binary mask containing only the target area; based on the binary mask, combined with the camera calibration parameters, the cross-sectional dimensions of the structural component are detected. The method of the present invention has the characteristics of high efficiency, accuracy, and stability, is suitable for the detection needs of structural components in various complex environments, and can significantly improve the automation and accuracy of the detection work.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent detection of engineering structures, and in particular to a method for detecting the cross-sectional dimensions of structural components based on a visual model and a local unit method. Background Art

[0002] Digital detection of structural component dimensions, as an important part of structural quality and safety assessment during the construction and operation and maintenance of engineering structures, has become an important research topic in the field of civil engineering. Automated reality capture technology, including various types of cameras and laser scanning technology, provides an effective way to quickly detect component dimensions. Unlike traditional labor-intensive manual detection methods that rely on human subjective consciousness, automated detection of components can ensure the reliability, objectivity and repeatability of quantitative analysis. Therefore, with the rapid development of information and intelligent technology, with the help of deep learning and computer vision technology, the implementation of fast and accurate intelligent detection of structural component dimensions and timely identification of the actual dimensions of structural components are of vital importance to improving the accuracy of structural quality evaluation and safety assessment.

[0003] However, most of the current component size detection methods based on image processing technology rely on manual feature extraction, which is complex and sensitive to factors such as image quality and noise. Although 3D laser scanning technology has high detection accuracy, it has high requirements for scanning equipment and is easily disturbed by the surrounding environment. The data processing process of the 3D point cloud model is relatively complex, with a lot of redundant information, and requires professional technicians, which limits the application and popularization of this technology. In addition, in the field of industrial manufacturing, photogrammetry, laser scanning, or a combination of the two is usually used on fixed devices or auxiliary platforms to quickly detect small-sized components. However, due to the limitations of the detection device and platform, only relatively small components can be detected at close range, and the portability is poor.

[0004] In addition, the existing detection methods only provide the overall size information of the component, and fail to detect the size corresponding to each cross-section. For components with variable cross-sections, the detection of cross-sectional size information at different positions is more important. Summary of the invention

[0005] The present invention provides a method for detecting the cross-sectional dimensions of a structural member based on a visual model and a local unit method, so as to solve the technical problems that the existing member dimension detection technology has a complex implementation process, a limited use environment, and the detected member dimension information is not comprehensive enough.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In one aspect, the present invention provides a method for detecting the cross-sectional dimensions of a structural member based on a visual model and a local element method, the method comprising:

[0008] Using a camera to photograph the measured structural component to obtain an image of the measured structural component;

[0009] Calibrate the camera to obtain calibration parameters of the camera;

[0010] Inputting the image of the structural component to be measured into a preset image segmentation model, using the image segmentation model to obtain an effective mask corresponding to the image of the structural component to be measured, and extracting a background mask from the effective mask;

[0011] The background mask is merged with the image of the structural component to generate a binary mask containing only the target area;

[0012] The local element method is adopted, based on the binary mask and combined with the calibration parameters of the camera, to realize the cross-sectional dimension detection of the structural component.

[0013] Furthermore, the preset image segmentation model is a SAM model.

[0014] Furthermore, the image of the structural component to be measured is input into a preset image segmentation model, and a valid mask corresponding to the image of the structural component to be measured is obtained by using the image segmentation model, including:

[0015] The image of the structural component to be measured is input into the SAM model, and the coordinate information of the cross intersection of the camera imaging is used as the input of the prompt module of the SAM model, and the effective mask corresponding to the image of the structural component to be measured is obtained by using the SAM model.

[0016] Furthermore, the local element method is used, based on the binary mask, in combination with the calibration parameters of the camera to realize the cross-sectional dimension detection of the structural component, including:

[0017] Calculating the minimum rectangular bounding box size of the binary mask by using a connected domain analysis method, and extracting the upper left corner coordinates, width and height of the minimum rectangular bounding box of the binary mask;

[0018] Performing least squares linear fitting on all pixel coordinates in the binary mask to obtain a straight line fitting all pixels in the binary mask, and obtaining its inclination angle through the slope of the straight line fitting all pixels in the binary mask; combining the width and height of the minimum rectangular bounding box of the binary mask and the inclination angle of the straight line fitting all pixels in the binary mask to calculate the size of the measured structural component;

[0019] According to the coordinates of the upper left corner and the lower right corner of the minimum rectangular bounding box of the binary mask, the minimum rectangular bounding box is divided into N local units; wherein N is a preset integer value greater than 1;

[0020] Calculate the pixel area of ​​the mask in each local unit respectively; at the same time, use the image contour detection technology to extract each pixel coordinate of the outermost contour of the mask in each local unit, and ensure that the distance between two adjacent pixel points in the outermost contour of the mask in each local unit does not exceed 1 pixel;

[0021] For each local unit, the coordinates of the center point of the mask pixel in the local unit are used as reference points, and the pixel coordinates of the outermost contour of the mask in the local unit are divided into boundaries to obtain the pixel coordinate points on each edge line;

[0022] The pixel coordinate points on the mask edge line in each local unit are respectively fitted by the least square method, so as to obtain the slope of the fitted straight line of each edge line, and convert it to obtain its inclination angle;

[0023] For each local unit, the mask size in the local unit, ie, the cross-sectional size of the structural component, is calculated based on the pixel area of ​​the mask in the local unit and the inclination angle of the fitting straight line of each edge line.

[0024] Furthermore, the measured structural component is a beam component or a column component.

[0025] Furthermore, when the measured structural component is a beam-type component, the formula for calculating the size of the measured structural component is as follows, combining the width and height of the minimum rectangular bounding box of the binary mask and the inclination angle of the fitted straight line of all pixels in the binary mask:

[0026]

[0027] Where L is the length of the beam component; W0 is the width of the minimum rectangular bounding box of the binary mask; θ is the inclination angle of the fitted straight line of all pixels in the binary mask; W m is the pixel calibration value in physical size;

[0028] When the measured structural component is a columnar component, the formula for calculating the size of the measured structural component is as follows, combining the width and height of the minimum rectangular bounding box of the binary mask and the inclination angle of the fitted straight line of all pixels in the binary mask:

[0029]

[0030] Wherein, H is the height of the columnar component; H0 is the height of the minimum rectangular bounding box of the binary mask.

[0031] Further, when the measured structural component is a beam-type component, the minimum rectangular bounding box is divided into N local units, specifically: the minimum rectangular bounding box is divided into N local units along the x-axis;

[0032] When the measured structural component is a columnar component, the minimum rectangular bounding box is divided into N local units, specifically: the minimum rectangular bounding box is divided into N local units along the y-axis.

[0033] Furthermore, when the measured structural component is a beam-type component, the coordinates of the center point of the mask pixel in the local unit are used as the reference point, and the pixel coordinates of the outermost contour of the mask in the local unit are divided into boundaries. Specifically, the coordinates of the center point of the mask pixel in the local unit are used as the reference point. If the y-axis coordinate value of the pixel point of the outermost contour of the mask in the local unit is less than the y-axis coordinate value of the center point of the mask pixel in the local unit, it is determined as a coordinate point on the upper edge line of the mask; otherwise, it is determined as a coordinate point on the lower edge line of the mask;

[0034] When the structural component to be measured is a columnar component, the center point coordinates of the mask pixels in the local unit are used as the reference point, and the pixel coordinates of the outermost contour of the mask in the local unit are divided into boundaries. Specifically, the center point coordinates of the mask pixels in the local unit are used as the reference point. If the x-axis coordinate value of the pixel point of the outermost contour of the mask in the local unit is less than the x-axis coordinate value of the center point of the mask pixels in the local unit, it is determined as a coordinate point on the left edge line of the mask; otherwise, it is determined as a coordinate point on the right edge line of the mask.

[0035] Furthermore, when the measured structural member is a beam member, the calculation formula for the cross-sectional size of the structural member in the local unit is:

[0036]

[0037] Among them, h i is the cross-sectional height of the component in the ith local unit; S i is the pixel area within the i-th local unit; W0 is the width of the minimum rectangular bounding box of the binary mask; W m is the pixel calibration value in physical size; α i is the inclination angle of the upper edge line of the mask in the i-th local unit; β i is the inclination angle of the lower edge line of the mask in the i-th local unit;

[0038] When the measured structural member is a columnar member, the calculation formula for the cross-sectional size of the structural member in the local unit is:

[0039]

[0040] Among them, w i is the cross-sectional width of the component in the ith local unit; H0 is the height of the minimum rectangular bounding box of the binary mask; γ i is the inclination angle of the left edge line of the mask in the i-th local unit; δ i is the inclination angle of the right edge line of the mask in the i-th local unit.

[0041] Furthermore, after the cross-sectional dimension detection of the structural member is realized, the cross-sectional dimension detection method of the structural member based on the visual model and the local element method further comprises:

[0042] The cross-sectional size corresponding to each local unit is mapped back to the image of the measured structural component in the form of a cloud map visualization, so as to intuitively display the cross-sectional size changes of each local unit of the measured structural component.

[0043] On the other hand, the present invention further provides an electronic device, comprising a processor and a memory; wherein the memory stores at least one instruction, and the instruction is loaded and executed by the processor to implement the above method.

[0044] In yet another aspect, the present invention further provides a computer-readable storage medium, wherein at least one instruction is stored in the storage medium, and the instruction is loaded and executed by a processor to implement the above method.

[0045] The beneficial effects brought about by the technical solution provided by the present invention include at least:

[0046] The present invention is based on the SAM model, and utilizes the principle that the cross intersection of the camera lens is aligned with the measured structural component during shooting. The coordinate information of the cross intersection of the camera imaging is used as the input of the SAM model prompt module, thereby automatically generating prompt information, realizing autonomous operation, without manual intervention and additional model training, thereby avoiding frequent human-computer interaction operations. At the same time, the effective mask generated by the model is fused with the original image, and a binary mask containing only structural components is quickly and automatically generated. On this basis, combined with the connected domain analysis and mask feature extraction technology, a method for automatic quantification of structural component size based on the local unit method is proposed. Through this method, all cross-sectional dimensions and height or length information of structural components of different component types can be quickly obtained, which is particularly suitable for unconventional structural component forms such as circular cross-sections and variable cross-sections, thereby significantly improving the accuracy and efficiency of detection. An effective detection means is provided for the construction and safe operation and maintenance stages of buildings. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0048] Figure 1 It is a schematic diagram of the process of automatically generating a binary mask of a specified structural component using the SAM model;

[0049] Figure 2 It is a flow chart of the cross-sectional dimension detection of structural members based on the local element method;

[0050] Figure 3 It is a system block diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0052] First of all, it should be noted that in the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "exemplarily" is intended to present the concept in a concrete way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0053] First embodiment

[0054] This embodiment provides a method for detecting the cross-sectional dimensions of a structural member based on a visual model and a local unit method. The method can be implemented by an electronic device. The execution flow of the method includes the following steps:

[0055] S1, photographing the measured structural component with a camera to obtain an image of the measured structural component;

[0056] Specifically, in this embodiment, the implementation process of the above S1 is: set up a tripod directly in front of the structural component to be measured, and install a camera on the tripod; make fine adjustments through the level bubble on the tripod and the digital level on the top of the camera to ensure that the main optical axis of the camera lens is perpendicular to the surface of the structural component to be measured, and automatically focus and capture the image; wherein, if the shooting space is limited and it is impossible to ensure that the main optical axis of the camera lens is perpendicular to the surface of the structural component to be measured, the camera tilt angle can be measured by the digital level, and the camera imaging can be corrected.

[0057] S2, calibrating the camera to obtain calibration parameters of the camera;

[0058] Specifically, in this embodiment, the implementation process of the above S2 is: measuring the distance between the camera lens and the surface of the measured structural component to obtain the conversion relationship between the image pixel unit and the physical size unit. The conversion relationship between the image pixel unit and the physical size unit can be calculated and obtained by the following two methods:

[0059] First, the pixel calibration value W is calculated using the measured object distance and camera internal parameters. m , the calculation formula is as follows:

[0060]

[0061] Among them, W m Indicates the pixel calibration value in physical size, W p Indicates the size in pixels, P c is the camera sensor parameter, d is the object distance, and f is the focal length.

[0062] The second is to calculate W using the fitting curve of the pixel calibration value of the camera at different object distances. m , taking the camera CMOS sensor size of 23.5x 15.7 mm and the focal length set to 18mm as an example, the fitting formula is as follows:

[0063] W m =6.75934×10 -4 ×d-0.00382 (2)

[0064] S3, inputting the image of the structural component to be measured into a preset image segmentation model, using the image segmentation model to obtain an effective mask corresponding to the image of the structural component to be measured, and extracting a background mask from the effective mask;

[0065] S4, fusing the background mask with the image of the measured structural component to generate a binary mask containing only the target area;

[0066] Among them, in this embodiment, the image segmentation model used is the visual large model SAM; wherein, the visual large model SAM is a human-computer interactive visual segmentation model (Segment Anything Model, referred to as SAM) based on prompt information, and no additional model training is required in downstream tasks. Based on this, this embodiment uses the strategy that when shooting the object under test with a camera, it is usually necessary to align the cross intersection of the lens with the object under test, input the image of the measured structural component into the SAM model, and use the coordinate information of the cross intersection of the camera imaging as the input of the prompt module of the SAM model, so that the model can autonomously segment the specified area in the image without human-computer interaction, thereby using the SAM model to automatically and quickly generate an effective mask corresponding to the image of the measured structural component, and its implementation process is as follows: Figure 1 As shown, specifically: after the image is input into the model, the encoding module is first entered to extract and encode the image features. At the same time, the coordinate information of the cross intersection of the camera imaging is passed to the prompt module of the model, so that the visual large model SAM can operate autonomously without human-computer interaction. Then, the mask decoding module combines the output result of the encoding module and the encoding of the prompt information to generate an effective segmentation mask for the target object. Furthermore, given that the effective mask output by the mask decoding module cannot be directly used for pixel processing in the target area, it is necessary to first extract the background mask from the effective mask and fuse it with the original image. By fusing the effective mask with the original image, the background pixels can be deleted, thereby quickly and automatically generating a binary mask image containing only pixels in the target area.

[0067] S5, using a local element method, based on the binary mask and in combination with the calibration parameters of the camera, to detect the cross-sectional dimensions of the structural component;

[0068] Specifically, in this embodiment, Figure 2 As shown, the implementation process of the above S5 is as follows:

[0069] S51, using a connected domain analysis method to calculate the minimum rectangular bounding box size of the binary mask, and then extracting feature information of the outer contour of the mask; wherein the feature information of the outer contour of the mask includes the upper left corner coordinates (x0, y0), width W0 and height H0 of the minimum rectangular bounding box of the binary mask;

[0070] S52, performing least squares linear fitting on all pixel coordinates in the binary mask to obtain a straight line fitted by all pixels in the binary mask, and obtaining an inclination angle thereof through the slope of the straight line fitted by all pixels in the binary mask; combining the width and height of the minimum rectangular bounding box of the binary mask and the inclination angle of the straight line fitted by all pixels in the binary mask, to quickly calculate the size of the measured structural component;

[0071] The measured structural member is a beam member or a column member; the size of the measured structural member calculated in S52 is the height or length of the measured structural member; wherein the length of the structural member corresponds to the length of the beam member, and the calculation formula thereof is as follows:

[0072]

[0073] Where L is the length of the beam component; W0 is the width of the minimum rectangular bounding box of the binary mask; θ is the inclination angle of the fitted straight line of all pixels in the binary mask; W m is the pixel calibration value in physical size;

[0074] The height of the structural member corresponds to the height of the column member, and its calculation formula is as follows:

[0075]

[0076] Wherein, H is the height of the columnar component; H0 is the height of the minimum rectangular bounding box of the binary mask.

[0077] S53, according to the coordinates of the upper left corner (x0, y0) and the lower right corner (x0+W0, y0+H0) of the minimum rectangular bounding box of the binary mask, divide the minimum rectangular bounding box into N local units, and number each local unit, where the unit number i=1, 2, ... N; wherein N is a preset integer value greater than 1;

[0078] Among them, when the measured structural component is a beam-type component, the minimum rectangular bounding box is divided into N local units, specifically: the minimum rectangular bounding box is divided into N local units along the x-axis; when the measured structural component is a column-type component, the minimum rectangular bounding box is divided into N local units, specifically: the minimum rectangular bounding box is divided into N local units along the y-axis.

[0079] S54, respectively calculate the pixel area of ​​the mask in each local unit; at the same time, use the image contour detection technology to extract each pixel coordinate (x ij ,y ij ), j = 1, 2, ... M, and ensure that the distance between two adjacent coordinate points in the vertical and horizontal directions does not exceed 1 pixel;

[0080] Among them, the image contour detection technology uses the edge search algorithm to detect the outermost contour pixel coordinates of the mask in the local unit, and ensures that the distance between two adjacent coordinate points in the vertical and horizontal directions does not exceed 1 pixel. The aforementioned ensuring that the distance between two adjacent coordinate points in the vertical and horizontal directions does not exceed 1 pixel is a restriction condition added to the edge search algorithm to overcome the large error that may be caused by the edge search algorithm when processing the edge line shape mutation part.

[0081] S55, for each local unit, using the coordinates of the center point of the mask pixel in the local unit as a reference point, dividing the boundary of the pixel coordinates of the outermost contour of the mask to obtain the pixel coordinate points on each edge line;

[0082] When the measured structural component is a beam-type component, the coordinates of the center point of the mask pixel in the local unit are used as the reference point to divide the boundary of the pixel coordinates of the outermost contour of the mask in the local unit. Specifically, the coordinates of the center point of the mask pixel in the local unit (x i0 ,y i0 ) as a reference point, if the y-axis coordinate value of the pixel point of the outermost contour of the mask in the local unit is less than y i0 , then it is determined as a coordinate point on the upper edge line of the mask; otherwise, it is determined as a coordinate point on the lower edge line of the mask;

[0083] When the measured structural component is a columnar component, the coordinates of the center point of the mask pixel in the local unit are used as the reference point to divide the boundary of the pixel coordinates of the outermost contour of the mask in the local unit. Specifically, the coordinates of the center point of the mask pixel in the local unit (x i0 ,y i0 ) as a reference point, if the x-axis coordinate value of the pixel point of the outermost contour of the mask in the local unit is less than x i0 , then it is determined as a coordinate point on the left edge line of the mask; otherwise, it is determined as a coordinate point on the right edge line of the mask.

[0084] S56, performing least squares linear fitting on the pixel coordinate points on the mask edge line in each local unit, thereby obtaining the slope of the fitting line of each edge line, and converting it to obtain its inclination angle;

[0085] S57, for each local unit, taking into account the influence of the inclination angle of each edge line of the mask in the local unit, making corrections thereto, and calculating the mask size in the local unit, that is, the cross-sectional size of the structural component, based on the pixel area of ​​the mask in the local unit and the inclination angle of the fitting straight line of each edge line;

[0086] When the measured structural component is a beam-type component, the influence of the horizontality of the upper and lower edge lines of the mask in the local unit needs to be considered. The mask height in each local unit, that is, the component section height h i The calculation formula is as follows:

[0087]

[0088] Among them, h i is the cross-sectional height of the component in the ith local unit; S i is the pixel area in the i-th local unit; αi is the inclination angle of the upper edge line of the mask in the i-th local unit; β i is the inclination angle of the lower edge line of the mask in the i-th local unit;

[0089] When the measured structural component is a columnar component, the influence of the inclination of the edge lines on the left and right sides of the mask in the local unit needs to be considered. The mask width in each local unit, that is, the component cross-sectional width w i The calculation formula is as follows:

[0090]

[0091] Among them, w i is the cross-sectional width of the component in the ith local unit; i is the inclination angle of the left edge line of the mask in the i-th local unit; δ i is the inclination angle of the right edge line of the mask in the i-th local unit.

[0092] S6, the cross-sectional dimensions corresponding to each local unit are mapped back to the original image in the form of cloud map visualization, so as to intuitively display the cross-sectional dimension changes of each local unit of the measured structural component, that is, the cross-sectional dimension changes at different positions of the component.

[0093] In summary, this embodiment provides a method for detecting the cross-sectional dimensions of structural components based on a visual model and a local unit method, which integrates the visual large model into the engineering construction and diagnosis and operation and maintenance direction, and can obtain holographic and global detection results, promoting objective, quantifiable and repeatable structural quality evaluation and safety assessment. The promotion and application of this technology can not only improve the efficiency and accuracy of detection, but also reduce the dependence on manual operation and professional equipment, thereby promoting further development in the field of civil engineering in terms of quality control and safety management.

[0094] Second embodiment

[0095] This embodiment provides an electronic device, such as Figure 3 As shown, the electronic device includes: a processor and a memory; wherein the processor and the memory can be connected via a communication bus; the memory stores at least one instruction, and the instruction is loaded and executed by the processor to implement the method of the first embodiment. In addition, the electronic device may also include a transceiver, the processor and the transceiver can be connected via a communication bus, and the transceiver is used to communicate with other devices.

[0096] Next, combine Figure 3 The following is a detailed introduction to the various components of the electronic device:

[0097] Among them, the processor is the control center of the electronic device, and the electronic device may include multiple processors, each of which may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may be a processor or a general term for multiple processing elements. For example, the processor is one or more central processing units (CPUs), or other general-purpose processors, application specific integrated circuits (ASICs), or one or more integrated circuits configured to implement an embodiment of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (field programmable gate arrays, FPGAs), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor may execute various functions of the electronic device by running or executing software programs stored in the memory and calling data stored in the memory.

[0098] In a specific implementation, as an embodiment, the processor may include one or more CPUs, such as Figure 3 The CPU0 and CPU1 shown in the figure are, of course, only exemplary.

[0099] The memory is used to store the software program for executing the solution of the present invention, and the execution is controlled by the processor. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0100] Optionally, the memory may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may be integrated with the processor or exist independently and accessed through the interface circuit ( Figure 3 (not shown) is coupled to the processor, which is not specifically limited in this embodiment of the present invention.

[0101] The transceiver may include a receiver and a transmitter ( Figure 3 The receiver is used to implement the receiving function, and the transmitter is used to implement the sending function. The transceiver can be integrated with the processor or exist independently and communicate with the electronic device through the interface circuit ( Figure 3 (not shown) is coupled to the processor, which is not specifically limited in this embodiment of the present invention.

[0102] In addition, it should be noted that Figure 3 The structure of the electronic device shown in the figure does not constitute a limitation on the device, and the actual device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. In addition, the technical effects achieved by the electronic device when executing the method of the first embodiment above can refer to the technical effects described in the first embodiment above, so they are not repeated here.

[0103] Third embodiment

[0104] This embodiment provides a computer-readable storage medium, which stores at least one instruction, and the instruction is loaded and executed by a processor to implement the method of the first embodiment. The computer-readable storage medium may be a ROM, a random access memory, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc. The instructions stored therein may be loaded by a processor in a terminal to execute the method.

[0105] In addition, it should be noted that the present invention can be provided as a method, an apparatus or a computer program product. Therefore, the embodiment of the present invention can be in the form of a full or partial hardware embodiment, a full or partial software embodiment or an embodiment combining software and hardware. Moreover, when implemented using software, the embodiment of the present invention can be in the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program codes. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center containing one or more available media sets. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium. The semiconductor medium may be a solid state hard disk.

[0106] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0107] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0108] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of more restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements. In addition, the term "and / or" is only an association relationship describing the associated objects, indicating that there can be three relationships, for example, A and / or B, which can represent: A exists alone, A and B exist at the same time, and B exists alone, wherein A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding. "At least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can be represented by: a, b, c, ab, ac, bc or abc, where a, b, c can be single or plural.

[0109] In addition, it can be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0110] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0111] In several embodiments provided by the present invention, it should be understood that the disclosed equipment, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of functional modules / units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The unit described as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or it may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, each functional unit in each embodiment of the present invention can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0112] If the method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0113] Finally, it should be noted that the above is only a preferred embodiment of the present invention. It should be pointed out that although the preferred embodiment of the present invention has been described, for ordinary technicians in this technical field, once the basic creative concept of the present invention is known, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the protection scope of the present invention. Therefore, the attached claims are intended to be interpreted as including the preferred embodiment and all changes and modifications that fall within the scope of the embodiments of the present invention.

Claims

1. A method for detecting the cross-sectional dimensions of a structural member based on a visual model and a local element method, characterized in that: The structural member cross-sectional dimension detection method based on the visual model and the local element method comprises: Using a camera to photograph the measured structural component to obtain an image of the measured structural component; Calibrate the camera to obtain calibration parameters of the camera; Inputting the image of the structural component to be measured into a preset image segmentation model, using the image segmentation model to obtain an effective mask corresponding to the image of the structural component to be measured, and extracting a background mask from the effective mask; The background mask is merged with the image of the structural component to generate a binary mask containing only the target area; Using a local element method, based on the binary mask and in combination with the calibration parameters of the camera, the cross-sectional dimensions of the structural component are detected; The local unit method is used, based on the binary mask, in combination with the calibration parameters of the camera to realize the cross-sectional dimension detection of the structural component, including: Calculating the minimum rectangular bounding box size of the binary mask by using a connected domain analysis method, and extracting the upper left corner coordinates, width and height of the minimum rectangular bounding box of the binary mask; Performing least squares linear fitting on all pixel coordinates in the binary mask to obtain a straight line fitting all pixels in the binary mask, and obtaining its inclination angle through the slope of the straight line fitting all pixels in the binary mask; combining the width and height of the minimum rectangular bounding box of the binary mask and the inclination angle of the straight line fitting all pixels in the binary mask to calculate the size of the measured structural component; According to the coordinates of the upper left corner and the lower right corner of the minimum rectangular bounding box of the binary mask, the minimum rectangular bounding box is divided into N local units; wherein N is a preset integer value greater than 1; Calculate the pixel area of ​​the mask in each local unit respectively; at the same time, use the image contour detection technology to extract each pixel coordinate of the outermost contour of the mask in each local unit, and ensure that the distance between two adjacent pixel points in the outermost contour of the mask in each local unit does not exceed 1 pixel; For each local unit, the coordinates of the center point of the mask pixel in the local unit are used as reference points, and the pixel coordinates of the outermost contour of the mask in the local unit are divided into boundaries to obtain the pixel coordinate points on each edge line; The pixel coordinate points on the mask edge line in each local unit are respectively fitted by the least square method, so as to obtain the slope of the fitted straight line of each edge line, and convert it to obtain its inclination angle; For each local unit, the mask size in the local unit, ie, the cross-sectional size of the structural component, is calculated based on the pixel area of ​​the mask in the local unit and the inclination angle of the fitting straight line of each edge line.

2. The method for detecting the cross-sectional dimensions of a structural member based on a visual model and a local element method according to claim 1, characterized in that: The preset image segmentation model is the SAM model.

3. The method for detecting the cross-sectional dimensions of a structural member based on a visual model and a local element method as claimed in claim 2, characterized in that: Inputting the image of the structural component to be measured into a preset image segmentation model, and using the image segmentation model to obtain a valid mask corresponding to the image of the structural component to be measured, comprising: The image of the structural component to be measured is input into the SAM model, and the coordinate information of the cross intersection of the camera imaging is used as the input of the prompt module of the SAM model, and the effective mask corresponding to the image of the structural component to be measured is obtained by using the SAM model.

4. The method for detecting the cross-sectional dimensions of a structural member based on a visual model and a local element method according to claim 1, characterized in that: The measured structural component is a beam component or a column component.

5. The method for detecting the cross-sectional dimensions of a structural member based on a visual model and a local element method as claimed in claim 4, characterized in that: When the measured structural component is a beam-type component, the formula for calculating the size of the measured structural component is as follows, combining the width and height of the minimum rectangular bounding box of the binary mask and the inclination angle of the fitted straight line of all pixels in the binary mask: Where L is the length of the beam component; W0 is the width of the minimum rectangular bounding box of the binary mask; θ is the inclination angle of the fitted straight line of all pixels in the binary mask; W m is the pixel calibration value in physical size; When the measured structural component is a columnar component, the formula for calculating the size of the measured structural component is as follows, combining the width and height of the minimum rectangular bounding box of the binary mask and the inclination angle of the fitted straight line of all pixels in the binary mask: Wherein, H is the height of the columnar component; H0 is the height of the minimum rectangular bounding box of the binary mask.

6. The method for detecting the cross-sectional dimensions of a structural member based on a visual model and a local element method as claimed in claim 4, characterized in that: When the measured structural component is a beam-type component, the minimum rectangular bounding box is divided into N local units, specifically: the minimum rectangular bounding box is divided into N local units along the x-axis; When the measured structural component is a columnar component, the minimum rectangular bounding box is divided into N local units, specifically: the minimum rectangular bounding box is divided into N local units along the y-axis.

7. The method for detecting the cross-sectional dimensions of a structural member based on a visual model and a local element method as claimed in claim 4, characterized in that: When the measured structural component is a beam-type component, the coordinates of the center point of the mask pixel in the local unit are used as the reference point, and the pixel coordinates of the outermost contour of the mask in the local unit are divided into boundaries. Specifically, the coordinates of the center point of the mask pixel in the local unit are used as the reference point. If the y-axis coordinate value of the pixel point of the outermost contour of the mask in the local unit is less than the y-axis coordinate value of the center point of the mask pixel in the local unit, it is determined as a coordinate point on the upper edge line of the mask; otherwise, it is determined as a coordinate point on the lower edge line of the mask; When the structural component to be measured is a columnar component, the center point coordinates of the mask pixels in the local unit are used as the reference point, and the pixel coordinates of the outermost contour of the mask in the local unit are divided into boundaries. Specifically, the center point coordinates of the mask pixels in the local unit are used as the reference point. If the x-axis coordinate value of the pixel point of the outermost contour of the mask in the local unit is less than the x-axis coordinate value of the center point of the mask pixels in the local unit, it is determined as a coordinate point on the left edge line of the mask; otherwise, it is determined as a coordinate point on the right edge line of the mask.

8. The method for detecting the cross-sectional dimensions of a structural member based on a visual model and a local element method as claimed in claim 7, characterized in that: When the measured structural member is a beam member, the calculation formula for the cross-sectional dimensions of the structural member in the local unit is: Among them, h i is the height of the component section in the i-th local unit; S i is the pixel area within the i-th local unit; W0 is the width of the minimum rectangular bounding box of the binary mask; W m is the pixel calibration value in physical size; α i is the inclination angle of the upper edge line of the mask in the i-th local unit; β i is the inclination angle of the lower edge line of the mask in the i-th local unit; When the measured structural member is a columnar member, the calculation formula for the cross-sectional size of the structural member in the local unit is: Among them, w i is the cross-sectional width of the component in the ith local unit; H0 is the height of the minimum rectangular bounding box of the binary mask; γ i is the inclination angle of the left edge line of the mask in the i-th local unit; δ i is the inclination angle of the right edge line of the mask in the i-th local unit.

9. The method for detecting the cross-sectional dimensions of a structural member based on a visual model and a local element method as claimed in claim 8, characterized in that: After the cross-sectional dimension detection of the structural member is realized, the cross-sectional dimension detection method of the structural member based on the visual model and the local element method further comprises: The cross-sectional size corresponding to each local unit is mapped back to the image of the measured structural component in the form of a cloud map visualization, so as to intuitively display the cross-sectional size changes of each local unit of the measured structural component.

Images