Building material stacking capacity monitoring and early warning method based on multi-modal model

Abstract

The invention discloses a building material stacking capacity monitoring and early warning method based on a multi-modal model. The method comprises the following steps: acquiring a front-end cross-section image IF and a top view image IT of building material stacking; inputting the IF into a pre-trained multi-modal model M, and outputting a construction material type C and a section type S; calculating the total section area A of the construction material; based on the top view IT, obtaining the pixel side length L and the pixel width B of the stacking area; calculating the actual area Aa of the constructed material stacking area; calculating the construction material weight W and the unit area weight w; and a grading early warning threshold value Wmax and wmax of the weight of the construction materials are preset for safety monitoring. According to the method, the image of the building material stacking area is collected, the type and the section type of the building material are accurately recognized through the multi-modal model, the weight of the building material is calculated, and real-time safety monitoring and early warning are carried out by setting the early warning threshold value, so that the intelligent level of building material stacking monitoring and the early warning precision are improved.

Description

Technical Field

[0001] The invention relates to a construction material stockpile load monitoring and early warning method based on a multi-modal model, and belongs to the technical field of building engineering construction. Background Art

[0002] Material stacking in construction sites is one of the common safety hazards at construction sites. Unreasonable stacking may lead to accidents such as the collapse of material carriers, threatening the lives of workers. Therefore, it is necessary to control the stacking of construction materials. Traditional construction material stacking control mainly relies on manual inspections and experience judgments, which has problems such as low supervision efficiency and delayed risk discovery.

[0003] At present, there are some relevant technical solutions for monitoring and early warning of construction material loading. For example, the construction material stacking safety monitoring system based on the Internet of Things can monitor the material stacking status in real time by deploying strain sensors in the material stacking area, and upload the monitoring data to the cloud platform for analysis and early warning, which can realize real-time monitoring of material stacking, but it is limited to the collection of strain data and lacks intuitive perception of the stacking status. There are also construction material stacking monitoring methods that use stereo vision, which obtain the three-dimensional point cloud data of the material stacking area through a binocular camera, and combine point cloud segmentation and clustering algorithms to realize automatic estimation of the material stacking volume, which to a certain extent realizes the monitoring and early warning of construction material loading, but there are still the following deficiencies: the identification and extraction of construction material types and key parameters need to be further strengthened, and there is a lack of intelligent analysis capabilities; the evaluation and early warning of the stacking status rely on preset thresholds, lacking adaptability and flexibility; the fusion analysis of multi-source heterogeneous data is not fully utilized, and the comprehensive effectiveness of monitoring and early warning needs to be improved; the push and presentation of early warning information is relatively single, and the real-time and interactivity are insufficient; there is a lack of indirect carbon emission control measures for construction materials during the construction process.

[0004] Therefore, there is an urgent need for a more intelligent and efficient construction material loading monitoring and early warning method to improve the level of informatization and intelligence of construction site safety management. Summary of the invention

[0005] In view of the problems existing in the existing construction material loading control, the present invention provides a construction material loading monitoring and early warning method based on a multimodal model, which identifies the type of construction materials by introducing a multimodal model and calculates the weight of the construction materials. W And unit area weight w As the basis for determining the loading amount of construction materials, the monitoring accuracy is improved.

[0006] In order to solve the above technical problems, the present invention includes the following technical solutions: A construction material pile load monitoring and early warning method based on a multimodal model comprises the following steps: S1. Deploy cameras to capture front-end cross-sectional images of construction material stacking I F and overhead image I T ; S2. Section image I F Input pre-trained multimodal model M , output construction material type C And the cross-section type S ; S3. Calculate the total cross-sectional area of ​​the construction material A ; S4. Based on overhead image I T , get the pixel side length of the stacking area L and pixel width B ; S5. Calculate the actual length of the construction material storage area L a and the actual width B a , calculate the actual area of ​​the construction material storage area A a ; S6. Based on the type of construction material C And the cross-section type S , select the density parameter r c , combined with A , L a , A a Calculate the weight of construction materials W And unit area weight w ; S7. Preset graded warning thresholds for construction material weight W max , w max ,like W or w When the corresponding warning threshold is exceeded, the system automatically triggers a warning.

[0007] Furthermore, the multimodal model M Set the construction material type prompt word set in T , prompt word set T Include qualifiers T 1 and output word T 2. Qualifiers T1Indicates the range of construction material types to be identified, output word T 2 Given a multimodal model M Output result format; multimodal model M Front section image based on the stacking of construction materials I F , determine the type of construction material C And the cross-section type S , the formula is: ( C , S )= M ( I F , T ); in, C ∈( C 1, C 2,…, C m ), C 1, C 2,…, C m Represents construction materials m Types, S ∈( S 1, S 2,…, S n ), S 1, S 2,…, S n Represents construction materials n Type of cross section.

[0008] Furthermore, a camera acquisition frequency f is preset, and at each acquisition moment, the camera continuously captures multiple frames of images, and averages the multiple frames of images to obtain an average image at the acquisition moment, and the average image is used for subsequent recognition and analysis.

[0009] Furthermore, a first camera and a second camera are respectively deployed at the front and top of the construction material stacking area, and the first camera captures the front cross-sectional image of the construction material stacking area. I F The second camera captures a bird's-eye view of the pile of construction materials I T ; The first camera and the second camera both use binocular cameras, which realize three-dimensional reconstruction of the stacking area through binocular vision technology to obtain three-dimensional point cloud data.

[0010] Furthermore, the warning threshold W max , wmax , according to the type of construction material C , dynamic setting of construction site conditions.

[0011] Further, in step S6, based on the type of building materials C , select the carbon emission factor of the corresponding material, according to the weight of the construction material W The indirect carbon emissions of the material are calculated, and in step S7, a warning threshold for carbon emissions of the construction material is set. If the indirect carbon emissions of the material exceed the corresponding warning threshold, the system automatically triggers a warning.

[0012] Further, in step S3, the total cross-sectional area of ​​the construction material is calculated A , including: Based on the type of construction material C With cross-section type S , determine the cross-sectional parameters of the construction material; Through the image I F Perform image processing and analysis to identify the number of cross-sections of construction materials N ; According to the cross-section parameters and quantity N , calculate the total cross-sectional area of ​​the building material A .

[0013] Further, the cross-sectional parameters of the construction material are determined, and the cross-sectional parameters of the construction material are extracted using an image recognition method, specifically including: S3-1. Front end cross-sectional image I F Perform preprocessing, including grayscale, median filtering and contrast enhancement, to obtain the preprocessed image I P ; S3-2. Image I P Perform Canny edge detection to obtain edge image I edge ; I edge =Canny(Ip,low T ,high T , s ); Among them, low T and high T are low threshold and high threshold respectively, s is the Gaussian filter scale; S3-3. According to the cross-section type S , respectively, to detect and extract parameters of circular or rectangular cross sections; (1) When S When the cross section is circular, the Hough transform principle is used to transform the edge image. I edge Detect the circle and get the center coordinates ( x c , y c ) and radius r ; [ x c , y c , r =HoughCircles( I edge , r min , r max ,△ r ,△ x ,△ y ); in, r min and r max To detect the minimum and maximum radius of a circle, △ r is the radius step length, △ x and y is the step size of the circle center coordinates; (2) When S When the cross section is rectangular, the edge image I edge Extract the contours from the R i}: { R i}=findContours( I edge ); For each contour C i , fit the minimum circumscribed rectangle and obtain the center coordinates of the rectangle ( x i , y i ),width w i and height h i :[ x i , y i , w i , h i]=minAreaRect( R i ).

[0014] Further, in step S4, the pixel side length of the stacking area is obtained L , including: S4-1: Stacking area image I T Convert to grayscale image I G ; Use adaptive threshold segmentation method to segment grayscale images I G Binarize to get a binary image I B , then for the binary image I B Invert it so that the material area is white and the background area is black; S4-2: Preprocessed binary image I B Mark the connected area and get the marked image I L and the number of connected regions M ; Extract the attributes of each connected region P i , including the bounding box, area, centroid and pixel list: ; According to the area range [ A min , A max ], aspect ratio r and outlier threshold T d The white areas that meet the material characteristics are screened out based on the above conditions to obtain a set of valid area attributes. S4-3: Envelope calculation and edge length extraction; Merge all pixels in the valid area into a point set P ; Point set P Calculate the envelope and get the envelope vertex set P c : P c =convhull( P x , P y ); in, P xand P y Point Sets P of x Coordinates and y coordinate; Calculate the vertical height of the envelope H c and horizontal width W c , and use it as the pixel side length of the stacking area L and width B An estimated value of L = H c =max( P c，y )-min( P c，y ); B = W c =max( P c，x )-min( P c，x ).

[0015] Further, in step S5, the actual length of the construction material storage area is calculated. L a , including: S5-1: Get camera parameters; Get the equivalent focal length of the camera f ; Get the equivalent sensor size of the camera, including the sensor height H s and width W s ; Get the camera's image resolution, including image height H r and width W r ; S5-2: Measure the vertical distance from the camera lens to the surface of the steel pipe stacking area d ; S5-3: Based on equivalent sensor height H s and image height resolution H r , calculate the equivalent sensor pixel size P s : P s = H s / H r ; S5-4: Calculate the imaging length of the object on the sensor L m : L m = L × P s ; S5-5: Based on the camera equivalent focal length f , the distance from the camera to the stacking area d And calculate the imaging length of the object on the sensor L m , use the principle of similar triangles to estimate the actual length of the stacking area L a : ; .

[0016] Due to the adoption of the above technical scheme, the present invention has the following advantages and positive effects compared with the prior art: the present invention provides a method for monitoring and warning the amount of construction material stacking based on a multimodal model, collects images of the construction material stacking area through a camera, uses a multimodal deep learning model to accurately identify the type of construction materials, and estimates the weight of the construction materials according to the type and volume of the construction materials, and realizes real-time safety monitoring and warning of the construction material stacking by setting the warning threshold, thereby improving the intelligence level of the construction material stacking monitoring and the accuracy of the warning. In addition, the present invention also solves the problems of the lack of adaptability and flexibility in the evaluation and warning of the stacking state of the prior art, excessive reliance on preset thresholds, insufficient real-time and interactive warning information, and lack of indirect carbon emission control means by integrating multimodal models, multi-perspective data fusion, adaptive warning strategies, mobile terminal interaction and other technologies, enabling the safe production management of construction projects, providing more efficient monitoring and warning means for the safety of construction material stacking and indirect carbon emission control of construction materials at the construction site, and ensuring the smooth implementation of construction projects. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flow chart of the construction material pile load monitoring and early warning method based on the multi-modal model in the present invention; Figure 2 A schematic diagram of camera settings in the present invention; Figure 3 It is a schematic diagram of extracting the cross-section parameters of the construction material in the present invention; Figure 4 It is a schematic diagram of extracting the side length parameters of construction materials in the present invention.

[0018] The numbers in the figure are as follows: 1-Building materials; 2-First camera; 3-Second camera. DETAILED DESCRIPTION

[0019] The following is a further detailed description of the construction material stacking monitoring and early warning method based on a multimodal model provided by the present invention in conjunction with the accompanying drawings and specific embodiments. The advantages and features of the present invention will become clearer in conjunction with the following description. It should be noted that the accompanying drawings are all in a very simplified form and are not in precise proportions, and are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention.

[0020] like Figure 1 As shown, the construction material pile load monitoring and early warning method based on the multimodal model provided in this embodiment includes the following steps: S1. Deploy cameras to capture front-end cross-sectional images of construction material stacking I F and overhead image I T .

[0021] like Figure 2 As shown, a first camera 2 and a second camera 3 are respectively deployed at the front end and above the stacking area of ​​the building material 1, and the first camera 2 captures the front cross-sectional image of the stacked building material. I F The second camera 3 captures a top view of the pile of construction materials. I T ,The image data collected by the first camera and the second camera are transmitted to the ,edge server and are recognized and processed by the collection system.

[0022] Furthermore, both the first camera and the second camera adopt binocular cameras, and realize three-dimensional reconstruction of the stacking area through binocular vision technology to obtain three-dimensional point cloud data, thereby further improving the accuracy of stacking volume estimation.

[0023] S2. Section image I F Input pre-trained multimodal model M , output construction material type C And the cross-section type S .

[0024] The multimodal model is an artificial intelligence model that can process data in multiple modalities. In this embodiment, the multimodal model is trained to enable it to identify the type of construction materials in the captured images. C And the cross-section type S , and represent it as a multimodal model M .

[0025] Multimodal Model M Set the construction material type prompt word set inT , prompt word set T Include qualifiers T 1 and output word T 2. Qualifiers T 1Indicates the range of construction material types to be identified, output word T 2 Given a multimodal model M Output result format. M Front section image of a pile of construction materials I F Identify and determine the type of construction material C And the cross-section type S . Multimodal Model M Make full use of image features and combine them with prompt word sets T , identify different types of construction materials C And the cross-section type S , as shown below: ( C , S )= M ( I F , T ); in, C ∈( C 1, C 2,…, C m ), C 1, C 2,…, C m Represents construction materials m There are many types of construction materials. Common types of construction materials include steel pipes, wood, steel bars, steel components, etc. S ∈( S 1, S 2,…, S n ), S 1, S 2,…, S n Represents construction materials n There are many cross-section types, the common cross-section types include circular hollow, circular solid, rectangular hollow, rectangular solid, etc.

[0026] S3. Calculate the total cross-sectional area of ​​the construction material A .

[0027] The type of construction material identified in step S2 C With cross-section type S , determine the cross-sectional parameters of the construction material, such as outer diameter, inner diameter, side length, etc.

[0028] By taking the image of the first camera I F Perform image processing and analysis to identify the number of cross-sections of construction materials N .

[0029] According to the cross-section parameters and quantity N , calculate the total cross-sectional area of ​​the building material A .

[0030] S4. Based on overhead image I T , get the pixel side length of the stacking area L and pixel width B .

[0031] Based on the top view image taken by the second camera I T , identify the construction material stacking area and obtain the pixel side length of the stacking area L and pixel width B .

[0032] S5. Calculate the actual length of the construction material storage area L a and the actual width B a , calculate the actual area of ​​the construction material storage area A a .

[0033] According to the pixel side length of the construction material stacking area L and pixel width B , combined with the internal and external parameters of the second camera such as focal length and object distance, estimate the actual length of the construction material stacking area L a and the actual width B a , and calculate the actual area of ​​the construction material storage area A a , A a = L a × B a .

[0034] S6. Based on the type of construction material C And the cross-section type S , select the density parameter r c , combined with A , L a , A a Calculate the weight of construction materialsW And unit area weight w .

[0035] Based on the type of construction material C With cross-section type S , select the corresponding density parameter r c , combined with the total cross-sectional area of ​​the building material A , Actual length of stacking area L a , and the area A a , estimate the weight of construction materials W , and weight per unit area w ,in: W = r c × L a × A ; w = W / A a .

[0036] S7. Preset graded warning thresholds for construction material weight W max , w max ,like W or w When the corresponding warning threshold is exceeded, the system automatically triggers an early warning and notifies management personnel to take corresponding measures in a timely manner.

[0037] In a specific embodiment, in step S1, a reasonable camera acquisition frequency f is set, such as once per minute. At each acquisition moment, the camera continuously captures multiple frames of images, such as continuously acquiring 10 frames. The multiple frames of images are averaged to obtain an average image at the acquisition moment as input for subsequent identification and analysis to improve the stability and reliability of the monitoring data.

[0038] In a specific embodiment, in step S7, the warning threshold W max According to the type of construction materials C , construction site conditions and other factors are dynamically set to achieve more refined safety early warning management.

[0039] In a specific embodiment, in step S6, based on the type of building materials C , select the carbon emission factor of the corresponding material, according to the weight of the construction material WThe indirect carbon emissions of the material are calculated, and in step S7, a warning threshold for carbon emissions of construction materials is set. If the indirect carbon emissions of the material exceed the corresponding warning threshold, the system automatically triggers a warning, thereby realizing the statistics and control of carbon emissions of construction materials.

[0040] In a specific embodiment, in step S3, based on the type of construction material C With cross-section type S , determine the cross-sectional parameters of the building materials, and the image recognition method can be used to extract the cross-sectional parameters of the building materials, specifically including: S3-1. Front end cross-sectional image I F Perform preprocessing, including grayscale, median filtering and contrast enhancement, to obtain the preprocessed image I P ; S3-2. Image I P Perform Canny edge detection to obtain edge image I edge ; I edge =Canny(Ip,low T ,high T , s ); Among them, low T and high T are low threshold and high threshold respectively, s is the Gaussian filter scale.

[0041] S3-3. According to the cross-section type S , respectively, to detect and extract parameters of circular or rectangular cross-sections.

[0042] (1) When S When the cross section is circular, the Hough transform principle can be used to I edge Detect the circle and get the center coordinates ( x c , y c ) and radius r ; [ x c , y c , r =HoughCircles( I edge , r min , rmax ,△ r ,△ x ,△ y ); in, r min and r max To detect the minimum and maximum radius of a circle, △ r is the radius step length, △ x and y is the step size of the circle center coordinates. If multiple circles are detected, the most likely circle is selected as the construction material section based on prior knowledge.

[0043] Calculate the total area of ​​a circular cross section A : A = Nπr 2 ;in N is the total number of circles detected.

[0044] (2) When S When the cross section is rectangular, the edge image I edge Extract the contours from the R i}: { R i}=findContours( I edge ); For each contour C i , fit the minimum circumscribed rectangle and obtain the center coordinates of the rectangle ( x i , y i ),width w i and height h i :[ x i , y i , w i , h i ]=minAreaRect( R i ); According to the characteristics of the rectangle such as aspect ratio and area, the most likely rectangle is selected as the construction material cross section based on prior knowledge. Calculate the total area of ​​the rectangular cross section A , A = Nwh ,in Nis the total number of rectangles detected.

[0045] In a specific embodiment, in step S4, the following image recognition method can be used to calculate the pixel side length of the stacking area: L extract: S4-1: Stacking area image I T Convert to grayscale image I G .like Figure 4 As shown, the adaptive threshold segmentation method is used to segment the grayscale image I G Binarize to get a binary image I B , then for the binary image I B Invert it so that the material area is white and the background area is black.

[0046] S4-2: Preprocessed binary image I B Mark the connected area and get the marked image I L and the number of connected regions M ; Extract the attributes of each connected region P i , including the bounding box, area, centroid and pixel list: ; According to the area range [ A min , A max ], aspect ratio r and outlier threshold T d The white areas that meet the material characteristics are screened out based on the above conditions to obtain a set of valid area attributes.

[0047] S4-3: Envelope calculation and edge length extraction.

[0048] Merge all pixels in the valid area into a point set P . Point Set P Calculate the envelope and get the envelope vertex set P c : P c =convhull( P x , P y ); in, P x and P y Point Sets P of x Coordinates and y coordinate.

[0049] Calculate the vertical height of the envelope H c and horizontal width W c , and use it as the pixel side length of the stacking area L and width B The estimated value of .

[0050] L = H c =max( P c，y )-min( P c，y ); B = W c =max( P c，x )-min( P c，x ); In a specific embodiment, in step S5, the following conversion method can be used to estimate the actual length, width and area of ​​the stacking area: S5-1: Get camera parameters.

[0051] Get the equivalent focal length of the camera f , in millimeters (mm), get the equivalent sensor size of the camera, including the sensor height H s and width W s , in millimeters (mm). Get the image resolution of the camera, including the image height H r and width W r , the unit is pixels.

[0052] S5-2: Use a laser rangefinder or other distance measuring equipment to measure the vertical distance from the camera lens to the surface of the steel pipe stacking area. d , the unit is millimeter (mm).

[0053] S5-3: Based on equivalent sensor height H s and image height resolutionH r , calculate the equivalent sensor pixel size P s : P s = H s / H r ; in, P s The unit is mm / pixel.

[0054] S5-4: Use pixel size P s and the pixel side length of the stacking area extracted in step S4 L , calculate the imaging length of the object on the sensor L m : L m = L × P s ; in, L m The unit is millimeter (mm).

[0055] S5-5: Based on the camera equivalent focal length f , the distance from the camera to the stacking area d And calculate the imaging length of the object on the sensor L m , use the principle of similar triangles to estimate the actual length of the stacking area L a : ; ; in, L a The unit is millimeter (mm).

[0056] Similarly, the actual length of the stacking area can be obtained B a , and area A a .

[0057] In a specific embodiment, in step S1, the first camera and the second camera have an equivalent focal length of 12 mm, the object is photographed vertically, the equivalent sensor size is 36 mm×24 mm, and the image resolution is 3072×4096 pixels; In step S2, set the construction material type qualifierT 1 = "Analysis diagram Construction material type and section type: Steel pipe C =1. Wood C =2. Steel bars C =3. Steel components C =4; round hollow S =1, round solid S =2, Rectangular hollow S =3, Rectangular solid S =4", output word T 2="The output format of construction material type and section type is "; Through multimodal models The image of the construction material taken by the first camera Classification is performed and the output result is: C , S ) = (1,1), indicating that the output construction material type and section type are: steel pipe round hollow; In step S3-1, the median filter uses a 3×3 filter window, and the specific parameters for contrast and brightness adjustment are set as follows: Input range: Brighten darker pixels and trim the darkest 30%; Brighten lighter pixels and trim the brightest 50%; Output range: The minimum brightness of the output range is increased to 0.4, making the image brighter overall; the maximum brightness of the output range is reduced to 0.8; In step S3-2, the low threshold value is low T =0.05, high threshold T =0.3, Gaussian filter scale s =3; In step S3-3, the minimum radius of the circle is detected r min =20, maximum radius r max =40, radius step △ r =1, the step length of the circle center coordinates △ x =1 and △ y =1; the number of detected circles is 46, such as Figure 3 As shown, based on prior knowledge, the disc-shaped steel pipe is selected as the construction material section. The outer diameter of a single steel pipe is 48.3mm and the wall thickness is 3.2mm. The total area of ​​the circular section is calculated. A : A = Nπr 2 =0.02024m 2 ; In step S4-2, the parameter value of the screening condition is set to: Amin , A max ]=[10000,20000], aspect ratio r =2, outlier threshold T d =1000 pixels; filter out the white area that meets the material characteristics and obtain the valid area attribute set; In step S4-3, ; ;

[0058] In step S5-1, the equivalent focal length of the camera f =12mm, sensor height H s =24mm and width W s =36mm; Image height H r =3072 pixels and width W r =4096 pixels; S5-2: The vertical distance from the camera lens to the surface of the steel pipe stacking area d =1500mm; S5-3: P s = H s / H r =1 / 128 (mm / pixels); S5-4: L m = L × P s =2063 / 128=16.117mm; S5-5: ; Likewise, ; ;

[0059] A a =2.614m 2 ; In step S6, the type of construction material identified in step S2 is C With cross-section type S , select the corresponding density parameter r c=7850kg / m 3 , and combined with the total cross-sectional area of ​​the building material obtained by S5 A And the actual length of the stacking area obtained by S5 L a , estimate the weight of construction materials W , and weight per unit area w , W = r c × L a × A =7850×2.01465×0.02024=320.09319kg; w = W / A a =122.5kg / m 2 ; In step S7, the graded warning threshold W max =2000kg, w max =500kg / m 2 , W , w If the corresponding warning threshold is not exceeded, no action is required.

[0060] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0061] The above-mentioned embodiments only express several implementation modes of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the attached claims.

Claims

1. A construction material pile load monitoring and early warning method based on a multimodal model, characterized in that: The steps include: S1. Deploy cameras to capture front-end cross-sectional images of construction material stacking I F and overhead image I T ; S2. Section image I F Input pre-trained multimodal model M , output construction material type C And the cross-section type S ; S3. Calculate the total cross-sectional area of ​​the construction material A ; S4. Based on overhead image I T , get the pixel side length of the stacking area L and pixel width B ; S5. Calculate the actual length of the construction material storage area L a and the actual width B a , calculate the actual area of ​​the construction material storage area A a ; S6. Based on the type of construction material C And the cross-section type S , select the density parameter ρ c , combined with A , L a , A a Calculate the weight of construction materials W And unit area weight w ; S7. Preset graded warning thresholds for construction material weight W max , w max ,like W or w When the corresponding warning threshold is exceeded, the system automatically triggers a warning.

2. The method for monitoring and early warning of construction material loading based on a multimodal model according to claim 1, characterized in that: Multimodal Model M Set the construction material type prompt word set in T , prompt word set T Include qualifiers T 1 and output word T 2. Qualifiers T 1Indicates the range of construction material types to be identified, output word T 2 Given a multimodal model M Output result format; multimodal model M Front section image based on the stacking of construction materials I F , determine the type of construction material C And the cross-section type S , the formula is: （ C , S ）= M ( I F , T )； in, C ∈( C 1, C 2,…, C m ), C 1, C 2,…, C m Represents construction materials m Types, S ∈( S 1, S 2,…, S n ), S 1, S 2,…, S n Represents construction materials n Type of cross section.

3. The method for monitoring and early warning of construction material loading based on a multimodal model as claimed in claim 1, characterized in that: The camera acquisition frequency f is preset. At each acquisition moment, the camera continuously captures multiple frames of images, and averages the multiple frames of images to obtain an average image at the acquisition moment. The average image is used for subsequent recognition and analysis.

4. The method for monitoring and early warning of construction material loading based on a multimodal model according to claim 1, characterized in that: A first camera and a second camera are respectively deployed at the front and top of the construction material stacking area, and the first camera captures the front cross-sectional image of the construction material stacking area. I F The second camera captures a bird's-eye view of the pile of construction materials I T ; The first camera and the second camera both use binocular cameras, which realize three-dimensional reconstruction of the stacking area through binocular vision technology to obtain three-dimensional point cloud data.

5. The method for monitoring and early warning of construction material loading based on a multimodal model according to claim 1, characterized in that: Warning threshold W max , w max , according to the type of construction material C , dynamic setting of construction site conditions.

6. The method for monitoring and early warning of construction material loading based on a multimodal model according to claim 1, characterized in that: In step S6, based on the type of building materials C , select the carbon emission factor of the corresponding material, according to the weight of the construction material W The indirect carbon emissions of the material are calculated, and in step S7, a warning threshold for carbon emissions of the construction material is set. If the indirect carbon emissions of the material exceed the corresponding warning threshold, the system automatically triggers a warning.

7. The method for monitoring and early warning of construction material loading based on a multimodal model according to claim 1, characterized in that: Step S3 calculates the total cross-sectional area of ​​the building material A , specifically including: Based on the type of construction material C With cross-section type S , determine the cross-sectional parameters of the construction material; Through the image I F Perform image processing and analysis to identify the number of cross-sections of construction materials N ; According to the cross-section parameters and quantity N , calculate the total cross-sectional area of ​​the building material A .

8. The method for monitoring and early warning of construction material loading based on a multimodal model as claimed in claim 7, characterized in that: When determining the cross-sectional parameters of the construction material, an image recognition method is used to extract the cross-sectional parameters of the construction material, specifically including: S3-1. Front end cross-sectional image I F Perform preprocessing, including grayscale, median filtering and contrast enhancement, to obtain the preprocessed image I P ; S3-2. Image I P Perform Canny edge detection to obtain edge image I edge ; I edge =Canny(Ip,low T ,high T , σ ); Among them, low T and high T are low threshold and high threshold respectively, σ is the Gaussian filter scale; S3-3. According to the cross-section type S , respectively, to detect and extract parameters of circular or rectangular cross sections; (1) When S When the cross section is circular, the Hough transform principle is used to transform the edge image. I edge Detect the circle and get the center coordinates ( x c , y c ) and radius r ; [ x c , y c , r ]=HoughCircles( I edge , r min , r max ,△ r ,△ x ,△ y ); in, r min and r max To detect the minimum and maximum radius of a circle, △ r is the radius step length, △ x and y is the step size of the circle center coordinates; (2) When S When the cross section is rectangular, the edge image I edge Extract the contours from the R i }: { R i }=findContours( I edge ); For each contour C i , fit the minimum circumscribed rectangle and obtain the center coordinates of the rectangle ( x i , y i ),width w i and height h i :[ x i , y i , w i , h i ]=minAreaRect( R i ).

9. The method for monitoring and early warning of construction material loading based on a multimodal model according to claim 1, characterized in that: In step S4, the pixel side length of the stacking area is obtained L , specifically including: S4-1: Stacking area image I T Convert to grayscale image I G ; Use adaptive threshold segmentation method to segment grayscale images I G Binarize to get a binary image I B , then for the binary image I B Invert it so that the material area is white and the background area is black; S4-2: Preprocessed binary image I B Mark the connected area and get the marked image I L and the number of connected regions M ; Extract the attributes of each connected region P i , including the bounding box, area, centroid and pixel list: ； According to the area range [ A min , A max ], aspect ratio r and outlier threshold T d The white areas that meet the material characteristics are screened out based on the above conditions to obtain a set of valid area attributes. S4-3: Envelope calculation and edge length extraction; Merge all pixels in the valid area into a point set P ; Point set P Calculate the envelope and get the envelope vertex set P c : P c =convhull( P x , P y ); in, P x and P y Point Sets P of x Coordinates and y coordinate; Calculate the vertical height of the envelope H c and horizontal width W c , and use it as the pixel side length of the stacking area L and width B An estimated value of L = H c =max( P c，y )-my( P c，y ); B = W c =max( P c，x )-my( P c，x )。 10. The method for monitoring and early warning of construction material loading based on a multimodal model according to claim 1, characterized in that: In step S5, the actual length of the construction material storage area is calculated L a , specifically including: S5-1: Get camera parameters; Get the equivalent focal length of the camera f ; Get the equivalent sensor size of the camera, including the sensor height H s and width W s ; Get the camera's image resolution, including image height H r and width W r ; S5-2: Measure the vertical distance from the camera lens to the surface of the steel pipe stacking area d ; S5-3: Based on equivalent sensor height H s and image height resolution H r , calculate the equivalent sensor pixel size P s : P s = H s / H r ; S5-4: Calculate the imaging length of the object on the sensor L m : L m = L × P s ; S5-5: Based on the camera equivalent focal length f , the distance from the camera to the stacking area d And calculate the imaging length of the object on the sensor L m , use the principle of similar triangles to estimate the actual length of the stacking area L a : ； 。

Images