A method and system for monitoring the safety of park construction materials with real-time processing of video streams

Abstract

The present invention discloses a method and system for monitoring the safety of park construction materials with real-time processing of video streams, including the following steps: first, obtaining macroscopic video image information of the construction material arrival acceptance area in real time, and inputting it into a trained first convolutional neural network model for rough identification; second, judging whether to enter the fine identification process, and if entering, collecting microscopic video image information of the construction materials arriving through a mobile terminal device and preprocessing; third, inputting feature vectors into a trained dual-stream deep learning network construction material recognition model for fine identification of the category and quantity of the construction materials; finally, displaying the category and quantity of the currently identified construction materials. This application adopts the first convolutional neural network model for rough identification, and adopts the dual-stream deep learning network construction material recognition model for fine identification of the category and quantity of the construction materials, which greatly improves the accuracy and efficiency of construction material identification and greatly increases the user experience.

Description

Technical Field

[0001] The present invention relates to the technical field of video stream image processing, and in particular to a method and system for monitoring the safety of park construction materials by real-time processing of video streams. Background Art

[0002] With the rapid development of industrialization and urbanization, the construction of large-scale construction projects and parks has become the norm. These projects usually involve the management of a large amount of construction materials and equipment. Ensuring their safety and efficiency is crucial to maintaining project progress and quality. Therefore, the safety monitoring of park construction materials has become particularly important, especially during the arrival, storage and use of materials. Traditional park construction material monitoring mainly relies on manual monitoring, such as regular inspections by security personnel or post-review of surveillance camera recordings. This method is not only inefficient, but also prone to human omissions, especially when faced with large-scale materials and rapidly changing construction sites. In addition, manual monitoring is difficult to achieve 24-hour all-weather coverage, and video playback cannot respond to potential safety issues in real time.

[0003] With the development of computer vision and machine learning technology, video surveillance systems have been greatly enhanced, making it possible to process video data in real time in an automated manner. However, existing systems usually rely only on simple motion detection or simple object recognition in specific scenarios, and lack the ability to identify diverse and accurately classified materials in complex scenarios. For example, in the monitoring of construction materials, existing technologies have difficulty distinguishing different types of materials and calculating the amount of materials, which is extremely critical in practical applications. In addition, existing systems usually fail to effectively utilize the combination of macro and micro image data, which limits their application effect in complex environments.

[0004] Although the existing park construction material recognition uses artificial intelligence methods for recognition statistics, the recognition efficiency and recognition accuracy are difficult to meet the existing needs. At present, in the recognition process of park construction materials or express delivery, only a single video image recognition is used without considering the influence of factors such as the large number of construction materials. In addition, the existing recognition algorithm uses machine learning and does not consider the effective recognition method for objects with low recognition accuracy, resulting in the dynamic adjustment in the calculation process being very insensitive or inaccurate. For example, the historical recognition of an item is low, and no more flexible video information is used for further judgment in order to be more in-depth, which leads to inaccurate quantity and category of construction materials provided; and there is no step in the existing technology to adjust the precise recognition according to the rough recognition results, which not only affects the recognition efficiency, but also causes the accuracy of building material recognition to develop in a worse direction; in view of the limitations of the existing park construction material recognition, a new solution is urgently needed for a high-accuracy and high-efficiency automated processing method to improve the processing efficiency and accuracy, so as to improve user satisfaction. Summary of the invention

[0005] In view of the above problems mentioned in the prior art, the present invention provides a method for monitoring the safety of construction materials in a park with real-time processing of video streams. The method first obtains macroscopic video image information of the construction material arrival acceptance area in real time, and inputs the macroscopic video image information into a trained first convolutional neural network model to roughly identify the initial type and quantity of the construction materials; secondly, it is determined whether to enter step S4 for a fine identification process. If entered, the microscopic video image information of the construction materials arriving is collected through a mobile terminal device and preprocessed; then, the feature vector f1 and the feature vector f2 are input into a trained dual-stream deep learning network construction material recognition model to finely identify the category and quantity of the construction materials; finally, the category and quantity of the currently identified construction materials are displayed. This application adopts the first convolutional neural network model for rough identification, and adopts the dual-stream deep learning network construction material recognition model for fine identification of the category and quantity of the construction materials, which greatly improves the accuracy and efficiency of construction material identification and greatly increases the user experience.

[0006] The present application provides a method for monitoring the safety of park construction materials by real-time processing of video streams, comprising the steps of:

[0007] S1: CCD camera acquires macroscopic video image information of the construction material arrival acceptance area in real time;

[0008] S2: Input the macro video image information into the trained first convolutional neural network model for rough recognition, and roughly identify the initial type and quantity of construction materials. The first convolutional neural network model uses the Swish activation function. Swish(x1) is expressed as follows:

[0009]

[0010] Among them, x1 is the input of the Swish activation function, e is the base of the natural logarithm, and α is a trainable parameter;

[0011] S3: The number of construction materials N that pass through per unit time t When the preset threshold is exceeded or there is a construction material with a historical recognition error rate greater than the set threshold in the current initial type of construction materials, the number of CPUs or GPUs executing the video image processing task is increased and the process of fine recognition is directly entered into step S4, otherwise, the process is entered into step S7;

[0012] S4: Collecting microscopic video image information of the construction materials arriving through a mobile terminal device;

[0013] S5: preprocessing the macroscopic video image information and the microscopic video image information, and extracting the feature vector f1 of the preprocessed macroscopic video image information and the feature vector f2 of the microscopic video image information respectively;

[0014] S6: Input the feature vector f1 and the feature vector f2 into the trained dual-stream deep learning network construction material recognition model for fine recognition, and identify the category and quantity of construction materials in real time. One stream of the dual-stream deep learning network construction material recognition model processes macroscopic video image information, and the other stream processes microscopic video image information; the dual-stream deep learning network construction material recognition model uses the Mish activation function:

[0015]

[0016] Among them, x2 is the input of the Mish activation function, N t is the number of construction materials passed per unit time, and β is the scaling factor used to adjust the impact of the number of construction materials on the activation function;

[0017] S7: Display the currently identified types and quantities of construction materials and the cumulative types and quantities of construction materials received through the arrival acceptance display screen or application.

[0018] Preferably, the macroscopic video image information and the microscopic video image information are preprocessed, and the preprocessing includes using Gaussian filtering to perform video image denoising and using histogram equalization to perform video image enhancement.

[0019] Preferably, the first convolutional neural network model includes: an input layer for receiving macroscopic video image information; the first convolutional layer contains multiple 3x3 convolution kernels and uses the Swish activation function, followed by a 2x2 maximum pooling layer; the second convolutional layer increases the number of convolution kernels and repeatedly uses the Swish activation function and maximum pooling; a flattening layer is used to convert the multi-dimensional data output by the convolutional layer into one-dimensional data; a fully connected layer uses the Swish activation function to integrate features and perform classification judgments; an output layer uses the Softmax activation function to output the classification probability and output quantity of construction materials.

[0020] Preferably, the extracting of the feature vector f1 of the preprocessed macroscopic video image information and the feature vector f2 of the microscopic video image information respectively includes using the scale-invariant feature transform (SIFT) method to extract key points and their descriptors, including corner points, edges, and brightness spots, from the preprocessed macroscopic video image information and the microscopic video image information; and using the histogram of oriented gradients (HOG) method to extract the gradient direction and intensity in the preprocessed macroscopic video image and the microscopic video image.

[0021] Preferably, the dual-stream deep learning network construction material recognition model includes: two independent input layers, respectively used to receive macroscopic video image information and microscopic video image information; 4 convolution layers are configured for the macroscopic video image information data stream and the microscopic video image information data stream, respectively, the first layer uses 32 3x3 convolution kernels, the second layer uses 64 3x3 convolution kernels, the third layer uses 128 3x3 convolution kernels, and the fourth layer uses 256 3x3 convolution kernels; each convolution layer is equipped with a Mish activation function; each convolution layer is followed by a 2x2 maximum pooling layer for reducing the feature dimension; the features of the macroscopic video image information data stream and the microscopic video image information data stream are merged by splicing after the fourth convolution layer; after the fusion features, three fully connected layers are connected, each layer contains 1024, 512 and 256 neurons respectively; a dropout ratio of 0.5 is used after each fully connected layer to prevent overfitting; the last fully connected layer is followed by a Softmax output layer for classification and a linear output unit for quantity estimation.

[0022] The present invention also provides a park construction material safety monitoring system for real-time processing of video streams, comprising:

[0023] Macro video image information acquisition module: CCD camera acquires macro video image information of the construction material arrival acceptance area in real time;

[0024] The first convolutional neural network model recognition module inputs the macro video image information into the trained first convolutional neural network model for rough recognition, and roughly recognizes the initial type and quantity of construction materials. The first convolutional neural network model uses the Swish activation function. Swish(x1) is expressed as follows:

[0025]

[0026] Among them, x1 is the input of the Swish activation function, e is the base of the natural logarithm, and α is a trainable parameter;

[0027] Fine identification process judgment module, when the number of construction materials N passing through per unit time t When the preset threshold is exceeded or there is a construction material with a historical recognition error rate greater than the set threshold in the current initial type of construction materials, the number of CPUs or GPUs executing the video image processing task is increased and the process of fine recognition is directly entered into step S4, otherwise, the process is entered into step S7;

[0028] A microscopic video image information acquisition module collects microscopic video image information of the incoming construction materials through a mobile terminal device;

[0029] A preprocessing module preprocesses the macroscopic video image information and the microscopic video image information, and extracts a feature vector f1 of the preprocessed macroscopic video image information and a feature vector f2 of the microscopic video image information;

[0030] The dual-stream deep learning network construction material recognition model recognition module inputs the feature vector f1 and the feature vector f2 into the trained dual-stream deep learning network construction material recognition model for fine recognition, and recognizes the category and quantity of construction materials in real time. One stream of the dual-stream deep learning network construction material recognition model processes macroscopic video image information, and the other stream processes microscopic video image information; the dual-stream deep learning network construction material recognition model adopts the Mish activation function:

[0031]

[0032] Among them, x2 is the input of the Mish activation function, N t is the number of construction materials passed per unit time, and β is the scaling factor used to adjust the impact of the number of construction materials on the activation function;

[0033] The display module displays the types and quantities of construction materials currently identified and the types and quantities of construction materials received cumulatively through an arrival acceptance display screen or an application.

[0034] Preferably, the macroscopic video image information and the microscopic video image information are preprocessed, and the preprocessing includes using Gaussian filtering to perform video image denoising and using histogram equalization to perform video image enhancement.

[0035] Preferably, the first convolutional neural network model includes: an input layer for receiving macroscopic video image information; the first convolutional layer contains multiple 3x3 convolution kernels and uses the Swish activation function, followed by a 2x2 maximum pooling layer; the second convolutional layer increases the number of convolution kernels and repeatedly uses the Swish activation function and maximum pooling; a flattening layer is used to convert the multi-dimensional data output by the convolutional layer into one-dimensional data; a fully connected layer uses the Swish activation function to integrate features and perform classification judgments; an output layer uses the Softmax activation function to output the classification probability and output quantity of construction materials.

[0036] Preferably, the extracting of the feature vector f1 of the preprocessed macroscopic video image information and the feature vector f2 of the microscopic video image information respectively includes using the scale-invariant feature transform (SIFT) method to extract key points and their descriptors, including corner points, edges, and brightness spots, from the preprocessed macroscopic video image information and the microscopic video image information; and using the histogram of oriented gradients (HOG) method to extract the gradient direction and intensity in the preprocessed macroscopic video image and the microscopic video image.

[0037] Preferably, the dual-stream deep learning network construction material recognition model includes: two independent input layers, respectively used to receive macroscopic video image information and microscopic video image information; 4 convolution layers are configured for the macroscopic video image information data stream and the microscopic video image information data stream, respectively, the first layer uses 32 3x3 convolution kernels, the second layer uses 64 3x3 convolution kernels, the third layer uses 128 3x3 convolution kernels, and the fourth layer uses 256 3x3 convolution kernels; each convolution layer is equipped with a Mish activation function; each convolution layer is followed by a 2x2 maximum pooling layer for reducing the feature dimension; the features of the macroscopic video image information data stream and the microscopic video image information data stream are merged by splicing after the fourth convolution layer; after the fusion features, three fully connected layers are connected, each layer contains 1024, 512 and 256 neurons respectively; a dropout ratio of 0.5 is used after each fully connected layer to prevent overfitting; the last fully connected layer is followed by a Softmax output layer for classification and a linear output unit for quantity estimation.

[0038] The present invention provides a method and system for monitoring the safety of park construction materials by real-time processing of video streams, which can achieve the following beneficial technical effects:

[0039] 1. The method of the present invention first obtains the macroscopic video image information of the construction material arrival acceptance area in real time, and inputs the macroscopic video image information into the trained first convolutional neural network model to roughly identify the initial type and quantity of the construction materials; secondly, it is determined whether to enter step S4 for fine identification process. If it is entered, the microscopic video image information of the construction materials arriving is collected and preprocessed through the mobile terminal device; thirdly, a query for the knowledge graph is constructed according to the semantic role annotation to obtain the information retrieved from the knowledge graph; then, the feature vector f1 and the feature vector f2 are input into the trained dual-stream deep learning network construction material recognition model to finely identify the category and quantity of the construction materials; finally, the category and quantity of the currently identified construction materials are displayed. This application adopts the first convolutional neural network model for rough identification, and adopts the dual-stream deep learning network construction material recognition model for fine identification of the category and quantity of the construction materials, which greatly improves the accuracy and efficiency of construction material identification and greatly increases the user experience.

[0040] 2. The present invention adopts a dual-stream deep learning network to construct a material recognition model using the Mish activation function:

[0041]

[0042] Among them, x2 is the input of the Mish activation function, N tis the number of construction materials passing through per unit time, and β is the scaling factor used to adjust the impact of the number of construction materials on the activation function. The Mish activation function makes full use of the rough recognition information by considering the number of construction materials passing through per unit time, and dynamically adjusts the calculation process according to the input number of construction materials, which greatly improves the accuracy and efficiency of construction material recognition.

[0043] 3. The present invention uses rough identification and sets a fine identification process judgment module. When the number of construction materials N passing through per unit time t When the preset threshold is exceeded or there is a construction material in the current initial type of construction material whose historical recognition error rate is greater than the set threshold, the number of CPUs or GPUs executing the video image processing task is increased and the process directly enters step S4 for fine recognition. Otherwise, it enters step S7, realizing the combination of rough recognition and fine recognition, which greatly improves the recognition efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 It is a schematic diagram of the steps of a method for monitoring the safety of park construction materials by real-time processing of video streams according to the present invention;

[0046] Figure 2 It is a schematic diagram of a park construction material safety monitoring system for real-time processing of video streams of the present invention. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] Embodiment 1:

[0049] In view of the above problems mentioned in the prior art, in order to solve the above technical problems, as shown in the attached Figure 1 As shown: This application provides a method for monitoring the safety of park construction materials by real-time processing of video streams, including the steps of:

[0050] S1: CCD camera acquires macroscopic video image information of the construction material arrival acceptance area in real time; in one embodiment, high-resolution CCD cameras are selected because they can provide high-quality images and are suitable for capturing complex scenes under macroscopic perspectives. CCD cameras have excellent light adaptability and color accuracy, and can maintain image clarity and details under different lighting conditions. The camera should be installed in a position covering the entire arrival acceptance area, usually with a high point of view to reduce line of sight obstruction and increase the monitoring range. Adjust the angle and focal length of the camera to ensure that the arrival area can be covered as widely as possible and the materials in the image can be clearly identified. The CCD camera needs to be set to real-time video mode, that is, continuously capture and transmit video streams for real-time image processing and analysis, and select appropriate resolution and frame rate to optimize the balance between image quality and transmission efficiency. Usually high resolution and moderate frame rate (e.g., 30 frames per second) are sufficient for effective monitoring. In one embodiment, in a material storage area of ​​a large construction project, multiple high-resolution CCD cameras are installed to monitor the arrival and storage of materials. Each camera is fixed at the top four corners of the warehouse, looking down to obtain a view with as little obstruction as possible. Industrial-grade CCD cameras with a resolution of 20 megapixels were selected to provide clear video images even under low light conditions. All cameras are connected to the central monitoring system via high-speed Ethernet, and real-time video streams are transmitted through a dedicated network to ensure the real-time and security of image data. In the night monitoring room, the input of all cameras is configured using video management software (VMS). The software supports real-time display, recording and playback of images, and sets motion detection alarms. Once new material arrival activities are detected, the system will automatically mark and notify warehouse managers.

[0051] S2: Input the macro video image information into the trained first convolutional neural network model for rough recognition, and roughly identify the initial type and quantity of construction materials. The first convolutional neural network model uses the Swish activation function. Swish(x1) is expressed as follows:

[0052]

[0053] Among them, x1 is the input of the Swish activation function, e is the base of the natural logarithm, and α is a trainable parameter;

[0054] In one embodiment, the input layer receives a preprocessed macro video image, and multiple convolutional layers use the Swish activation function, with each layer using a different number of convolution kernels to extract features of the image. A pooling layer (such as maximum pooling) is used after the convolutional layer to reduce the number of parameters and the amount of computation while extracting the main features. After multiple layers of convolution and pooling, a fully connected layer is used to integrate features and perform final classification and quantity estimation. The output layer may use a Softmax function for classification, and a linear unit for quantity estimation. A construction project requires real-time monitoring of different types of building materials delivered to the site, such as cement, sand, and steel bars. The project uses a convolutional neural network model based on the Swish activation function to automatically identify and count these materials. CCD cameras are installed in the material unloading area to capture macro images of incoming goods in real time. The images are transmitted to the processing center at a rate of 30 frames per second. The images are first normalized and resized to meet the network input specifications. The image data is input into the convolutional neural network. The convolutional layer in the network uses the Swish activation function to extract key features. By learning the labeled training data, the network has been able to identify different types of building materials and their quantities. The network output includes the type and quantity of building materials. This information is displayed on the monitoring screen in real time and is used for subsequent logistics and storage management.

[0055] In neural networks using the Swish activation function, α is a key trainable parameter that affects the shape of the activation function and the behavior of the network. Here is how to obtain and train α Before starting training, α needs to be initialized. Usually, it is a reasonable choice to initialize α to 1, because this value provides a neutral starting point and makes the Swish function close to the behavior of the ReLU function in the initial stage. However, according to different application scenarios and network designs, the initial value of α can also be selected based on experience. As a parameter in the network, α participates in the training process together with other parameters such as weights and biases. It will be directly integrated into the calculation of the activation function and affect the output of each layer of the network. During network training, α is updated through the backpropagation algorithm. After the network performs forward propagation for each input to generate an output, the difference between the output and the actual label, i.e., the loss, will be calculated. During the backpropagation process, the gradient of the loss relative to α will be calculated, and how α needs to be adjusted to reduce future losses. Use gradient descent or other optimization algorithms to adjust the value of α. This means that according to its gradient (i.e., the direction and magnitude of the impact on the loss), α is gradually adjusted to make the network's predictions more accurate. The learning rate determines the step size of each update to adjust α. Throughout the training process, α is continuously adjusted, and its value may be fine-tuned at each training iteration. This process gradually stabilizes as the network learns different training data. During the training process, it is very important to monitor the impact of α on the performance of the model. If you observe that adjusting α causes performance degradation or training instability, you may need to adjust the learning rate or re-examine the initial setting of α. In this way, α can be adaptively adjusted during training, helping the model better learn and adapt to data characteristics. This dynamic adjustment mechanism makes the Swish activation function more flexible and efficient when dealing with different types of tasks.

[0056] S3: The number of construction materials N that pass through per unit time t When the preset threshold is exceeded or there is a construction material with a historical recognition error rate greater than the set threshold in the current initial type of construction materials, the number of CPUs or GPUs executing the video image processing task is increased and the process of fine recognition is directly entered into step S4, otherwise, the process is entered into step S7;

[0057] In one embodiment, the number of construction materials passing through the monitoring area exceeds a preset threshold: this means that within a certain period of time, such as one minute or one hour, the number of construction materials entering the monitoring area exceeds the amount that the system usually handles. Exceeding this threshold may indicate that a large amount of materials have arrived, which requires the system to process faster to maintain the efficiency of real-time monitoring. There are construction materials with a historical recognition error rate greater than the set threshold in the current initial type of construction materials: this means that the historical error rate of the system when identifying certain types of construction materials exceeds the acceptable threshold. In this case, the system needs more sophisticated analysis to reduce false recognition and ensure the accuracy of monitoring data. When any of these conditions is met, the system will automatically increase the required computing resources, such as increasing the use of CPU or GPU, to handle higher data traffic or perform more complex data analysis. If these conditions are not triggered, the system continues to operate normally without additional resource allocation. In one embodiment, a monitoring system for a large construction site, which is responsible for monitoring the arrival of various construction materials, such as cement, steel and wood, is equipped with multiple CCD cameras on the site to transmit real-time video of the material unloading area. The threshold of the number of materials per unit time set by the system is 100 items per minute. The threshold for the historical recognition error rate is set at 10%. Most of the time, the number of materials arriving per minute is around 50, and the system uses conventional computing resources for processing. At one point, due to the simultaneous arrival of multiple suppliers, the monitoring system detected 150 materials arriving in one minute. This number exceeded the threshold, and the system automatically increased GPU resources and started a more efficient image processing algorithm to maintain real-time monitoring without delay. The system found that the error rate reached 12% when identifying a specific batch of wood. Because it exceeded the set threshold, the system automatically switched to a more refined recognition mode, increasing computing resources to improve recognition accuracy.

[0058] S4: Collect microscopic video image information of incoming construction materials through mobile terminal devices; In construction projects, macroscopic images can provide an overview of the arrival of materials, but microscopic images can provide more detailed information, such as the texture, markings, and minor damage of the materials. These details are crucial to ensuring the quality and suitability of the materials. Mobile terminal devices (such as smartphones or tablets) equipped with high-resolution cameras can be used to obtain these microscopic images. Mobile terminal devices are particularly suitable for this task due to their portability and functional diversity (high-resolution video, instant data transmission, etc.). These devices can be operated by on-site workers to directly approach the materials for shooting, ensuring that the pictures are clear and focused on the required details. In one embodiment, a large construction project needs to strictly monitor the quality of steel bars, because steel bars are key materials for structural safety. A fixed camera system for macroscopic monitoring is set up on the construction site, but for microscopic inspections such as rust and cracks of steel bars, more detailed observations are required. The fixed camera continuously monitors the unloading and storage of steel bars to ensure that the quantity and storage method meet the requirements. When the steel bars are unloaded to the designated area, a field worker uses a smartphone equipped with a high-resolution camera to approach the steel bars for shooting. They pay special attention to the rust on the surface of the steel bars and any possible cracks. The microscopic images taken by the staff are instantly uploaded to the central monitoring system. The system uses advanced image processing algorithms to analyze the images and identify defects such as rust and cracks. If the system detects a serious problem, it will immediately send an alert to the project manager and quality control department, who can take further inspections or replace materials.

[0059] S5: Preprocess the macroscopic video image information and the microscopic video image information, and extract the feature vector f1 of the preprocessed macroscopic video image information and the feature vector f2 of the microscopic video image information respectively; in one embodiment, the pixel values ​​of the image data are normalized to the same scale (usually between 0 and 1) to eliminate the influence of illumination changes and unify the processing standards, and apply image denoising technology (such as Gaussian filtering) to reduce random noise in the image capture process, which can help improve the accuracy of subsequent image analysis and feature extraction, and enhance the image, such as using histogram equalization to improve the contrast of the image, so that important features such as the texture and edge of the material are more prominent. For example, the feature vector is extracted from the processed image using techniques such as histogram of oriented gradients (HOG) or scale-invariant feature transform (SIFT). These features include but are not limited to image characteristics such as edges, corners, and textures, which are important bases for classification and recognition. Use pre-trained convolutional neural networks (such as VGG or ResNet) to automatically extract high-level features from the image. This method can directly learn complex feature representations from raw pixels, which is usually more effective than traditional methods. In a large construction project, the quality of steel bars and concrete blocks needs to be monitored. A high-resolution CCD camera is used to capture the incoming steel bars and concrete blocks from two different angles (macro and micro). The macro camera captures the overall arrival, while the micro camera focuses on the details of the material surface. The background noise is removed by Gaussian filtering, and then histogram equalization is applied to enhance the image contrast to make the overall layout clearer. The micro image is normalized, the lighting effect is adjusted, and the cracks and textures on the material surface are highlighted by sharpening enhancement. HOG is used from the macro image to extract the feature vector of the global layout, which helps to identify the distribution and stacking of the material. SIFT is used to extract local features from the micro image, focusing on capturing cracks, holes and other potential defects. These feature vectors are input into a two-stream deep learning network for further analysis to identify the material type and assess the quality.

[0060] S6: Input the feature vector f1 and the feature vector f2 into the trained dual-stream deep learning network construction material recognition model for fine recognition, and identify the category and quantity of construction materials in real time. One stream of the dual-stream deep learning network construction material recognition model processes macroscopic video image information, and the other stream processes microscopic video image information; the dual-stream deep learning network construction material recognition model uses the Mish activation function:

[0061]

[0062] Among them, x2 is the input of the Mish activation function, N t is the number of construction materials passed per unit time, and β is the scaling factor used to adjust the impact of the number of construction materials on the activation function;

[0063] In one embodiment, the two-stream deep learning network contains two independent data processing streams, one stream processes macroscopic video image information and the other processes microscopic video image information. This design allows the network to consider both the overall layout and detailed features of the material, improving the accuracy and reliability of recognition. The network fuses the features of the two streams in the deep structure, which can be achieved by splicing, weighted summation or other fusion strategies to reasonably utilize the complementary advantages of macroscopic and microscopic information. The network uses the Mish activation function, which is a smooth and non-monotonic activation function suitable for enhancing the learning ability and expressiveness of the model. In this case, the shape of the Mish activation function is adjusted by a trainable scaling factor that is related to the amount of construction materials passing through per unit time, so that the response of the activation function is dynamically adjusted with the material flow. A system for monitoring and managing materials at a large construction site, which specifically needs to identify different types of construction materials (such as cement bags, bricks, steel bars, etc.) and estimate their quantity. A fixed CCD camera is used to collect macro images to capture the overall arrival of materials. A handheld device is used to collect micro images of each material, such as detailed images of cement bags, to detect potential damage or moisture issues. The macro and micro images are preprocessed, including normalization, denoising, and contrast enhancement. Feature vectors are extracted from the processed images using advanced image processing algorithms such as HOG and SIFT. Two-stream network processing The extracted feature vectors are input into two different streams of the two-stream network respectively, and these features are fused in the deep structure of the network. The fused features are processed using the Mish activation function. Here, the scaling factor of the Mish function is dynamically adjusted to reflect the current material flow, thereby optimizing the responsiveness of the network. The network outputs the classification results and quantity estimates of the materials, which are displayed in real time at the monitoring center for logistics planning and instant inventory management.

[0064] S7: Display the currently identified construction material categories and quantities and the cumulative construction material categories and quantities received through the arrival acceptance display screen or application. In one embodiment, the arrival acceptance display screen is usually a large screen located in the material receiving area or control center, which displays the arrival of materials in real time, including the type and quantity of each material. This display method makes it easy for on-site staff to quickly view and check the arrival materials. A specially designed application can run on a variety of devices (such as tablets, smartphones, and computers) to provide more detailed data analysis and visualization. The application not only displays current data, but also provides historical data comparison, trend analysis, and early warning notifications. The display system needs to be able to update in real time to reflect the latest material reception situation, and should support user interaction, such as zooming in to view detailed information on a certain type of material, or quickly searching historical data. The system should be able to integrate data from different cameras and sensors to provide a comprehensive monitoring view. In one embodiment, a large commercial construction project needs to accurately track the arrival of different construction materials, including cement, bricks, and steel bars. A large LED display screen is installed in the main material arrival area to display the arrival quantity and type of various materials in real time. The installation location is selected in a central location that is easy for staff to view to ensure that the information can be received quickly. A customized application is developed to run on the project manager's smartphone or tablet. The application includes functions such as user login interface, real-time data display, historical data access, and notification settings. When a new batch of materials passes through the entrance, the system automatically updates the type and quantity of the materials through a dual-stream deep learning network. The data is instantly transmitted to the display screen and application. Site managers and material managers can see the latest data and adjust the daily operations of the site accordingly. The application sets an automatic notification function. When the arrival quantity of a certain material falls below a preset threshold, the procurement department is immediately notified. Real-time data is compared with budget and plan data to help project managers make faster decisions.

[0065] Preferably, the macroscopic video image information and the microscopic video image information are preprocessed, and the preprocessing includes using Gaussian filtering to perform video image denoising and using histogram equalization to perform video image enhancement. In one embodiment, Gaussian filtering is used to perform video image denoising. Gaussian filtering is a commonly used image smoothing technology that improves image quality by reducing random noise in the image. It uses a normal distribution (Gaussian) as a filter kernel to perform weighted averaging on each pixel in the image, where the center pixel obtains the largest weight. In video monitoring of construction materials, images are often affected by a variety of noises, such as dust, raindrops, or electronic noise from the camera itself. Applying Gaussian filtering can effectively reduce these noises and improve the usability of the image. Histogram equalization is used to perform video image enhancement. Histogram equalization is a technology for improving the global contrast of an image by adjusting the histogram of the image to achieve a more uniform brightness distribution. This technology is particularly suitable for images with very dark backgrounds and foregrounds or very bright images. In the video monitoring of construction materials, the contrast of the video image may not be ideal due to the changes in lighting conditions at different time periods. The use of histogram equalization can make the details of the image clearer and help to better identify the type and status of the material. In a large construction project, the management team needs to monitor the cement bags stored in the outdoor warehouse to ensure that they are not damaged by moisture or other damage. High-definition CCD cameras are installed in the warehouse to continuously monitor the storage status of the cement bags. The real-time video stream is first processed by Gaussian filtering to remove visual noise caused by dust or other tiny particles caused by wind. Then, histogram equalization is applied to improve the image contrast problem caused by changes in sunlight, especially at sunrise and sunset. The pre-processed video is used to automatically detect the integrity of the cement bags, such as whether there are signs of damage or moisture. The video analysis results are fed back to the project management system in real time.

[0066] Preferably, the first convolutional neural network model includes: an input layer for receiving macroscopic video image information; the first convolutional layer contains multiple 3x3 convolution kernels and uses the Swish activation function, followed by a 2x2 maximum pooling layer; the second convolutional layer increases the number of convolution kernels and repeatedly uses the Swish activation function and maximum pooling; a flattening layer is used to convert the multi-dimensional data output by the convolutional layer into one-dimensional data; a fully connected layer uses the Swish activation function to integrate features and perform classification judgments; an output layer uses the Softmax activation function to output the classification probability and output quantity of construction materials.

[0067] Preferably, the extracting of the feature vector f1 of the preprocessed macroscopic video image information and the feature vector f2 of the microscopic video image information respectively includes using the scale-invariant feature transform (SIFT) method to extract key points and their descriptors, including corner points, edges, and brightness spots, from the preprocessed macroscopic video image information and the microscopic video image information; and using the histogram of oriented gradients (HOG) method to extract the gradient direction and intensity in the preprocessed macroscopic video image and the microscopic video image.

[0068] Scale Invariant Feature Transform (SIFT) and Histogram of Oriented Gradients (HOG) are two widely used image feature extraction methods that play an important role in the field of computer vision and image processing. These methods are able to extract useful features from images for various applications such as object recognition, image matching, and video surveillance. The following is a detailed description of these two techniques and how to use them to extract feature vectors from macro and micro video images. Scale Invariant Feature Transform SIFT aims to extract some key points (feature points) and their descriptors from an image that are scale invariant. These key points remain unchanged when the image scale changes, and remain stable even when rotated, brightness changes, etc. Scale space extremum detection uses the Gaussian difference (DoG) method to find potential points of interest in different scale spaces. These points are local extrema in scale and space. Key points are located at each candidate position. A fitting model is used to determine the position and scale of the key points. Points with low contrast and strong edge responses are removed to enhance matching stability and noise resistance. Direction assignment assigns one or more directions to each key point based on the direction of the local image gradient. This step is the basis of scale and rotation invariance. Key point descriptor generation. In the area around each key point, a descriptor is constructed based on the direction and magnitude of the image gradient. The descriptor is a vector, usually with 128 elements. Applying the above steps to macro and micro images, the key points of each image and their descriptors constitute a feature vector. Directed gradient histogram HOG is a method to describe the distribution of local gradient directions and intensity of an image. HOG features are constructed by counting the gradient directions of local areas of the image. Gradient calculation first calculates the gradient magnitude and direction of each pixel in the image. When constructing the gradient direction histogram, the image is first divided into small connected areas, called cells. The gradient direction of each cell is divided into several intervals according to a certain angle interval, and then the gradient direction of each pixel is accumulated into the corresponding interval. In order to improve the resistance to changes in illumination and shadows, all histograms within the larger area composed of cells (called blocks) are normalized, and the normalized histograms of all blocks are connected to form the final feature descriptor. For images obtained from macro and micro videos, the HOG method is applied to extract the gradient direction histogram of the entire image and generate a feature vector for subsequent processing. In one embodiment, in a construction site safety monitoring system, SIFT and HOG techniques are used to extract features from macro and micro video images obtained from different cameras. SIFT is implemented to capture key points and descriptors from micro videos to accurately identify specific construction materials on the construction site (such as the detailed texture of bricks). HOG is implemented to analyze the overall flow of construction materials from macro videos, such as the arrangement pattern of stacked cement bags and steel bars.

[0069] Preferably, the dual-stream deep learning network construction material recognition model includes: two independent input layers, respectively used to receive macroscopic video image information and microscopic video image information; 4 convolution layers are configured for the macroscopic video image information data stream and the microscopic video image information data stream, respectively, the first layer uses 32 3x3 convolution kernels, the second layer uses 64 3x3 convolution kernels, the third layer uses 128 3x3 convolution kernels, and the fourth layer uses 256 3x3 convolution kernels; each convolution layer is equipped with a Mish activation function; each convolution layer is followed by a 2x2 maximum pooling layer for reducing the feature dimension; the features of the macroscopic video image information data stream and the microscopic video image information data stream are merged by splicing after the fourth convolution layer; after the fusion features, three fully connected layers are connected, each layer contains 1024, 512 and 256 neurons respectively; a dropout ratio of 0.5 is used after each fully connected layer to prevent overfitting; the last fully connected layer is followed by a Softmax output layer for classification and a linear output unit for quantity estimation.

[0070] The present invention also provides a park construction material safety monitoring system for real-time processing of video streams, such as Figure 2As shown, it includes: for a park construction material safety monitoring system with real-time processing of video streams, the system is mainly composed of a variety of hardware, which are interconnected through a network to realize real-time transmission and processing of data. The following is a detailed description of the main hardware components of the system and how they are connected. CCD cameras capture macro and micro video images in the park. These cameras should have high resolution and night vision functions to ensure clear images under various lighting conditions. An appropriate number of cameras are installed according to the size of the park and the number of areas that need to be covered. Mobile terminal devices are used to collect more detailed microscopic image information, such as detailed images of materials taken by workers using handheld devices, including smartphones and tablets, which need to be equipped with dedicated applications to facilitate data upload and access. The server processes the video stream from the camera and runs video analysis software, such as feature extraction, image preprocessing and deep learning models, high-performance CPUs and GPUs, large-capacity storage devices, and high-speed memory to ensure that large amounts of data and complex computing tasks can be processed. The display screen displays real-time video streams and analysis results in the control room, such as the classification and quantity of materials, large LED display screens or multi-screen display systems to support multi-window and high-resolution display. Network equipment ensures that all devices can operate in a high-speed and secure network environment, including routers, switches and possible wireless access points, supporting Ethernet and Wi-Fi connections. The camera is connected to the network via high-speed Ethernet or Wi-Fi. In the network, all video data streams are routed to the server, which is responsible for data processing and analysis. The mobile terminals and the mobile terminals used by the server workers are connected to the same network via Wi-Fi. The data uploaded by these devices is also sent to the server for further processing. The server outputs the processed results to the display screen in the control room through the HDMI or VGA interface so that the staff can monitor the system status and analysis results in real time. A local area network is established using routers and switches to ensure the speed and security of data transmission. For remote access, VPN can be configured or a secure connection can be made over the Internet.

[0071] Macro video image information acquisition module: CCD camera acquires macro video image information of the construction material arrival acceptance area in real time;

[0072] The first convolutional neural network model recognition module inputs the macro video image information into the trained first convolutional neural network model for rough recognition, and roughly recognizes the initial type and quantity of construction materials. The first convolutional neural network model uses the Swish activation function. Swish(x1) is expressed as follows:

[0073]

[0074] Among them, x1 is the input of the Swish activation function, e is the base of the natural logarithm, and α is a trainable parameter;

[0075] Fine identification process judgment module, when the number of construction materials N passing through per unit time t When the preset threshold is exceeded or there is a construction material with a historical recognition error rate greater than the set threshold in the current initial type of construction materials, the number of CPUs or GPUs executing the video image processing task is increased and the process of fine recognition is directly entered into step S4, otherwise, the process is entered into step S7;

[0076] A microscopic video image information acquisition module collects microscopic video image information of the incoming construction materials through a mobile terminal device;

[0077] A preprocessing module preprocesses the macroscopic video image information and the microscopic video image information, and extracts a feature vector f1 of the preprocessed macroscopic video image information and a feature vector f2 of the microscopic video image information;

[0078] The dual-stream deep learning network construction material recognition model recognition module inputs the feature vector f1 and the feature vector f2 into the trained dual-stream deep learning network construction material recognition model for fine recognition, and recognizes the category and quantity of construction materials in real time. One stream of the dual-stream deep learning network construction material recognition model processes macroscopic video image information, and the other stream processes microscopic video image information; the dual-stream deep learning network construction material recognition model adopts the Mish activation function:

[0079]

[0080] Among them, x2 is the input of the Mish activation function, N t is the number of construction materials passed per unit time, and β is the scaling factor used to adjust the impact of the number of construction materials on the activation function;

[0081] The display module displays the types and quantities of construction materials currently identified and the types and quantities of construction materials received cumulatively through an arrival acceptance display screen or an application.

[0082] Preferably, the macroscopic video image information and the microscopic video image information are preprocessed, and the preprocessing includes using Gaussian filtering to perform video image denoising and using histogram equalization to perform video image enhancement.

[0083] Preferably, the first convolutional neural network model includes: an input layer for receiving macroscopic video image information; the first convolutional layer contains multiple 3x3 convolution kernels and uses the Swish activation function, followed by a 2x2 maximum pooling layer; the second convolutional layer increases the number of convolution kernels and repeatedly uses the Swish activation function and maximum pooling; a flattening layer is used to convert the multi-dimensional data output by the convolutional layer into one-dimensional data; a fully connected layer uses the Swish activation function to integrate features and perform classification judgments; an output layer uses the Softmax activation function to output the classification probability and output quantity of construction materials.

[0084] Preferably, the extracting of the feature vector f1 of the preprocessed macroscopic video image information and the feature vector f2 of the microscopic video image information respectively includes using the scale-invariant feature transform (SIFT) method to extract key points and their descriptors, including corner points, edges, and brightness spots, from the preprocessed macroscopic video image information and the microscopic video image information; and using the histogram of oriented gradients (HOG) method to extract the gradient direction and intensity in the preprocessed macroscopic video image and the microscopic video image.

[0085] Preferably, the dual-stream deep learning network construction material recognition model includes: two independent input layers, respectively used to receive macroscopic video image information and microscopic video image information; 4 convolution layers are configured for the macroscopic video image information data stream and the microscopic video image information data stream, respectively, the first layer uses 32 3x3 convolution kernels, the second layer uses 64 3x3 convolution kernels, the third layer uses 128 3x3 convolution kernels, and the fourth layer uses 256 3x3 convolution kernels; each convolution layer is equipped with a Mish activation function; each convolution layer is followed by a 2x2 maximum pooling layer for reducing the feature dimension; the features of the macroscopic video image information data stream and the microscopic video image information data stream are merged by splicing after the fourth convolution layer; after the fusion features, three fully connected layers are connected, each layer contains 1024, 512 and 256 neurons respectively; a dropout ratio of 0.5 is used after each fully connected layer to prevent overfitting; the last fully connected layer is followed by a Softmax output layer for classification and a linear output unit for quantity estimation.

[0086] The present invention provides a method and system for monitoring the safety of park construction materials by real-time processing of video streams, which can achieve the following beneficial technical effects:

[0087] 1. The method of the present invention first obtains the macroscopic video image information of the construction material arrival acceptance area in real time, and inputs the macroscopic video image information into the trained first convolutional neural network model to roughly identify the initial type and quantity of the construction materials; secondly, it is determined whether to enter step S4 for fine identification process. If it is entered, the microscopic video image information of the construction materials arriving is collected and preprocessed through the mobile terminal device; thirdly, a query for the knowledge graph is constructed according to the semantic role annotation to obtain the information retrieved from the knowledge graph; then, the feature vector f1 and the feature vector f2 are input into the trained dual-stream deep learning network construction material recognition model to finely identify the category and quantity of the construction materials; finally, the category and quantity of the currently identified construction materials are displayed. This application adopts the first convolutional neural network model for rough identification, and adopts the dual-stream deep learning network construction material recognition model for fine identification of the category and quantity of the construction materials, which greatly improves the accuracy and efficiency of construction material identification and greatly increases the user experience.

[0088] 2. The present invention adopts a dual-stream deep learning network to construct a material recognition model using the Mish activation function:

[0089]

[0090] Among them, x2 is the input of the Mish activation function, N t is the number of construction materials passing through per unit time, and β is the scaling factor used to adjust the impact of the number of construction materials on the activation function. The Mish activation function makes full use of the rough recognition information by considering the number of construction materials passing through per unit time, and dynamically adjusts the calculation process according to the input number of construction materials, which greatly improves the accuracy and efficiency of construction material recognition.

[0091] 4. The present invention uses rough identification and sets a fine identification process judgment module. When the number of construction materials N passing through per unit time t When the preset threshold is exceeded or there is a construction material in the current initial type of construction material whose historical recognition error rate is greater than the set threshold, the number of CPUs or GPUs executing the video image processing task is increased and the process directly enters step S4 for fine recognition. Otherwise, it enters step S7, realizing the combination of rough recognition and fine recognition, which greatly improves the recognition efficiency and accuracy.

[0092] The above is a detailed introduction to a method and system for monitoring the safety of campus construction materials with real-time processing of video streams. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the core idea of ​​the present invention; at the same time, for general technical personnel in this field, according to the ideas and methods of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for monitoring the safety of park construction materials by real-time processing of video streams, characterized in that: Includes steps: S1: CCD camera acquires macroscopic video image information of the construction material arrival acceptance area in real time; S2: Input the macro video image information into the trained first convolutional neural network model for rough recognition, and roughly identify the initial type and quantity of construction materials. The first convolutional neural network model uses the Swish activation function. Swish(x1) is expressed as follows: Among them, x1 is the input of the Swish activation function, e is the base of the natural logarithm, and α is a trainable parameter; S3: The number of construction materials N that pass through per unit time t When the preset threshold is exceeded or there is a construction material with a historical recognition error rate greater than the set threshold in the current initial type of construction materials, the number of CPUs or GPUs executing the video image processing task is increased and the process of fine recognition is directly entered into step S4, otherwise, the process is entered into step S7; S4: Collecting microscopic video image information of the construction materials arriving through a mobile terminal device; S5: preprocessing the macroscopic video image information and the microscopic video image information, and extracting the feature vector f1 of the preprocessed macroscopic video image information and the feature vector f2 of the microscopic video image information respectively; S6: Input the feature vector f1 and the feature vector f2 into the trained dual-stream deep learning network construction material recognition model for fine recognition, and identify the category and quantity of construction materials in real time. One stream of the dual-stream deep learning network construction material recognition model processes macroscopic video image information, and the other stream processes microscopic video image information; the dual-stream deep learning network construction material recognition model uses the Mish activation function: Among them, x2 is the input of the Mish activation function, N t is the number of construction materials passed per unit time, and β is the scaling factor used to adjust the impact of the number of construction materials on the activation function; S7: Display the currently identified types and quantities of construction materials and the cumulative types and quantities of construction materials received through the arrival acceptance display screen or application.

2. A method for monitoring the safety of park construction materials by real-time processing of video streams as claimed in claim 1, characterized in that: The macroscopic video image information and the microscopic video image information are preprocessed, and the preprocessing includes using Gaussian filtering to perform video image denoising and using histogram equalization to perform video image enhancement.

3. The method for monitoring the safety of park construction materials by real-time processing of video streams as claimed in claim 1, characterized in that: The first convolutional neural network model includes: an input layer for receiving macroscopic video image information; the first convolutional layer contains multiple 3x3 convolution kernels and uses the Swish activation function, followed by a 2x2 maximum pooling layer; the second convolutional layer increases the number of convolution kernels and repeatedly uses the Swish activation function and maximum pooling; a flattening layer is used to convert the multi-dimensional data output by the convolutional layer into one-dimensional data; a fully connected layer uses the Swish activation function to integrate features and perform classification judgments; an output layer uses the Softmax activation function to output the classification probability and output quantity of construction materials.

4. The method for monitoring the safety of park construction materials by real-time processing of video streams as claimed in claim 1, characterized in that: The extracting the feature vector f1 of the preprocessed macroscopic video image information and the feature vector f2 of the microscopic video image information respectively comprises extracting key points and their descriptors, including corner points, edges, and brightness spots, from the preprocessed macroscopic video image information and the microscopic video image information using a scale-invariant feature transform (SIFT) method; The Histogram of Oriented Gradients (HOG) method is used to extract the gradient direction and intensity in the preprocessed macroscopic and microscopic video images.

5. The method for monitoring the safety of park construction materials by real-time processing of video streams as claimed in claim 1, characterized in that: The dual-stream deep learning network construction material recognition model includes: two independent input layers, respectively used to receive macroscopic video image information and microscopic video image information; 4 convolution layers are configured for the macroscopic video image information data stream and the microscopic video image information data stream, respectively, the first layer uses 32 3x3 convolution kernels, the second layer uses 64 3x3 convolution kernels, the third layer uses 128 3x3 convolution kernels, and the fourth layer uses 256 3x3 convolution kernels; each convolution layer is equipped with a Mish activation function; each convolution layer is followed by a 2x2 maximum pooling layer for reducing the feature dimension; the features of the macroscopic video image information data stream and the microscopic video image information data stream are merged by splicing after the fourth convolution layer; after the fusion features, three fully connected layers are connected, each layer contains 1024, 512 and 256 neurons respectively; a dropout ratio of 0.5 is used after each fully connected layer to prevent overfitting; the last fully connected layer is followed by a Softmax output layer for classification, and a linear output unit for quantity estimation.

6. A park construction material safety monitoring system with real-time video stream processing, characterized in that: include: Macro video image information acquisition module: CCD camera acquires macro video image information of the construction material arrival acceptance area in real time; The first convolutional neural network model recognition module inputs the macro video image information into the trained first convolutional neural network model for rough recognition, and roughly recognizes the initial type and quantity of construction materials. The first convolutional neural network model uses the Swish activation function. Swish(x1) is expressed as follows: Among them, x1 is the input of the Swish activation function, e is the base of the natural logarithm, and α is a trainable parameter; Fine identification process judgment module, when the number of construction materials N passing through per unit time t When the preset threshold is exceeded or there is a construction material with a historical recognition error rate greater than the set threshold in the current initial type of construction materials, the number of CPUs or GPUs executing the video image processing task is increased and the process of fine recognition is directly entered into step S4, otherwise, the process is entered into step S7; A microscopic video image information acquisition module collects microscopic video image information of the incoming construction materials through a mobile terminal device; A preprocessing module preprocesses the macroscopic video image information and the microscopic video image information, and extracts a feature vector f1 of the preprocessed macroscopic video image information and a feature vector f2 of the microscopic video image information; The dual-stream deep learning network construction material recognition model recognition module inputs the feature vector f1 and the feature vector f2 into the trained dual-stream deep learning network construction material recognition model for fine recognition, and recognizes the category and quantity of construction materials in real time. One stream of the dual-stream deep learning network construction material recognition model processes macroscopic video image information, and the other stream processes microscopic video image information; the dual-stream deep learning network construction material recognition model adopts the Mish activation function: Among them, x2 is the input of the Mish activation function, N t is the number of construction materials passed per unit time, and β is the scaling factor used to adjust the impact of the number of construction materials on the activation function; The display module displays the types and quantities of construction materials currently identified and the types and quantities of construction materials received cumulatively through an arrival acceptance display screen or an application.

7. A park construction material safety monitoring system for real-time video stream processing as claimed in claim 6, characterized in that: The macroscopic video image information and the microscopic video image information are preprocessed, and the preprocessing includes using Gaussian filtering to perform video image denoising and using histogram equalization to perform video image enhancement.

8. A park construction material safety monitoring system for real-time video stream processing as claimed in claim 6, characterized in that: The first convolutional neural network model includes: an input layer for receiving macroscopic video image information; the first convolutional layer contains multiple 3x3 convolution kernels and uses the Swish activation function, followed by a 2x2 maximum pooling layer; the second convolutional layer increases the number of convolution kernels and repeatedly uses the Swish activation function and maximum pooling; a flattening layer is used to convert the multi-dimensional data output by the convolutional layer into one-dimensional data; a fully connected layer uses the Swish activation function to integrate features and perform classification judgments; an output layer uses the Softmax activation function to output the classification probability and output quantity of construction materials.

9. A park construction material safety monitoring system for real-time video stream processing as claimed in claim 6, characterized in that: The extracting the feature vector f1 of the preprocessed macroscopic video image information and the feature vector f2 of the microscopic video image information respectively comprises extracting key points and their descriptors, including corner points, edges, and brightness spots, from the preprocessed macroscopic video image information and the microscopic video image information using a scale-invariant feature transform (SIFT) method; The Histogram of Oriented Gradients (HOG) method is used to extract the gradient direction and intensity in the preprocessed macroscopic and microscopic video images.

10. A park construction material safety monitoring system for real-time video stream processing as claimed in claim 6, characterized in that: The dual-stream deep learning network construction material recognition model includes: two independent input layers, respectively used to receive macroscopic video image information and microscopic video image information; 4 convolution layers are configured for the macroscopic video image information data stream and the microscopic video image information data stream, respectively, the first layer uses 32 3x3 convolution kernels, the second layer uses 64 3x3 convolution kernels, the third layer uses 128 3x3 convolution kernels, and the fourth layer uses 256 3x3 convolution kernels; each convolution layer is equipped with a Mish activation function; each convolution layer is followed by a 2x2 maximum pooling layer for reducing the feature dimension; the features of the macroscopic video image information data stream and the microscopic video image information data stream are merged by splicing after the fourth convolution layer; after the fusion features, three fully connected layers are connected, each layer contains 1024, 512 and 256 neurons respectively; a dropout ratio of 0.5 is used after each fully connected layer to prevent overfitting; the last fully connected layer is followed by a Softmax output layer for classification, and a linear output unit for quantity estimation.

Images