Project field multi-modal data acquisition system and method based on AI Internet of Things

Abstract

The invention relates to an AI Internet of Things-based project field multi-modal data acquisition system and method, provides access control, and particularly relates to the field of a set or interface system of a database management system. The system comprises a warehouse-in acquisition interface for acquiring warehouse-in content of a set construction project site in a current time interval by adopting an Internet of Things connection mode; and the intelligent identification device is used for intelligently identifying the proportion of personnel wearing safety helmets in the current time interval and the maximum noise decibel value of the nearby residential building position in the current time interval by adopting an AI state identification model based on the warehousing content of the current time interval. According to the invention, the technical problem that the proportion of personnel wearing safety helmets in any time interval and the maximum noise decibel value of the positions of nearby residential buildings are difficult to synchronously identify is solved by simultaneously introducing an access control mode of database input and output, an Internet of Things connection mode and an intelligent identification mode.

Description

Technical Field

[0001] The access control proposed in this invention relates more specifically to the field of database management system collections or interface systems, and more specifically to a multimodal data acquisition system and method based on AI IoT project site. Background Technology

[0002] Access control is most commonly used in database management systems (DBMS) for access control of collections or interface systems. It controls the storage and retrieval of data within the DBMS, enabling the use of various data management mechanisms to ensure the performance of critical data, such as security and traceability, preventing data loss or leakage, and guaranteeing the reliability and stability of subsequent data processing results. The types of critical data vary depending on the specific application area. For example, the data collected synchronously by various data acquisition arrays installed on a construction project site provides crucial information for safety management, noise management, and other aspects of construction project management.

[0003] For example, Chinese invention patent publication CN114742530A proposes an Internet of Things (IoT)-based project construction management system. The system includes an IoT module, a construction management module, a data storage module, and a visualization operation module. The IoT module collects data from the project construction site and sends the collected data to the data storage module. The construction management module manages and analyzes the entire construction process and sends the analysis results to the visualization operation module. The data storage module stores user identity information, project construction data, and pre-stored project construction standards. The visualization operation module displays various parameters of the project construction management system according to instructions from the construction management module. This application overcomes the limitations of dispersed project construction areas, enabling comprehensive project construction management while improving management efficiency. The data is displayed in report form, facilitating real-time monitoring of project construction status by staff and aiding in subsequent project construction management.

[0004] For example, Chinese invention patent publication CN106709703A proposes a method and system for managing the construction process of decoration projects based on Internet of Things (IoT) technology. This method standardizes decoration product data and establishes a connection with a DPM (Design for Product Management) system. It uses IoT technology to achieve on-site data collection and real-time monitoring, including monitoring the entry, exit, and usage of construction materials, tracking construction progress, monitoring and analyzing construction quality, monitoring and analyzing the completion of on-site workers' tasks, and tracking dynamic construction costs. This application employs IoT technology, using two-dimensional barcode technology, RFID technology, laser scanners, and wireless sensors. Through the ZigBee protocol, it aggregates information on personnel, materials, and machinery at the construction site to a routing device and transmits it to a central information system via the internet. This enables real-time transmission and sharing of information such as decoration project costs, progress, and quality, solving the timeliness and accuracy issues of information collection in the DPM system.

[0005] Therefore, it is evident that although the various technical solutions of the aforementioned existing technologies all involve the collection, storage, and application of various data acquisition devices installed inside the construction project site in project management, these applications are all conventional and employ conventional data processing modes. For example, they are project construction progress management or project construction cost and quality management. They cannot simultaneously identify the proportion of personnel wearing safety helmets at the project construction site or the maximum noise decibel value experienced by nearby residential buildings. In particular, they cannot perform simultaneous identification of various states over a period of time, but only perform routine analysis of the state at a certain moment for some conventional applications. As a result, the level of automation and intelligence in project construction progress management cannot be further improved. Summary of the Invention

[0006] To address the technical problems in existing technologies, this invention provides an AI IoT-based multimodal data acquisition system and method for construction site projects. By employing IoT communication, it acquires various data sets synchronously collected by arrays of various acquisition devices at a designated construction site within a given time interval. Based on these data sets, the location information of various devices within the designated construction site, multiple sets of on-site configuration data, the reference pattern corresponding to safety helmets, and the 3D point cloud data of a designated residential building, it simultaneously and intelligently identifies the proportion of personnel wearing safety helmets at the designated construction site within the given time interval and the maximum noise level experienced by the designated residential building within the given time interval. This allows for the simultaneous and intelligent assessment of two different data sets. Intelligent analysis of similar on-site conditions provides valuable reference data for the customization of subsequent safety management strategies and dynamic noise control strategies, improving the automation and intelligence levels of on-site management of construction projects. In particular, by setting up the access interface system of the database management system for inbound and outbound operations, it is possible to ensure the performance of various data sets of various acquisition device arrays synchronously collected by the construction project site within the current time interval based on the various data management mechanisms provided within the database management system. This avoids data loss or leakage and ensures the reliability and stability of subsequent synchronous intelligent identification results.

[0007] According to one aspect of the present invention, a multimodal data acquisition system based on AI IoT project site is provided, the system comprising:

[0008] The data acquisition interface is connected via IoT to the image acquisition device array, audio acquisition device array, light illuminance sensor array, and vibration sensor array that perform synchronous data acquisition on-site within the designated construction project. It is used to store the sets of project site images, audio signal amplitude sets, light illuminance sets, and real-time vibration data sets that are synchronously acquired at uniform intervals at various times within the current time interval. The data to be stored is the database management system of the designated construction project.

[0009] Configure analytical devices to analyze the location information of various devices and multiple sets of on-site configuration data for the construction project.

[0010] The target is to construct a device for performing various training actions on a convolutional neural network to obtain an AI state identification model.

[0011] The outbound data acquisition interface connects to the database management system of the designated construction project and is used to retrieve the inbound content corresponding to the current time interval.

[0012] The intelligent identification device connects to the outbound data acquisition interface, configuration analysis device, and target construction device, respectively. It uses an AI-based state identification model based on the reference pattern corresponding to the safety helmet, the 3D point cloud data of the residential building, the duration of each time interval, the data stored for the current time interval, the location information of various devices in the construction project, and multiple sets of on-site configuration data. It intelligently identifies the proportion of personnel wearing safety helmets at the construction site within the current time interval and the maximum noise level experienced by the residential building within the current time interval.

[0013] According to another aspect of the present invention, a method for acquiring multimodal data at the site of an AI IoT project is provided, the method comprising:

[0014] An input acquisition interface is used to connect the image acquisition device array, audio acquisition device array, light illuminance sensor array, and vibration sensor array to the Internet of Things for synchronous data acquisition at various time points within the current time interval. This interface is used to input the data into the database management system of the specified construction project.

[0015] Analyze and set the location information of various components and multiple sets of on-site configuration data for the construction project;

[0016] Perform various training actions on the convolutional neural network to obtain an AI state identification model;

[0017] An outbound data acquisition interface connected to the database management system of the designated construction project is used to retrieve the inbound content corresponding to the current time interval.

[0018] The AI-based status assessment model uses a baseline pattern corresponding to the safety helmet, three-dimensional point cloud data of the residential building, the duration of each time interval, the content stored in the database corresponding to the current time interval, the location information of various components of the construction project, and multiple sets of on-site configuration data to intelligently assess the proportion of personnel wearing safety helmets at the construction site within the current time interval, as well as the maximum noise level that the residential building experiences within the current time interval.

[0019] Therefore, it can be seen that the present invention has at least the following main inventive points:

[0020] Invention Point A: Based on the various data sets synchronously collected by the array of various acquisition devices at the construction site within the current time interval, the location information of various devices in the construction project, multiple sets of on-site configuration data, the reference pattern corresponding to the safety helmet, and the three-dimensional point cloud data of the residential building, the invention can simultaneously and intelligently identify the proportion of personnel wearing safety helmets at the construction site within the current time interval and the maximum noise decibel value experienced by the residential building within the current time interval. This allows for the simultaneous intelligent analysis of two different types of on-site conditions, providing valuable reference data for the customization of subsequent safety management strategies and dynamic noise control strategies, and improving the automation and intelligence level of the construction site management.

[0021] Invention Point B: By setting up the access interface system of the database management system for inbound and outbound operations, the system can ensure the performance of various data sets of the key data used for synchronous intelligent identification, namely the various data sets of the various acquisition device arrays that are synchronously acquired at the construction site within the current time interval, based on the various data management mechanisms provided within the database management system. This avoids data loss or leakage and ensures the reliability and stability of the subsequent synchronous intelligent identification results.

[0022] Invention Point C: To perform synchronous intelligent identification of the proportion of personnel wearing safety helmets at a construction site within the current time interval and the maximum noise level experienced by a residential building within the current time interval, an AI state identification model with a customized structure for the construction project is introduced. The AI ​​state identification model is a convolutional neural network after various training actions. The number of training actions is positively correlated with the area occupied by the construction project. The convolutional neural network used has a single input layer, multiple convolutional layers, multiple pooling layers, and a single fully connected layer. The number of multiple convolutional layers is positively correlated with the total number of data acquisition devices in the construction project. All convolutional layers use the Sigmoid function as the activation function. Thus, different customized AI state identification models are designed for different construction projects. Through the above-mentioned customized structural designs, the reliability and stability of the synchronous intelligent identification results are further guaranteed.

[0023] Invention Point D: In each training action of the convolutional neural network, the proportion of personnel wearing safety helmets at the construction site during a known past time interval and the maximum noise level experienced by the residential building during the same past time interval are used as two outputs of the convolutional neural network. The reference pattern corresponding to the safety helmets, the 3D point cloud data of the residential building, the duration of each time interval, the data stored in the database corresponding to the past time interval, the location information of various components in the construction project, and multiple sets of on-site configuration data are used as inputs of the convolutional neural network to complete the training action of the convolutional neural network, thereby ensuring the training effect of each training action of the convolutional neural network.

[0024] Invention Point E: To perform synchronous intelligent identification of the proportion of personnel wearing safety helmets at the construction site within the current time interval and the maximum noise decibel value experienced by the residential building within the current time interval, various basic elements are introduced, including the reference pattern corresponding to the safety helmet, the three-dimensional point cloud data of the residential building, the duration of each time interval, the content stored in the database corresponding to the current time interval, the location information of various components in the construction project, and multiple sets of on-site configuration data. The comprehensive introduction of the above basic elements further ensures the reliability and stability of the synchronous intelligent identification results.

[0025] Invention Point F: Specifically, the data stored in the current time interval consists of a set of project site images, a set of audio signal amplitudes, a set of light intensity, and a set of real-time vibration data, all collected synchronously at uniform intervals within the current time interval. It also includes the three-dimensional coordinate data of the locations of all devices in various device arrays, used as the location information of each device in the construction project. Simultaneously, the land area, total number of registered construction personnel, building height, and single-floor area of ​​the construction project are used as multiple sets of on-site configuration data for the construction project. This completes the customized design of the data structure for the various basic contents used in synchronous intelligent identification. Attached Figure Description

[0026] The embodiments of the present invention will now be described with reference to the accompanying drawings, wherein:

[0027] Figure 1 This is a schematic diagram of a working scenario for a multimodal data acquisition system and method based on AI IoT projects according to the present invention.

[0028] Figure 2 This is an internal structure diagram of a multimodal data acquisition system for an AI IoT project site, as shown in the first embodiment of the present invention.

[0029] Figure 3This is an internal structure diagram of a multimodal data acquisition system for an AI IoT project site, as shown in the second embodiment of the present invention.

[0030] Figure 4 This is an internal structure diagram of a multimodal data acquisition system for an AI IoT project site, as shown in the third embodiment of the present invention.

[0031] Figure 5 This is an internal structure diagram of an AI IoT project site multimodal data acquisition system according to the fourth embodiment of the present invention.

[0032] Figure 6 This is an internal structure diagram of a multimodal data acquisition system for an AI IoT project site, as shown in the fifth embodiment of the present invention.

[0033] Figure 7 The present invention provides a flowchart illustrating the steps of a multimodal data acquisition method for an AI IoT project site, as shown in the sixth embodiment of the present invention. Detailed Implementation

[0034] like Figure 1 The diagram illustrates a working scenario of a multimodal data acquisition system and method for AI IoT projects based on the present invention. The access control proposed in this invention more specifically relates to the field of database management system collections or interface systems.

[0035] To achieve the present invention, the following technical processes are specifically proposed:

[0036] Technical Process 1: To simultaneously and intelligently assess the proportion of personnel wearing safety helmets at the construction site within the current time interval and the maximum noise level experienced by the residential building within the current time interval, an AI-based condition assessment model customized for the construction project's structural design is introduced. Figure 1 As shown;

[0037] Specifically, different customized AI status assessment models are designed for different construction projects. The customized structural design of the AI ​​status assessment model corresponding to the construction project is reflected in the following aspects:

[0038] First, the AI ​​status identification model corresponding to the construction project is set as a convolutional neural network after each training action. The convolutional neural network used has a single input layer, multiple convolutional layers, multiple pooling layers and a single fully connected layer.

[0039] Secondly, the number of training actions of the selected convolutional neural network is positively correlated with the land area of ​​the construction project.

[0040] For example, if the construction project covers an area of ​​5,000 square meters, the number of training movements selected is 500; if the construction project covers an area of ​​7,000 square meters, the number of training movements selected is 700; if the construction project covers an area of ​​10,000 square meters, the number of training movements selected is 1,000; if the construction project covers an area of ​​20,000 square meters, the number of training movements selected is 2,000, and so on.

[0041] Furthermore, the number of selected convolutional layers is positively correlated with the total number of data acquisition devices in the set construction project, and all convolutional layers use the Sigmoid function as the activation function;

[0042] For example, the construction project is set to have 20 data acquisition devices and 2 selected convolutional layers; the construction project is set to have 30 data acquisition devices and 3 selected convolutional layers; the construction project is set to have 40 data acquisition devices and 4 selected convolutional layers; the construction project is set to have 50 data acquisition devices and 5 selected convolutional layers, and so on.

[0043] Finally, in each training action of the convolutional neural network, the proportion of personnel wearing safety helmets at the construction site during a certain past time interval and the maximum noise level experienced by the residential building during the same past time interval are used as the two outputs of the convolutional neural network. The reference pattern corresponding to the safety helmets, the 3D point cloud data of the residential building, the duration of each time interval, the data stored in the database corresponding to the past time interval, the location information of various components in the construction project, and multiple sets of on-site configuration data are used as the inputs of the convolutional neural network to complete the training action of the convolutional neural network, thereby ensuring the training effect of each training action of the convolutional neural network.

[0044] In this way, the reliability and stability of the synchronous intelligent identification results are guaranteed through the above-mentioned customized structural designs.

[0045] Technical Process 2: To perform synchronous intelligent assessment of the proportion of personnel wearing safety helmets at the construction site within the current time interval and the maximum noise decibel value that the residential building will experience within the current time interval, various basic elements were introduced;

[0046] Specifically, such as Figure 1As shown, the basic content includes the data to be entered into the database corresponding to the current time interval, the three-dimensional point cloud data of the residential building and other auxiliary data. The other auxiliary data includes the reference pattern corresponding to the safety helmet, the duration of each time interval, the location information of various components of the construction project and multiple sets of on-site configuration data. The comprehensive and full introduction of the above basic content further ensures the reliability and stability of the synchronous intelligent identification results.

[0047] More specifically, the data to be stored in the current time interval consists of a set of project site images, a set of audio signal amplitudes, a set of light intensity, and a set of real-time vibration data, all collected synchronously at evenly spaced intervals within the current time interval. It also includes the three-dimensional coordinate data of the locations of all devices in various device arrays as the location information of each device in the construction project. At the same time, the land area, the total number of registered construction personnel, the building height, and the area of ​​a single floor of the construction project are used as multiple sets of on-site configuration data for the construction project. This completes the customized design of the data structure for the basic content of synchronous intelligent identification.

[0048] In this way, the reliability and stability of the synchronous intelligent identification results are further guaranteed through the targeted screening of the above basic elements.

[0049] Technical Process 3: Using the AI ​​status assessment model of the customized structural design of the construction project as set by Technical Process 1, and based on the basic content selected by Technical Process 2, the proportion of personnel wearing safety helmets on the construction site of the set construction project within the current time interval and the maximum noise decibel value that the residential building is subjected to within the current time interval are simultaneously intelligently assessed.

[0050] like Figure 1 As shown, safety status data and noise status data are obtained. The former is the proportion of personnel wearing safety helmets at the construction site within the current time interval, and the latter is the maximum noise decibel value that the residential building is subjected to within the current time interval.

[0051] Obviously, as can be seen from this invention, the acquired state data is state data within a time interval, rather than state data at a single moment in the prior art. The two types of state data acquired simultaneously are two unconventional types of state data, and the two types of state data are acquired simultaneously through synchronous intelligent identification.

[0052] Technical Process Four: Simultaneously display the two status data of the three-synchronization intelligent identification process on-site;

[0053] For example, such as Figure 1 As shown, a real-time display panel can be selected and installed in the control room of a designated construction project. The real-time display panel can be... Figure 1 The LCD display array in the middle can also be a liquid crystal display screen, used to receive and synchronously display the proportion of people wearing safety helmets at the construction site within the current time interval, as well as the maximum noise decibel value that the residential building will experience within the current time interval.

[0054] Technical Process 5: Based on the two states of the three-synchronization intelligent identification in the technical process, formulate corresponding security control strategies and noise response strategies respectively;

[0055] Specifically, the corresponding safety control strategies are as follows: when the proportion of personnel wearing safety helmets at the construction site is less than or equal to a preset limit within the current time interval, all construction devices associated with fall accidents within the construction project will be shut down in the next time interval of the current time interval; and when the proportion of personnel wearing safety helmets at the construction site is greater than a preset limit within the current time interval, all construction devices associated with fall accidents within the construction project will be kept running in the next time interval of the current time interval.

[0056] Specifically, the corresponding noise response strategy is as follows: when the maximum noise level received by the residential building in the current time interval is higher than or equal to the set decibel threshold, the number of building components with excessive rated output noise amplitude within the construction project to be shut down in the next time interval is determined based on the difference between the maximum noise level and the set decibel threshold. When the maximum noise level received by the residential building in the current time interval is lower than the set decibel threshold, the operation of building components with excessive rated output noise amplitude within the construction project is maintained. The larger the difference between the maximum noise level and the set decibel threshold, the more building components with excessive rated output noise amplitude within the construction project to be shut down in the next time interval.

[0057] Therefore, through the coordinated operation of the above-mentioned technical processes, it is possible to simultaneously and intelligently identify the proportion of personnel wearing safety helmets at the construction site and the maximum noise level experienced by the residential building within the current time interval, based on various data sets synchronously collected by the arrays of various acquisition devices at the construction site within the current time interval, the location information of various devices at the construction site and multiple sets of on-site configuration data, the reference pattern corresponding to the safety helmet, and the 3D point cloud data of the residential building. This allows for the simultaneous intelligent analysis of two different types of on-site conditions, providing valuable reference data for the customization of subsequent safety management strategies and dynamic noise control strategies, and improving the automation and intelligence levels of the construction site management. In particular, by setting up the access interface system of the database management system for data entry and exit, the various data management mechanisms provided within the database management system can ensure the performance of the key data used for synchronous intelligent identification, namely the various data sets synchronously collected by the arrays of various acquisition devices at the construction site within the current time interval, avoiding data loss or leakage, and ensuring the reliability and stability of the subsequent synchronous intelligent identification results.

[0058] The key points of this invention are: synchronous intelligent identification of the safety status of workers wearing safety helmets at construction sites and the noise status of nearby residential buildings within any given time interval; AI status identification models with different customized structures designed for different construction projects; access control mode for database entry and exit; and dynamic formulation of safety control strategies and noise response strategies based on the two status data of synchronous intelligent identification.

[0059] The following will describe in detail an AI IoT project site multimodal data acquisition system and method of the present invention through embodiments.

[0060] First Embodiment

[0061] Figure 2 This is an internal structure diagram of a multimodal data acquisition system for an AI IoT project site, as shown in the first embodiment of the present invention.

[0062] like Figure 2 As shown, the AI ​​IoT-based on-site multimodal data acquisition system includes the following components:

[0063] The data acquisition interface is connected via IoT to the image acquisition device array, audio acquisition device array, light illuminance sensor array, and vibration sensor array that perform synchronous data acquisition on-site within the designated construction project. It is used to store the sets of project site images, audio signal amplitude sets, light illuminance sets, and real-time vibration data sets that are synchronously acquired at uniform intervals at various times within the current time interval. The data to be stored is the database management system of the designated construction project.

[0064] Specifically, the database management system for the construction project includes Oracle database management system, MySQL database management system, or Microsoft SQL Server database management system;

[0065] For example, the current time is 4:00 PM, the current time interval is from 4:00 PM to 4:20 PM, and the evenly spaced time positions within the current time interval are 4:00 PM, 4:02 PM, 4:04 PM, 4:06 PM, 4:08 PM, 4:10 PM, 4:12 PM, 4:14 PM, 4:16 PM, and 4:18 PM, for a total of 10 time positions;

[0066] Configure analytical devices to analyze the location information of various devices and multiple sets of on-site configuration data for the construction project.

[0067] For example, a first parsing device can be selected to parse the location information of various components in the set construction project, while a second parsing device can be selected to parse multiple sets of on-site configuration data for the set construction project.

[0068] The target is to construct a device for performing various training actions on a convolutional neural network to obtain an AI state identification model.

[0069] Specifically, the MATLAB toolbox can be used to simulate and test the modeling process of performing various training actions on the convolutional neural network to obtain the AI ​​state identification model. In fact, the obtained AI state identification model is the convolutional neural network after performing each training action.

[0070] The outbound data acquisition interface connects to the database management system of the designated construction project and is used to retrieve the inbound content corresponding to the current time interval.

[0071] Specifically, the content to be stored in the database corresponding to the current time interval is the collection of project site images, audio signal amplitudes, light intensity, and real-time vibration data, which are collected synchronously at various time points at even intervals within the current time interval.

[0072] The intelligent identification device is connected to the outbound acquisition interface, the configuration analysis device, and the target construction device, respectively. It is used to adopt the AI ​​state identification model based on the reference pattern corresponding to the safety helmet, the three-dimensional point cloud data of the residential building, the occupation duration of each time interval, the content entered into the database corresponding to the current time interval, the location information of various devices in the construction project, and multiple sets of on-site configuration data. It intelligently identifies the proportion of personnel wearing safety helmets at the construction site of the current time interval and the maximum noise decibel value that the residential building is subjected to in the current time interval.

[0073] Obviously, as can be seen from this invention, the acquired state data is state data within a time interval, rather than state data at a single moment in the prior art. The two types of state data acquired simultaneously are two unconventional types of state data, and the two types of state data are acquired simultaneously through synchronous intelligent identification.

[0074] Among them, the analysis of the location information of various components and multiple sets of on-site configuration data of the construction project includes: analyzing the three-dimensional coordinate data of the location of all components in various component arrays as the location information of various components in the construction project, and using the land area, total number of registered construction personnel, building height and single-floor area of ​​the construction project as multiple sets of on-site configuration data of the construction project.

[0075] Specifically, the three-dimensional coordinate data of the location of all devices in various device arrays and the three-dimensional point cloud data of the previously set residential building are all based on the same three-dimensional coordinate system, such as a three-dimensional coordinate system with the midpoint of the geographical area occupied by the three-dimensional point cloud data of the set residential building as the origin.

[0076] The content to be stored in the current time interval includes the collection of project site images, audio signal amplitudes, light intensity, and real-time vibration data, which are collected synchronously at various time points at even intervals within the current time interval.

[0077] Among them, the number of time positions at even intervals within the current time interval is proportional to the total number of registered construction personnel for the set construction project, and the current time interval starts from the current time, and the duration of each time interval is equal to the preset time length.

[0078] For example, if the current time is 4:00 PM and the current time interval is from 4:00 PM to 4:20 PM, then the duration of each time interval, i.e. the preset time length, is 20 minutes.

[0079] Among them, performing various training actions on the convolutional neural network to obtain the AI ​​state identification model includes: performing various training actions on the convolutional neural network to obtain the convolutional neural network after performing various training actions as the AI ​​state identification model, and the number of training actions is positively correlated with the land area of ​​the set construction project.

[0080] For example, a positive correlation between the number of training actions and the area of ​​the set construction project includes: setting the area of ​​the construction project to be 5,000 square meters and selecting 500 training actions; setting the area of ​​the construction project to be 7,000 square meters and selecting 700 training actions; setting the area of ​​the construction project to be 10,000 square meters and selecting 1,000 training actions; setting the area of ​​the construction project to be 20,000 square meters and selecting 2,000 training actions, and so on.

[0081] The process of performing various training actions on the convolutional neural network to obtain the AI ​​state identification model also includes: the convolutional neural network used has a single input layer, multiple convolutional layers, multiple pooling layers and a single fully connected layer. The number of image acquisition devices in the image acquisition device array, the number of audio acquisition devices in the audio acquisition device array, the number of light illuminance sensors in the light illuminance sensor array and the number of vibration sensors in the vibration sensor array are accumulated to obtain the total number of data acquisition devices for the set construction project. The number of multiple convolutional layers is positively correlated with the total number of data acquisition devices for the set construction project, and multiple convolutional layers all use the Sigmoid function as the activation function.

[0082] For example, the number of multiple convolutional layers is positively correlated with the total number of data acquisition devices in the set construction project as follows: if the set construction project has 20 data acquisition devices, the number of multiple convolutional layers selected is 2; if the set construction project has 30 data acquisition devices, the number of multiple convolutional layers selected is 3; if the set construction project has 40 data acquisition devices, the number of multiple convolutional layers selected is 4; if the set construction project has 50 data acquisition devices, the number of multiple convolutional layers selected is 5, and so on.

[0083] In each training action performed on the convolutional neural network, the proportion of personnel wearing safety helmets at the construction site during a known past time interval and the maximum noise level experienced by the residential building during the same past time interval are used as two outputs of the convolutional neural network. The reference pattern corresponding to the safety helmet, the 3D point cloud data of the residential building, the duration of each time interval, the data stored in the database corresponding to the past time interval, the location information of various components in the construction project, and multiple sets of on-site configuration data are used as inputs of the convolutional neural network to complete the training action of the convolutional neural network.

[0084] Second Embodiment

[0085] Figure 3 This is an internal structure diagram of a multimodal data acquisition system for an AI IoT project site, as shown in the second embodiment of the present invention.

[0086] like Figure 3 As shown, compared to Figure 2 The AI ​​IoT-based multimodal data acquisition system for project sites also includes:

[0087] The strategy-customized device, connected to the intelligent identification device, is used to shut down all construction devices associated with fall accidents in the construction project in the next time interval if the proportion of people wearing safety helmets at the construction site is less than or equal to a preset proportion limit within the current time interval.

[0088] For example, the selected preset scale limit can be any specific scale value between 1 / 50 and 1 / 100;

[0089] The strategy customization device is also used to maintain the operation of each building device associated with a fall from height accident in the next time interval of the set construction project when the proportion of people wearing safety helmets at the construction site is greater than a preset limit within the current time interval.

[0090] Third Embodiment

[0091] Figure 4 This is an internal structure diagram of a multimodal data acquisition system for an AI IoT project site, as shown in the third embodiment of the present invention.

[0092] like Figure 4 As shown, compared to Figure 2 The AI ​​IoT-based multimodal data acquisition system for project sites also includes:

[0093] The status response device, connected to the intelligent identification device, is used to determine the number of building devices whose rated output noise amplitude exceeds the limit in the next time interval of the set construction project when the maximum noise decibel value received by the set residential building in the current time interval is higher than or equal to the set decibel threshold, based on the difference between the maximum noise decibel value and the set decibel threshold.

[0094] For example, the selected decibel threshold can be any specific decibel value between 65 and 80 decibels;

[0095] The status response device is also used to maintain the operation of building components whose rated output noise amplitude exceeds the limit within the set construction project when the maximum noise decibel value received by the set residential building in the current time interval is lower than the set decibel threshold.

[0096] The determination of the number of building components whose rated output noise amplitude exceeds the limit in the next time interval of the set construction project, based on the difference between the maximum noise decibel value and the set decibel threshold, includes the following: the greater the difference between the maximum noise decibel value and the set decibel threshold, the more building components whose rated output noise amplitude exceeds the limit in the next time interval of the set construction project will be determined to be shut down.

[0097] Fourth embodiment

[0098] Figure 5 This is an internal structure diagram of an AI IoT project site multimodal data acquisition system according to the fourth embodiment of the present invention.

[0099] like Figure 5 As shown, compared to Figure 2 The AI ​​IoT-based multimodal data acquisition system for project sites also includes:

[0100] The real-time display panel, located in the control room of the set construction project, is connected to the intelligent identification device to receive the proportion of people wearing safety helmets at the set construction project site within the current time interval, as well as the maximum noise decibel value experienced by the set residential building within the current time interval.

[0101] For example, a liquid crystal display screen or an LCD display array can be used to realize a real-time display panel;

[0102] The real-time display panel is also used to simultaneously display the proportion of personnel wearing safety helmets at the construction site within the current time interval, as well as the maximum noise level experienced by the residential building within the current time interval.

[0103] Fifth embodiment

[0104] Figure 6 This is an internal structure diagram of a multimodal data acquisition system for an AI IoT project site, as shown in the fifth embodiment of the present invention.

[0105] like Figure 6 As shown, compared to Figure 2 The AI ​​IoT-based multimodal data acquisition system for project sites also includes:

[0106] The timing server, connected to the data acquisition interface, provides timing reference signals for the data acquisition of various sets of project site images, audio signal amplitudes, light intensity, and real-time vibration data collected synchronously at uniform intervals within the current time interval.

[0107] The timing server, connected to the data acquisition interface, is used to provide timing reference signals for the data acquisition of various project site image sets, audio signal amplitude sets, light intensity sets, and real-time vibration data sets that are synchronously acquired at uniform intervals within the current time interval. The timing server has a built-in quartz oscillator unit for generating clock pulse signals required to generate the timing reference signals.

[0108] For example, the timing server device has a built-in quartz oscillator unit for generating clock pulse signals required to generate timing reference signals, including: the clock pulse signals being pulse signals of a sinusoidal waveform with a set frequency.

[0109] Next, various embodiments of the present invention will be further described.

[0110] Optionally, within the above embodiments, in the AI ​​IoT project site multimodal data acquisition system:

[0111] The three-dimensional coordinate data of the location of all devices in various device arrays are analyzed to serve as the location information of each device in the construction project. At the same time, the land area, total number of registered construction personnel, building height and single-floor area of ​​the construction project are used as multiple sets of on-site configuration data for the construction project. These include: the image acquisition device array is composed of each image acquisition device, the audio acquisition device array is composed of each audio acquisition device, the light intensity sensor array is composed of each light intensity sensor, and the vibration sensor array is composed of each vibration sensor.

[0112] For example, the number of each image acquisition device in the image acquisition device array, the number of each audio acquisition device in the audio acquisition device array, the number of each light illuminance sensor in the light illuminance sensor array, and the number of each vibration sensor in the vibration sensor array are not exactly equal.

[0113] For example, each building floor of a construction project can be equipped with more than one image acquisition device, more than one audio acquisition device, more than one light intensity sensor, and more than one vibration sensor.

[0114] Among them, the analysis of the three-dimensional coordinate data of the location of all devices in various device arrays is used as the location information of each device in the construction project. At the same time, the land area, total number of registered construction personnel, building height and single-floor area of ​​the construction project are used as multiple sets of on-site configuration data for the construction project. It also includes the three-dimensional coordinate data of the location of all devices in various device arrays, namely the three-dimensional coordinate data of the location of each image acquisition device, the three-dimensional coordinate data of the location of each audio acquisition device, the three-dimensional coordinate data of the location of each light illuminance sensor and the three-dimensional coordinate data of the location of each vibration sensor.

[0115] The database includes sets of project site images, audio signal amplitudes, light intensity, and real-time vibration data that are synchronously collected at evenly spaced intervals within the current time interval. The database management system for the construction project includes: a single set of project site images synchronously collected at each time point within the current time interval, which is composed of frames of project site images synchronously collected by each image acquisition device at each time point within the current time interval; and a single set of audio signal amplitudes synchronously collected at each time point within the current time interval, which is composed of audio signal amplitudes synchronously collected by each audio acquisition device at each time point within the current time interval.

[0116] For example, the current time is 4:00 PM, the current time interval is from 4:00 PM to 4:20 PM, and the evenly spaced time positions within the current time interval are 4:00 PM, 4:02 PM, 4:04 PM, 4:06 PM, 4:08 PM, 4:10 PM, 4:12 PM, 4:14 PM, 4:16 PM, and 4:18 PM, for a total of 10 time positions;

[0117] The database management system, which stores data on project site images, audio signal amplitudes, light intensity, and real-time vibration data collected synchronously at evenly spaced intervals within the current time interval, also includes: a single light intensity set collected synchronously at each time point within the current time interval, composed of light intensity data collected synchronously by various light intensity sensors at each time point within the current time interval; and a single real-time vibration data set collected synchronously at each time point within the current time interval, composed of real-time vibration data collected synchronously by various vibration sensors at each time point within the current time interval.

[0118] And, optionally, within the above embodiments, in the AI ​​IoT project site multimodal data acquisition system:

[0119] The AI-based status assessment model is based on a reference pattern corresponding to the safety helmet, 3D point cloud data of the residential building, the duration of each time interval, the content stored in the database corresponding to the current time interval, the location information of various components of the construction project, and multiple sets of on-site configuration data. It intelligently assesses the proportion of personnel wearing safety helmets at the construction site within the current time interval and the maximum noise level (in decibels) experienced by the residential building within the current time interval. The model inputs the reference pattern corresponding to the safety helmet, 3D point cloud data of the residential building, the duration of each time interval, the content stored in the database corresponding to the current time interval, the location information of various components of the construction project, and multiple sets of on-site configuration data in parallel into the AI ​​status assessment model.

[0120] For example, editable logic devices can be used to implement the reference pattern corresponding to the safety helmet body, set the three-dimensional point cloud data of the residential building body, the occupation duration of each time interval, the content to be entered into the database corresponding to the current time interval, set the location information of various devices in the construction project, and input multiple sets of on-site configuration data into the AI ​​status identification model in parallel.

[0121] The AI-based status assessment model, based on the reference pattern corresponding to the safety helmet, the 3D point cloud data of the residential building, the duration of each time interval, the content stored in the database corresponding to the current time interval, the location information of various components of the construction project, and multiple sets of on-site configuration data, intelligently assesses the proportion of personnel wearing safety helmets at the construction site and the maximum noise level experienced by the residential building during the current time interval. This also includes running the AI ​​status assessment model to obtain the output of the AI ​​status assessment model: the proportion of personnel wearing safety helmets at the construction site and the maximum noise level experienced by the residential building during the current time interval.

[0122] In addition, the following are set as follows: the proportion of personnel wearing safety helmets at the construction site within the current time interval; the maximum noise decibel value that the residential building will experience within the current time interval; the reference pattern corresponding to the safety helmet; the three-dimensional point cloud data of the residential building; the duration of each time interval; the content to be entered into the database corresponding to the current time interval; the location information of various components of the construction project; and multiple sets of on-site configuration data are all represented in binary numerical form.

[0123] Sixth Embodiment

[0124] Figure 7 The present invention provides a flowchart illustrating the steps of a multimodal data acquisition method for an AI IoT project site, as shown in the sixth embodiment of the present invention.

[0125] like Figure 7 As shown, the method for acquiring multimodal data on-site in AI IoT projects includes the following steps:

[0126] An input acquisition interface is used to connect the image acquisition device array, audio acquisition device array, light illuminance sensor array, and vibration sensor array to the Internet of Things for synchronous data acquisition at various time points within the current time interval. This interface is used to input the data into the database management system of the specified construction project.

[0127] Specifically, the database management system for the construction project includes Oracle database management system, MySQL database management system, or Microsoft SQL Server database management system;

[0128] For example, the current time is 4:00 PM, the current time interval is from 4:00 PM to 4:20 PM, and the evenly spaced time positions within the current time interval are 4:00 PM, 4:02 PM, 4:04 PM, 4:06 PM, 4:08 PM, 4:10 PM, 4:12 PM, 4:14 PM, 4:16 PM, and 4:18 PM, for a total of 10 time positions;

[0129] Analyze and set the location information of various components and multiple sets of on-site configuration data for the construction project;

[0130] For example, a first parsing device can be selected to parse the location information of various components in the set construction project, while a second parsing device can be selected to parse multiple sets of on-site configuration data for the set construction project.

[0131] Perform various training actions on the convolutional neural network to obtain an AI state identification model;

[0132] Specifically, the MATLAB toolbox can be used to simulate and test the modeling process of performing various training actions on the convolutional neural network to obtain the AI ​​state identification model. In fact, the obtained AI state identification model is the convolutional neural network after performing each training action.

[0133] An outbound data acquisition interface, connected to the database management system of the designated construction project, is used to retrieve inbound data corresponding to the current time interval.

[0134] Specifically, the content to be stored in the database corresponding to the current time interval is the collection of project site images, audio signal amplitudes, light intensity, and real-time vibration data, which are collected synchronously at various time points at even intervals within the current time interval.

[0135] The AI ​​status identification model is based on the reference pattern corresponding to the safety helmet, the three-dimensional point cloud data of the residential building, the occupation duration of each time interval, the content entered into the database corresponding to the current time interval, the location information of various components of the construction project, and multiple sets of on-site configuration data. It can intelligently identify the proportion of personnel wearing safety helmets on the construction site of the current time interval and the maximum noise decibel value that the residential building can withstand in the current time interval.

[0136] Obviously, as can be seen from this invention, the acquired state data is state data within a time interval, rather than state data at a single moment in the prior art. The two types of state data acquired simultaneously are two unconventional types of state data, and the two types of state data are acquired simultaneously through synchronous intelligent identification.

[0137] Among them, the analysis of the location information of various components and multiple sets of on-site configuration data of the construction project includes: analyzing the three-dimensional coordinate data of the location of all components in various component arrays as the location information of various components in the construction project, and using the land area, total number of registered construction personnel, building height and single-floor area of ​​the construction project as multiple sets of on-site configuration data of the construction project.

[0138] Specifically, the three-dimensional coordinate data of the location of all devices in various device arrays and the three-dimensional point cloud data of the previously set residential building are all based on the same three-dimensional coordinate system, such as a three-dimensional coordinate system with the midpoint of the geographical area occupied by the three-dimensional point cloud data of the set residential building as the origin.

[0139] The content to be stored in the current time interval includes the collection of project site images, audio signal amplitudes, light intensity, and real-time vibration data, which are collected synchronously at various time points at even intervals within the current time interval.

[0140] Among them, the number of time positions at even intervals within the current time interval is proportional to the total number of registered construction personnel for the set construction project, and the current time interval starts from the current time, and the duration of each time interval is equal to the preset time length.

[0141] For example, if the current time is 4:00 PM and the current time interval is from 4:00 PM to 4:20 PM, then the duration of each time interval, i.e. the preset time length, is 20 minutes.

[0142] Among them, performing various training actions on the convolutional neural network to obtain the AI ​​state identification model includes: performing various training actions on the convolutional neural network to obtain the convolutional neural network after performing various training actions as the AI ​​state identification model, and the number of training actions is positively correlated with the land area of ​​the set construction project.

[0143] For example, a positive correlation between the number of training actions and the area of ​​the set construction project includes: setting the area of ​​the construction project to be 5,000 square meters and selecting 500 training actions; setting the area of ​​the construction project to be 7,000 square meters and selecting 700 training actions; setting the area of ​​the construction project to be 10,000 square meters and selecting 1,000 training actions; setting the area of ​​the construction project to be 20,000 square meters and selecting 2,000 training actions, and so on.

[0144] The process of performing various training actions on the convolutional neural network to obtain the AI ​​state identification model also includes: the convolutional neural network used has a single input layer, multiple convolutional layers, multiple pooling layers and a single fully connected layer. The number of image acquisition devices in the image acquisition device array, the number of audio acquisition devices in the audio acquisition device array, the number of light illuminance sensors in the light illuminance sensor array and the number of vibration sensors in the vibration sensor array are accumulated to obtain the total number of data acquisition devices for the set construction project. The number of multiple convolutional layers is positively correlated with the total number of data acquisition devices for the set construction project, and multiple convolutional layers all use the Sigmoid function as the activation function.

[0145] For example, the number of multiple convolutional layers is positively correlated with the total number of data acquisition devices in the set construction project as follows: if the set construction project has 20 data acquisition devices, the number of multiple convolutional layers selected is 2; if the set construction project has 30 data acquisition devices, the number of multiple convolutional layers selected is 3; if the set construction project has 40 data acquisition devices, the number of multiple convolutional layers selected is 4; if the set construction project has 50 data acquisition devices, the number of multiple convolutional layers selected is 5, and so on.

[0146] In each training action performed on the convolutional neural network, the proportion of personnel wearing safety helmets at the construction site during a known past time interval and the maximum noise level experienced by the residential building during the same past time interval are used as two outputs of the convolutional neural network. The reference pattern corresponding to the safety helmet, the 3D point cloud data of the residential building, the duration of each time interval, the data stored in the database corresponding to the past time interval, the location information of various components in the construction project, and multiple sets of on-site configuration data are used as inputs of the convolutional neural network to complete the training action of the convolutional neural network.

[0147] Furthermore, in the AI ​​IoT project site multimodal data acquisition system and method according to the present invention:

[0148] The number of time positions at uniform intervals within the current time interval is proportional to the total number of registered construction workers for the set construction project, and the current time interval starts at the current time, and the duration of each time interval is equal to the preset time length. This includes: using a first data mapping function to represent the data mapping relationship between the number of time positions at uniform intervals within the current time interval and the total number of registered construction workers for the set construction project.

[0149] Among them, the convolutional neural network is trained through each training action to obtain the convolutional neural network after each training action is completed as an AI state identification model. The positive correlation between the number of training actions and the land area of ​​the set construction project includes: using a second data mapping function to represent the data mapping relationship between the number of training actions and the land area of ​​the set construction project.

[0150] For example, different ASIC devices can be used to test and simulate the data mapping process of the first data mapping function and the second data mapping function, respectively.

[0151] While only exemplary embodiments of the invention have been described in detail above, those skilled in the art will immediately understand that many variations can be made in these exemplary embodiments without substantially departing from the novel principles and advantages of the invention. Accordingly, all such variations are to be included within the scope of the invention as defined in the following claims. In the claims, the means and function clauses are intended to include the structures described herein as implementing the stated functions, and include not only structural equivalents but also equivalent structures.

Claims

1. A multimodal data acquisition system for AI IoT project sites, characterized in that, The system includes: The data acquisition interface is connected via IoT to the image acquisition device array, audio acquisition device array, light illuminance sensor array, and vibration sensor array that perform synchronous data acquisition on-site within the designated construction project. It is used to store the sets of project site images, audio signal amplitude sets, light illuminance sets, and real-time vibration data sets that are synchronously acquired at uniform intervals at various times within the current time interval. The data to be stored is the database management system of the designated construction project. Configure analytical devices to analyze the location information of various devices and multiple sets of on-site configuration data for the construction project. The target is to construct a device for performing various training actions on a convolutional neural network to obtain an AI state identification model. The outbound data acquisition interface connects to the database management system of the designated construction project and is used to retrieve the inbound content corresponding to the current time interval. The intelligent identification device is connected to the outbound acquisition interface, the configuration analysis device, and the target construction device, respectively. It is used to adopt the AI ​​state identification model based on the reference pattern corresponding to the safety helmet, the three-dimensional point cloud data of the residential building, the occupation duration of each time interval, the content entered into the database corresponding to the current time interval, the location information of various devices in the construction project, and multiple sets of on-site configuration data. It intelligently identifies the proportion of personnel wearing safety helmets at the construction site of the set construction project within the current time interval and the maximum noise decibel value that the residential building is subjected to within the current time interval.

2. The multimodal data acquisition system based on AI IoT project site as described in claim 1, characterized in that: The analysis of the location information of various components and multiple sets of on-site configuration data for the construction project includes: analyzing the three-dimensional coordinate data of the location of all components in various component arrays as the location information of various components in the construction project; and using the land area, total number of registered construction personnel, building height and single-floor area of ​​the construction project as multiple sets of on-site configuration data for the construction project. The content to be stored in the current time interval includes the collection of project site images, audio signal amplitudes, light intensity, and real-time vibration data, which are collected synchronously at various time points at even intervals within the current time interval. The number of time positions at even intervals within the current time interval is proportional to the total number of registered construction workers for the set construction project. The current time interval starts at the current time, and the duration of each time interval is equal to the preset time length.

3. The multimodal data acquisition system based on AI IoT project site as described in claim 2, characterized in that: The process of performing various training actions on a convolutional neural network to obtain an AI state identification model includes: performing various training actions on the convolutional neural network to obtain the convolutional neural network after performing various training actions as an AI state identification model, and the number of training actions is positively correlated with the land area of ​​the set construction project. The process of performing various training actions on the convolutional neural network to obtain the AI ​​state identification model also includes: the convolutional neural network used has a single input layer, multiple convolutional layers, multiple pooling layers and a single fully connected layer. The number of image acquisition devices in the image acquisition device array, the number of audio acquisition devices in the audio acquisition device array, the number of light illuminance sensors in the light illuminance sensor array and the number of vibration sensors in the vibration sensor array are accumulated to obtain the total number of data acquisition devices for the set construction project. The number of multiple convolutional layers is positively correlated with the total number of data acquisition devices for the set construction project, and multiple convolutional layers all use the Sigmoid function as the activation function. In each training action of the convolutional neural network, the proportion of personnel wearing safety helmets at the construction site during a known past time interval and the maximum noise level experienced by the residential building during the same past time interval are used as the two outputs of the convolutional neural network. The reference pattern corresponding to the safety helmet, the 3D point cloud data of the residential building, the duration of each time interval, the data stored in the database corresponding to the past time interval, the location information of various components in the construction project, and multiple sets of on-site configuration data are used as the inputs of the convolutional neural network to complete this training action.

4. The multimodal data acquisition system for AI IoT project sites as described in claim 3, characterized in that, The system also includes: The strategy-customized device, connected to the intelligent identification device, is used to shut down all construction devices associated with fall accidents in the construction project in the next time interval if the proportion of personnel wearing safety helmets at the construction site is less than or equal to a preset proportion limit within the current time interval. The strategy customization device is also used to maintain the operation of each building device associated with a fall accident in the next time interval of the set construction project when the proportion of people wearing safety helmets at the construction site is greater than a preset limit within the current time interval.

5. The multimodal data acquisition system for AI IoT project sites as described in claim 3, characterized in that, The system also includes: The status response device, connected to the intelligent identification device, is used to determine the number of building devices whose rated output noise amplitude exceeds the limit in the next time interval of the set construction project when the maximum noise decibel value received by the set residential building in the current time interval is higher than or equal to the set decibel threshold, based on the difference between the maximum noise decibel value and the set decibel threshold. The status response device is also used to maintain the operation of building components whose rated output noise amplitude exceeds the limit within the set construction project when the maximum noise decibel value received by the set residential building in the current time interval is lower than the set decibel threshold. The determination of the number of building components whose rated output noise amplitude exceeds the limit in the next time interval of the set construction project, based on the difference between the maximum noise decibel value and the set decibel threshold, includes the following: the greater the difference between the maximum noise decibel value and the set decibel threshold, the more building components whose rated output noise amplitude exceeds the limit in the next time interval of the set construction project will be determined to be shut down.

6. The multimodal data acquisition system for AI IoT project sites as described in claim 3, characterized in that, The system also includes: The real-time display panel, located in the control room of the set construction project, is connected to the intelligent identification device to receive the proportion of people wearing safety helmets at the set construction project site within the current time interval, as well as the maximum noise decibel value experienced by the set residential building within the current time interval. The real-time display panel is also used to simultaneously display the proportion of personnel wearing safety helmets at the construction site within the current time interval, as well as the maximum noise level experienced by the residential building within the current time interval.

7. The multimodal data acquisition system for AI IoT project sites as described in claim 3, characterized in that, The system also includes: The timing server, connected to the data acquisition interface, provides timing reference signals for the data acquisition of various sets of project site images, audio signal amplitudes, light intensity, and real-time vibration data collected synchronously at uniform intervals within the current time interval. The timing server, connected to the data acquisition interface, is used to provide timing reference signals for the data acquisition of various project site image sets, audio signal amplitude sets, light intensity sets, and real-time vibration data sets that are synchronously acquired at uniform intervals within the current time interval. The timing server has a built-in quartz oscillator unit for generating clock pulse signals required to generate the timing reference signals.

8. The multimodal data acquisition system for AI IoT project sites as described in any one of claims 3-7, characterized in that: The three-dimensional coordinate data of the location of all devices in various device arrays are analyzed to serve as the location information of each device in the construction project. At the same time, the land area, total number of registered construction personnel, building height and single-floor area of ​​the construction project are used as multiple sets of on-site configuration data for the construction project. These include: the image acquisition device array is composed of each image acquisition device, the audio acquisition device array is composed of each audio acquisition device, the light intensity sensor array is composed of each light intensity sensor, and the vibration sensor array is composed of each vibration sensor. Among them, the analysis of the three-dimensional coordinate data of the location of all devices in various device arrays is used as the location information of each device in the construction project. At the same time, the land area, total number of registered construction personnel, building height and single-floor area of ​​the construction project are used as multiple sets of on-site configuration data for the construction project. It also includes the three-dimensional coordinate data of the location of all devices in various device arrays, namely the three-dimensional coordinate data of the location of each image acquisition device, the three-dimensional coordinate data of the location of each audio acquisition device, the three-dimensional coordinate data of the location of each light illuminance sensor and the three-dimensional coordinate data of the location of each vibration sensor. The database includes sets of project site images, audio signal amplitudes, light intensity, and real-time vibration data that are synchronously collected at evenly spaced intervals within the current time interval. The database management system for the construction project includes: a single set of project site images synchronously collected at each time point within the current time interval, which is composed of frames of project site images synchronously collected by each image acquisition device at each time point within the current time interval; and a single set of audio signal amplitudes synchronously collected at each time point within the current time interval, which is composed of audio signal amplitudes synchronously collected by each audio acquisition device at each time point within the current time interval. The database management system, which stores data on project site images, audio signal amplitudes, light intensity, and real-time vibration data collected synchronously at evenly spaced intervals within the current time interval, also includes: a single light intensity set collected synchronously at each time point within the current time interval, composed of light intensity data collected synchronously by various light intensity sensors at each time point within the current time interval; and a single real-time vibration data set collected synchronously at each time point within the current time interval, composed of real-time vibration data collected synchronously by various vibration sensors at each time point within the current time interval.

9. The multimodal data acquisition system for AI IoT project sites as described in any one of claims 3-7, characterized in that: The AI-based status assessment model is based on a reference pattern corresponding to the safety helmet, 3D point cloud data of the residential building, the duration of each time interval, the content stored in the database corresponding to the current time interval, the location information of various components of the construction project, and multiple sets of on-site configuration data. It intelligently assesses the proportion of personnel wearing safety helmets at the construction site within the current time interval and the maximum noise level (in decibels) experienced by the residential building within the current time interval. The model inputs the reference pattern corresponding to the safety helmet, 3D point cloud data of the residential building, the duration of each time interval, the content stored in the database corresponding to the current time interval, the location information of various components of the construction project, and multiple sets of on-site configuration data in parallel into the AI ​​status assessment model. The AI-based status assessment model, based on the reference pattern corresponding to the safety helmet, the 3D point cloud data of the residential building, the duration of each time interval, the content stored in the database corresponding to the current time interval, the location information of various components of the construction project, and multiple sets of on-site configuration data, intelligently assesses the proportion of personnel wearing safety helmets at the construction site and the maximum noise level experienced by the residential building during the current time interval. This also includes running the AI ​​status assessment model to obtain the output of the AI ​​status assessment model: the proportion of personnel wearing safety helmets at the construction site and the maximum noise level experienced by the residential building during the current time interval. Among them, the proportion of personnel wearing safety helmets at the construction site within the current time interval, the maximum noise decibel value that the residential building will experience within the current time interval, the reference pattern corresponding to the safety helmet, the three-dimensional point cloud data of the residential building, the duration of each time interval, the content to be entered into the database corresponding to the current time interval, the location information of various components of the construction project, and multiple sets of on-site configuration data are all represented in binary numerical form.

10. A method for acquiring multimodal data at the site of an AI IoT project, characterized in that, The method includes: An input acquisition interface is used to connect the image acquisition device array, audio acquisition device array, light illuminance sensor array, and vibration sensor array to the Internet of Things for synchronous data acquisition at various time points within the current time interval. This interface is used to input the data into the database management system of the specified construction project. Analyze and set the location information of various components and multiple sets of on-site configuration data for the construction project; Perform various training actions on the convolutional neural network to obtain an AI state identification model; An outbound data acquisition interface connected to the database management system of the designated construction project is used to retrieve the inbound content corresponding to the current time interval. The AI-based status assessment model uses a baseline pattern corresponding to the safety helmet, three-dimensional point cloud data of the residential building, the duration of each time interval, the content stored in the database corresponding to the current time interval, the location information of various components of the construction project, and multiple sets of on-site configuration data to intelligently assess the proportion of personnel wearing safety helmets at the construction site within the current time interval, as well as the maximum noise level that the residential building experiences within the current time interval.

Images