Knowledge-data dual-drive-based construction personnel quantity statistical method and system

Abstract

The invention relates to the field of computer vision and deep learning, and discloses a knowledge-data dual-drive-based constructor quantity statistical method and system, and the method comprises the steps: synchronously obtaining visible light image data and thermal imaging data of a same scene through a multi-modal sensing device; carrying out normalization, denoising and segmentation processing on the sum, and dividing to obtain superpixel regions; according to the obtained superpixel region, constructing a hyperedge and a weight thereof to obtain a hypergraph; learning multi-modal features of the hypergraph by using a hypergraph convolutional network; the superpixel importance weight is calculated through an attention mechanism, and the number of people is calculated and counted by using a density estimation function. Visible light images and thermal imaging data are fused, and a high-order correlation model is constructed by using a hypergraph technology, so that high-precision and robust counting of the number of crowds in a complex scene is realized.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and deep learning, specifically to a method and system for statistical analysis of the number of construction workers based on a dual knowledge-data driven approach. Background Technology

[0002] Crowd counting is a key task in computer vision, widely used in public safety monitoring, traffic management, construction management, and other fields. Traditional crowd counting methods typically rely on single-modal data, such as visible light images or videos. However, in environments with insufficient lighting, severe occlusion, or complex backgrounds, these methods often struggle to achieve accurate counts. In recent years, thermal imaging technology has been gradually introduced to compensate for the limitations of visible light data in low-light or occluded environments. However, using thermal imaging data alone can still be affected by noise and resolution limitations.

[0003] To overcome these shortcomings, multimodal data fusion technology has been proposed, combining visible light images and thermal imaging data to improve the accuracy of crowd detection and counting. However, most existing multimodal fusion methods employ simple feature stitching or weighted fusion strategies, failing to fully utilize the deep-seated correlation information between the two modalities, resulting in counting errors when processing complex scenes. Furthermore, how to efficiently extract and fuse complementary information from different modal data, especially in the presence of occlusion, low contrast, and noise interference, remains a challenge that urgently needs to be addressed.

[0004] Hypergraph technology, as a modeling tool capable of capturing high-order interactions between multimodal data, has stronger expressive power than traditional graph models. Hypergraphs can be used to establish complex relationships between features of multiple modalities, thereby achieving more comprehensive feature fusion. However, no research has yet applied hypergraph technology to multimodal population counting tasks. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a method for statistically analyzing the number of construction workers based on a knowledge-data dual-drive approach, comprising the following steps: Step 1: Simultaneously acquire visible light image data of the same scene using multimodal sensing devices. and thermal imaging data ; Step two, for visible light image data of the same scene and thermal imaging data After normalization, denoising, and segmentation, the resulting data is divided into... Superpixel region ; Step 3, based on the obtained superpixel region Constructing hyperedges and their weights To obtain a hypergraph ; Step 4: Learn the multimodal features of the hypergraph using a hypergraph convolutional network. ; Step 5: Calculate using the attention mechanism As superpixel importance weights, and using the density estimation function The number of people was calculated and statistically obtained. .

[0006] Furthermore, the visible light image data of the same scene... and thermal imaging data After normalization, denoising, and segmentation, the resulting data is divided into... Superpixel region ,include: Visible light image data and thermal imaging data Noise was removed by normalization and adaptive median filtering, and then a superpixel segmentation algorithm was used. Noise-removed and normalized visible light image data and thermal imaging data Divided into Superpixel region .

[0007] Furthermore, the obtained superpixel region... Constructing hyperedges and their weights To obtain a hypergraph ,include: Let the hypergraph be: The node set Corresponding to the superpixel region, the superedge set This represents a subset of superpixels that have similar color, texture, or thermal distribution characteristics. For the superedge weight set; Super Edge and their weights for: in, Let be the number of nodes in the hyperedge. and Superpixel regions and eigenvectors; calculate the Laplacian matrix of the hypergraph: ,in Let be the degree matrix of the hypergraph. This is the hyperedge weight matrix.

[0008] Furthermore, the method of learning multimodal features of a hypergraph using a hypergraph convolutional network is described. ,include: The calculation method for each hypergraph convolution layer is as follows: in, For the first The input feature matrix of the layer, and These are the learnable weights and bias parameters, respectively. It is a non-linear activation function. The Laplacian matrix of the hypergraph is used to obtain the fused multimodal features after passing through multiple hypergraph convolutions. .

[0009] Furthermore, the aforementioned calculation via attention mechanism As superpixel importance weights, and using the density estimation function The number of people was calculated and statistically obtained. ,include: An attention mechanism is used to weight the importance of superpixel regions, and the weight of each superpixel is calculated. : in, For learnable parameters, Superpixel region The fusion features are based on the density estimation function. Calculate the population density in each superpixel region and obtain the population count by weighted summation. : .

[0010] The construction worker quantity statistics system based on knowledge-data dual-drive applies the aforementioned construction worker quantity statistics method based on knowledge-data dual-drive, including a data acquisition unit, a data preprocessing unit, a hypergraph construction unit, a feature extraction and fusion unit, a crowd counting unit, a result optimization unit, and a data processing unit. The data acquisition unit, data preprocessing unit, hypergraph construction unit, feature extraction and fusion unit, crowd counting unit, and result optimization unit are all connected to the data processing unit. The data acquisition unit is used to acquire visible light images and thermal imaging data; The data preprocessing unit is used to normalize and remove noise from the collected data, and to divide the input data into multiple superpixel regions. The hypergraph construction unit is used to construct a hypergraph based on the superpixel segmentation results and to calculate the hypergraph Laplacian matrix; The feature extraction and fusion unit is used to extract and fuse multimodal features based on a hypergraph convolutional network; The crowd counting unit is used to perform regression based on the fusion features using an attention mechanism to obtain preliminary crowd information.

[0011] The beneficial effects of this invention are: This invention proposes a method for counting the number of construction workers based on a knowledge-data dual-drive approach, which can make full use of the complementary information between visible light images and thermal imaging modalities to construct a high-order correlation model of multimodal data, thereby achieving high-precision and robust personnel counting. Attached Figure Description

[0012] Figure 1 This is a flowchart illustrating a knowledge- and data-driven method for statistically analyzing the number of construction workers. Figure 2 This is a schematic diagram of an embodiment of the construction worker quantity statistics method based on knowledge-data dual-drive of the present invention; Figure 3 (a) represents visible light images and thermal imaging data acquired synchronously through a multimodal sensing device; where (a) is a visible light image and (b) is thermal imaging data. Figure 4 Feature maps of unfused visible light images and thermal imaging data; Figure 5 This is the feature map after knowledge-data dual-driven correlation modeling. Detailed Implementation

[0013] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings, but the scope of protection of the present invention is not limited to the following description.

[0014] The features and performance of the present invention will be further described in detail below with reference to embodiments.

[0015] like Figure 1 As shown, the knowledge-data dual-driven method for counting the number of construction workers includes the following steps: Step 1: Simultaneously acquire visible light image data of the same scene using multimodal sensing devices. and thermal imaging data ; Step two, for visible light image data of the same scene and thermal imaging data After normalization, denoising, and segmentation, the resulting data is divided into... Superpixel region ; Step 3, based on the obtained superpixel region Constructing hyperedges and their weights To obtain a hypergraph ; Step 4: Learn the multimodal features of the hypergraph using a hypergraph convolutional network. ; Step 5: Calculate using the attention mechanism As superpixel importance weights, and using the density estimation function The number of people was calculated and statistically obtained. .

[0016] The visible light image data of the same scene and thermal imaging data After normalization, denoising, and segmentation, the resulting data is divided into... Superpixel region ,include: Visible light image data and thermal imaging data Noise was removed by normalization and adaptive median filtering, and then a superpixel segmentation algorithm was used. Noise-removed and normalized visible light image data and thermal imaging data Divided into Superpixel region .

[0017] The obtained superpixel region Constructing hyperedges and their weights To obtain a hypergraph ,include: Let the hypergraph be: The node set Corresponding to the superpixel region, the superedge set This represents a subset of superpixels that have similar color, texture, or thermal distribution characteristics. For the superedge weight set; Super Edge and their weights for: in, Let be the number of nodes in the hyperedge. and Superpixel regions and eigenvectors; calculate the Laplacian matrix of the hypergraph: ,in Let be the degree matrix of the hypergraph. This is the hyperedge weight matrix.

[0018] The method described above utilizes hypergraph convolutional networks to learn multimodal features of hypergraphs. ,include: The calculation method for each hypergraph convolution layer is as follows: in, For the first The input feature matrix of the layer, and These are the learnable weights and bias parameters, respectively. It is a non-linear activation function. The Laplacian matrix of the hypergraph is used to obtain the fused multimodal features after passing through multiple hypergraph convolutions. .

[0019] The aforementioned calculation via attention mechanism As superpixel importance weights, and using the density estimation function The number of people was calculated and statistically obtained. ,include: An attention mechanism is used to weight the importance of superpixel regions, and the weight of each superpixel is calculated. : in, For learnable parameters, Superpixel region The fusion features are based on the density estimation function. Calculate the population density in each superpixel region and obtain the population count by weighted summation. : .

[0020] The construction worker quantity statistics system based on knowledge-data dual-drive applies the aforementioned construction worker quantity statistics method based on knowledge-data dual-drive, including a data acquisition unit, a data preprocessing unit, a hypergraph construction unit, a feature extraction and fusion unit, a crowd counting unit, a result optimization unit, and a data processing unit. The data acquisition unit, data preprocessing unit, hypergraph construction unit, feature extraction and fusion unit, crowd counting unit, and result optimization unit are all connected to the data processing unit. The data acquisition unit is used to acquire visible light images and thermal imaging data; The data preprocessing unit is used to normalize and remove noise from the collected data, and to divide the input data into multiple superpixel regions. The hypergraph construction unit is used to construct a hypergraph based on the superpixel segmentation results and to calculate the hypergraph Laplacian matrix; The feature extraction and fusion unit is used to extract and fuse multimodal features based on a hypergraph convolutional network; The crowd counting unit is used to perform regression based on the fusion features using an attention mechanism to obtain preliminary crowd information.

[0021] Specifically, such as Figure 2 As shown, this embodiment provides a knowledge-data dual-driven method for counting the number of construction workers, used to construct a high-order correlation model between visible light images and thermal imaging data, achieving high-precision and robust counting of the number of construction workers, including the following steps: Step 1: Acquire visible light image and thermal imaging data As input; First, visible light images are acquired synchronously using a multimodal sensing device. (like Figure 3 (a) shown) and thermal imaging data (like Figure 3 (b) shows that visible light images provide rich color and texture information, while thermal imaging data can detect human thermal radiation signals in low-light and occluded environments. The combination of the two modalities provides more comprehensive information support for crowd counting in complex scenes.

[0022] Step 2: Normalize, denoise, and segment the data; Acquired visible light images and thermal imaging data After normalization, the pixel values ​​are scaled to... Within a certain range, to eliminate dimensional differences between different data sources. Subsequently, Gaussian filtering or median filtering is used to denoise the data, reducing background noise interference. Next, a superpixel segmentation algorithm is employed. Divide the input image into Superpixel region Each superpixel region represents a locally consistent image patch, which facilitates subsequent feature extraction and processing.

[0023] Step 3: Hypergraph construction, which divides the input data into superpixel regions. Constructing hyperedges and their weights To reflect the similarity of features in local areas; Based on superpixel segmentation, a supergraph is constructed. ,in: This represents the set of nodes corresponding to a superpixel region. Let be a set of superedges, and the superedges Connecting superpixel regions with similar characteristics reflects the higher-order correlation between them; Let be the set of weights for the hyperedges, and let the hyperedge weights be... Calculated using the following formula: in, This indicates the number of nodes contained in the superedge. and These represent superpixel regions. and The feature vectors. The superstructure construction step can effectively capture the complex relationships in multimodal data. Preliminary image features are as follows: Figure 4 As shown in (a), the thermal imaging features are as follows Figure 4 As shown in (b).

[0024] Step 4: Feature extraction and fusion, using hypergraph convolution. Learning multimodal features ; The calculation method for each hypergraph convolution layer is as follows: in, For the first The input feature matrix of the layer, and These are the learnable weights and bias parameters, respectively. It is a non-linear activation function. The Laplacian matrix of the hypergraph is used to obtain the fused multimodal features after passing through multiple hypergraph convolutions. This effectively combines the advantages of visible light images and thermal imaging data. The fused features include... Figure 5 As shown.

[0025] Step 5: Crowd count regression to calculate using attention mechanisms. As superpixel importance weights, and using the density estimation function Calculate the final number of people: An attention mechanism is used to weight the importance of superpixel regions, and the weight of each superpixel is calculated. : in, For learnable parameters, Superpixel region The fusion features. Next, based on the density estimation function... Calculate the crowd density in each superpixel region to obtain the final crowd count. CWe obtain the result through weighted summation: This example also provides a construction worker quantity statistics system based on a knowledge-data dual-drive approach, including: a data acquisition unit, a data preprocessing unit, a hypergraph construction unit, a feature extraction and fusion unit, a crowd counting unit, and a result optimization unit; its key feature is: The data acquisition unit is used to acquire visible light images and thermal imaging data; The data preprocessing unit is used to normalize and remove noise from the collected data, and to divide the input data into multiple superpixel regions; The hypergraph construction unit is used to construct a hypergraph based on the superpixel segmentation results and to calculate the hypergraph Laplacian matrix; The feature extraction and fusion unit is used to extract and fuse multimodal features based on hypergraph convolutional networks; The crowd counting unit is used to perform regression based on the fusion features using an attention mechanism to obtain preliminary crowd information. This embodiment provides a construction worker quantity statistics system based on knowledge and data dual-drive, which can be used to implement a construction worker quantity statistics method based on knowledge and data dual-drive.

[0026] The above description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the concept described herein through the above teachings or related technologies or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A method for statistically analyzing the number of construction workers based on a knowledge-data dual-drive approach, characterized in that: Includes the following steps: Step 1: Simultaneously acquire visible light image data of the same scene using multimodal sensing devices. and thermal imaging data ; Step two, for visible light image data of the same scene and thermal imaging data After normalization, denoising, and segmentation, the resulting data is divided into... Superpixel region ; Step 3, based on the obtained superpixel region Constructing hyperedges and their weights To obtain a hypergraph ; Step 4: Learn the multimodal features of the hypergraph using a hypergraph convolutional network. ; Step 5: Calculate using the attention mechanism As superpixel importance weights, and using the density estimation function The number of people was calculated and statistically obtained. .

2. The method for statistically analyzing the number of construction workers based on a knowledge-data dual-drive approach as described in claim 1, characterized in that, The visible light image data of the same scene and thermal imaging data After normalization, denoising, and segmentation, the resulting data is divided into... Superpixel region ,include: Visible light image data and thermal imaging data Noise was removed by normalization and adaptive median filtering, and then a superpixel segmentation algorithm was used. Noise-removed and normalized visible light image data and thermal imaging data Divided into Superpixel region .

3. The method for statistically analyzing the number of construction workers based on a knowledge-data dual-drive approach according to claim 1, characterized in that, The obtained superpixel region Constructing hyperedges and their weights To obtain a hypergraph ,include: Let the hypergraph be: The node set Corresponding to the superpixel region, the superedge set This represents a subset of superpixels that have similar color, texture, or thermal distribution characteristics. For the superedge weight set; Super Edge and their weights for: in, Let be the number of nodes in the hyperedge. and Superpixel regions and eigenvectors; calculate the Laplacian matrix of the hypergraph: ,in Let be the degree matrix of the hypergraph. This is the hyperedge weight matrix.

4. The method for counting the number of construction workers based on a knowledge-data dual-drive approach according to claim 1, characterized in that, The method described above utilizes hypergraph convolutional networks to learn multimodal features of hypergraphs. ,include: The calculation method for each hypergraph convolution layer is as follows: in, For the first The input feature matrix of the layer, and These are the learnable weights and bias parameters, respectively. It is a non-linear activation function. The Laplacian matrix of the hypergraph is used to obtain the fused multimodal features after passing through multiple hypergraph convolutions. .

5. The method for counting the number of construction workers based on a knowledge-data dual-drive approach according to claim 1, characterized in that, The aforementioned calculation via attention mechanism As superpixel importance weights, and using the density estimation function The number of people was calculated and statistically obtained. ,include: An attention mechanism is used to weight the importance of superpixel regions, and the weight of each superpixel is calculated. : in, For learnable parameters, Superpixel region The fusion features are based on the density estimation function. Calculate the population density in each superpixel region and obtain the population count by weighted summation. : 。 6. A construction worker quantity statistics system based on knowledge-data dual-drive, characterized in that, The method for counting the number of construction workers based on knowledge-data dual-drive as described in any one of claims 1-5 includes a data acquisition unit, a data preprocessing unit, a hypergraph construction unit, a feature extraction and fusion unit, a crowd counting unit, a result optimization unit, and a data processing unit. The data acquisition unit, data preprocessing unit, hypergraph construction unit, feature extraction and fusion unit, crowd counting unit, and result optimization unit are all connected to the data processing unit. The data acquisition unit is used to acquire visible light images and thermal imaging data; The data preprocessing unit is used to normalize and remove noise from the collected data, and to divide the input data into multiple superpixel regions. The hypergraph construction unit is used to construct a hypergraph based on the superpixel segmentation results and to calculate the hypergraph Laplacian matrix; The feature extraction and fusion unit is used to extract and fuse multimodal features based on a hypergraph convolutional network; The crowd counting unit is used to perform regression based on the fusion features using an attention mechanism to obtain preliminary crowd information.

Images