Irregular curtain wall panel recognition method and device based on saliency detection network
Through the method based on channel-space combined significant detection network, the problem of difficulty in identifying irregular curtain wall panels in the prior art is solved, and the accurate identification and positioning of customized panels is achieved, with good applicability and recognition accuracy.
Patent Information
- Application Number
- CN202310179247.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-02-27
- Publication Date
- 2025-04-29
- Estimated Expiration
- 2043-02-27
AI Technical Summary
The existing mainstream intelligent recognition algorithms are difficult to achieve accurate identification of irregular curtain wall panels, especially the shape of the board actually used in the construction site is different from the board data sets trained by the network, and it goes beyond the learning and characterization ability of neural networks.
Using the identification method based on channel-space combined significance detection network, RGB images and depth maps are collected by depth cameras, cross-modal significance detection data sets are constructed, and detection networks with channel-through complete branches and space-undercomplete branches are designed, and the disclosed significance data sets are used for training to obtain optimal parameters.
The accurate identification and positioning of constantly changing customized non-regular curtain wall panels has been achieved, and the production of a large number of multi-type curtain wall panel data sets has been avoided. The recognition accuracy is not affected by the changes in the board appearance, and it has good applicability.
Smart Images
Figure CN116310808B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of building curtain wall installation, and relates to a method and a device for identifying irregular curtain wall panels based on a channel-space joint saliency detection network. Background Art
[0002] Curtain wall panels have become a comprehensive carrier of materials, craftsmanship and aesthetics. Due to the continuous enrichment and maturity of architectural design and panel forming technology, highly independent multi-type curtain wall panels have also sprung up. Customized irregular curtain wall panels have become a new trend in large-scale building design and construction.
[0003] However, in the face of the ever-changing development of curtain wall panels, it is difficult for existing mainstream intelligent recognition algorithms to achieve accurate recognition, such as segmentation and detection algorithms based on deep learning. First of all, the obvious difficulty is that this type of algorithm needs to collect a large number of curtain wall panel images of various types and make corresponding annotations to form a data set. Secondly, since mainstream panel manufacturing uses customized solutions, this method of achieving target recognition based on large amounts of data training is not applicable. Specifically, the shape of the panels actually used on the construction site is likely to be different from the panel data set trained by the network, which exceeds the learning representation ability of the neural network.
[0004] In summary, there are relatively few studies on the automatic identification of curtain wall panels, and the existing mainstream intelligent identification methods cannot be directly applied to curtain wall panels. This is a key issue that needs to be urgently solved in the field of building automation. Summary of the invention
[0005] In view of this, the object of the present invention is to provide a method and device for identifying irregular curtain wall panels based on a channel-space joint saliency detection network. Salient object detection (SOD) can identify visually salient target areas in an image. Since the training of saliency networks often uses fixed public data, the problem of collecting and producing curtain wall panel training data sets is avoided. In addition, since the accuracy of the saliency detection algorithm is mainly related to the saliency relationship of the target, it has little to do with changes in the appearance characteristics of the target object. Therefore, the method of the present invention can effectively achieve accurate identification and positioning for customized irregular curtain wall panels that are constantly evolving.
[0006] In order to achieve the above object, the present invention provides the following technical solutions:
[0007] Solution 1:
[0008] A method for identifying irregular curtain wall panels based on a saliency detection network, characterized in that the method specifically comprises the following steps:
[0009] S1: Use a depth camera to collect RGB images and depth maps of the curtain wall panels in the target area;
[0010] S2: Create a cross-modal saliency detection dataset for curtain wall panels;
[0011] S3: Construct a channel-space joint saliency detection network for accurate identification and positioning of irregular curtain wall panels;
[0012] S4: Use three public saliency datasets to build a training set for the detection network, and use this training set to input the channel-space joint saliency detection network for training to obtain the optimal parameters;
[0013] S5: Use the trained network and parameters to predict the saliency image of the curtain wall panel.
[0014] Furthermore, in step S2, a black and white segmentation label is used to make a label map of the data set; in the depth map, the closer the scene is to the observer, the more it tends to be black, and the farther the scene is from the observer, the more it tends to be white; in the label map, the significant plate is white, and the non-significant background is black.
[0015] Further, in step S3, the constructed channel-space joint saliency detection network includes a channel overcomplete branch and a space undercomplete branch; the channel overcomplete branch includes a channel overcomplete architecture and an SC module; the space undercomplete branch includes a space undercomplete architecture and an MC module; the channel overcomplete architecture is used to capture low-level features of curtain wall panels; the space undercomplete architecture is used to extract high-level features; the SC module is used to correct the spatial divergence of multi-scale features; the MC module is used to correct the modal divergence of cross-modal features; wherein SC represents spatial correction and MC represents modal correction.
[0016] Further, in step S3, the channel overcomplete architecture includes a channel slicing (CS) operation, an anti-channel slicing (ACS) operation and an expanded channel residual (ECR) block; the CS operation is to split the input features in space and then sequentially concatenate them in the channel; the ACS operation is the reverse operation of the channel slicing operation; the ECR block maps the input features to a higher channel dimension, uses deep convolution to independently extract features from each channel, and fuses and restores the channel features.
[0017] Further, in step S3, the CS operation specifically includes: considering the principle of central symmetry of the salient object, the input feature map is split into four equal-sized parts in a "cross" shape; then the four split features are spliced clockwise in the channel dimension to ensure that the channel features are arranged from small to large according to the center distance; each time the channel slicing operation is performed, the height and width of the input feature are halved, and the number of channels is enlarged by 4 times;
[0018] The ACS operation specifically includes: splitting the feature into four parts from the channel dimension and splicing them in the spatial dimension, which is the reverse operation of the channel slicing operation; each time through the reverse channel slicing operation, the input feature height and width are enlarged by 2 times, and the number of channels is reduced to 1 / 4 of the original;
[0019] The ECR block specifically includes: first, using 1×1×1 convolution to map the input features to a size of C×i, where C represents the number of input channels and i represents the expansion ratio value; the channel expansion ratio value i corresponds to a given set of input ( or ) and output Secondly, a 3×3×3 deep convolution is used to extract features from each channel respectively; a 1×1×1 convolution is used to fuse the extracted channel features and map them to the same size as the input features; finally, the fused channel features are added pixel by pixel with the input features through a residual link and output;
[0020] The outputs of the four ECR blocks in the encoder of the channel overcomplete branch are expressed as Their sizes are {1, 1 / 2, 1 / 4, 1 / 8} of the input image respectively; the outputs of the four ECR blocks in the decoder are expressed as It corresponds to the encoder having the same resolution; similarly, the outputs of the encoder and decoder of the spatial incomplete branch are expressed as and Specifically, the mathematical expression of the channel overcomplete branch is:
[0021]
[0022]
[0023] Among them, F represents the joint input of RGB image and depth image, ECR i (·) represents the dilated channel residual block, SC i (·) represents the spatial correction module, represents a convolution with kernel size k×j×j; S(·) and S -1 (·) denote channel slicing and anti-channel slicing operations, respectively.
[0024] Further, in step S3, the spatially incomplete architecture includes an encoder based on ConvNeXt and a basic decoding (BD) block based on a non-expanded channel residual block; wherein ConvNeXt adopts a 3D version of the Tiny model architecture to ensure basic performance and minimum parameters; the basic decoding (BD) block is a non-expanded version of the expanded channel residual block, and the expansion ratio value is 1.
[0025] The mathematical expression of the space incomplete branch is:
[0026]
[0027]
[0028] Where F represents the joint input of RGB image and depth image, σ g (·) and δ l (·) denotes GELU activation function and layer normalization, CN i (·) represents the ConvNeXt block, δ d (·) indicates the downsampling operation using the maximum pooling layer, MC i (·) represents the modal correction module, Represents a basic decoding block.
[0029] Furthermore, in step S3, four SC modules with shared weights are used in the channel-space joint saliency detection network to realize spatial correction of multi-scale features. Each SC module uses a 3×7×7 three-dimensional large convolution and a Sigmoid activation function to generate a weight map from high-level features from the spatial undercomplete architecture; trilinear interpolation and an appropriate number of channel slicing operations are used to convert the weight map into a feature vector that matches the feature dimension in the channel overcomplete architecture; finally, pixel-by-pixel addition and multiplication operations are used to realize the fusion of high-level features and low-level features.
[0030] The output of the SC module is expressed as Specifically, for a set of input and The mathematical expression of the SC module is:
[0031]
[0032]
[0033] Among them, σ s (·) represents the Sigmoid function, M is the attention weight map, ⊙ represents the matrix dot product, S j (·) represents j-times channel slicing operation, δ u (·) represents the upsampling operation using trilinear interpolation, δ l (·) indicates layer normalization.
[0034] Further, in step S3, four MC modules with shared weights are used in the channel-space joint saliency detection network; in each MC module, the input features from the channel over-complete architecture are first restored to the dimension matching the features in the spatial incomplete architecture through an appropriate number of inverse channel slicing operations and upsampling operations; then, the features are split into a feature V1 biased towards RGB and a feature V2 biased towards depth using a splitting operation in the visual dimension; the feature V1 and the feature V2 are subtracted pixel by pixel, and their absolute values are taken to obtain a modal difference feature; the modal difference feature is compensated to the feature V1 and the feature V2 to ensure the original information distribution of the two features; finally, the two compensated features are spliced in the visual dimension to obtain a feature V3; two 1×1×1 convolutions are used to fuse the features of the channel dimension, and are added pixel by pixel with the features from the spatial incomplete architecture.
[0035] The output of the MC module is expressed as Specifically, for a set of input and The mathematical expression of the modal correction module is:
[0036]
[0037] V3=c v (|V1-V2|+V1,|V1-V2|+V2) (8)
[0038]
[0039] Among them, V1 and V2 represent two feature vectors in the visual dimension, V3 represents the correction vector; S j - 1 (·) represents j times of reverse channel slicing operations; s v (·) and c v (·) represent the splitting and connection in the visual dimension, σ g (·) represents the GELU activation function.
[0040] Further, in step S4, the training set of the network includes 1485 images in the NJUD dataset, 700 images in the NLPR dataset, and 800 images in the DUT dataset. The method for obtaining the optimal parameters is: using the maximum F-measure, the maximum E-measure, the structure-measure, the mean absolute error (MAE), the PR (Precision-Recall) curve and the F-measure curve to comprehensively evaluate the trained results, and selecting the result with the best indicator evaluation as the optimal result.
[0041] Option 2:
[0042] A device for identifying irregular curtain wall panels based on a saliency detection network, comprising:
[0043] The image acquisition module uses a depth camera to collect RGB images and depth maps of curtain wall panels for network prediction.
[0044] Data processing and network training module, which is used to create a cross-modal curtain wall panel dataset and set a saliency training dataset, and to design and train a channel-space joint saliency detection network.
[0045] Operation prediction module for testing curtain wall panel dataset on channel-space joint based saliency detection network.
[0046] The data display module uses a small-sized display screen to display the color image, depth image, and saliency prediction image of the object area.
[0047] The beneficial effects of the present invention are as follows: the present invention designs a new channel-space joint-based saliency detection network for the recognition of irregular curtain wall panels, which can avoid the production of a large number of multi-type curtain wall panel data sets, while achieving accurate recognition and positioning of curtain wall panels, and the accuracy of the method is not affected by the shape of the developing and changing irregular curtain wall panels, has good applicability, and is easy to promote and apply.
[0048] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:
[0050] Figure 1 It is an overall flow chart of the irregular curtain wall panel recognition method based on the channel-space joint saliency detection network of the present invention;
[0051] Figure 2 The overall structure diagram of the channel-spatial feature combined saliency detection network of the present invention;
[0052] Figure 3 This is a diagram illustrating the operation of channel slicing (CS) in the present invention;
[0053] Figure 4 FIG. 1 is a block structure diagram of the extended channel residual (ECR) in the present invention;
[0054] Figure 5It is a structural diagram of the spatial correction (SC) module and the modal correction (MC) module in the present invention;
[0055] Figure 6 It is a block diagram of the curtain wall panel identification device of the present invention. DETAILED DESCRIPTION
[0056] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.
[0057] Embodiment 1:
[0058] This embodiment provides a method for identifying irregular curtain wall panels based on a channel-space joint saliency detection network. Figure 1 As shown, the curtain wall panel recognition method is as follows: Figure 1 As shown, the specific steps include:
[0059] Step 1: Use the depth camera to collect the RGB image and depth map of the curtain wall panels in the target area. Panel types include: aluminum panels, wood panels, ceramic panels, color steel panels, glass panels, stainless steel panels, stone panels, and polycarbonate panels.
[0060] Step 2: Use the labelme tool to annotate the collected curtain wall panel data and construct a test dataset. The label map of the cross-modal saliency detection dataset of curtain wall panels uses black and white segmentation labels. In the depth map, the closer the scene is to the observer, the more it tends to be black, and the farther the scene is from the observer, the more it tends to be white. In the label map, the salient panels are white and the non-salient background is black.
[0061] Step 3: Design a channel-space joint saliency detection network for accurate recognition and positioning of irregular curtain wall panels. The detailed structure is as follows: Figure 2As shown in the figure. The network consists of a channel overcomplete architecture, a spatial undercomplete architecture, a spatial correction (SC) module, and a modal correction (MC) module. The channel overcomplete architecture uses channel slicing (CS) operations, anti-channel slicing (ACS) operations, and dilated channel residual (ECR) blocks to capture low-level features of curtain wall panels; the spatial undercomplete architecture uses a ConvNeXt-based encoder and a basic decoding (BD) block based on a non-dilated ECR block to extract high-level features; the SC module uses a 3×7×7 three-dimensional large convolution with shared weights to correct the spatial divergence of multi-scale features; the MC module uses the absolute value of the modal difference to compensate for the two modalities, thereby correcting the modal divergence of cross-modal features.
[0062] like Figure 3 The following is an illustration of the channel slice (CS) operation. Considering the principle of central symmetry of salient objects, the input feature map is split into four equal-sized parts in a "cross" shape; then the four split features are spliced clockwise in the channel dimension to ensure that the channel features are arranged from small to large according to the center distance.
[0063] like Figure 4 As shown in Figure 1, the structure of the ECR block is shown. First, a 1×1×1 convolution is used to map the input features to a size of C×i, where C represents the number of input channels and i represents the expansion ratio value; the channel expansion ratio value i corresponds to a given set of input (( or ) and output Secondly, a 3×3×3 deep convolution is used to extract features from each channel respectively; a 1×1×1 convolution is used to fuse the extracted channel features and map them to the same size as the input features; finally, the fused channel features are added pixel by pixel with the input features through a residual link and output.
[0064] like Figure 5The figure shows the structure of the spatial correction (SC) module and the modal correction (MC) module. Four SC modules with shared weights are used in the network to realize the spatial correction of multi-scale features; each module uses a 3×7×7 three-dimensional large convolution and a Sigmoid activation function to generate a weight map from high-level features from the spatially incomplete architecture; trilinear interpolation and a suitable number of channel slicing operations are used to convert the weight map into a feature vector that matches the feature dimension in the channel overcomplete architecture; finally, pixel-by-pixel addition and multiplication operations are used to achieve the fusion of high-level features with low-level features. Four MC modules with shared weights are used in the network to achieve modal correction. In each module, first, an appropriate number of inverse channel slicing operations and upsampling operations are performed to restore the input features from the channel overcomplete architecture to a dimension that matches the features in the spatial incomplete architecture. The features are split into feature V1 that is biased towards RGB and feature V2 that is biased towards depth using a split operation in the visual dimension. Feature V1 and feature V2 are subtracted pixel by pixel and their absolute values are taken to obtain the modal difference feature. The modal difference feature is compensated to feature V1 and feature V2 to ensure the original information distribution of the two features. The two compensated features are concatenated in the visual dimension to obtain feature V3. Two 1×1×1 convolutions are used to fuse the features of the channel dimension and are added pixel by pixel with the features from the spatial incomplete architecture.
[0065] Step 4: Use three public saliency datasets to form the training set of the proposed network, and use the training set to input the channel-space joint saliency detection network to obtain the optimal parameters. The training set of the network includes 1485 images in the NJUD dataset, 700 images in the NLPR dataset, and 800 images in the DUT dataset. We use the maximum F-measure, maximum E-measure, structure-measure, mean absolute error (MAE), and PR and F-measure curves to comprehensively evaluate the trained results, and select the result with the best indicator evaluation as the optimal result. RGB images and depth maps are spliced in one visual dimension as the input of the network. The network uses a mixed loss function BCELoss+DiceLoss. The network is trained using the AdamW optimizer and cosine decay strategy.
[0066] Step 5: Use the trained model and parameters to predict the curtain wall panel saliency image. The predicted image is a black and white image, with white representing the salient target panel and black representing the background. In the predicted scene, the panel is placed in the target area, and there is no interference such as large-scale occlusion that affects the salient properties of the panel.
[0067] Embodiment 2:
[0068] This embodiment provides a block diagram of an irregular curtain wall panel recognition device based on a channel-space joint saliency detection network. Figure 6As shown, the curtain wall panel identification device specifically includes:
[0069] Image acquisition module: Place the curtain wall panels in the target area and use the depth camera to collect the RGB image and depth map of the curtain wall panels for network prediction.
[0070] Data processing and network training module: Use this module to create a cross-modal curtain wall panel dataset and set a saliency training dataset. Design and train a channel-space joint saliency detection network based on the PyTorch framework.
[0071] Operation prediction module: used for testing curtain wall panel dataset based on channel-space joint saliency detection network.
[0072] Data display module: Use a small-size display screen to display the color image, depth image, and saliency prediction image of the object area.
[0073] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.
Claims
1. A method for identifying irregular curtain wall panels based on a saliency detection network, characterized in that: The method specifically comprises the following steps: S1: Use a depth camera to collect RGB images and depth maps of the curtain wall panels in the target area; S2: Create a cross-modal saliency detection dataset for curtain wall panels; S3: Construct a channel-space joint saliency detection network for accurate recognition and positioning of irregular curtain wall panels, including a channel overcomplete branch and a space undercomplete branch; the channel overcomplete branch includes a channel overcomplete architecture and an SC module; the space undercomplete branch includes a space undercomplete architecture and an MC module; the channel overcomplete architecture is used to capture low-level features of curtain wall panels; the space undercomplete architecture is used to extract high-level features; the SC module is used to correct the spatial divergence of multi-scale features; the MC module is used to correct the modal divergence of cross-modal features; wherein SC represents spatial correction and MC represents modal correction; The channel overcomplete architecture includes CS operation, ACS operation and ECR block; the CS operation is to split the input features in space and then splice them in order in the channel; the ACS operation is the reverse operation of the channel slicing operation; the ECR block is to map the input features to a higher channel dimension, use deep convolution to extract features from each channel independently, fuse and restore the channel features; where CS represents channel slicing, ACS represents anti-channel slicing, and ECR represents dilated channel residual; The spatially incomplete architecture includes a ConvNeXt-based encoder and a basic decoding block based on a non-expanded channel residual block; the mathematical expression of the spatially incomplete branch is: in, and represents the output of the encoder and decoder of the spatially incomplete branch; F represents the joint input of the RGB image and the depth map, σ g (·) and δ l (·) denotes GELU activation function and layer normalization, CN i (·) represents the ConvNeXt block, δ d (·) indicates the downsampling operation using the maximum pooling layer, MC i (·) represents the modal correction module, represents a basic decoding block; represents a convolution with a kernel size of k×j×j; Represents the output of the four ECR blocks in the decoder; S4: Use three public saliency datasets to build a training set for the detection network, and use the training set to input the channel-space joint saliency detection network for training to obtain the optimal parameters; S5: Use the trained network and parameters to predict the saliency image of the curtain wall panel.
2. The irregular curtain wall panel identification method according to claim 1, characterized in that: In step S2, the black and white segmentation labels used in the label map of the data set are produced; in the depth map, the closer the scene is to the observer, the more it tends to be black, and the farther the scene is from the observer, the more it tends to be white; in the label map, the significant plate is white, and the non-significant background is black.
3. The irregular curtain wall panel identification method according to claim 1, characterized in that: In step S3, the CS operation specifically includes: considering the principle of central symmetry of the salient object, the input feature map is split into four equal-sized parts in a "cross" shape; then the four split features are spliced clockwise in the channel dimension to ensure that the channel features are arranged from small to large according to the center distance; each time the channel slicing operation is performed, the height and width of the input feature are halved, and the number of channels is enlarged by 4 times; The ACS operation specifically includes: splitting the feature into four parts from the channel dimension and splicing them in the spatial dimension, which is the reverse operation of the channel slicing operation; each time through the reverse channel slicing operation, the input feature height and width are enlarged by 2 times, and the number of channels is reduced to 1 / 4 of the original; The ECR block specifically includes: first, using 1×1×1 convolution to map the input features to a size of C×i, where C represents the number of input channels and i represents the expansion ratio value; the channel expansion ratio value i corresponds to a given set of input and output Secondly, a 3×3×3 deep convolution is used to extract features from each channel respectively; a 1×1×1 convolution is used to fuse the extracted channel features and map them to the same size as the input features; finally, the fused channel features are added pixel by pixel with the input features through a residual link and output; The outputs of the four ECR blocks in the encoder of the channel overcomplete branch are expressed as Their sizes are {1, 1 / 2, 1 / 4, 1 / 8} of the input image respectively; the outputs of the four ECR blocks in the decoder are expressed as It corresponds to the encoder having the same resolution; the outputs of the encoder and decoder of the spatial incomplete branch are respectively expressed as and The mathematical expression of the channel overcomplete branch is: Among them, F represents the joint input of RGB image and depth image, ECR i (·) represents the dilated channel residual block, SC i (·) represents the spatial correction module, represents a convolution with kernel size k×j×j; S(·) and S -1 (·) denote channel slicing and anti-channel slicing operations, respectively.
4. The irregular curtain wall panel identification method according to claim 3, characterized in that: In step S3, four SC modules with shared weights are used in the channel-space joint saliency detection network, each of which uses a 3×7×7 three-dimensional large convolution and a Sigmoid activation function to generate a weight map from high-level features from the spatial undercomplete architecture; trilinear interpolation and an appropriate number of channel slicing operations are used to convert the weight map into a feature vector that matches the feature dimension in the channel overcomplete architecture; finally, pixel-by-pixel addition and multiplication operations are used to achieve the fusion of high-level features with low-level features.
5. The irregular curtain wall panel identification method according to claim 1, characterized in that: In step S3, four MC modules with shared weights are used in the channel-space joint saliency detection network; in each MC module, the input features from the channel over-complete architecture are first restored to the dimension matching the features in the spatial incomplete architecture through multiple inverse channel slicing operations and upsampling operations; then, the features are split into a feature V1 biased towards RGB and a feature V2 biased towards depth using a splitting operation in the visual dimension; the features V1 and V2 are subtracted pixel by pixel, and their absolute values are taken to obtain the modal difference feature; the modal difference feature is compensated to the features V1 and V2 to ensure the original information distribution of the two features; finally, the two compensated features are spliced in the visual dimension to obtain the feature V3; two 1×1×1 convolutions are used to fuse the features of the channel dimension, and are added pixel by pixel with the features from the spatial incomplete architecture.
6. The irregular curtain wall panel identification method according to claim 1, characterized in that: In step S4, the method for obtaining the optimal parameters is: using the maximum F-metric, the maximum E-metric, the structure-metric, the mean absolute error, the PR curve and the F-metric curve to comprehensively evaluate the training results, and selecting the result with the best indicator evaluation as the optimal result.
7. An irregular curtain wall panel recognition device based on a saliency detection network, characterized in that: The device includes: Image acquisition module, which uses a depth camera to collect RGB images and depth maps of curtain wall panels for network prediction; A data processing and network training module, using the module to create a cross-modal curtain wall panel data set and set a saliency training data set, and to design and train a channel-space joint saliency detection network as described in any one of claims 1 to 6; The computation prediction module is used to test the curtain wall panel dataset on the channel-space joint saliency detection network; The data display module uses a small-sized display screen to display the color image, depth image, and saliency prediction image of the object area.