Monocular panoramic vision three-dimensional reconstruction method for forest scene
Through monocular panoramic cameras and deep learning technology, the shallow and deep features of forest area images are extracted, and depth information is generated in combination with attention mechanisms, which realizes efficient three-dimensional reconstruction of forest area, solving the problems of high cost and complexity of traditional methods.
Patent Information
- Application Number
- CN202411188074.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-28
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-08-28
AI Technical Summary
Traditional forest area three-dimensional reconstruction methods rely on multi-eye vision systems or lidars, with high equipment costs, complex deployment and large data processing volume, making it difficult to apply in remote or difficult-to-reach forest areas.
A monocular panoramic camera is used to collect panoramic images of the forest area, and the combined perception of multi-level feature extraction, feature distribution gradient mask prominent device and attention mechanism are used to generate depth information decoded values to realize the reconstruction of the three-dimensional model.
It reduces the cost of three-dimensional reconstruction equipment, simplifies system deployment, improves the efficiency and accuracy of three-dimensional reconstruction in complex forest areas, and provides support for accurate monitoring of forestry resources and intelligent robot navigation.
Smart Images

Figure CN119027593A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of three-dimensional reconstruction of forest areas, and specifically to a monocular panoramic vision three-dimensional reconstruction method for forest scenes. Background Art
[0002] At present, my country's artificial forest area ranks first in the world. Actively developing forest information theory is an important way to implement the development concept. Improving the robot's perception and understanding of the forest environment, and on this basis, collecting forest resource information and intelligent robot navigation, positioning and operation target recognition are important directions for forest information theory research under the new normal. Three-dimensional reconstruction is a key technology for expressing objective objects in computers through relevant sensors. Its value lies in implicitly modeling the relative positions between spatial objects. For forestry robots, obtaining three-dimensional structural information of forest scenes is the key to improving robots' perception and understanding of the forest environment.
[0003] Traditional 3D reconstruction methods for forest areas mostly rely on multi-camera vision systems or laser radar (LiDAR) technology. Although these methods can provide high-precision 3D models, they often have limitations such as high equipment cost, complex deployment, and large data processing volume. Especially in remote or difficult-to-reach forest areas, their application is greatly restricted.
[0004] Therefore, a monocular panoramic vision 3D reconstruction method for forest scenes is desired. Summary of the invention
[0005] This application is made in consideration of the above problems. One purpose of this application is to provide a monocular panoramic vision 3D reconstruction method for forest scenes.
[0006] The embodiment of the present application provides a monocular panoramic vision 3D reconstruction method for forest scenes, which includes: Obtain a panoramic image of the forest area collected by a monocular panoramic camera; Performing multi-level feature extraction on the forest area panoramic image to obtain a forest area panoramic shallow feature map and a forest area panoramic deep semantic feature map; Inputting the forest area panoramic shallow feature map into a feature distribution gradient mask salient device to obtain a forest area panoramic shallow salient feature map; Generate a depth information decoding value of the forest area image based on the joint perception information of the forest area panoramic deep semantic feature map and the forest area panoramic shallow salient feature map; Based on the depth information decoding value, the forest area panoramic image is projected into a three-dimensional space to obtain a three-dimensional model of the forest area scene.
[0007] For example, according to the monocular panoramic vision 3D reconstruction method for forest scenes in an embodiment of the present application, multi-level feature extraction is performed on the forest panoramic image to obtain a forest panoramic shallow feature map and a forest panoramic deep semantic feature map, including: The forest area panoramic image is input into a panoramic image multi-scale feature extractor based on a pyramid network to obtain the forest area panoramic shallow feature map and the forest area panoramic deep semantic feature map.
[0008] For example, according to the monocular panoramic vision 3D reconstruction method for forest scenes in an embodiment of the present application, the forest panoramic shallow feature map is input into a feature distribution gradient mask salient device to obtain a forest panoramic shallow salient feature map, including: Calculating the multi-directional gradient value distribution of each position in the forest area panoramic shallow feature map, and determining the gradient amplitude value of each position in the forest area panoramic shallow feature map based on the multi-directional gradient value distribution of each position to obtain a forest area panoramic shallow feature gradient amplitude distribution map; Calculating the local description operator of the gradient amplitude at each position in the forest area panoramic shallow feature gradient amplitude distribution map to obtain a local significant distribution map of the forest area panoramic shallow feature gradient amplitude; Inputting the forest area panoramic shallow feature gradient amplitude local significant distribution map into a gated masker based on a GELU function to obtain a gradient amplitude local significant gated mask map; The forest area panoramic shallow salient feature map is obtained by calculating the point-by-point multiplication between the gradient amplitude local salient gated mask map and the forest area panoramic shallow feature map.
[0009] For example, according to the monocular panoramic vision 3D reconstruction method for forest scenes in an embodiment of the present application, the local description operator of the gradient amplitude at each position in the forest panoramic shallow feature gradient amplitude distribution map is calculated to obtain a local significant distribution map of the forest panoramic shallow feature gradient amplitude, including: Determine the scale of the local neighborhood, calculate the average of the difference between the gradient amplitude value of a predetermined position in the forest area panoramic shallow feature gradient amplitude distribution map and the gradient amplitude values of other positions in the local neighborhood to obtain a gradient amplitude local description operator corresponding to the predetermined position.
[0010] For example, according to the monocular panoramic vision 3D reconstruction method for forest scenes in an embodiment of the present application, the depth information decoding value of the forest image is generated based on the joint perception information of the forest panoramic deep semantic feature map and the forest panoramic shallow significant feature map, including: Performing joint perception on the forest area panoramic deep semantic feature map and the forest area panoramic shallow salient feature map to obtain a forest area panoramic shallow-deep salient joint perception feature map; The forest area panoramic shallow-deep significant joint perceptual feature map is input into a decoder-based depth estimation module to obtain the depth information decoding value.
[0011] For example, according to the monocular panoramic vision 3D reconstruction method for forest scenes in an embodiment of the present application, the forest panoramic deep semantic feature map and the forest panoramic shallow significant feature map are jointly perceived to obtain a forest panoramic shallow-deep significant joint perception feature map, including: The panoramic deep semantic feature map of the forest area and the panoramic shallow salient feature map of the forest area are input into a salient joint perception module based on an attention mechanism to obtain the panoramic shallow-deep salient joint perception feature map of the forest area.
[0012] For example, according to the monocular panoramic vision 3D reconstruction method for forest scenes in an embodiment of the present application, the forest panoramic deep semantic feature map and the forest panoramic shallow salient feature map are input into a salient joint perception module based on an attention mechanism to obtain the forest panoramic shallow-deep salient joint perception feature map, including: Performing feature shape reshaping on the forest area panoramic shallow salient feature map and the forest area panoramic deep semantic feature map to obtain a forest area panoramic shallow feature shape reshaping matrix and a forest area panoramic deep semantic feature shape reshaping matrix; Input the forest area panoramic shallow feature shape reshaping matrix and the forest area panoramic deep semantic feature shape reshaping matrix into the feature channel-by-channel interactive perception module to obtain the forest area panoramic shallow detail-deep semantic dependency matrix and the forest area panoramic deep semantic-shallow detail dependency matrix; Input the forest area panoramic shallow detail-deep semantic dependency matrix and the forest area panoramic deep semantic-shallow detail dependency matrix into a random dropout module to obtain a pruned forest area panoramic shallow detail-deep semantic dependency matrix and a pruned forest area panoramic deep semantic-shallow detail dependency matrix; Based on the pruned forest area panoramic shallow detail-deep semantic dependency matrix and the pruned forest area panoramic deep semantic-shallow detail dependency matrix, feature optimization is performed on the forest area panoramic deep semantic feature shape reshaping matrix and the forest area panoramic shallow feature shape reshaping matrix to obtain a dependency optimized forest area panoramic deep semantic feature matrix and a dependency optimized forest area panoramic shallow feature matrix; Reshaping the dependency relationship optimized forest area panoramic shallow feature matrix and the dependency relationship optimized forest area panoramic deep semantic feature matrix to obtain an optimized forest area panoramic shallow salient feature map and an optimized forest area panoramic deep semantic feature map; The weighted sum of the optimized forest area panoramic shallow salient feature map and the optimized forest area panoramic deep semantic feature map is calculated to obtain the forest area panoramic shallow-deep salient joint perceptual feature map.
[0013] For example, according to the monocular panoramic vision 3D reconstruction method for forest scenes in an embodiment of the present application, the forest panoramic shallow feature shape reshaping matrix and the forest panoramic deep semantic feature shape reshaping matrix are input into a feature channel-by-channel interactive perception module to obtain a forest panoramic shallow detail-deep semantic dependency matrix and a forest panoramic deep semantic-shallow detail dependency matrix, including: Calculate the forest area panoramic shallow feature shape reshaping matrix multiplied by the transposed matrix of the forest area panoramic deep semantic feature shape reshaping matrix to obtain the forest area panoramic shallow detail-deep semantic association representation matrix; The forest panorama shallow detail-deep semantic association representation matrix is divided by the scale of the forest panorama shallow feature shape reshaping matrix and then input into the softmax function to obtain the forest panorama shallow detail-deep semantic dependency relationship matrix; Calculate the transposed matrix of the forest area panoramic deep semantic feature shape reshaping matrix and multiply it by the forest area panoramic shallow feature shape reshaping matrix to obtain the forest area panoramic deep semantic-shallow detail association representation matrix; The forest area panoramic deep semantics-shallow detail association representation matrix is divided by the scale of the forest area panoramic shallow feature shape reshaping matrix and then input into the softmax function to obtain the forest area panoramic deep semantics-shallow detail dependency relationship matrix.
[0014] For example, according to the monocular panoramic vision 3D reconstruction method for forest scenes of an embodiment of the present application, based on the pruned forest panorama shallow detail-deep semantic dependency matrix and the pruned forest panorama deep semantic-shallow detail dependency matrix, the forest panorama deep semantic feature shape reshaping matrix and the forest panorama shallow feature shape reshaping matrix are feature optimized to obtain a dependency optimized forest panorama deep semantic feature matrix and a dependency optimized forest panorama shallow feature matrix, including: Performing matrix multiplication of the pruned forest area panoramic shallow detail-deep semantic dependency matrix and the forest area panoramic deep semantic feature shape reshaping matrix to obtain the dependency optimized forest area panoramic deep semantic feature matrix; The pruned forest area panoramic deep semantic-shallow detail dependency matrix is matrix-multiplied with the forest area panoramic shallow feature shape reshaping matrix to obtain the dependency-optimized forest area panoramic shallow feature matrix.
[0015] According to the monocular panoramic vision 3D reconstruction method for forest scenes of the embodiment of the present application, a monocular panoramic camera is used to collect panoramic images of the forest area, and image processing technology based on deep learning is used to analyze the panoramic images of the forest area, respectively extracting shallow detail features and deep semantic features of the panoramic images of the forest area, and then optimizing and fusing the shallow and deep features based on the bidirectional dependency between the shallow and deep features, so as to realize multi-level joint perception of the panoramic images of the forest area, thereby intelligently predicting the depth information of each position of the panoramic images of the forest area, and reconstructing the three-dimensional images thereof. In this way, the equipment cost of the three-dimensional reconstruction can be effectively reduced, the system deployment can be simplified, and the efficiency and accuracy of the three-dimensional reconstruction in the complex forest environment can be improved, providing strong support for the precise monitoring of forestry resources, intelligent robot navigation and operation. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings of the embodiments of the present application are briefly introduced below. Obviously, the drawings described below only relate to some embodiments of the present application, and are not intended to limit the present application.
[0017] Figure 1 A schematic diagram of the application architecture of a monocular panoramic vision 3D reconstruction method for forest scenes in an embodiment of the present application is shown; Figure 2 A flow chart of a monocular panoramic vision 3D reconstruction method for a forest scene in an embodiment of the present application is shown; Figure 3 A flowchart of sub-step S530 of the monocular panoramic vision 3D reconstruction method for forest scenes in an embodiment of the present application is shown; Figure 4 A flowchart of sub-step S540 of the monocular panoramic vision 3D reconstruction method for forest scenes in an embodiment of the present application is shown; Figure 5 A schematic diagram of the structure of a monocular panoramic vision 3D reconstruction system for forest scenes in an embodiment of the present application is shown; Figure 6 An application scenario diagram of a monocular panoramic vision 3D reconstruction method for a forest scene in an embodiment of the present application is shown; Figure 7 A flowchart showing a monocular panoramic vision 3D reconstruction method for a forest scene in another embodiment of the present application is shown; and Figure 8 A schematic diagram showing the acquisition of a panoramic depth map in a monocular panoramic vision 3D reconstruction method for a forest scene in another embodiment of the present application is shown. DETAILED DESCRIPTION
[0018] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work also fall within the scope of protection of the present application.
[0019] The terms used in this specification are those common terms currently widely used in the art in consideration of the functions of the present application, but these terms may vary according to the intentions of those skilled in the art, precedents, or new technologies in the art. In addition, specific terms may be selected, and in this case, their detailed meanings will be described in the detailed description of the present application. Therefore, the terms used in the specification should not be understood as simple names, but rather as a general description based on the meaning of the terms and the present application.
[0020] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.
[0021] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. At the same time, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.
[0022] Figure 1 A schematic diagram of the application architecture of a monocular panoramic vision 3D reconstruction method for forest scenes in an embodiment of the present application is shown, including a server 100 and a terminal device 200.
[0023] The terminal device 200 and the server 100 can be connected via the Internet to achieve mutual communication. Optionally, the above-mentioned Internet uses standard communication technology and / or protocol. The Internet is usually the Internet, but it can also be any network, including but not limited to any combination of a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a private network or a virtual private network. In some embodiments, the data exchanged through the network can be represented by technologies and / or formats including Hyper Text Markup Language (HTML), Extensible Markup Language (XML), etc. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec), etc. can be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above-mentioned data communication technologies.
[0024] The server 100 can provide various network services for the terminal device 200, wherein the server 100 can be a single server, a server cluster consisting of several servers, or a cloud computing center. Specifically, the server 100 may include a processor 110 (Center Processing Unit, CPU), a memory 120, an input device 130, and an output device 140, etc. The input device 130 may include a keyboard, a mouse, a touch screen, etc., and the output device 140 may include a display device, such as a liquid crystal display (Liquid Crystal Display, LCD), a cathode ray tube (Cathode Ray Tube, CRT), etc.
[0025] The memory 120 may include a read-only memory (ROM) and a random access memory (RAM), and provides the processor 110 with program instructions and data stored in the memory 120. In the embodiment of the present application, the memory 120 may be used to store the program of the monocular panoramic vision 3D reconstruction method for forest scenes in the embodiment of the present application.
[0026] The processor 110 calls the program instructions stored in the memory 120, and the processor 110 is used to execute the steps of any one of the monocular panoramic vision three-dimensional reconstruction methods for forest scenes in the embodiments of the present application according to the obtained program instructions.
[0027] In addition, the application architecture diagram in the embodiment of the present application is intended to more clearly illustrate the technical solution in the embodiment of the present application, and does not constitute a limitation on the technical solution provided in the embodiment of the present application. Of course, for other application architectures and business applications, the technical solution provided in the embodiment of the present application is also applicable to similar problems.
[0028] The following is a non-restrictive explanation of the monocular panoramic vision three-dimensional reconstruction method for forest scenes provided according to at least one embodiment of the present application through several examples or embodiments. As described below, different features in these specific examples or embodiments can be combined with each other without conflicting with each other to obtain new examples or embodiments, and these new examples or embodiments also fall within the scope of protection of the present application.
[0029] In response to the above technical problems, the technical concept of this application is to collect panoramic images of forest areas through a monocular panoramic camera, and use deep learning-based image processing technology to analyze the panoramic images of forest areas, respectively extract shallow detail features and deep semantic features of the panoramic images of forest areas, and then optimize and fuse the shallow and deep features based on the bidirectional dependency between the shallow and deep features, so as to achieve multi-level joint perception of the panoramic images of forest areas, and thus intelligently predict the depth information of each position of the panoramic images of forest areas, so as to reconstruct them in three dimensions. In this way, the equipment cost of three-dimensional reconstruction can be effectively reduced, the system deployment can be simplified, and the efficiency and accuracy of three-dimensional reconstruction in complex forest environments can be improved, providing strong support for the precise monitoring of forest resources, intelligent robot navigation and operations.
[0030] Based on this, Figure 2 The flowchart of the monocular panoramic vision 3D reconstruction method for forest scenes in an embodiment of the present application is shown. For example, the monocular panoramic vision 3D reconstruction method for forest scenes can be executed by a server, which can be Figure 1 The server 100 shown in FIG. Figure 2As shown, according to the embodiment of the present application, the monocular panoramic vision three-dimensional reconstruction method for forest scenes includes the following steps: S510, acquiring a panoramic image of a forest area collected by a monocular panoramic camera; S520, performing multi-level feature extraction on the panoramic image of the forest area to obtain a panoramic shallow feature map of the forest area and a panoramic deep semantic feature map of the forest area; S530, inputting the panoramic shallow feature map of the forest area into a feature distribution gradient mask salient device to obtain a panoramic shallow salient feature map of the forest area; S540, generating a depth information decoding value of the forest area image based on the joint perception information of the panoramic deep semantic feature map of the forest area and the panoramic shallow salient feature map of the forest area; S550, based on the depth information decoding value, projecting the panoramic image of the forest area into a three-dimensional space to obtain a three-dimensional model of the forest scene.
[0031] Specifically, in the technical solution of the present application, first, a panoramic image of the forest area captured by a monocular panoramic camera is obtained. It should be understood that traditional monocular or binocular cameras can only capture images with limited viewing angles, and multiple captured images need to be stitched together, which not only increases the computational complexity, but may also introduce stitching errors due to differences in lighting, viewing angles, etc. A monocular panoramic camera can capture all-round images of 360 degrees horizontally and a certain angle (such as 180 degrees) vertically, achieving panoramic coverage of the complex environment of the forest area, directly providing a continuous and seamless scene view, avoiding the image stitching process required after using multiple cameras or multiple shots, thereby reducing data redundancy and errors, and facilitating the subsequent three-dimensional reconstruction process.
[0032] Considering the large scale differences of objects such as trees and shrubs in the forest area, everything from tiny branches and leaves to the global spatial distribution of trees plays an important role in the three-dimensional reconstruction of the forest area. Therefore, in order to simultaneously consider the depth and breadth of feature extraction, in the technical solution of the present application, a panoramic image multi-scale feature extractor based on a pyramid network is used to process the panoramic image of the forest area. The multi-level feature extraction capability of the pyramid network is utilized to construct feature maps of different scales through layer-by-layer downsampling, effectively capturing the shallow detail information and deep semantic information of the panoramic image of the forest area, and generating a shallow feature map of the panoramic view of the forest area and a deep semantic feature map of the panoramic view of the forest area. Among them, the shallow feature map of the panoramic view of the forest area mainly focuses on the detailed information such as local texture and edges in the image, while the deep semantic feature map of the panoramic view of the forest area focuses more on the high-level semantic information such as the global structure of the image and object categories, providing a rich information basis for the prediction of forest depth information.
[0033] Correspondingly, in step S520, multi-level feature extraction is performed on the panoramic image of the forest area to obtain a shallow feature map of the panoramic image of the forest area and a deep semantic feature map of the panoramic image of the forest area, including: inputting the panoramic image of the forest area into a panoramic image multi-scale feature extractor based on a pyramid network to obtain the shallow feature map of the panoramic image of the forest area and the deep semantic feature map of the panoramic image of the forest area.
[0034] Next, in order to further highlight the key information such as the main objects, edge contours, texture details, etc. in the image, the present application further introduces a feature distribution gradient mask salient device to perform feature enhancement processing on the shallow feature map of the panoramic forest area, and analyzes the pixel change intensity in the shallow feature map of the panoramic forest area to enhance the expression of edge contours and detail areas in the feature map, and more clearly highlight the local details of the forest area. Specifically, the feature distribution gradient mask salient device first quantifies the local change intensity of the pixels at each position by calculating the gradient amplitude value at each position in the shallow feature map of the panoramic forest area. Then, the gradient amplitude local description operator is calculated by measuring the significance of the gradient amplitude value at each position relative to the gradient amplitude values at other positions in the local neighborhood, and the feature importance of each position is evaluated. Then, the GELU function is used to perform nonlinear activation on the gradient amplitude local description operator at each position to generate a gated mask to guide the enhancement of key information in the shallow feature map of the panoramic forest area. Finally, based on the generated gated mask, the shallow feature map of the forest area panorama is weighted element by element, so as to focus on the important visual clues in the image, remove background noise, irrelevant details and other information, and generate a shallow salient feature map of the forest area panorama.
[0035] Accordingly, in step S530, if Figure 3 As shown, the panoramic shallow feature map of the forest area is input into the feature distribution gradient mask salient device to obtain the panoramic shallow salient feature map of the forest area, including: S531, calculating the multi-directional gradient value distribution of each position in the panoramic shallow feature map of the forest area, and determining the gradient amplitude value of each position in the panoramic shallow feature map of the forest area based on the multi-directional gradient value distribution of each position to obtain the panoramic shallow feature gradient amplitude distribution map of the forest area; S532, calculating the gradient amplitude local description operator of each position in the panoramic shallow feature gradient amplitude distribution map of the forest area to obtain the panoramic shallow feature gradient amplitude local significant distribution map of the forest area; S533, inputting the panoramic shallow feature gradient amplitude local significant distribution map of the forest area into the gated masker based on the GELU function to obtain the gradient amplitude local significant gated mask map; S534, calculating the position point multiplication between the gradient amplitude local significant gated mask map and the panoramic shallow feature map of the forest area to obtain the panoramic shallow salient feature map of the forest area.
[0036] Among them, in step S532, the local description operator of the gradient amplitude of each position in the panoramic shallow feature gradient amplitude distribution map of the forest area is calculated to obtain the local significant distribution map of the panoramic shallow feature gradient amplitude of the forest area, including: determining the scale of the local neighborhood, calculating the mean of the difference between the gradient amplitude value of the predetermined position in the panoramic shallow feature gradient amplitude distribution map of the forest area and the gradient amplitude values of other positions in the local neighborhood to obtain the local description operator of the gradient amplitude corresponding to the predetermined position.
[0037] In one example, the forest area panoramic shallow feature map is input into a feature distribution gradient mask salient device to obtain a forest area panoramic shallow salient feature map, including: processing the forest area panoramic shallow feature map with the following feature distribution gradient mask formula to obtain the forest area panoramic shallow salient feature map, wherein the feature distribution gradient mask formula is: in, , , , , and They respectively represent the grayscale values of the corresponding positions of the shallow feature map of the forest panorama, In the shallow feature map of the forest area, The gradient value in the horizontal direction of the pixel point. In the shallow feature map of the forest area, The gradient value in the ordinate direction of the pixel point. The shallow characteristic map of the forest area is shown in The channel direction gradient value of the position pixel point, The first The gradient magnitude value at the position, The shallow characteristic gradient amplitude distribution map of the forest area is shown in FIG. The set of gradient magnitude values in the local neighborhood centered at position, represents the number of gradient magnitude values in the set of gradient magnitude values in the local neighborhood, , and Respectively represent the offset in the horizontal axis direction, the vertical axis direction and the channel direction, The first The local description operator of the gradient magnitude of the position, represents the GELU function, The first part of the gradient magnitude local significant gated mask map is represented by The local description operator of the gradient magnitude of the position, represents the local significant gated mask map of the gradient magnitude, It means point multiplication by position. A panoramic shallow-layer significant feature map of the forest area is shown.
[0038] Furthermore, since the forest area panoramic deep semantic feature map and the forest area panoramic shallow salient feature map are feature representations of different levels and focuses of the forest area panoramic image, there is a natural dependency and information complementarity between the two. Therefore, in order to fully understand the overall content and spatial contextual relationship in the forest area panoramic image, the present application introduces a significant joint perception module based on the attention mechanism to process the forest area panoramic deep semantic feature map and the forest area panoramic shallow salient feature map, and models the bidirectional dependency between the two through a dual interactive attention mechanism to achieve the optimized fusion of deep and shallow features. Specifically, first, the module first reshapes the forest area panoramic deep semantic feature map and the forest area panoramic shallow salient feature map into a feature matrix so that its feature dimension and shape are adapted to the input requirements of the attention mechanism. Then, the bidirectional dependency between the two sets of feature matrices is calculated through the feature channel-by-channel interactive perception module, the interaction between the features is quantified, and the shallow detail-deep semantic and deep semantic-shallow detail dependency matrices are generated, which serve as a guiding basis for the optimized fusion of feature attention. At the same time, the sensitivity to image noise is reduced and overfitting is prevented. The two sets of dependency matrices are further input into the random dropout module for pruning, and the generalization ability of the model is enhanced by selectively ignoring or suppressing some feature interactions. Then, based on the pruned two sets of dependency matrices, the two sets of feature matrices after shape reshaping are weighted optimized, and restored to the original feature map form through shape reshaping and weighted fusion to generate a shallow-deep significant joint perception feature map of the forest panorama.
[0039] Then, the forest area panoramic shallow-deep significant joint perception feature map is input into the decoder-based depth estimation module to obtain the depth information decoding value. In the technical solution of the present application, the decoder uses the trained and optimized network structure and parameters to perform multi-layer feature analysis and information recovery on the forest area panoramic shallow-deep significant joint perception feature map, gradually restores the feature size of the forest area panoramic shallow-deep significant joint perception feature map to the same size as the original image, and generates a corresponding depth prediction value at each pixel position, thereby accurately representing the three-dimensional depth information of each position in the forest area panoramic image.
[0040] Accordingly, in step S540, if Figure 4As shown, based on the joint perception information of the forest area panoramic deep semantic feature map and the forest area panoramic shallow salient feature map, a depth information decoding value of the forest area image is generated, including: S541, jointly perceiving the forest area panoramic deep semantic feature map and the forest area panoramic shallow salient feature map to obtain a forest area panoramic shallow-deep salient joint perception feature map; S542, inputting the forest area panoramic shallow-deep salient joint perception feature map into a decoder-based depth estimation module to obtain the depth information decoding value.
[0041] Among them, in step S541, the forest area panoramic deep semantic feature map and the forest area panoramic shallow salient feature map are jointly perceived to obtain the forest area panoramic shallow-deep salient joint perception feature map, including: inputting the forest area panoramic deep semantic feature map and the forest area panoramic shallow salient feature map into a salient joint perception module based on the attention mechanism to obtain the forest area panoramic shallow-deep salient joint perception feature map.
[0042] Specifically, the forest area panoramic deep semantic feature map and the forest area panoramic shallow salient feature map are input into a salient joint perception module based on an attention mechanism to obtain the forest area panoramic shallow-deep salient joint perception feature map, including: performing feature shape reshaping on the forest area panoramic shallow salient feature map and the forest area panoramic deep semantic feature map to obtain a forest area panoramic shallow feature shape reshaping matrix and a forest area panoramic deep semantic feature shape reshaping matrix; inputting the forest area panoramic shallow feature shape reshaping matrix and the forest area panoramic deep semantic feature shape reshaping matrix into a feature channel-by-channel interactive perception module to obtain a forest area panoramic shallow detail-deep semantic dependency matrix and a forest area panoramic deep semantic-shallow detail dependency matrix; inputting the forest area panoramic shallow detail-deep semantic dependency matrix and the forest area panoramic deep semantic-shallow detail dependency matrix into a random dropout module to obtain a pruned forest area panoramic shallow detail-deep semantic dependency matrix. A deep semantic dependency matrix and a pruned forest area panoramic deep semantic-shallow detail dependency matrix; based on the pruned forest area panoramic shallow detail-deep semantic dependency matrix and the pruned forest area panoramic deep semantic-shallow detail dependency matrix, feature optimization is performed on the forest area panoramic deep semantic feature shape reshaping matrix and the forest area panoramic shallow feature shape reshaping matrix to obtain a dependency optimized forest area panoramic deep semantic feature matrix and a dependency optimized forest area panoramic shallow feature matrix; feature shape reshaping is performed on the dependency optimized forest area panoramic shallow feature matrix and the dependency optimized forest area panoramic deep semantic feature matrix to obtain an optimized forest area panoramic shallow salient feature map and an optimized forest area panoramic deep semantic feature map; a weighted sum of the optimized forest area panoramic shallow salient feature map and the optimized forest area panoramic deep semantic feature map is calculated to obtain the forest area panoramic shallow-deep salient joint perceptual feature map.
[0043] Furthermore, the forest area panoramic shallow feature shape reshaping matrix and the forest area panoramic deep semantic feature shape reshaping matrix are input into the feature channel-by-channel interactive perception module to obtain the forest area panoramic shallow detail-deep semantic dependency matrix and the forest area panoramic deep semantic-shallow detail dependency matrix, including: calculating the forest area panoramic shallow feature shape reshaping matrix multiplied by the transposed matrix of the forest area panoramic deep semantic feature shape reshaping matrix to obtain the forest area panoramic shallow detail-deep semantic association representation matrix; dividing the forest area panoramic shallow detail-deep semantic association representation matrix by the forest area panoramic The scale of the shallow feature shape reshaping matrix is input into the softmax function to obtain the forest area panoramic shallow detail-deep semantic dependency matrix; the transposed matrix of the forest area panoramic deep semantic feature shape reshaping matrix is calculated and multiplied by the forest area panoramic shallow feature shape reshaping matrix to obtain the forest area panoramic deep semantic-shallow detail association representation matrix; the forest area panoramic deep semantic-shallow detail association representation matrix is divided by the scale of the forest area panoramic shallow feature shape reshaping matrix and then input into the softmax function to obtain the forest area panoramic deep semantic-shallow detail dependency matrix.
[0044] Furthermore, based on the pruned forest area panoramic shallow details-deep semantic dependency matrix and the pruned forest area panoramic deep semantics-shallow details dependency matrix, the forest area panoramic deep semantic feature shape reshaping matrix and the forest area panoramic shallow feature shape reshaping matrix are feature optimized to obtain a dependency optimized forest area panoramic deep semantic feature matrix and a dependency optimized forest area panoramic shallow feature matrix, including: matrix multiplying the pruned forest area panoramic shallow details-deep semantic dependency matrix with the forest area panoramic deep semantic feature shape reshaping matrix to obtain the dependency optimized forest area panoramic deep semantic feature matrix; matrix multiplying the pruned forest area panoramic deep semantics-shallow details dependency matrix with the forest area panoramic shallow feature shape reshaping matrix to obtain the dependency optimized forest area panoramic shallow feature matrix.
[0045] In a specific example, the forest area panoramic deep semantic feature map and the forest area panoramic shallow salient feature map are input into a salient joint perception module based on an attention mechanism to obtain the forest area panoramic shallow-deep salient joint perception feature map, including: processing the forest area panoramic shallow salient feature map and the forest area panoramic deep semantic feature map with the following joint perception formula to obtain the forest area panoramic shallow-deep salient joint perception feature map, wherein the joint perception formula is: in, A panoramic shallow-layer significant feature map of the forest area is shown, represents the panoramic deep semantic feature map of the forest area, represents the feature shape reshaping, Represents the shallow characteristic shape reshaping matrix of the forest panorama, Represents the shape reshaping matrix of the deep semantic features of the forest panorama, represents matrix multiplication, represents the transpose of a matrix, represents the scale of the forest area panoramic shallow feature shape reshaping matrix, that is, the width of the forest area panoramic shallow feature shape reshaping matrix multiplied by the height, and the scale of the forest area panoramic shallow feature shape reshaping matrix and the forest area panoramic deep semantic feature shape reshaping matrix are the same, is the normalized exponential function, represents the shallow detail-deep semantic dependency matrix of the forest panorama, represents the forest panorama deep semantics-shallow detail dependency matrix, represents random dropout processing, Represents the shallow detail-deep semantic dependency matrix of the pruned forest panorama, Represents the deep semantics-shallow details dependency matrix of the pruned forest panorama, Represents the dependency relationship to optimize the shallow feature matrix of the forest panorama, Representing dependency relationship to optimize the forest panorama deep semantic feature matrix, It represents the optimized shallow-layer significant feature map of the forest panorama. Represents the optimized forest panorama deep semantic feature map, and Represent different weight coefficients, Represents the shallow-deep joint significant perceptual feature map of the panoramic view of the forest area.
[0046] Then, based on the depth information decoded value, the panoramic image of the forest area is projected into a three-dimensional space to obtain a three-dimensional model of the forest area scene. That is, by combining the depth information decoded value with the two-dimensional pixel coordinates of the panoramic image of the forest area, the two-dimensional image pixels are matched with the corresponding depth information decoded value using a depth mapping algorithm to construct a three-dimensional point cloud model of the forest area scene, thereby effectively restoring the three-dimensional coordinates of each point in the forest environment, including trees, shrubs, ground and other terrain features, and realizing three-dimensional fine reconstruction of complex forest areas.
[0047] In a preferred example, the forest area panoramic shallow-deep significant joint perception feature map is input into a decoder-based depth estimation module to obtain the depth information decoding value, including the steps of: clustering all eigenvalues of the forest area panoramic shallow-deep significant joint perception feature map based on the L2 distance between the eigenvalues, and arranging the cluster features into a forest area panoramic shallow-deep significant joint perception cluster vector; determining the clustering ratio of the number of eigenvalues of the forest area panoramic shallow-deep significant joint perception cluster vector to the number of eigenvalues of the forest area panoramic shallow-deep significant joint perception feature map value; dividing the bi-norm of the shallow-deep significant joint perception clustering vector of the forest area panorama by the bi-norm of the shallow-deep significant joint perception feature vector of the forest area panorama obtained after the shallow-deep significant joint perception feature map of the forest area panorama is expanded to obtain the shallow-deep significant joint perception conflict representation value of the forest area panorama; dividing the first power value of the one-norm of the shallow-deep significant joint perception clustering vector of the forest area panorama with the cluster ratio value as the exponent by the second power value of the one-norm of the shallow-deep significant joint perception feature vector of the forest area panorama with the cluster ratio value as the exponent to obtain the forest area panorama shallow-deep significant joint perception confrontation representation value; for each eigenvalue of the forest area panoramic shallow-deep significant joint perception clustering vector, multiply it by the inverse of the difference between the forest area panoramic shallow-deep significant joint perception conflict representation value and the forest area panoramic shallow-deep significant joint perception confrontation representation value to obtain the optimized eigenvalue of the forest area panoramic shallow-deep significant joint perception clustering vector; for each eigenvalue outside the cluster in the forest area panoramic shallow-deep significant joint perception feature map, multiply it by the difference between the forest area panoramic shallow-deep significant joint perception conflict representation value and the forest area panoramic shallow-deep significant joint perception confrontation representation value The inverse of the sum of the forest area panoramic shallow-deep significant joint perceptual adversarial representation values is obtained to obtain the optimized out-of-class eigenvalue of the forest area panoramic shallow-deep significant joint perceptual feature map; the optimized eigenvalue of the forest area panoramic shallow-deep significant joint perceptual clustering vector and the optimized out-of-class eigenvalue of the forest area panoramic shallow-deep significant joint perceptual feature map are combined to form an optimized forest area panoramic shallow-deep significant joint perceptual feature map; the optimized forest area panoramic shallow-deep significant joint perceptual feature map is input into the decoder-based depth estimation module to obtain the depth information decoding value.
[0048] The optimization process of the shallow-deep significant joint perceptual feature map of the forest panorama is expressed as follows: in, is the shallow-deep joint salient perceptual feature vector of the forest panorama, is the number of eigenvalues of the shallow-deep joint significant perceptual eigenvector of the forest panorama, is the shallow-deep joint significant perceptual clustering vector of the forest panorama, is the number of eigenvalues of the shallow-deep significant joint perceptual clustering vector of the forest panorama, is the clustering ratio value, represents the clustering feature set corresponding to the shallow-deep significant joint perception clustering vector of the forest panorama, and Represents the binary norm and the uninorm of the vector respectively. Power, is each eigenvalue of the shallow-deep joint significant perceptual feature map of the forest panorama, It is each eigenvalue of the optimized forest area panoramic shallow-deep significant joint perceptual feature map.
[0049] That is, considering that the panoramic deep semantic feature map of the forest area and the panoramic shallow salient feature map of the forest area respectively represent the shallow features and deep semantic features of the image based on the feature distribution gradient mask saliency of the panoramic image of the forest area, when further performing the salient joint perception based on the attention mechanism, the difference in the shallow-deep fusion attention weights of the image will cause the image semantic salient joint alignment conflict, thereby causing the loss of aggregated key information and affecting the expression effect of the panoramic shallow-deep salient joint perception feature map of the forest area.
[0050] Therefore, in order to avoid the loss of key suffix semantic information relative to the original feature set as a whole due to aggregation conflict based on the aggregation features of the forest area panoramic shallow-deep significant joint perception feature map, the clustering ratio of the number of eigenvalues of the forest area panoramic shallow-deep significant joint perception clustering vector relative to the number of eigenvalues of the forest area panoramic shallow-deep significant joint perception feature map is used as a decision function to perform an adversarial decision on the set absolute representation of the norm of the forest area panoramic shallow-deep significant joint perception clustering vector and the forest area panoramic shallow-deep significant joint perception feature vector, and respectively compare it with the forest area panoramic shallow-deep significant joint perception clustering vector and the forest area panoramic shallow-deep significant joint perception feature vector. The clustering intrinsic conflict representation of the binary norm of the forest area panoramic shallow-deep significant joint perceptual feature vector is positively and negatively interacted to construct the optimized forest area panoramic shallow-deep significant joint perceptual feature map based on the aggregated features, which is a solid alignment fence with the original feature set as a whole, thereby achieving the intentional mitigation of harmful information loss based on the aggregated risk transferability of the optimized forest area panoramic shallow-deep significant joint perceptual feature map, improving the expression effect of the optimized forest area panoramic shallow-deep significant joint perceptual feature map, and thus improving the accuracy of the depth information decoding value obtained by inputting the forest area panoramic shallow-deep significant joint perceptual feature map into the decoder-based depth estimation module.
[0051] Based on the above embodiments, see Figure 5 As shown, it is a structural schematic diagram of a monocular panoramic vision 3D reconstruction system 800 for forest scenes in an embodiment of the present application. The monocular panoramic vision 3D reconstruction system 800 for forest scenes includes: an image acquisition module 810, which is used to acquire a panoramic image of the forest area collected by a monocular panoramic camera; a multi-level feature extraction module 820, which is used to perform multi-level feature extraction on the panoramic image of the forest area to obtain a panoramic shallow feature map of the forest area and a panoramic deep semantic feature map of the forest area; a mask calculation module 830, which is used to input the panoramic shallow feature map of the forest area into a feature distribution gradient mask salient device to obtain a panoramic shallow salient feature map of the forest area; a decoding value generation module 840, which is used to generate a depth information decoding value of the forest area image based on the joint perception information of the panoramic deep semantic feature map of the forest area and the panoramic shallow salient feature map of the forest area; and a three-dimensional model generation module 850, which is used to project the panoramic image of the forest area into a three-dimensional space based on the depth information decoding value to obtain a three-dimensional model of the forest scene.
[0052] Here, those skilled in the art can understand that the specific functions and operations of each module in the above-mentioned monocular panoramic vision 3D reconstruction system 800 for forest scenes have been described in the above reference. Figures 2 to 4 It has been introduced in detail in the description of the monocular panoramic vision 3D reconstruction method for forest scenes, and therefore, its repeated description will be omitted.
[0053] Figure 6 FIG. 1 is an application scenario diagram of a monocular panoramic vision 3D reconstruction method for forest scenes according to an embodiment of the present application. Figure 6 As shown, in this application scenario, first, a panoramic image of a forest area captured by a monocular panoramic camera is obtained (for example, Figure 6 Then, the forest area panoramic image is input to a server (for example, Figure 6 In S) as shown in , the server is able to use the monocular panoramic vision three-dimensional reconstruction algorithm for forest scenes to process the forest panoramic image to generate a depth information decoding value of the forest image, and then, based on the depth information decoding value, project the forest panoramic image into a three-dimensional space to obtain a three-dimensional model of the forest scene.
[0054] Furthermore, in another example of the present application, a monocular panoramic vision 3D reconstruction method for forest scenes is provided, referring to Figure 7 As shown, it includes: S610, obtaining panoramic image data of a typical forest area under different environmental factors such as light and climate collected by a panoramic camera; S620, counting the relevant features of the panoramic image of the forest area, and exploring the benefits of relevant clues such as light on the accuracy of panoramic image depth estimation; S630, using a monocular panoramic vision-depth clue fusion method to establish a depth estimation model for forest area images; S640, using the required depth information to project the panoramic image into a three-dimensional space and perform three-dimensional reconstruction of the forest scene. Further, the process of obtaining the panoramic depth map refers to Figure 8 As shown in the figure, it encodes and decodes the implicit depth clues from the panoramic image based on the convolutional neural network, and encodes the explicit depth clues from the copied image based on the multi-scale wavelet transform to obtain a multi-scale depth map, and then fuses them through the attention fusion module to obtain a panoramic depth map. It also involves the process of spectral domain-spatial domain transformation and multi-scale mask calculation.
[0055] Based on the above embodiments, another exemplary embodiment of an electronic device is also provided in the embodiments of the present application. In some possible implementations, the electronic device in the embodiments of the present application may include a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor can implement the steps of the monocular panoramic vision 3D reconstruction method for forest scenes in the above embodiments when executing the program.
[0056] For example, in the case of electronic equipment Figure 1 Taking the server 100 in the example as an example, the processor in the electronic device is the processor 110 in the server 100, and the memory in the electronic device is the memory 120 in the server 100.
[0057] The embodiment of the present application also provides a computer-readable storage medium, on which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, the monocular panoramic vision three-dimensional reconstruction method for forest scenes according to the embodiment of the present application described with reference to the above figures can be executed. The computer-readable storage medium includes, but is not limited to, for example, volatile memory and / or non-volatile memory. The volatile memory may, for example, include a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may, for example, include a read-only memory (ROM), a hard disk, a flash memory, etc.
[0058] The embodiment of the present application also provides a computer program product or a computer program, which includes computer executable instructions, and the computer executable instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer executable instructions from the computer-readable storage medium, and the processor executes the computer executable instructions, so that the computer device executes the monocular panoramic vision three-dimensional reconstruction method for forest scenes according to the embodiment of the present application.
[0059] Those skilled in the art will appreciate that the contents disclosed in this application may be subject to various variations and improvements. For example, the various devices or components described above may be implemented by hardware, or by software, firmware, or a combination of some or all of the three.
[0060] In addition, although the present application makes various references to certain units in the system according to embodiments of the present application, any number of different units can be used and run on the client and / or server. The units are only illustrative, and different aspects of the system and method can use different units.
[0061] Those skilled in the art will appreciate that all or part of the steps in the above method can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, such as a read-only memory, a disk or an optical disk. Optionally, all or part of the steps in the above embodiment can also be implemented using one or more integrated circuits. Accordingly, each module / unit in the above embodiment can be implemented in the form of hardware or in the form of software function modules. The present application is not limited to any particular form of combination of hardware and software.
[0062] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art to which this application belongs. It should also be understood that terms such as those defined in common dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an idealized or extremely formal sense, unless explicitly defined as such herein.
[0063] The above is an explanation of the present application and should not be considered as limiting thereof. Although several exemplary embodiments of the present application have been described, those skilled in the art will readily appreciate that many modifications may be made to the exemplary embodiments without departing from the novel teachings and advantages of the present application.
Claims
1. A monocular panoramic vision 3D reconstruction method for forest scenes, characterized in that: include: Obtain a panoramic image of the forest area collected by a monocular panoramic camera; Performing multi-level feature extraction on the forest area panoramic image to obtain a forest area panoramic shallow feature map and a forest area panoramic deep semantic feature map; Inputting the forest area panoramic shallow feature map into a feature distribution gradient mask salient device to obtain a forest area panoramic shallow salient feature map; Generate a depth information decoding value of the forest area image based on the joint perception information of the forest area panoramic deep semantic feature map and the forest area panoramic shallow salient feature map; Based on the depth information decoding value, the forest area panoramic image is projected into a three-dimensional space to obtain a three-dimensional model of the forest area scene.
2. The monocular panoramic vision 3D reconstruction method for forest scenes according to claim 1 is characterized in that: Performing multi-level feature extraction on the forest area panoramic image to obtain a forest area panoramic shallow feature map and a forest area panoramic deep semantic feature map, including: The forest area panoramic image is input into a panoramic image multi-scale feature extractor based on a pyramid network to obtain the forest area panoramic shallow feature map and the forest area panoramic deep semantic feature map.
3. The monocular panoramic vision 3D reconstruction method for forest scenes according to claim 2 is characterized in that: Inputting the forest area panoramic shallow feature map into a feature distribution gradient mask salient device to obtain a forest area panoramic shallow salient feature map, including: Calculating the multi-directional gradient value distribution of each position in the forest area panoramic shallow feature map, and determining the gradient amplitude value of each position in the forest area panoramic shallow feature map based on the multi-directional gradient value distribution of each position to obtain a forest area panoramic shallow feature gradient amplitude distribution map; Calculating the local description operator of the gradient amplitude at each position in the forest area panoramic shallow feature gradient amplitude distribution map to obtain a local significant distribution map of the forest area panoramic shallow feature gradient amplitude; Inputting the forest area panoramic shallow feature gradient amplitude local significant distribution map into a gated masker based on a GELU function to obtain a gradient amplitude local significant gated mask map; The forest area panoramic shallow salient feature map is obtained by calculating the point-by-point multiplication between the gradient amplitude local salient gated mask map and the forest area panoramic shallow feature map.
4. The monocular panoramic vision 3D reconstruction method for forest scenes according to claim 3 is characterized in that: Calculating the local description operator of the gradient amplitude at each position in the forest area panoramic shallow feature gradient amplitude distribution map to obtain the forest area panoramic shallow feature gradient amplitude local significant distribution map, including: Determine the scale of the local neighborhood, calculate the average of the difference between the gradient amplitude value of a predetermined position in the forest area panoramic shallow feature gradient amplitude distribution map and the gradient amplitude values of other positions in the local neighborhood to obtain a gradient amplitude local description operator corresponding to the predetermined position.
5. The monocular panoramic vision 3D reconstruction method for forest scenes according to claim 4, characterized in that: Based on the joint perception information of the forest area panoramic deep semantic feature map and the forest area panoramic shallow salient feature map, a depth information decoding value of the forest area image is generated, including: Performing joint perception on the forest area panoramic deep semantic feature map and the forest area panoramic shallow salient feature map to obtain a forest area panoramic shallow-deep salient joint perception feature map; The forest area panoramic shallow-deep significant joint perceptual feature map is input into a decoder-based depth estimation module to obtain the depth information decoding value.
6. The monocular panoramic vision 3D reconstruction method for forest scenes according to claim 5, characterized in that: The forest area panoramic deep semantic feature map and the forest area panoramic shallow salient feature map are jointly perceived to obtain a forest area panoramic shallow-deep salient joint perception feature map, including: The panoramic deep semantic feature map of the forest area and the panoramic shallow salient feature map of the forest area are input into a salient joint perception module based on an attention mechanism to obtain the panoramic shallow-deep salient joint perception feature map of the forest area.
7. The monocular panoramic vision 3D reconstruction method for forest scenes according to claim 6, characterized in that: Inputting the forest area panoramic deep semantic feature map and the forest area panoramic shallow salient feature map into a salient joint perception module based on an attention mechanism to obtain the forest area panoramic shallow-deep salient joint perception feature map, including: Performing feature shape reshaping on the forest area panoramic shallow salient feature map and the forest area panoramic deep semantic feature map to obtain a forest area panoramic shallow feature shape reshaping matrix and a forest area panoramic deep semantic feature shape reshaping matrix; Input the forest area panoramic shallow feature shape reshaping matrix and the forest area panoramic deep semantic feature shape reshaping matrix into the feature channel-by-channel interactive perception module to obtain the forest area panoramic shallow detail-deep semantic dependency matrix and the forest area panoramic deep semantic-shallow detail dependency matrix; Input the forest area panoramic shallow detail-deep semantic dependency matrix and the forest area panoramic deep semantic-shallow detail dependency matrix into a random dropout module to obtain a pruned forest area panoramic shallow detail-deep semantic dependency matrix and a pruned forest area panoramic deep semantic-shallow detail dependency matrix; Based on the pruned forest area panoramic shallow detail-deep semantic dependency matrix and the pruned forest area panoramic deep semantic-shallow detail dependency matrix, feature optimization is performed on the forest area panoramic deep semantic feature shape reshaping matrix and the forest area panoramic shallow feature shape reshaping matrix to obtain a dependency optimized forest area panoramic deep semantic feature matrix and a dependency optimized forest area panoramic shallow feature matrix; Reshaping the dependency relationship optimized forest area panoramic shallow feature matrix and the dependency relationship optimized forest area panoramic deep semantic feature matrix to obtain an optimized forest area panoramic shallow salient feature map and an optimized forest area panoramic deep semantic feature map; The weighted sum of the optimized forest area panoramic shallow salient feature map and the optimized forest area panoramic deep semantic feature map is calculated to obtain the forest area panoramic shallow-deep salient joint perceptual feature map.
8. The monocular panoramic vision 3D reconstruction method for forest scenes according to claim 7, characterized in that: The forest area panoramic shallow feature shape reshaping matrix and the forest area panoramic deep semantic feature shape reshaping matrix are input into the feature channel-by-channel interactive perception module to obtain the forest area panoramic shallow detail-deep semantic dependency matrix and the forest area panoramic deep semantic-shallow detail dependency matrix, including: Calculate the forest area panoramic shallow feature shape reshaping matrix multiplied by the transposed matrix of the forest area panoramic deep semantic feature shape reshaping matrix to obtain the forest area panoramic shallow detail-deep semantic association representation matrix; The forest panorama shallow detail-deep semantic association representation matrix is divided by the scale of the forest panorama shallow feature shape reshaping matrix and then input into the softmax function to obtain the forest panorama shallow detail-deep semantic dependency relationship matrix; Calculate the transposed matrix of the forest area panoramic deep semantic feature shape reshaping matrix and multiply it by the forest area panoramic shallow feature shape reshaping matrix to obtain the forest area panoramic deep semantic-shallow detail association representation matrix; The forest area panoramic deep semantics-shallow detail association representation matrix is divided by the scale of the forest area panoramic shallow feature shape reshaping matrix and then input into the softmax function to obtain the forest area panoramic deep semantics-shallow detail dependency matrix.
9. The monocular panoramic vision 3D reconstruction method for forest scenes according to claim 8, characterized in that: Based on the pruned forest area panoramic shallow detail-deep semantic dependency matrix and the pruned forest area panoramic deep semantic-shallow detail dependency matrix, the forest area panoramic deep semantic feature shape reshaping matrix and the forest area panoramic shallow feature shape reshaping matrix are feature optimized to obtain a dependency optimized forest area panoramic deep semantic feature matrix and a dependency optimized forest area panoramic shallow feature matrix, including: Performing matrix multiplication of the pruned forest area panoramic shallow detail-deep semantic dependency matrix and the forest area panoramic deep semantic feature shape reshaping matrix to obtain the dependency optimized forest area panoramic deep semantic feature matrix; The pruned forest area panoramic deep semantic-shallow detail dependency matrix is matrix-multiplied with the forest area panoramic shallow feature shape reshaping matrix to obtain the dependency-optimized forest area panoramic shallow feature matrix.
Citation Information
Patent Citations
Remote sensing image semantic segmentation method combining deep learning and random forest
CN111476170A
Image generation method, device and equipment and storage medium
CN112270745A
Three-dimensional reconstruction method based on attention mechanism and monocular multi-view angle
CN113838191A
Forest region positioning and three-dimensional reconstruction method and system based on multi-sensor fusion
CN116228969A
Instance-aware monocular semantic scene completion method, medium and equipment
CN117422629A
Cited By
Wetland species classification method based on aerospace remote sensing fusion image
CN118762237A
Health data management platform based on cloud computing
CN119742048A
Garden map updating method and system based on live-action image sharing and storage medium
CN120510486A
Landscape map updating method and system based on real scene image sharing and storage medium
CN120510486B
