A 3D space reconstruction method for complex jungle environments based on image recognition

Abstract

The present application provides a three-dimensional space reconstruction method for a complex jungle environment based on image recognition, which relates to the field of three-dimensional reconstruction technology. The method includes: arranging an environmental monitoring network in combination with the initial detection information of the target jungle to obtain a jungle environmental monitoring network; determining M sets of effective image acquisition spaces; determining K sets of image acquisition overlapping spaces; obtaining K sets of overlapping space splicing paths; determining a set of matching overlapping space splicing paths; extracting the acquisition images of M image acquisition devices in the current reconstruction window to obtain M real-time acquisition images, performing three-dimensional reconstruction based on the M real-time acquisition images, and splicing the reconstruction space in combination with the set of matching overlapping space splicing paths to determine the three-dimensional space of the target reconstruction environment of the target jungle. It solves the technical problem of low accuracy in three-dimensional space reconstruction in the existing technology for complex jungle environments, and achieves the effect of improving the accuracy and quality of three-dimensional reconstruction.

Description

Technical Field

[0001] This application relates to the field of three-dimensional reconstruction technology, and particularly to a three-dimensional space reconstruction method for complex jungle environments based on image recognition. Background Art

[0002] Traditional single-view or few-device acquisition methods often cannot comprehensively cover all details in the jungle, resulting in limited image data collection. In particular, information loss is likely to occur in areas with dense vegetation or complex terrain. In addition, due to large differences in lighting, occlusion, and vegetation height in different areas of the jungle, even when multiple devices are used for simultaneous acquisition, improper handling of the overlapping field of view areas between devices may lead to redundant data or inaccurate overlaps in the spatial reconstruction results, thereby affecting the accuracy of the overall three-dimensional model.

[0003] In summary, there is a technical problem of low accuracy in three-dimensional space reconstruction in complex jungle environments in the prior art. Summary of the Invention

[0004] The purpose of this application is to provide a three-dimensional space reconstruction method for complex jungle environments based on image recognition to solve the technical problem of low accuracy in three-dimensional space reconstruction in complex jungle environments in the prior art.

[0005] In view of the above problems, this application provides a three-dimensional space reconstruction method for complex jungle environments based on image recognition. Among them, the method includes:

[0006] Combining the initial detection information of the target jungle to deploy an environmental monitoring network to obtain a jungle environmental monitoring network, where the jungle environmental monitoring network includes M image acquisition devices, M device deployment positions, and M device deployment angles, and M is an integer greater than or equal to 1;

[0007] Interact with the device basic information of the M image acquisition devices to determine M effective image acquisition space sets of the M image acquisition devices under K monitoring scenarios and the M device deployment angles. Each effective image acquisition space is the area range where the image quality of an image acquisition device meets the requirements under a monitoring scenario, where K is an integer greater than or equal to 1;

[0008] Using the K monitoring scenarios as indexes, perform overlapping space recognition on the M effective image acquisition space sets and the M device deployment positions to determine K image acquisition overlapping space sets, where each image acquisition overlapping space includes multiple overlapping interested targets and multiple superimposed pixel point gradients;

[0009] Traverse the K image acquisition overlapping space sets to perform interested target space fusion recognition, and determine the space splicing path according to the recognition result to obtain K overlapping space splicing path sets;

[0010] Collect the node monitoring scene features of the jungle environment monitoring network in the current reconstruction window, match them with the K monitoring scenes to obtain the matching monitoring scenes, and determine the matching overlapping space splicing path set according to the matching monitoring scenes and the K overlapping space splicing path sets;

[0011] Extract the captured images of the M image capture devices in the current reconstruction window to obtain M real-time captured images, perform 3D reconstruction based on the M real-time captured images, and splice the reconstructed space in combination with the matching overlapping space splicing path set to determine the 3D space of the target reconstruction environment of the target jungle.

[0012] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0013] In this application, the environmental monitoring network is deployed by combining the initial detection information of the target jungle to obtain the jungle environmental monitoring network. Among them, the jungle environmental monitoring network includes M image capture devices, M device deployment positions, and M device deployment angles, where M is an integer greater than or equal to 1. Then, the device basic information of the M image capture devices is interacted to determine the M effective image capture space sets of the M image capture devices under the K monitoring scenes and the M device deployment angles. Each effective image capture space is the area range where the image quality of an image capture device meets the requirements in a monitoring scene. Among them, K is an integer greater than or equal to 1. Furthermore, taking the K monitoring scenes as indexes, the overlapping space recognition is performed on the M effective image capture space sets and the M device deployment positions to determine the K image capture overlapping space sets. Each image capture overlapping space includes multiple overlapping interested targets and multiple superimposed pixel point gradients. Then, traverse the K image capture overlapping space sets for interested target space fusion recognition, determine the space splicing path according to the recognition result to obtain the K overlapping space splicing path sets. By collecting the node monitoring scene features of the jungle environmental monitoring network in the current reconstruction window, matching them with the K monitoring scenes to obtain the matching monitoring scenes, and determining the matching overlapping space splicing path set according to the matching monitoring scenes and the K overlapping space splicing path sets. Then, extract the captured images of the M image capture devices in the current reconstruction window to obtain M real-time captured images, perform 3D reconstruction based on the M real-time captured images, and splice the reconstructed space in combination with the matching overlapping space splicing path set to determine the 3D space of the target reconstruction environment of the target jungle. The technical effect of improving the 3D reconstruction accuracy and quality can be achieved.

[0014] The above description is only an overview of the technical solution of the present application. In order to understand the technical means of the present application more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features and advantages of the present application more obvious and understandable, the following specifically gives the specific implementation manners of the present application. It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become easily understood through the following specification. Brief Description of the Drawings

[0015] In order to more clearly illustrate the technical solutions in the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings described below are only exemplary. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to the provided drawings.

[0016] Figure 1 It is a schematic flowchart of a three-dimensional space reconstruction method for a complex jungle environment based on image recognition in the present application;

[0017] Figure 2 It is a schematic flowchart of determining M effective image acquisition space sets in a three-dimensional space reconstruction method for a complex jungle environment based on image recognition in the present application. Detailed Description of the Embodiments

[0018] The present application provides a three-dimensional space reconstruction method for a complex jungle environment based on image recognition, and solves the technical problem of low accuracy in three-dimensional space reconstruction in the prior art for a complex jungle environment.

[0019] Next, the technical solutions in the present application will be clearly and completely described with reference to the drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. It should be understood that the present application is not limited by the exemplary embodiments described here. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application. Additionally, it should be noted that for the sake of description, only the parts related to the present application are shown in the drawings rather than all.

[0020] Embodiment, as Figure 1 shown, the present application provides a three-dimensional space reconstruction method for a complex jungle environment based on image recognition, and the method specifically includes the following steps:

[0021] Step S100: Arrange an environmental monitoring network in combination with the initial detection information of the target jungle to obtain a jungle environmental monitoring network, where the jungle environmental monitoring network includes M image acquisition devices, M device arrangement positions, and M device arrangement angles, and M is an integer greater than or equal to 1;

[0022] In a possible embodiment, the target jungle is any jungle that requires three-dimensional space reconstruction. The initial detection information reflects the overall characteristics of the target jungle, including topographic features, vegetation density, tree height, and other information. The jungle environmental monitoring network is used to collect images of the target jungle, thereby providing image data support for subsequent three-dimensional space reconstruction.

[0023] Preferably, use remote sensing devices (such as drones, LIDAR, satellite remote sensing) to conduct preliminary detection on the target jungle to obtain the overall characteristics of the jungle, thereby obtaining the initial detection information. Optionally, the initial detection information includes topographic features (such as mountains, hills, plains, etc., which mainly affect the arrangement position and height of the devices), vegetation density and distribution (such as tree height, plant density, which determine the field of view range and perspective selection of the devices), lighting conditions and occlusion situations (used to understand the lighting intensity in different areas and the occlusion of vegetation or other obstacles). According to the initial detection information, those skilled in the art analyze and determine the type of image acquisition device (such as a high-resolution camera, a multispectral camera, or a device with infrared function, etc.), and determine the device arrangement position and device arrangement angle according to the area of the target jungle.

[0024] Preferably, by analyzing the initial detection data, calculate the coverage range of the target jungle and determine the number of devices to be arranged. The value of M must be sufficient to cover the entire area of the target jungle and ensure that the overlapping area can provide sufficient image data for three-dimensional reconstruction. Exemplarily, divide the target jungle into several monitoring areas to ensure that at least one image acquisition device covers each area. In flat areas, the devices should be evenly distributed; in complex terrains such as slopes and valleys, the devices should be arranged at high or low positions to ensure that topographic features at different heights can be captured. In areas with dense vegetation, the devices can be arranged at high positions (such as drones or installed on tree canopies) to obtain a wider field of view and avoid blind spots caused by canopy occlusion. The arrangement position of the devices should ensure that each area can be covered by multiple devices, especially key jungle areas (such as special landforms or areas with dense vegetation). By reasonably setting the overlapping field of view between the devices, ensure that there is enough image overlap for subsequent stitching. When setting the device arrangement angle, the horizontal angle of each device should cover the largest surrounding monitoring area. The device can adjust the horizontal angle around its arrangement point to ensure that enough jungle features in the field of view are captured.

[0025] By combining the initial detection information of the jungle environment, determine the layout positions and angles of M image acquisition devices, ensuring that the devices can effectively cover all important areas in the jungle and have sufficient field-of-view overlap areas to provide data support for subsequent 3D reconstruction.

[0026] Step S200: Interact with the device basic information of the M image acquisition devices to determine M sets of effective image acquisition spaces of the M image acquisition devices under K monitoring scenarios and the M device layout angles. Each effective image acquisition space is the area range where the image quality of an image acquisition device meets the requirements under a monitoring scenario, where K is an integer greater than or equal to 1.

[0027] Further, as Figure 2 shown, when interacting with the device basic information of the M image acquisition devices to determine M sets of effective image acquisition spaces of the M image acquisition devices under K monitoring scenarios and the M device layout angles, step S200 of the embodiment of the present application further includes:

[0028] Obtain K sets of jungle environment characteristics of the K monitoring scenarios;

[0029] Interact with the jungle environment monitoring network to obtain M sets of device basic information of the M image acquisition devices, and combine the M device layout angles and the K sets of jungle environment characteristics to perform test image acquisition to obtain M sets of test images, where each set of test images includes K test images, and each test image corresponds to a monitoring scenario;

[0030] Traverse the M sets of test images to perform pixel point gradient calculation, and determine M sets of image effective area according to the calculation results;

[0031] Extract the target of interest from the M sets of image effective areas, and perform spatial coordinate mapping analysis in combination with the extraction results, and determine M sets of effective image acquisition spaces according to the analysis results.

[0032] Further, each set of jungle environment characteristics in the K sets of jungle environment characteristics includes lighting conditions, weather conditions, and monitoring seasons.

[0033] In an embodiment, obtain the device basic information of the M image acquisition devices, so as to master the image shooting capabilities of the M image acquisition devices, and combine the K monitoring scenarios and each device layout angle to determine the spatial range where the M image acquisition devices can capture effective images under different monitoring scenarios, and obtain the M sets of effective image acquisition spaces. Where each effective image acquisition space is the area range where the image quality of an image acquisition device meets the requirements under a monitoring scenario.

[0034] Optionally, in step S200 of the present application, through the interaction of device basic information, monitoring scenarios, and deployment angles, and by using pixel point gradient calculation and extraction of targets of interest, the effective image acquisition space of each device in different monitoring scenarios is gradually determined. This process ensures that the complex environment of the jungle can be completely covered through image acquisition from multiple angles and multiple scenarios, and provides high-quality image data for subsequent three-dimensional space reconstruction.

[0035] In one embodiment, the K monitoring scenarios are determined based on the initial detection information of the target jungle, where the K monitoring scenarios are the environmental conditions with the top K frequencies of occurrence in the target jungle. The K jungle environmental feature sets are determined based on the terrain and climate conditions in the initial detection information of the target jungle, where each jungle environmental feature set includes lighting conditions, weather conditions, and monitoring seasons.

[0036] Exemplarily, the lighting conditions include the light intensity at different time periods (morning, noon, evening). The weather conditions include the impacts of different weathers such as sunny, cloudy, rainy, or foggy days on image acquisition. The monitoring seasons include the impacts of seasonal changes such as dense vegetation in spring and fallen leaves in autumn on the jungle environment.

[0037] Optionally, interact with the jungle environment monitoring network to obtain M sets of device basic information of the M image acquisition devices. Each set of device basic information includes device parameters (such as resolution, focal length, field of view angle, sensor characteristics, etc.), device service life, and device manufacturers, etc. Image acquisition tests are respectively conducted on the M image acquisition devices at the M device deployment angles under the K jungle environmental feature sets, and the image with the best image quality in the test is selected as the test image to obtain the M sets of test images. For the convenience of subsequent three-dimensional reconstruction, the test images of each image acquisition device in different monitoring scenarios are classified into one category.

[0038] Optionally, in order to be able to identify the clear regions in the M sets of test images, it is necessary to calculate the gradients of the pixel points of each test image in the M sets of test images. The larger the pixel point gradient, the clearer the image at that position, and the smaller the pixel point gradient, the blurrier the image at that position. The edges of the clear images are identified based on the calculation results to obtain the M sets of image effective region sets. Optionally, the Canny edge detection algorithm is used to calculate the gradients of the pixel points in each image, and further, the clarity of different regions in the image is evaluated.

[0039] Preferably, the M sets of image effective regions include the cases where the target jungle is captured at different positions. By extracting the targets of interest in the images, the positions where the targets of interest to be analyzed are distributed in the target jungle are determined, so as to determine the spatial range where the M image acquisition devices can capture effective images, that is, the M sets of effective image acquisition spaces.

[0040] Further, traverse the M sets of test images to calculate the pixel point gradients, and determine the M sets of image effective regions according to the calculation results. Step S200 of the embodiment of the present application further includes:

[0041] Randomly select a first test image from the M sets of test images, and calculate the gradients of the pixel points in the first test image to obtain a plurality of first pixel point gradients;

[0042] Take the pixel points corresponding to the N first pixel point gradients among the plurality of first pixel point gradients that are ranked in the top N as the centers of N first effective sub-regions. Starting from the centers of the N first effective sub-regions, perform effective sub-region diffusion according to a preset region diffusion step length until a preset diffusion stop condition is met, to obtain N first effective sub-regions, where N is an integer greater than or equal to 1, and the preset region diffusion step length is the maximum pixel point gradient difference allowed for a single diffusion;

[0043] Perform a union operation on the N first effective sub-regions to obtain a first image effective region;

[0044] Calculate the pixel point gradients of the M sets of test images according to the preset region diffusion step length to determine the M sets of image effective regions.

[0045] Further, step S200 of the embodiment of the present application further includes:

[0046] Starting from the centers of the N first effective sub-regions, perform effective sub-region diffusion according to a preset region diffusion step length to obtain N first initial diffusion sub-regions;

[0047] Perform edge diffusion on the N first initial diffusion sub-regions again according to the preset region diffusion step length to obtain N first stage diffusion sub-regions;

[0048] Judge whether the difference between the pixel point gradient means of the N first stage diffusion sub-regions and the pixel point gradient means of the N first initial diffusion sub-regions meets a preset difference. If so, the preset diffusion stop condition is met, and the N first stage diffusion sub-regions are used as the N first effective sub-regions;

[0049] If not, edge diffusion is performed on the N first-stage diffusion sub-regions again according to the preset region diffusion step length until the preset diffusion stop condition is met, and the N first effective sub-regions are obtained.

[0050] In one embodiment, an image (referred to as the first test image) is randomly selected from the M test image sets as the initial sample for subsequent gradient calculation and effective region identification. Preferably, preprocessing is performed on the first test image, including denoising and enhancing contrast, to ensure the accuracy of gradient calculation. Furthermore, using the Canny edge detection gradient algorithm, the gradient value of each pixel point in the first test image is calculated to obtain the gradient intensity of each pixel point, thereby obtaining the plurality of first pixel point gradients. The gradient reflects the gradient intensity and direction of each pixel point in the image.

[0051] Preferably, the pixel points corresponding to the N first pixel point gradients among the plurality of first pixel point gradients that are ranked in the top N are used as the centers of the N first effective sub-regions. Since the N first pixel point gradients are relatively large, the corresponding first pixel points can be used as the starting points for effective sub-region diffusion, thereby improving the efficiency of effective region screening.

[0052] Starting from the centers of the N first effective sub-regions, effective sub-region diffusion is performed according to the preset region diffusion step length until the preset diffusion stop condition is met, and N first effective sub-regions are obtained, where N is an integer greater than or equal to 1, and the preset region diffusion step length is the maximum pixel point gradient difference allowed for a single diffusion set by those skilled in the art.

[0053] Preferably, the N first effective sub-regions are relatively clear sub-regions formed by diffusing outward with the centers of the N first effective sub-regions as the centers. By taking the union of the N first effective sub-regions, the first image effective region with relatively high quality in the first test image can be obtained. Based on the same principle as obtaining the first image effective region, pixel point gradient calculation is performed on the M test image sets according to the preset region diffusion step length to determine the M image effective region sets. The goal of effective region identification for the M test image sets is achieved.

[0054] Preferably, first, starting from the centers of the N first effective sub-regions, effective sub-region diffusion is performed according to the preset region diffusion step length to obtain N first initial diffusion sub-regions. That is to say, the N first initial diffusion sub-regions are the regions surrounded by the outermost first pixel points in the set of first pixel points corresponding to the plurality of first pixel point gradients obtained after pixel point gradient screening that satisfies the preset region diffusion step length in all directions with the centers of the N first effective sub-regions as the starting points.

[0055] Furthermore, taking multiple outermost first pixel points in the first initial diffusion sub-region as diffusion starting points, diffusing according to the preset region diffusion step length to obtain the N first-stage diffusion sub-regions. Calculate the mean value of the pixel gradients within the N first-stage diffusion sub-regions and the N first initial diffusion sub-regions respectively. Then, determine whether the difference between the mean value of the pixel gradients of the N first-stage diffusion sub-regions and the mean value of the pixel gradients of the N first initial diffusion sub-regions meets the preset difference. If so, it indicates that the edge between the clear region and the blurred region has been reached, and the preset diffusion stop condition is satisfied. Take the N first-stage diffusion sub-regions as N first effective sub-regions.

[0056] If not, it indicates that the edge between the clear region and the blurred region has not been reached yet. At this time, perform edge diffusion on the N first-stage diffusion sub-regions again according to the preset region diffusion step length until the preset diffusion stop condition is satisfied, and obtain the N first effective sub-regions.

[0057] The above steps calculate through pixel gradients, gradually diffuse and refine, and finally determine the set of effective regions for each image. This method ensures that only high-quality regions in the image are recognized and extracted, avoiding the influence of blurred regions on subsequent target extraction and spatial analysis.

[0058] Furthermore, step S200 of the embodiment of the present application further includes:

[0059] Perform deep learning processing on the M sets of image effective regions to extract M clusters of interested targets, and each cluster of interested targets includes K sets of interested targets;

[0060] Traverse the M clusters of interested targets for spatial coordinate mapping analysis to determine M clusters of target spatial coordinates;

[0061] Based on the M clusters of target spatial coordinates and the M device layout positions, perform spatial edge analysis to obtain the M sets of effective image acquisition spaces.

[0062] In one embodiment, the M sets of image effective regions obtained in the previous steps are used as inputs and input into a deep learning model. The set of effective regions contains the high-quality parts screened from images collected by different devices. Use a pre-trained object detection model (such as YOLO) to detect the interested targets in the images. These models should be able to detect and classify different targets in a complex jungle environment, such as jungle features like trees, shrubs, and stones.

[0063] Preferably, object detection is performed on each effective image region to extract clusters of objects of interest. Each cluster of objects of interest includes K sets of objects of interest (K is an integer greater than or equal to 1), representing the sets of objects identified from each perspective in each monitoring scenario. Objects of interest include trees, shrubs, grasslands, stones, etc. The positions of each object of interest in the target jungle are determined based on the M clusters of objects of interest, and M clusters of target space coordinates are obtained.

[0064] Preferably, based on the positions, deployment angles, and imaging geometric parameters of each image acquisition device, the two-dimensional pixel coordinates in the image are mapped to the coordinates in the actual three-dimensional space. This step maps the object from the image plane to the actual jungle environment space through inverse projection or multi-view geometry methods. Preferably, the positions of the objects of interest in the target jungle can also be determined manually, and then the corresponding space coordinates can be obtained.

[0065] Optionally, with the M device deployment positions as M spatial centers, combined with the M clusters of target space coordinates, the target space coordinates with the farthest distance from each angle to the M spatial centers are extracted and used as an edge point on the spatial edge. Furthermore, the obtained multiple edge points are connected to obtain M sets of effective image acquisition spaces. Preferably, in combination with the deployment positions of each device, the edges of the object clusters are analyzed to ensure that the coverage range of the device can completely capture the boundaries of the object clusters. If the boundaries of some object clusters are in the acquisition blind area of the device, the system will prompt to readjust the device deployment angle or add new devices. Through the above steps, all key objects in the complex jungle environment are accurately detected and mapped, providing reliable data support for subsequent three-dimensional space reconstruction.

[0066] Step S300: Using the K monitoring scenarios as indexes, overlapping space recognition is performed on the M sets of effective image acquisition spaces and the M device deployment positions to determine K sets of image acquisition overlapping spaces, where each image acquisition overlapping space includes multiple overlapping objects of interest and multiple superimposed pixel point gradients;

[0067] In one embodiment, using the K monitoring scenarios as indexes, combined with the M sets of effective image acquisition spaces, the effective image acquisition spaces belonging to the same monitoring scenario are aggregated into one set to obtain K sets of effective image acquisition spaces. Thus, the consistency and synchronization of information such as device positions and angles in each monitoring scenario are achieved.

[0068] Extract the first set of effective image acquisition spaces from the K sets of effective image acquisition spaces. Since the M device deployment positions are the spatial centers of the corresponding spaces in the first set of effective image acquisition spaces, the first set of effective image acquisition spaces can be placed in the same coordinate system according to the M device deployment positions, and overlapping space recognition is performed on the first set of effective image acquisition spaces to obtain the K sets of image acquisition overlapping spaces.

[0069] Preferably, in combination with the M sets of target space coordinates corresponding to the M clusters of targets of interest in step S200 above, and the K sets of image acquisition overlapping spaces, it is possible to determine the targets of interest existing in each image acquisition overlapping space, and use them as multiple overlapping targets of interest in each image acquisition overlapping space. Since each overlapping target of interest may exist in multiple effective image acquisition spaces, and the pixel point gradients are different in different effective image acquisition spaces, the larger the value obtained by superimposing the multiple pixel point gradients in the multiple effective image acquisition spaces, the higher the clarity of the overlapping target of interest in the image acquisition overlapping space, and the higher the accuracy and tolerance of using this as the stitching path for spatial stitching. Therefore, by obtaining that each image acquisition overlapping space includes multiple overlapping targets of interest and multiple superimposed pixel point gradients, the technical effect of laying the foundation for determining the spatial stitching path in the subsequent process is achieved.

[0070] Step S400: Traverse the K sets of image acquisition overlapping spaces to perform spatial fusion recognition of the targets of interest, determine the spatial stitching path according to the recognition result, and obtain the K sets of overlapping space stitching path sets;

[0071] Furthermore, when traversing the K sets of image acquisition overlapping spaces to perform spatial fusion recognition of the targets of interest and determining the spatial stitching path according to the recognition result to obtain the K sets of overlapping space stitching path sets, step S400 of the embodiment of the present application further includes:

[0072] Randomly extract the first image acquisition overlapping space from the K sets of image acquisition overlapping spaces, where the first image acquisition overlapping space includes multiple overlapping targets of interest and multiple superimposed pixel point gradients;

[0073] Extract the overlapping target of interest located at the spatial edge and corresponding to the minimum value of the superimposed pixel point gradients from the multiple overlapping targets of interest, and use it as the starting point of the first path;

[0074] Based on the distances from the multiple overlapping targets of interest to the starting point of the first path, determine the first starting neighborhood of the starting point of the first path, where the first starting neighborhood is a set of overlapping targets of interest whose distances to the starting point of the first path satisfy a preset distance threshold;

[0075] Take the overlapping interesting target corresponding to the maximum gradient of the superimposed pixel points in the first starting neighborhood as the first path stage point, and construct the neighborhood of the first path stage point in combination with the multiple overlapping interesting targets. Determine the next first path stage point according to the construction result until reaching the edge of the first image acquisition overlapping space, and obtain multiple first path stage points;

[0076] Connect the first path starting point and the multiple first path stage points in sequence according to the obtained order to obtain the first overlapping space stitching path;

[0077] Perform spatial stitching path recognition on the K image acquisition overlapping space sets to obtain a set of K overlapping space stitching paths.

[0078] In one embodiment, according to the multiple overlapping interesting targets and multiple superimposed pixel point gradients included in each image acquisition overlapping space in the K image acquisition overlapping space sets, after determining that the two-dimensional images collected by different image acquisition devices in each image acquisition overlapping space are converted into three-dimensional point clouds, obtain the paths for stitching the three-dimensional point clouds formed after image conversion of different image acquisition devices in each image acquisition overlapping space, and obtain the set of K overlapping space stitching paths. It achieves the technical effect of facilitating the effective fusion of data collected by multiple devices and perspectives in the jungle environment and providing a complete spatial path planning for subsequent three-dimensional reconstruction.

[0079] Preferably, randomly select one from the K overlapping space sets as the first image acquisition overlapping space for the initial planning of the spatial stitching path. The first image acquisition overlapping space contains multiple overlapping interesting targets and multiple superimposed pixel point gradients. These information will help to judge the target distribution and image quality in this overlapping area.

[0080] In the first image acquisition overlapping space, preferentially extract the interesting targets located at the edge of the space. These targets usually represent the geographical boundaries, key feature points in the jungle or the targets in the boundary area of the image. Furthermore, select the target with the minimum superimposed pixel point gradient as the first path starting point. The target with the minimum pixel point gradient represents that the image quality in this area is lower or there are fewer details, and it is suitable as the starting point of the path.

[0081] Taking the first path starting point as the center, calculate the distances between this starting point and other overlapping interesting targets. According to the preset distance threshold (the maximum distance from the overlapping interesting target in the neighborhood preset by those skilled in the art to the center of the first path), determine the targets with relatively close distances to the starting point as the first starting neighborhood. Preferably, the targets in the first starting neighborhood are a set of interesting targets that meet the distance threshold requirements, and these targets will be used as potential path stage points.

[0082] Furthermore, within the first starting neighborhood, select the target corresponding to the maximum gradient value of the superimposed pixel points as the first path stage point. A higher gradient value indicates better image quality and clearer edges, so it is suitable as an intermediate path point. Based on the same principle, for the selected path stage points, repeat the neighborhood construction process, and continue to expand the path neighborhood according to the distance from other overlapping target regions of interest. The distance and gradient difference between each stage point are used to determine the selection of the next path stage point. Continuously select the next path stage point until reaching the edge of the first image acquisition overlapping space, obtaining multiple first path stage points. Connect the multiple first path stage points in the order they are obtained to generate the first overlapping space stitching path. This path represents the complete space stitching process from the edge to the edge of the overlapping space. Repeat the above path planning process for each overlapping space set to determine the starting point, path stage points, and path expansion of each overlapping space until obtaining the stitching paths of all overlapping spaces, that is, the set of K overlapping space stitching paths. Each stitching path set represents a part of the jungle environment, and the images taken by different devices and at different angles are jointly stitched. This achieves the technical effect of improving the accuracy of three-dimensional space reconstruction.

[0083] Step S500: Collect the node monitoring scene features of the jungle environment monitoring network in the current reconstruction window, match them with the K monitoring scenes, obtain the matching monitoring scenes, and determine the matching overlapping space stitching path set according to the matching monitoring scenes and the set of K overlapping space stitching paths;

[0084] In a possible embodiment, the current reconstruction window is the time point for three-dimensional reconstruction of the target jungle. At this time point, collect the environmental conditions of the jungle environment monitoring network to obtain the node monitoring scene features. Among them, the node monitoring scene features are used to describe the environmental conditions of the target jungle in the current reconstruction window, including elements such as lighting, weather, terrain, and vegetation. Optionally, use the previously set image acquisition devices or other sensors to obtain the feature data of these nodes.

[0085] Analyze the monitoring scene features collected in the current reconstruction window, and compare them with the set of features of the K monitoring scenes defined in the previous steps. Respectively use the cosine similarity calculation formula to determine the similarity between the node monitoring scene features and the K jungle environment feature sets of the K monitoring scenes, and use the monitoring scene corresponding to the maximum similarity as the matching monitoring scene. According to the mapping relationship between the monitoring scene and the set of K overlapping space stitching paths, search based on the matching monitoring scene to determine the corresponding matching overlapping space stitching path set. This achieves the technical effect of providing a stitching path for subsequent three-dimensional space reconstruction.

[0086] Step S600: Extract the acquired images of the M image acquisition devices in the current reconstruction window to obtain M real-time acquired images, perform three-dimensional reconstruction based on the M real-time acquired images, and splice the reconstruction space in combination with the set of matching overlapping space splicing paths to determine the three-dimensional space of the target reconstruction environment of the target jungle.

[0087] Further, step S600 of the embodiment of the present application further includes:

[0088] Obtain M window angle information of the M image acquisition devices under the current reconstruction window;

[0089] Authenticate the M window angle information based on the M device layout angles and the M angle tolerance intervals to obtain M authentication results;

[0090] If all of the M authentication results are passed, obtain an image acquisition instruction;

[0091] Perform image acquisition of the M image acquisition devices based on the image acquisition instruction to obtain the M real-time acquired images.

[0092] In a possible embodiment, extract the images acquired by the M image acquisition devices in the current reconstruction window to obtain the M real-time acquired images. Among them, the M real-time acquired images reflect the real-time environmental conditions of the target jungle. Perform three-dimensional reconstruction based on the M real-time acquired images, and splice the overlapping spaces in the reconstruction space in combination with the set of matching overlapping space splicing paths to determine the three-dimensional space of the target reconstruction environment of the target jungle. The technical effect of improving the reliability and accuracy of three-dimensional space reconstruction is achieved.

[0093] Preferably, connect the M image acquisition devices, extract the camera angle information of the M image acquisition devices, and obtain M window angle information of the M image acquisition devices under the current reconstruction window. Furthermore, authenticate the M window angle information based on the M device layout angles and the M angle tolerance intervals (the angle range that the cameras of the M image acquisition devices can be offset set by those skilled in the art) to obtain M authentication results.

[0094] Optionally, when the difference between the M window angle information and the M device layout angles satisfies the M angle tolerance intervals, the authentication result is passed. When the difference between the M window angle information and the M device layout angles does not satisfy the M angle tolerance intervals, the authentication result is not passed. If all of the M authentication results are passed, obtain an image acquisition instruction, and perform image acquisition of the M image acquisition devices based on the image acquisition instruction to obtain the M real-time acquired images. Thus, the technical effect of improving the reliability of the acquired images is achieved.

[0095] In one embodiment, preliminary preprocessing is performed on M real-time acquired images, including denoising, contrast enhancement, and image correction, to ensure that the quality of each image meets the requirements of 3D reconstruction. Feature points are extracted from each real-time image. The SIFT algorithm can be used to extract key points in the images, and these feature points will be used for matching and alignment between images. Feature point matching is performed among the M real-time images, and the FLANN algorithm is used to match feature points from different perspectives. The successfully matched feature points will be used to generate a 3D spatial point cloud. Based on the matched feature points, a multi-view stereo (MVS) technique is used to generate a 3D point cloud. This step combines the different perspective information of the M real-time acquired images to generate the 3D geometric structure of the jungle environment. Exemplarily, a sparse point cloud model is first generated, which reflects the 3D spatial positions of the main features in the jungle and forms the preliminary geometric framework of the entire environment. Based on the sparse point cloud, a multi-view stereo algorithm is used for dense point cloud reconstruction to generate a more refined 3D structure. The dense point cloud will contain more details and reflect the minute features of elements such as plants and terrain in the jungle.

[0096] Furthermore, the set of matching overlapping space stitching paths obtained from step S500 contains the stitching path information of multiple overlapping spaces of different image acquisition devices. The generated 3D point cloud is stitched using the set of matching overlapping space stitching paths, and the point cloud data in the overlapping regions acquired by the M image acquisition devices is seamlessly docked. Ensure that the point cloud models generated by each device are spatially aligned to form a continuous 3D reconstruction model. Redundant points in the overlapping regions are processed to eliminate duplicate data caused by multi-device shooting. Through weighted averaging or feature fusion techniques, the stitching quality of these overlapping regions is optimized to ensure the continuity and smoothness of the visual effect. Thus, a complete 3D space model of the jungle is constructed. This model contains the terrain in the jungle.

[0097] In step S600, a target 3D space model of the jungle environment is generated through the acquisition of real-time images, 3D reconstruction, and spatial stitching. This process ensures the high precision and detailed completeness of the generated 3D model through image preprocessing, feature point matching, point cloud generation, and the integration of stitching paths, and finally presents a complete 3D space of the jungle environment.

[0098] In summary, a 3D space reconstruction method for a complex jungle environment based on image recognition provided by the present application has the following technical effects:

[0099] 1. Through efficient multi-device image acquisition, real-time 3D reconstruction, automated spatial stitching, and flexible scene adaptation, the present application realizes high-precision reconstruction of complex jungle environments. This method not only improves the accuracy and efficiency of 3D models but also enhances the adaptability to complex environments.

[0100] 2. By identifying and fusing the target of interest in the overlapping space, and based on pixel point gradient calculation and spatial feature matching, a stitching path is automatically generated, effectively reducing manual intervention, improving the efficiency of spatial stitching, and achieving the technical effect of improving the accuracy of three-dimensional space reconstruction.

[0101] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments shown herein, but rather will be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0102] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalent technologies, the present application is also intended to include these changes and modifications.

Claims

1. A method for reconstructing three-dimensional space in a complex jungle environment based on image recognition, characterized in that: The method comprises: Deploy an environmental monitoring network based on the initial detection information of the target jungle to obtain a jungle environmental monitoring network, wherein the jungle environmental monitoring network includes M image acquisition devices, M device deployment positions, and M device deployment angles, where M is an integer greater than or equal to 1; Interacting the basic device information of the M image acquisition devices, determining M effective image acquisition space sets of the M image acquisition devices under K monitoring scenes and the M device deployment angles, each effective image acquisition space is an area range of an image acquisition device in a monitoring scene where the image quality meets the requirements, wherein K is an integer greater than or equal to 1; Using the K monitoring scenes as indexes, performing overlapping space identification on the M valid image acquisition space sets and the M equipment deployment positions, and determining K image acquisition overlapping space sets, wherein each image acquisition overlapping space includes multiple overlapping objects of interest and multiple superimposed pixel point gradients; Traversing the K image acquisition overlapping space sets to perform spatial fusion recognition of the target of interest, determining a spatial splicing path according to the recognition result, and obtaining a set of K overlapping spatial splicing paths; Collecting node monitoring scene features of the jungle environment monitoring network in the current reconstruction window, matching them with the K monitoring scenes to obtain a matching monitoring scene, and determining a matching overlapping space splicing path set according to the matching monitoring scene and the K overlapping space splicing path sets; The captured images of M image acquisition devices in the current reconstruction window are extracted to obtain M real-time captured images, three-dimensional reconstruction is performed based on the M real-time captured images, and the reconstructed space is spliced ​​in combination with the matching overlapping space splicing path set to determine the three-dimensional space of the target reconstruction environment of the target jungle.

2. The method for reconstructing a three-dimensional space of a complex jungle environment based on image recognition as claimed in claim 1, characterized in that: Interacting the basic device information of the M image acquisition devices to determine M effective image acquisition space sets of the M image acquisition devices under K monitoring scenes and the M device deployment angles, including: Obtaining K jungle environment feature sets of the K monitoring scenes; Interacting with the jungle environment monitoring network to obtain M sets of basic information of the M image acquisition devices, and performing test image acquisition in combination with the M device layout angles and the K jungle environment feature sets to obtain M test image sets, wherein each test image set includes K test images, and each test image corresponds to a monitoring scene; Traversing the M test image sets to perform pixel point gradient calculations, and determining M image valid area sets according to the calculation results; The objects of interest are extracted from the M sets of effective image regions, and spatial coordinate mapping analysis is performed in combination with the extraction results, and the M sets of effective image acquisition spaces are determined according to the analysis results.

3. The method for reconstructing a complex jungle environment three-dimensional space based on image recognition as claimed in claim 2, characterized in that: Traversing the M test image sets to perform pixel point gradient calculations, and determining M image valid area sets according to the calculation results, including: Randomly extracting a first test image from the set of M test images, and performing gradient calculation on pixel points in the first test image to obtain a plurality of first pixel point gradients; Taking the pixel points corresponding to the N first pixel point gradients located in the first N positions among the multiple first pixel point gradients as the center points of N first effective sub-regions, taking the center points of the N first effective sub-regions as the starting point, performing effective sub-region diffusion according to a preset area diffusion step length until a preset diffusion stop condition is met, and obtaining N first effective sub-regions, wherein N is an integer greater than or equal to 1, and the preset area diffusion step length is the maximum pixel point gradient difference allowed in a single diffusion; Taking a union of the N first valid sub-regions to obtain a first image valid region; Pixel point gradient calculation is performed on the M test image sets according to the preset area diffusion step length to determine the M image valid area sets.

4. The method for reconstructing a three-dimensional space of a complex jungle environment based on image recognition as claimed in claim 3, characterized in that: include: Taking the center points of the N first effective sub-regions as starting points, effective sub-region diffusion is performed according to a preset region diffusion step length to obtain N first initial diffusion sub-regions; Performing edge diffusion on the N first initial diffusion sub-regions again according to the preset region diffusion step length to obtain N first-stage diffusion sub-regions; Determine whether a difference between a pixel point gradient mean value of the N first stage diffusion sub-regions and a pixel point gradient mean value of the N first initial diffusion sub-regions satisfies a preset difference value, and if so, the preset diffusion stop condition is satisfied, and the N first stage diffusion sub-regions are used as N first effective sub-regions; If not, edge diffusion is performed again on the N first-stage diffusion sub-regions according to the preset region diffusion step length until a preset diffusion stop condition is met, thereby obtaining the N first effective sub-regions.

5. The method for reconstructing a complex jungle environment three-dimensional space based on image recognition as claimed in claim 2, characterized in that: include: Performing deep learning processing on the M sets of valid image regions to extract M clusters of objects of interest, each cluster of objects of interest including K sets of objects of interest; Traversing the M clusters of interest to perform spatial coordinate mapping analysis to determine M target spatial coordinate clusters; Spatial edge analysis is performed based on the M target space coordinate clusters and the M equipment layout positions to obtain the M effective image acquisition space sets.

6. The method for reconstructing a three-dimensional space of a complex jungle environment based on image recognition as claimed in claim 1, characterized in that: Traversing the K image acquisition overlapping space sets to perform spatial fusion recognition of the target of interest, determining the spatial splicing path according to the recognition result, and obtaining K overlapping spatial splicing path sets, including: Randomly extracting a first image acquisition overlapping space from the K image acquisition overlapping space sets, wherein the first image acquisition overlapping space includes a plurality of overlapping objects of interest and a plurality of superimposed pixel point gradients; Extracting an overlapping object of interest located at the edge of the space and corresponding to the minimum value of the gradient of the superimposed pixel point from the multiple overlapping objects of interest, and taking it as the starting point of the first path; Determine a first starting neighborhood of the first path starting point based on the distances of the multiple overlapping interested objects to the first path starting point, wherein the first starting neighborhood is a set of overlapping interested objects whose distances to the first path starting point meet a preset distance threshold; Taking the overlapping interested object corresponding to the maximum value of the gradient of the superimposed pixel point in the first starting neighborhood as the first path stage point, and constructing the neighborhood of the first path stage point in combination with the multiple overlapping interested objects, and determining the next first path stage point according to the construction result, until reaching the edge of the first image acquisition overlapping space, and obtaining multiple first path stage points; Connecting the first path starting point and the plurality of first path stage points in sequence according to the order in which they are obtained, to obtain a first overlapping space splicing path; Perform spatial splicing path identification on the K image acquisition overlapping space sets to obtain K overlapping space splicing path sets.

7. The method for reconstructing a three-dimensional space of a complex jungle environment based on image recognition as claimed in claim 2, characterized in that: Each of the K jungle environment feature sets includes lighting conditions, weather conditions and monitoring seasons.

8. The method for reconstructing a complex jungle environment three-dimensional space based on image recognition as claimed in claim 1, characterized in that: include: Obtaining M window angle information of M image acquisition devices under the current reconstruction window; Based on the M device deployment angles and the M angle tolerance intervals, the M window angle information is authenticated to obtain M authentication results; If the M authentication results are all authenticated, obtaining an image acquisition instruction; The M image acquisition devices are used to acquire images based on the image acquisition instruction to obtain the M real-time acquired images.

Images