Target identification sample automatic generation method based on oblique photography modeling

Abstract

The invention belongs to the technical field of image processing, and discloses a target identification sample automatic generation method based on oblique photography modeling, which comprises the steps of oblique photography modeling, target oblique photography data inputting, and data processing of the oblique photography data to obtain an integral oblique photography model. And oblique photography monomerization: extracting a target contour of the target real shot image, and carrying out cutting monomerization on the oblique photography overall model based on the target contour to obtain a target oblique photography monomer model. And three-dimensional simulation scene construction: constructing a three-dimensional simulation scene through a modeling technology, and selecting a position to place the target model. And fusing the target monomer model with the three-dimensional simulation scene, and adjusting the pose angle and environmental parameters of the model. And generating a sample, setting state information in the three-dimensional simulation scene, rendering the three-dimensional scene in real time, and automatically outputting target sample data. The method overcomes the limitation of a traditional sample generation method, and meets the urgent demands of the target recognition field for high-quality and diversified samples.

Description

Technical Field

[0001] The present invention relates to the technical fields of computer vision, three-dimensional modeling and image processing, and in particular to a method for automatically generating target recognition samples based on oblique photography modeling. Background Art

[0002] This section merely provides background information related to the present disclosure and is not necessarily prior art.

[0003] As a core task in the field of computer vision, the accuracy of object recognition directly depends on the quality and quantity of training samples. However, traditional sample generation methods face many challenges. On the one hand, manually annotating samples is not only time-consuming and laborious, but also costly and difficult to meet the needs of large-scale datasets. On the other hand, samples generated by simple image processing methods often lack realism and diversity, and cannot fully simulate the complex situations in actual application scenarios.

[0004] In recent years, the rise of oblique photography technology has provided new ideas for 3D modeling. Through images taken from multiple angles and precise location information, oblique photography technology can build high-precision, realistic 3D models. At the same time, the development of 3D simulation technology has also provided new possibilities for sample generation. By constructing 3D simulation scenes under different background conditions, various situations that may occur in actual applications of the target can be simulated. However, how to combine oblique photography technology with 3D simulation technology to achieve the fusion of the target monomer model and the simulation scene and automatically generate sample data still lacks corresponding technical approaches. Summary of the invention

[0005] In view of the deficiencies in the prior art, the purpose of the present invention is to provide an innovative solution to overcome the limitations of traditional sample generation methods and meet the urgent demand for high-quality and diversified samples in the field of target recognition. Technical solution:

[0006] A method for automatically generating target recognition samples based on oblique photography modeling comprises the following steps:

[0007] Step 1, oblique photography modeling, select an area that meets the requirements as the oblique photography target area, the requirements include rich terrain features and diverse types of land objects, and use drones to collect high-precision images of the area; in order to obtain a high-precision model of the target object, the target object can be supplemented with samples to obtain complete oblique photography data of the scene; the oblique photography data is processed to obtain the overall oblique photography model.

[0008] Step 2, oblique photography singulation, cutting and singulation processing is performed on the oblique photography overall model obtained in step 1, and physically cutting the three-dimensional model grid constructed based on oblique photography to obtain an independent target oblique photography singulation model.

[0009] Step 3, 3D simulation scene construction, import the oblique photography model and the target monomer model into the 3D space processing engine, and realize the visualization display of the target in the oblique photography scene through the function of the 3D space processing engine.

[0010] Step 4: Fusing the target monomer model with the three-dimensional simulation scene. Fusing the target monomer model obtained in step 2 with the three-dimensional simulation scene obtained in step 3. Select multiple locations with different terrain features, place the target models respectively, and adjust the scene light brightness and shadow direction through the display parameters of the three-dimensional space processing engine, so that the target and scene are close to the real world.

[0011] Step 5, sample generation, set parameters through the software interface or configuration file, the parameters include state parameters and other parameters, each determined by a set of values, including the start value, end value and adjustment step; the other parameters include the format of sample annotation. According to the parameter settings, the program automatically adjusts the viewing angle of the three-dimensional space processing engine and outputs the simulation sample data in batches.

[0012] Furthermore, the step 1 comprises:

[0013] Step 1-1, data collection, collects oblique photography data of the target area. The oblique photography data includes a large range of terrain features, and in particular, encrypted sampling is performed on the target object to obtain clear shape, contour, color, and light and dark features.

[0014] Step 1-2, data processing, pre-processing the oblique photography data collected in step 1-1, the pre-processing includes aerial triangulation, automatic modeling and mapping, and generating an oblique photography model of the scene.

[0015] Furthermore, the step 2 includes:

[0016] Step 2-1, extracting the target contour, based on the appearance difference between the target and the surrounding ground, the difference includes height difference and color difference, extracting the target contour through edge detection technology, etc. The edge detection technology includes gradient calculation, threshold processing and connectivity analysis, etc.

[0017] Furthermore, the three-dimensional space processing engine functions in step 3 include coordinate conversion and visualization display functions, and the operations include placement, copying and movement of the target in the oblique photography scene.

[0018] Step 2-2, cutting and individualizing processing, preprocessing the extracted target contour, using Boolean operations to cut the oblique photography overall model to obtain the oblique photography monomer model, and enhancing the model details through subdivision modeling and texture mapping.

[0019] Step 2-3, model trimming, by comparing the generated oblique photography monomer model with the actual target, adjust the color difference caused by light exposure to make the model color and texture uniform and reasonable.

[0020] Furthermore, the oblique photography scene in step 3 is an overall scene, from which multiple areas representing different terrain types can be selected to place the target monomer models obtained in step 2 respectively, or three-dimensional models from third-party sources can be placed to form multiple three-dimensional simulation scenes.

[0021] Furthermore, the step 4 includes:

[0022] Step 4-1: Adjust the model pose. Adjust the pose of the target model according to the scene requirements, including position, angle, and scale.

[0023] Step 4-2: For 3D models from third-party sources, you can also adjust the editable components of the model to display the model in various postures.

[0024] Step 4-3, according to user interaction or scene changes, adjust environmental parameters in real time through the three-dimensional space engine, and the environmental parameters include illumination, light, and shadow.

[0025] Furthermore, the target monomer model in step 4 is a three-dimensional model from a third party source.

[0026] Further, step 5 includes,

[0027] Step 5-1, setting parameters, setting parameters through the software interface or configuration file, is used to schedule the viewing angle of the three-dimensional space processing engine. The status information includes the camera viewing angle direction, inclination angle, and height.

[0028] Step 5-2, output the target sample data, the program automatically schedules the camera parameters of the 3D space engine according to the settings, renders the 3D scene in real time, and saves the image; at the same time, it automatically solves the pixel position of the target in the output image and outputs the corresponding annotation file in the annotation format.

[0029] Compared with the prior art, the advantages of the present invention are:

[0030] The present invention proposes a method for generating simulation samples using oblique photography data. The three-dimensional scene generated by the method has high-precision and realistic texture information, and the target model has real color, shape, shadow and light and dark tones, which can accurately reflect the real form and appearance characteristics of the scene and the target. To enhance the authenticity of the sample, the high-precision three-dimensional model and real texture information obtained by oblique photography technology are combined with the background scene constructed by three-dimensional simulation technology to generate a target recognition sample with high fidelity.

[0031] The present invention proposes a method for automatically generating simulation samples, which integrates advanced algorithms and automated processing technologies, realizes automated processing from oblique photography data construction to target recognition sample output, and automatically sets image viewing angle information (such as height, rotation angle, pitch angle, lighting, etc.) and background conditions (such as grassland, bare land, road, forest, etc.), eliminating the manual collection process of real-shot target images and significantly improving the efficiency of sample generation. By setting different target state information (such as height, angle, lighting, etc.), sample data covering a variety of scenes and conditions is generated, improving the generalization ability and recognition accuracy of the target recognition model.

[0032] The user of the present invention can flexibly configure sample generation parameters according to actual needs, such as target type, quantity, status information, etc., to meet the needs of different application scenarios. The system architecture and algorithm design of the present invention have good scalability and can easily integrate new algorithms and technologies to adapt to future technological developments and changes.

[0033] The present invention can be applied to multiple fields such as military reconnaissance, intelligent monitoring, and autonomous driving, and can provide strong support for target recognition tasks in these fields. Through high-quality, multi-angle oblique photography models, it can generate sample data covering a variety of scenarios and conditions, which helps to enhance the generalization ability and robustness of the target recognition model.

[0034] The present invention combines oblique photography technology, three-dimensional simulation technology and automated processing technology to automatically generate target simulation samples that are close to real-life images. In summary, compared with the prior art, the present invention has significant advantages in terms of efficiency, authenticity, diversity, and application prospects. These advantages make the present invention more competitive and practical in the field of automatic generation of target recognition samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more clear.

[0036] Figure 1 It is a schematic flow diagram of the method of the present invention.

[0037] Figure 2It is a sample that automatically generates a software module composition diagram.

[0038] Figure 3 This is a schematic diagram of the automatic sample generation method. DETAILED DESCRIPTION

[0039] The following is a clear and complete description of the technical methods and processes in the embodiments of the present invention. There are many different methods for generating simulation samples. The embodiments described in this article for using simulation to generate simulation samples for three-dimensional scenes and target models are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0040] The purpose of the present invention is to provide a method for generating training samples of a target recognition model, which can improve the generation speed of training samples and the diversity of scenarios.

[0041] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0042] Example:

[0043] The present invention constructs a method for automatically generating samples based on oblique photography, such as Figure 1 As shown; and designed a set of sample automatic generation software, the software module composition is as follows Figure 2 The specific steps for using the software to generate samples are as follows:

[0044] Step 1: Place the target, such as a car, on a field, and then perform a 1km 2 Oblique photography images are taken in the surrounding areas of the area. The image quality is guaranteed to be clear and include rich terrain and objects, such as buildings, trees, grass, asphalt roads, and cement roads. Cars are encrypted and sampled to collect information such as the color and texture of the cars. Using the oblique photography model building tool, the acquired oblique photography data is preprocessed, feature extracted and matched, 3D point cloud generated, triangulated network model built, texture mapped, and a complete 3D model of the area is built.

[0045] Step 2: Use the target model monomerization module to extract the edge contours between the car and the ground by setting a height threshold, and then cut out the monomer model of the car through the contours; perform color correction on the hue and brightness differences of the car's three-dimensional model caused by the influence of light, so that the hue and brightness of the three-dimensional model remain consistent.

[0046] Step 3, import the 3D scene as the background into the 3D space processing engine, and search for terrain areas with typical features through space roaming and browsing; place the target 3D models in these areas respectively, and record the positions of the 3D models through the program.

[0047] Step 4: adjust the size of the target 3D model in the 3D space processing engine so that the size ratio of the car model is consistent with reality, and then adjust the attributes such as the orientation and posture of the car model to obtain a variety of simulation scenes.

[0048] Step 5, set the parameters in the sample attribute setting visualization interface. The initial angle of the viewing direction is 0 degrees, the end angle is 350 degrees, and the interval is 10 degrees; the initial angle of the inclination is 45 degrees, the end angle is 90 degrees, and the interval is 15 degrees; the initial value of the height is 100 meters, the end value is 200 meters, and the interval is 25 meters. The meaning of each parameter is as follows: Figure 3 As shown, then set the annotation file format to coco format. The 3D space processing engine automatically adjusts the camera position according to the above parameters, renders and generates simulation scenes, and saves the pictures at that position in sequence. At the same time, the 3D space processing engine automatically calculates the pixel range of the target 3D model in the picture and saves it as an annotation file according to the sample annotation format.

[0049] The present invention proposes a method for automatically generating target recognition samples based on oblique photography modeling. The method aims to combine oblique photography technology, three-dimensional simulation technology and automated processing technology to automatically generate target simulation samples that are close to real-life images. Through the method of the present invention, the annotation cost can be reduced, the efficiency and accuracy of sample generation can be improved, and rich sample data can be provided for the training of target recognition models. At the same time, the present invention also has broad application prospects and can be applied to multiple fields such as military reconnaissance, intelligent monitoring, and automatic driving.

[0050] The present invention provides a method and idea for automatically generating target recognition samples based on oblique photography modeling. There are many methods and ways to implement the technical solution. The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the protection scope of the present invention. All components not specified in this embodiment can be implemented by existing technologies.

Claims

1. A method for automatically generating target recognition samples based on oblique photography modeling, characterized in that: The following steps are included: Step 1: Oblique photography modeling: select an area that meets the requirements as the oblique photography target area, collect images of the area; process the oblique photography data to obtain the overall oblique photography model; Step 2, individualization of oblique photography, cutting and individualizing the oblique photography overall model obtained in step 1, physically cutting the three-dimensional model grid constructed based on oblique photography, and obtaining an independent oblique photography monomer model of the target; Step 3: constructing a 3D simulation scene, importing the oblique photography model and the target monomer model into the 3D space processing engine, and realizing the visualization display of the target in the oblique photography scene through the function of the 3D space processing engine; Step 4, the target monomer model is integrated with the three-dimensional simulation scene. The target monomer model obtained in step 2 is integrated with the three-dimensional simulation scene obtained in step 3. Multiple locations with different terrain features are selected to place the target models respectively, and the light brightness and shadow direction of the scene are adjusted through the display parameters of the three-dimensional space processing engine. Step 5, sample generation, set parameters through the software interface or configuration file; according to the parameter settings, the program automatically adjusts the observation angle of the three-dimensional space processing engine and outputs simulation sample data in batches.

2. The method for automatically generating target recognition samples based on oblique photography modeling according to claim 1, characterized in that: The step 1 comprises: Step 1-1, data collection, collecting oblique photography data of the target area, which includes a wide range of terrain features and target object features, and performing encrypted sampling on the target object to obtain the shape, contour, color, and light and dark features of the target object; Step 1-2, data processing, pre-processing the oblique photography data collected in step 1-1, the pre-processing includes aerial triangulation, automatic modeling and mapping, and generating an oblique photography model of the scene.

3. The method for automatically generating target recognition samples based on oblique photography modeling according to claim 1, characterized in that: In step 1, image acquisition performs additional sampling of the target object as required.

4. The method for automatically generating target recognition samples based on oblique photography modeling according to claim 1, characterized in that: The step 2 comprises: Step 2-1, extracting the target contour, based on the appearance difference between the target and the surrounding ground, the difference includes height difference and color difference, and extracting the target contour by edge detection technology; Step 2-2, cutting and individualizing processing, pre-processing the extracted target contour, cutting the oblique photography overall model by Boolean operation to obtain the oblique photography monomer model, and enhancing the model details by subdivision modeling and texture mapping; Step 2-3, model trimming, by comparing the generated oblique photography monomer model with the actual target, adjust the color difference caused by light exposure.

5. The method for automatically generating target recognition samples based on oblique photography modeling according to claim 1, characterized in that: The edge detection technology described in step 2-1 includes gradient calculation, threshold processing and connectivity analysis.

6. The method for automatically generating target recognition samples based on oblique photography modeling according to claim 1, characterized in that: The oblique photography scene in step 3 is an overall scene, from which multiple areas representing different terrain types can be selected to place the target monomer models obtained in step 2 respectively, or three-dimensional models from third-party sources can be placed to form multiple three-dimensional simulation scenes.

7. The method for automatically generating target recognition samples based on oblique photography modeling according to claim 6, characterized in that: The three-dimensional space processing engine functions in step 3 include coordinate conversion and visualization display functions, and the operations include placement, copying and movement of the target in the oblique photography scene.

8. The method for automatically generating target recognition samples based on oblique photography modeling according to claim 1, characterized in that: The step 4 comprises: Step 4-1, adjusting the model posture, adjusting the posture of the target monomer model according to the scene requirements, the posture including position, angle and scale; Step 4-2: For a 3D model from a third party, adjust the editable components of the model to display the model in various postures; Step 4-3, according to user interaction or scene changes, adjust environmental parameters in real time through the three-dimensional space engine, and the environmental parameters include illumination, light, and shadow.

9. The method for automatically generating target recognition samples based on oblique photography modeling according to claim 1, characterized in that: Step 5 includes, Step 5-1, setting parameters, setting parameters through the software interface or configuration file, for scheduling the viewing angle of the three-dimensional space processing engine, the state information includes the camera viewing angle direction, inclination angle and height; Step 5-2, output the target sample data, the program automatically schedules the camera parameters of the 3D space engine according to the settings, renders the 3D scene in real time, and saves the image; at the same time, it automatically solves the pixel position of the target in the output image and outputs the corresponding annotation file in the annotation format.

10. The method for automatically generating target recognition samples based on oblique photography modeling according to claim 9, characterized in that: The parameters in step 5 include state parameters and other parameters. The state parameters include observation angle, inclination and height, which are respectively determined by a set of values, including a starting value, an ending value and an adjustment step; the other parameters include the format of sample annotation.

Images