Model generation method and device

Abstract

This application discloses a model generation method, comprising: constructing a virtual scene based on preset scene material requirements; generating synthetic images from different perspectives and environments based on the virtual scene; generating annotated image data corresponding to the synthetic images based on the synthetic images; enhancing the annotated image data using a preset generative adversarial network to obtain enhanced annotated image data; and training a preset model using the enhanced annotated image data to obtain a target model. This method can rapidly synthesize the training data required for computer vision tasks in various aircraft operation scenarios, addressing the high cost of annotating real datasets and the inability to obtain real data for specific scenarios. This method allows for rapid dataset expansion in a short time and at a low cost, thereby improving the robustness and accuracy of computer vision models.

Description

Technical Field

[0001] The present application relates to the field of model generation technology, and in particular to a model generation method and device. Background Art

[0002] In the object detection task, deep learning-based object detection algorithms have completely surpassed those based on non-deep learning methods and have become the mainstream object detection algorithms. However, deep learning-based object detection algorithms rely on large-scale annotated datasets.

[0003] Constructing an object detection dataset is a key technology in the object detection task. The sample size and variety of an object detection dataset largely determine the effectiveness of the object detection algorithm. During the object detection dataset construction process, a large number of images of the target objects to be detected are acquired and then annotated. This process involves identifying each object in the image, assigning its corresponding category, and labeling each object with a box. The data annotation process for dataset construction is tedious and costly. While object detection algorithm research often relies on public datasets, which provide a large number of annotated images and reduce research costs, the limited variety of public datasets often fails to meet specific requirements for specific tasks. Furthermore, in certain scenarios, such as dangerous scenes in autonomous driving and certain non-public targets, target images are difficult to obtain, making data acquisition expensive. Therefore, how to quickly acquire target images, perform data annotation, and complete the object detection dataset construction remains a pressing challenge in the object detection task. Summary of the Invention

[0004] This application provides a model generation method and device that enables the rapid synthesis of training data required for computer vision tasks in various aircraft operation scenarios. It solves the problems of high cost of labeling real data sets and the inability to obtain real data in specific scenarios. It expands the data set in a short time and at a low cost, thereby improving the robustness and accuracy of the computer vision model.

[0005] In a first aspect, the present application provides a model generation method, the method comprising:

[0006] Build virtual scenes according to preset scene material requirements;

[0007] Based on the virtual scene, generating synthetic images in different perspectives and different environments;

[0008] generating, based on the composite image, annotated image data corresponding to the composite image;

[0009] Performing enhancement processing on the annotated image data using a preset generative adversarial network to obtain enhanced annotated image data;

[0010] The enhanced annotated image data is used to train a preset model to obtain a target model.

[0011] Optionally, constructing a virtual scene according to preset scene material requirements includes:

[0012] Construct a local virtual scene of the airport based on the scene near the airport ground;

[0013] Use the Mapbox for Unity plug-in to generate a ground map based on satellite remote sensing images, and obtain airport runway vector graphics from a preset map to match the ground map to generate a virtual airport ground scene;

[0014] generating a three-dimensional model of the airport according to the local virtual scene of the airport and the ground virtual scene of the airport;

[0015] Dynamic actions and behaviors are added to the three-dimensional model of the airport, and the environment parameters of the three-dimensional model of the airport are adjusted using a Unity plug-in to obtain a virtual scene.

[0016] Optionally, generating synthetic images in different perspectives and environments based on the virtual scene includes:

[0017] In a constructed virtual environment, synthetic images in different perspectives and environments are obtained by moving and / or rotating the camera angle.

[0018] Optionally, generating, based on the synthetic image, annotated image data corresponding to the synthetic image includes:

[0019] The synthetic image is annotated using an instance semantic segmentation algorithm, a normal map and depth information acquisition algorithm, and a target detection algorithm to generate annotated image data corresponding to the synthetic image.

[0020] Optionally, the enhancing the annotated image data using a preset generative adversarial network to obtain enhanced annotated image data includes:

[0021] The preset generative adversarial network is used to perform fitting processing on the annotated image data to simulate the difference between the synthetic image and the real image, thereby obtaining enhanced annotated image data.

[0022] In a second aspect, the present application provides a model generation device, the device comprising:

[0023] A scene construction unit, used to construct a virtual scene according to preset scene material requirements;

[0024] An image generating unit, configured to generate synthetic images in different viewing angles and environments based on the virtual scene;

[0025] an annotation generating unit, configured to generate, based on the composite image, annotation image data corresponding to the composite image;

[0026] An image enhancement unit, configured to perform enhancement processing on the annotated image data using a preset generative adversarial network to obtain enhanced annotated image data;

[0027] The model training unit is used to train a preset model using the enhanced annotated image data to obtain a target model.

[0028] Optionally, the scene construction unit is used to:

[0029] Construct a local virtual scene of the airport based on the scene near the airport ground;

[0030] Use the Mapbox for Unity plug-in to generate a ground map based on satellite remote sensing images, and obtain airport runway vector graphics from a preset map to match the ground map to generate a virtual airport ground scene;

[0031] generating a three-dimensional model of the airport according to the local virtual scene of the airport and the ground virtual scene of the airport;

[0032] Dynamic actions and behaviors are added to the three-dimensional model of the airport, and the environment parameters of the three-dimensional model of the airport are adjusted using a Unity plug-in to obtain a virtual scene.

[0033] Optionally, the image generating unit is configured to:

[0034] In a constructed virtual environment, synthetic images in different perspectives and environments are obtained by moving and / or rotating the camera angle.

[0035] Optionally, the annotation generating unit is configured to:

[0036] The synthetic image is annotated using an instance semantic segmentation algorithm, a normal map and depth information acquisition algorithm, and a target detection algorithm to generate annotated image data corresponding to the synthetic image.

[0037] Optionally, the image enhancement unit is configured to:

[0038] The preset generative adversarial network is used to perform fitting processing on the annotated image data to simulate the difference between the synthetic image and the real image, thereby obtaining enhanced annotated image data.

[0039] In a third aspect, the present application provides a readable medium comprising execution instructions. When a processor of an electronic device executes the execution instructions, the electronic device executes any method described in the second aspect.

[0040] In a fourth aspect, the present application provides an electronic device comprising a processor and a memory storing execution instructions. When the processor executes the execution instructions stored in the memory, the processor executes any method described in the first aspect.

[0041] As can be seen from the above technical solution, this application provides a model generation method, which includes: constructing a virtual scene based on preset scene material requirements; generating synthetic images from different perspectives and in different environments based on the virtual scene; generating annotated image data corresponding to the synthetic image based on the synthetic image; enhancing the annotated image data using a preset generative adversarial network to obtain enhanced annotated image data; and training a preset model using the enhanced annotated image data to obtain a target model. This method can quickly synthesize the training data required for computer vision tasks in various aircraft operation scenarios, solving the problems of high cost of annotating real data sets and the inability to obtain real data in specific scenarios. This allows for the expansion of data sets in a short time and at a low cost, thereby improving the robustness and accuracy of computer vision models.

[0042] The further effects of the above-mentioned non-conventional preferred embodiment will be described below in conjunction with specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present application or the existing technical solutions, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0044] Figure 1 A flow chart of a model generation method provided in one embodiment of the present application;

[0045] Figure 2 A flow chart of a model generation method provided in one embodiment of the present application;

[0046] Figure 3 A schematic structural diagram of another model generation device provided in one embodiment of the present application;

[0047] Figure 4 A schematic structural diagram of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0048] To make the objectives, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0049] Various non-limiting embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0050] See also Figure 1 , shows a model generation method in an embodiment of the present application. In this embodiment, the method may include the following steps:

[0051] S101: Construct a virtual scene according to preset scene material requirements.

[0052] In this embodiment, a simulated basic 3D environment can be constructed in the game engine based on the purpose of the visual task. The preset scene material requirements can include scenes near the airport ground, computer vision tasks such as flight, takeoff and landing, and landscape data.

[0053] Specifically, we can first construct a local virtual scene of the airport based on scenes near the airport ground (such as drones flying around for inspection, airport aircraft identification, and other scenes near the airport ground); for example, we can construct a local virtual scene of the airport such as airport ground facilities, buildings, and lights for scenes near the airport ground such as drones flying around for inspection, airport aircraft identification, and other scenes.

[0054] Then, the Mapbox for Unity plugin can be used to generate a ground map based on satellite remote sensing imagery. Vector graphics of the airport runway can be obtained from a pre-set map and matched with the ground map to create a virtual airport ground scene. It is understandable that computer vision tasks such as flight and takeoff and landing require the introduction of a large amount of landscape data. Therefore, the Mapbox for Unity plugin can be used to generate a ground map based on satellite remote sensing imagery and obtain vector graphics of the airport runway from OSM to match the map to meet the needs of building a virtual scene.

[0055] The three-dimensional airport model is generated based on the local virtual scene of the airport and the virtual scene of the airport ground. That is, the local virtual scene of the airport and the virtual scene of the airport ground can be combined according to actual needs to generate the three-dimensional airport model.

[0056] Dynamic actions and behaviors are added to the 3D airport model, and the environmental parameters of the 3D airport model (e.g., weather conditions, lighting conditions, texture colors, and model parameters) are adjusted using Unity plugins to create a virtual scene. It is understood that, after constructing the 3D airport model, dynamic actions and behaviors, such as those for aircraft, ground vehicles, and ground personnel, can be added, and a degree of randomness can be added to their behaviors, depending on the mission requirements. Furthermore, Unity plugins such as Weather can be used to randomly generate special weather conditions and lighting conditions, change texture colors, and parameterize the model, improving the randomness and robustness of the synthesized data.

[0057] S102: Based on the virtual scene, generate synthetic images in different perspectives and different environments.

[0058] Specifically, in a constructed virtual environment, by moving and / or rotating the camera angle, synthetic images from different perspectives and environments can be obtained. It is understood that in a constructed virtual environment, the camera angle can be adjusted as needed or randomly moved and rotated to fully obtain synthetic images from different perspectives and environments.

[0059] S103: Generate annotated image data corresponding to the synthesized image according to the synthesized image.

[0060] In this embodiment, the composite image may be annotated using an instance semantic segmentation algorithm, a normal map and depth information acquisition algorithm, and a target detection algorithm to generate annotated image data corresponding to the composite image.

[0061] For example, you can use the following methods to synthesize various types of annotation data in the Unity engine by adjusting different shader methods:

[0062] A) Principles of instance segmentation / semantic segmentation algorithms:

[0063] Each instance in the randomly generated composite image is assigned an independent ID number and configured in a different layer. A color RGB hash value is generated based on the ID information, where the three RGB colors generate a unique vector based on the ID, which can handle the color encoding of 16,581,375 instances of 255*255*255. The transparency a value indicates the layer it is in. Therefore, when outputting instance segmentation / semantic segmentation annotations, a shader is used to convert the camera rendering into the RGB value corresponding to the object, thereby generating instance segmentation and semantic segmentation annotation data. This method can simultaneously output instance segmentation and semantic segmentation mask annotations, and is highly efficient.

[0064] B) Normal map and depth information:

[0065] The normal map and depth information are directly obtained by using Unity's normals and deepinfo interfaces to obtain the corresponding annotation information of the synthesized image.

[0066] C) Object Detection:

[0067] This paper aims to address the problem of poor target detection annotation generation accuracy in traditional methods. This application combines pixel-level high-precision instance segmentation annotation synthetic data. First, based on the vertex coordinates of the object to be identified in the synthetic image, the vertex world coordinates are converted into screen coordinates to obtain the approximate range of the detection target; then, the instance segmentation annotation data is used to obtain the position information of each instance in the image, so as to correct the Boundingbox coordinates obtained based on the target vertex. Through this method, it is possible to obtain BBOX coordinate points with higher precision, and it is possible to avoid the problem of poor algorithm robustness and lack of associativity caused by obtaining coordinates only from the instance segmentation dataset.

[0068] S104: Using a preset generative adversarial network to perform enhancement processing on the annotated image data to obtain enhanced annotated image data.

[0069] Specifically, a preset generative adversarial network may be used to perform fitting processing on the annotated image data to simulate the difference between the synthetic image and the real image, thereby obtaining enhanced annotated image data.

[0070] Understandably, while virtual environments are often meticulously designed to mimic real-world environments as closely as possible, differences in rendering style, lighting, reflections, materials, and other details often lead to differences between virtual and real-world scenes. This can lead to overfitting computer vision models to the synthetic datasets when training them, impacting model performance. To address this issue, this method employs a generative adversarial network (CycleGAN) to approximate the differences between simulated and real-world images.

[0071] In addition, you can also use practical tools such as dataset export and control to obtain enhanced annotated image data. According to different task requirements, practical function scripts such as random multi-object generation, random walk behavior, video recording or single image generation control UI, synthetic dataset management, etc. have been developed to meet different task requirements.

[0072] S105: Using the enhanced annotated image data to train a preset model to obtain a target model.

[0073] It is understandable that the preset model can be trained using enhanced annotated image data so that a target model can be obtained, which can then be used to identify situations in the airport, such as identifying individual instances, identifying the airport environment, identifying aircraft in the image, etc.

[0074] It is understandable that if Figure 2 As shown in the figure, the implementation process of this method is mainly divided into three stages: the first stage is the construction of the virtual scene, including the construction of the environment, maps, objects, behaviors, etc.; after the scene construction is completed, the shader rendering technology of the Unity engine and the self-developed ray-based algorithm are used to synthesize the labeled data set required for the visual task, and the generative adversarial network is used to enhance the display effect of the simulation environment; the third stage is to use the synthesized data set to train and test the deep learning model of the computer vision task, and quickly generate the model required for the task.

[0075] As can be seen from the above technical solution, this application provides a model generation method, which includes: constructing a virtual scene based on preset scene material requirements; generating synthetic images from different perspectives and in different environments based on the virtual scene; generating annotated image data corresponding to the synthetic image based on the synthetic image; enhancing the annotated image data using a preset generative adversarial network to obtain enhanced annotated image data; and training a preset model using the enhanced annotated image data to obtain a target model. This method can quickly synthesize the training data required for computer vision tasks in various aircraft operation scenarios, solving the problems of high cost of annotating real data sets and the inability to obtain real data in specific scenarios. This allows for the expansion of data sets in a short time and at a low cost, thereby improving the robustness and accuracy of computer vision models.

[0076] In other words, this application proposes a method to construct a simulation environment that approximates various scenes of real commercial aircraft to generate simulated synthetic image datasets in various scenes. Among them, this proposal uses Unity3D rendering software to simulate environmental changes in artificial scenes and automatically generates real labels, including semantic / instance segmentation, object bounding boxes, depth maps, normal maps, etc. In addition, after generating simulated synthetic images or videos, the classic adversarial generative network model (CycleGAN) is innovatively used to reduce the difference between synthetic images and real images and improve the fidelity of synthetic images. The method of this application can quickly synthesize the training data required for computer vision tasks in various aircraft operation scenarios, solving the problem of high cost of real data set annotation and the inability to obtain real data in specific scenarios, thereby expanding the data set in a short time and at low cost to improve the robustness and accuracy of computer vision models. It can be understood that this application adopts a synthetic image / video generation method based on a game engine, a simulation image enhancement method based on a generative adversarial network, and an image synthesis method for various aircraft scenes, such as the combination of simulated images and remote sensing data in flight scenes; in addition, it can support continuous dynamic scenes and synthesize video data; the picture effect can be improved using a generative adversarial network; in addition, it has a wide range of uses and can support the output of 6 types of synthetic data annotations: target detection, strength segmentation, semantic segmentation, normals, depth information, and synthetic images.

[0077] like Figure 3 The device described in this embodiment is a physical device for executing the model generation method described in the above embodiment, and is used to Figure 1 The model generation system shown in FIG. Its technical solution is essentially consistent with the above embodiment, and the corresponding description in the above embodiment is also applicable to this embodiment. The device in this embodiment includes:

[0078] The scene construction unit 301 is used to construct a virtual scene according to preset scene material requirements;

[0079] An image generating unit 302 is configured to generate synthetic images in different viewing angles and environments based on the virtual scene;

[0080] An annotation generating unit 303, configured to generate annotated image data corresponding to the composite image based on the composite image;

[0081] The image enhancement unit 304 is configured to perform enhancement processing on the annotated image data using a preset generative adversarial network to obtain enhanced annotated image data;

[0082] The model training unit 305 is used to train a preset model using the enhanced annotated image data to obtain a target model.

[0083] Optionally, the scene construction unit 301 is configured to:

[0084] Construct a local virtual scene of the airport based on the scene near the airport ground;

[0085] Use the Mapbox for Unity plug-in to generate a ground map based on satellite remote sensing images, and obtain airport runway vector graphics from a preset map to match the ground map to generate a virtual airport ground scene;

[0086] generating a three-dimensional model of the airport according to the local virtual scene of the airport and the ground virtual scene of the airport;

[0087] Dynamic actions and behaviors are added to the three-dimensional model of the airport, and the environment parameters of the three-dimensional model of the airport are adjusted using a Unity plug-in to obtain a virtual scene.

[0088] Optionally, the image generating unit 302 is configured to:

[0089] In a constructed virtual environment, synthetic images in different perspectives and environments are obtained by moving and / or rotating the camera angle.

[0090] Optionally, the annotation generating unit 303 is configured to:

[0091] The synthetic image is annotated using an instance semantic segmentation algorithm, a normal map and depth information acquisition algorithm, and a target detection algorithm to generate annotated image data corresponding to the synthetic image.

[0092] Optionally, the image enhancement unit 304 is configured to:

[0093] The preset generative adversarial network is used to perform fitting processing on the annotated image data to simulate the difference between the synthetic image and the real image, thereby obtaining enhanced annotated image data.

[0094] Figure 4 : This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. Among them, the memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0095] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0096] Memory is used to store execution instructions. Specifically, execution instructions are computer programs that can be executed. Memory can include internal memory and non-volatile memory, and provides execution instructions and data to the processor.

[0097] In one possible implementation, a processor reads corresponding execution instructions from a non-volatile memory into a memory and then executes them. Alternatively, the processor may obtain corresponding execution instructions from another device to form a model generation device at a logical level. The processor executes the execution instructions stored in the memory to implement the model generation method provided in any embodiment of the present application through the execution of the execution instructions.

[0098] The above application Figure 1 The method performed by the model generation device provided in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or instructions in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The various methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0099] The steps of the method disclosed in the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0100] An embodiment of the present application also proposes a readable storage medium, which stores execution instructions. When the stored execution instructions are executed by a processor of an electronic device, the electronic device can execute the model generation method provided in any embodiment of the present application, and is specifically used to execute the above-mentioned model generation method.

[0101] The electronic device described in each of the aforementioned embodiments may be a computer.

[0102] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods or computer program products. Therefore, the present application may adopt a completely hardware embodiment, a completely software embodiment, or a combination of software and hardware.

[0103] The various embodiments in this application are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences from other embodiments. In particular, the device embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the partial description of the method embodiments.

[0104] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0105] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A model generation method, characterized in that: The method comprises: Build virtual scenes according to preset scene material requirements; Based on the virtual scene, generating synthetic images in different perspectives and different environments; generating, based on the composite image, annotated image data corresponding to the composite image; Performing enhancement processing on the annotated image data using a preset generative adversarial network to obtain enhanced annotated image data; Using the enhanced annotated image data to train a preset model to obtain a target model; The step of generating annotated image data corresponding to the composite image according to the composite image includes: Annotating the composite image using an instance semantic segmentation algorithm, a normal map and depth information acquisition algorithm, and an object detection algorithm to generate annotated image data corresponding to the composite image; Use the following method to synthesize various types of labeled data: Each instance in the randomly generated synthetic image is assigned an independent ID number and placed in a different layer. A color RGB hash value is generated based on the ID information, where the three RGB colors generate a unique vector based on the ID, which can support the color encoding of 16,581,375 instances with a 255*255*255 color space. The transparency a value indicates the layer it is in. Therefore, when outputting instance segmentation / semantic segmentation annotations, a shader is used to convert the camera rendering into the RGB value corresponding to the object, thereby generating instance segmentation and semantic segmentation annotation data. The normal map and depth information are directly obtained by using Unity's normals and deepinfo interfaces to obtain the corresponding annotation information of the synthesized image; First, based on the vertex coordinates of the object to be identified in the synthetic image, the vertex world coordinates are converted to screen coordinates to obtain the approximate range of the detection target; then, the instance segmentation annotation data is used to obtain the position information of each instance in the image, thereby correcting the bounding box coordinates obtained based on the target vertex.

2. The method according to claim 1, characterized in that The virtual scene is constructed according to the preset scene material requirements, including: Construct a local virtual scene of the airport based on the scene near the airport ground; Use the Mapbox for Unity plug-in to generate a ground map based on satellite remote sensing images, and obtain airport runway vector graphics from a preset map to match the ground map to generate a virtual airport ground scene; generating a three-dimensional model of the airport according to the local virtual scene of the airport and the ground virtual scene of the airport; Dynamic actions and behaviors are added to the three-dimensional model of the airport, and the environment parameters of the three-dimensional model of the airport are adjusted using a Unity plug-in to obtain a virtual scene.

3. The method according to claim 1, characterized in that Generating synthetic images in different perspectives and environments based on the virtual scene includes: In a constructed virtual environment, synthetic images in different perspectives and environments are obtained by moving and / or rotating the camera angle.

4. The method according to claim 1, wherein The method of using a preset generative adversarial network to enhance the annotated image data to obtain enhanced annotated image data includes: The preset generative adversarial network is used to perform fitting processing on the annotated image data to simulate the difference between the synthetic image and the real image, thereby obtaining enhanced annotated image data.

5. A model generation device, characterized in that: The device comprises: A scene construction unit, used to construct a virtual scene according to preset scene material requirements; An image generating unit, configured to generate synthetic images in different viewing angles and environments based on the virtual scene; an annotation generating unit, configured to generate, based on the composite image, annotation image data corresponding to the composite image; An image enhancement unit, configured to perform enhancement processing on the annotated image data using a preset generative adversarial network to obtain enhanced annotated image data; A model training unit, configured to train a preset model using the enhanced annotated image data to obtain a target model; The step of generating annotated image data corresponding to the composite image according to the composite image includes: Annotating the composite image using an instance semantic segmentation algorithm, a normal map and depth information acquisition algorithm, and an object detection algorithm to generate annotated image data corresponding to the composite image; Use the following method to synthesize various types of labeled data: Each instance in the randomly generated synthetic image is assigned an independent ID number and placed in a different layer. A color RGB hash value is generated based on the ID information, where the three RGB colors generate a unique vector based on the ID, which can support the color encoding of 16,581,375 instances with a 255*255*255 color space. The transparency a value indicates the layer it is in. Therefore, when outputting instance segmentation / semantic segmentation annotations, a shader is used to convert the camera rendering into the RGB value corresponding to the object, thereby generating instance segmentation and semantic segmentation annotation data. The normal map and depth information are directly obtained by using Unity's normals and deepinfo interfaces to obtain the corresponding annotation information of the synthesized image; First, based on the vertex coordinates of the object to be identified in the synthetic image, the vertex world coordinates are converted to screen coordinates to obtain the approximate range of the detection target; then, the instance segmentation annotation data is used to obtain the position information of each instance in the image, thereby correcting the bounding box coordinates obtained based on the target vertex.

6. The device according to claim 5, characterized in that The scene construction unit is used to: Construct a local virtual scene of the airport based on the scene near the airport ground; Use the Mapbox for Unity plug-in to generate a ground map based on satellite remote sensing images, and obtain airport runway vector graphics from a preset map to match the ground map to generate a virtual airport ground scene; generating a three-dimensional model of the airport according to the local virtual scene of the airport and the ground virtual scene of the airport; Dynamic actions and behaviors are added to the three-dimensional model of the airport, and the environment parameters of the three-dimensional model of the airport are adjusted using a Unity plug-in to obtain a virtual scene.

7. The device according to claim 5, characterized in that The image generating unit is configured to: In a constructed virtual environment, synthetic images in different perspectives and environments are obtained by moving and / or rotating the camera angle.

8. A readable medium, characterized in that The method comprises an execution instruction. When a processor of an electronic device executes the execution instruction, the electronic device executes the method according to any one of claims 1 to 4.

9. An electronic device, characterized in that: The method comprises a processor and a memory storing execution instructions. When the processor executes the execution instructions stored in the memory, the processor executes the method according to any one of claims 1 to 4.

Images