Method and system for generating image data set, and storage medium
By constructing three-dimensional map and dynamic layer models, the image data generation model is generated, and the problems of low efficiency and poor consistency of image data set generation in the existing technology are solved, and efficiently generating image data sets with high applicability in the virtual world are achieved.
Patent Information
- Application Number
- CN202010591846.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-06-24
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2040-11-26
AI Technical Summary
The existing image dataset generation methods have problems such as weak real-time, poor consistency and low generation efficiency, and it is difficult to quickly obtain rich image datasets in various scenarios in the real physical world.
By acquiring map data, building a three-dimensional map model and dynamic layer model, using training data and image data for model training, generating image data generation models, realizing simulation shooting of infinite scenes in the virtual world, and improving the real-time and consistency of image data sets sex.
The generation efficiency and quality of image data sets are improved, and the rich and highly applicable image data sets for testing can be quickly generated in the virtual world, solving the problem of insufficient real-time and consistency in the prior art.
Smart Images

Figure CN111784600B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of virtual reality technology, and in particular to a method and system for generating an image data set, and a storage medium. Background Art
[0002] At present, when testing the shooting function of a terminal, it is necessary to first collect an image dataset, and then adjust and optimize the shooting function of the terminal based on the image dataset. The generation of image datasets mainly includes artificial real-life scene shooting and smartphone camera noise reduction dataset (Smartphone Image Denoising Dataset, SIDD).
[0003] However, the image datasets obtained by artificial on-site real-scene shooting have defects such as poor consistency, limited time and equipment resources, and inability to reproduce the same scene. At the same time, the number and scenes of the image datasets obtained through SIDD are also very limited, and the real-time performance is also relatively poor.
[0004] It can be seen that it is unrealistic to complete countless shots in the real physical world to obtain image datasets in various scenarios. The existing methods of generating image datasets generally have problems such as weak real-time performance, poor consistency, and low generation efficiency. Summary of the invention
[0005] The embodiments of the present application provide a method and system for generating an image data set, and a storage medium, which can ensure the real-time and consistency of the image data when generating the image data set, and can greatly improve the generation efficiency and data quality of the image data set.
[0006] The technical solution of the embodiment of the present application is implemented as follows:
[0007] In a first aspect, an embodiment of the present application provides a method for generating an image data set, the method comprising:
[0008] Acquire map data, and model the static layer according to the map data to obtain a three-dimensional map model;
[0009] Modeling the dynamic layer based on the three-dimensional map model and the dynamic parameters to obtain a first virtual reality model;
[0010] The first virtual reality model is trained using the first training data and the first image data to obtain a second virtual reality model; wherein the first image data is a real image captured by the first terminal based on the first training data;
[0011] The second virtual reality model is trained using the second training data and the second image data to obtain an image data generation model; wherein the second image data is a real image captured by a second terminal based on the second training data; and the second terminal is a terminal other than the first terminal;
[0012] An image data set is generated by the image data generation model; wherein the image data set is used to test a shooting function.
[0013] In a second aspect, an embodiment of the present application provides a system for generating an image data set, wherein the system for generating an image data set comprises: an acquisition unit, a construction unit, a training unit, a generation unit,
[0014] The acquisition unit is used to acquire map data;
[0015] The construction unit is used to model the static layer according to the map data to obtain a three-dimensional map model; and to model the dynamic layer based on the three-dimensional map model and dynamic parameters to obtain a first virtual reality model;
[0016] The training unit is used to train the first virtual reality model using first training data and first image data to obtain a second virtual reality model; wherein the first image data is a real image captured by the first terminal based on the first training data; and to train the second virtual reality model using second training data and second image data to obtain an image data generation model; wherein the second image data is a real image captured by the second terminal based on the second training data; and the second terminal is a terminal other than the first terminal;
[0017] The generating unit is used to generate an image data set by using the image data generating model; wherein the image data set is used to test a shooting function.
[0018] In a third aspect, an embodiment of the present application provides a system for generating an image dataset, wherein the system for generating an image dataset comprises a processor and a memory storing instructions executable by the processor, and when the instructions are executed by the processor, the method for generating an image dataset as described above is implemented.
[0019] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a program stored thereon, which is applied to a system for generating an image data set, and when the program is executed by a processor, the method for generating an image data set as described above is implemented.
[0020] The embodiment of the present application provides a method and system for generating an image data set, and a storage medium. The system for generating the image data set acquires map data, and models a static layer according to the map data to obtain a three-dimensional map model; models a dynamic layer based on the three-dimensional map model and dynamic parameters to obtain a first virtual reality model; trains the first virtual reality model using first training data and first image data to obtain a second virtual reality model; wherein the first image data is a real image captured by a first terminal based on the first training data; trains the second virtual reality model using second training data and second image data to obtain an image data generation model; wherein the second image data is a real image captured by a second terminal based on the second training data; the second terminal is a terminal other than the first terminal; generates an image data set through the image data generation model; wherein the image data set is used to test a shooting function. It can be seen that in the embodiments of the present application, the image data set generation system can perform static layer model and dynamic layer model training based on map data and dynamic parameters to obtain a first virtual reality model, and then use the first training data and the corresponding first image data, and the second training data and the corresponding second image data to perform model training, and then obtain an image data generation model, wherein the image data generation model can be used to simulate shooting of infinite scenes in the virtual world, so the image data model can be used to quickly generate a rich and highly applicable image data set for testing shooting functions. Therefore, when generating an image data set, the real-time and consistency of the image data can be guaranteed, and the generation efficiency and data quality of the image data set can be greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Figure 1 This is a flow chart of SIDD establishment;
[0022] Figure 2 It is a test process diagram;
[0023] Figure 3 Schematic diagram of the implementation process of the method for generating image datasets Figure 1 ;
[0024] Figure 4 Schematic diagram of the implementation process of the method for generating image datasets Figure 2 ;
[0025] Figure 5 Schematic diagram of the implementation process of the method for generating image datasets Figure 3 ;
[0026] Figure 6 Schematic diagram of the implementation process of the method for generating image datasets Figure 4 ;
[0027] Figure 7Schematic diagram of the implementation process of the method for generating image datasets Figure 5 ;
[0028] Figure 8 Schematic diagram of the implementation process of the method for generating image datasets Figure 6 ;
[0029] Fig. 9 Schematic diagram of the implementation process of the method for generating image datasets Figure 7 ;
[0030] Fig.10 Schematic diagram of the composition structure of the system for generating image datasets Figure 1 ;
[0031] Fig.11 Schematic diagram of the composition structure of the system for generating image datasets Figure 2 . DETAILED DESCRIPTION
[0032] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. It is understood that the specific embodiments described herein are only used to explain the related applications, rather than to limit the applications. It should also be noted that, for ease of description, only the parts related to the related applications are shown in the drawings.
[0033] With the continuous development of terminal technology and image processing technology, shooting technology has undergone tremendous changes, from the original professional digital single lens reflex camera (DSLR) to a simpler point-and-shoot camera on a smart terminal. Compared with DSLR, smart terminals will generate more noise than SLRs due to the size of the aperture and sensor. To solve this problem, it is necessary to study the noise reduction problem of cameras in smart terminals. Specifically, before the terminal is put into use, its shooting function should be extensively tested using image data sets.
[0034] Currently, there are two main ways to generate image datasets for testing terminal shooting functions. One method is to manually take photos and videos on the spot, such as DXO Mark, using different terminals to capture images of the same scene and object to obtain multiple frames of images corresponding to the scene and object.
[0035] Another way to generate image datasets is to directly use public datasets, such as the smartphone camera noise reduction dataset SIDD, in which SIDD uses 5 representative mobile phones to shoot about 10 scenes with a total of 30,000 noisy images under different lighting conditions, as well as corresponding "noise-free" real images. Specifically, Figure 1The schematic diagram of SIDD establishment process is as follows: Figure 1 As shown in the figure, first you need to input the acquired image data, and then perform defective pixel correction, outlier image removal, intensity alignment, dense local image alignment and other processing on the input image sequence in turn, so as to remove erroneous pixels and erroneous images, and complete the image alignment. Finally, you can generate a "noise-free" real image through robust average image estimation.
[0036] Input image sequence, defective pixel correction, outlier image removal, intensity alignment, dense local image alignment, robust average image estimation, ground truth image
[0037] At present, in the actual testing process of the shooting function, the terminal shooting function is often adjusted and calibrated by combining the image data sets obtained by the above two methods. Figure 2 The test flow diagram is as follows: Figure 2 As shown, the shooting function of the terminal can be tested based on SIDD first, and the shooting parameters of the shooting device in the terminal can be adjusted according to the test results. Then, the image of the actual scene can be captured based on the adjusted shooting parameters to obtain image data. Finally, the shooting parameters of the shooting device in the terminal can be further adjusted based on the image data and SIDD, thereby completing the testing and optimization of the shooting function of the terminal.
[0038] However, on the one hand, field shooting under different conditions of time, people, places, things, light, weather, etc. often takes a long time, such as several years or even decades, and in the process of image acquisition, there are many uncontrollable factors such as environment and climate, which will cause inconsistencies in other shooting conditions besides variables; that is, shooting images of real scenes on the spot has defects such as poor consistency, limited time and equipment resources, and inability to reproduce the same scene.
[0039] On the other hand, the number and scenes of image data sets obtained through SIDD are also very limited, and the real-time performance of image data obtained by this method is also relatively poor.
[0040] It can be seen that it is unrealistic to complete countless shots in the real physical world to obtain image datasets in various scenarios. The existing methods of generating image datasets generally have problems such as weak real-time performance, poor consistency, and low generation efficiency.
[0041] In order to solve the problems existing in the existing decoding process, in an embodiment of the present application, the image data set generation system can train the static layer model and the dynamic layer model based on the map data and dynamic parameters to obtain the first virtual reality model, and then use the first training data and the corresponding first image data, and the second training data and the corresponding second image data to perform model training, so as to obtain the image data generation model, wherein the image data generation model can be used to simulate shooting of infinite scenes in the virtual world, so the image data model can be used to quickly generate a rich and highly applicable image data set for testing the shooting function. Therefore, when generating the image data set, the real-time and consistency of the image data can be guaranteed, and the generation efficiency and data quality of the image data set can be greatly improved.
[0042] Specifically, the image data set generation method proposed in this application can realize the shooting and processing of unlimited scenes in virtual reality by constructing an image data generation model, so as to generate an image data set that can be used to test the shooting function of the terminal. Among them, the image data set generation system can integrate technologies such as simultaneous localization and mapping (SLAM), virtual reality (VR) and machine learning (ML) to establish an image data generation model, so as to create various permutation and combination scenes in the virtual world, and then simulate shooting to obtain unlimited image data.
[0043] It can be seen that the image data generation model can be used to quickly and efficiently input complete and comprehensive image data in the virtual world according to the needs of actual occasions, thereby improving the efficiency and quality of image data generation, saving time and improving image quality.
[0044] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.
[0045] An embodiment of the present application provides a method for generating an image data set. Figure 3 Schematic diagram of the implementation process of the method for generating image datasets Figure 1 ,like Figure 3 As shown, in an embodiment of the present application, the method for the system to generate an image data set may include the following steps:
[0046] Step 101: Obtain map data, and model the static layer according to the map data to obtain a three-dimensional map model.
[0047] In an embodiment of the present application, the system for generating an image data set may first acquire map data, and then perform modeling processing on the static layer based on the map data, thereby completing the construction of a three-dimensional map model.
[0048] It can be understood that, in the embodiment of the present application, the map data is real-world geographic location map data.
[0049] It should be noted that in the embodiments of the present application, the image data set generation system can obtain map data in a variety of ways. For example, the image data set generation system can obtain map data through a geographic information system (GIS). Among them, the geographic information system GIS is a computer system used to input, store, query, analyze and display geographic data. The geographic information system combines geography and cartography and is widely used in more than 100 fields such as surveying and mapping, resource management, disaster monitoring, urban and rural planning, national defense, environmental protection, and macro decision support.
[0050] Furthermore, in the embodiments of the present application, the image data set generation system can also combine the data of maps and satellite images to obtain map data. Specifically, in addition to professional geographic information systems, with the booming development of the Internet and mobile Internet, more and more commercial geographic information systems are available to people in their daily lives, such as Google Maps, Bing Maps, Baidu Maps, and AutoNavi. These systems provide great convenience to our lives.
[0051] Furthermore, in an embodiment of the present application, when the image data set generation system models the static layer according to the map data to obtain the three-dimensional map model, it can first mark the different feature data in the map data to obtain the marked data, and then build the static layer model based on the marked data to obtain the three-dimensional map model. The marked data can reflect the relationship between the number, characteristics and spatial positions of the objects in the real world.
[0052] That is to say, in an embodiment of the present application, after acquiring the map data, the image data set generation system needs to first label the different feature data in the map data, for example, label the data representing buildings, roads, lane markings, plants, scenery and other features in the map data, obtain the labeled data, and then use the labeled data to construct a three-dimensional map model, so that the obtained three-dimensional map model can more realistically restore the buildings, roads, lane markings, plants, scenery and other features in the real world.
[0053] It should be noted that, in the embodiments of the present application, the three-dimensional map model constructed by the image data set generation system can reflect the real world and simulate static objects in the real world to the greatest extent. Specifically, the three-dimensional map model is a stereoscopic digital map of the real world.
[0054] It should be noted that in the embodiments of the present application, the image data set generation system can use simultaneous localization and mapping (SLAM), also known as real-time localization and mapping, when constructing a three-dimensional map model. The problem can be described as: if a robot is placed in an unknown location in an unknown environment, is there a way to allow the robot to gradually draw a complete map of the environment while moving, where a complete map means that the robot can move to every accessible corner of the room without obstacles.
[0055] Furthermore, in an embodiment of the present application, when constructing a three-dimensional map model, the system for generating an image data set can complete the modeling of a static layer based on the map data through computer vision and deep learning algorithms.
[0056] Computer Vision (CV) is a science that studies how to make machines "see". To put it more specifically, it refers to the use of cameras and computers to replace human eyes to identify, track and measure targets, and further process images, using computers to process them into images that are more suitable for human observation or transmission to instruments for detection. Among them, computer vision can also be regarded as a science that studies how to make artificial systems "perceive" from images or multidimensional data.
[0057] Machine Learning (ML) is a multi-disciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. It is the core of artificial intelligence and the fundamental way to make computers intelligent. Among them, artificial intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence.
[0058] It can be seen that through the acquired map data, the image dataset generation system uses computer vision and machine learning to restore the real world as closely as possible.
[0059] It should be noted that, in an embodiment of the present application, a system for generating an image data set that executes a method for generating an image data set may be a system integrated by at least one terminal device and supporting a shooting function in a virtual reality (VR) environment, wherein the at least one terminal device refers to any terminal with communication and storage functions, such as: a tablet computer, a mobile phone, an e-reader, a remote control, a personal computer (PC), a laptop computer, a vehicle-mounted device, an Internet TV, a wearable device, a personal digital assistant (PDA), a portable media player (PMP), a navigation device, and other terminals.
[0060] Among them, virtual reality (VR) uses computer simulation to create a three-dimensional virtual world, providing users with simulations of vision and other senses, allowing users to feel as if they are in the real world and observe things in the three-dimensional space instantly and without restrictions. This technology integrates the latest developments in computer graphics, computer simulation, artificial intelligence, sensing, display and network parallel processing, and is a high-tech simulation system generated with the assistance of computer technology.
[0061] That is to say, the system for generating an image data set proposed in the present application can obtain an image data set by providing a shooting function in virtual reality.
[0062] Step 102: Model the dynamic layer based on the three-dimensional map model and the dynamic parameters to obtain a first virtual reality model.
[0063] In an embodiment of the present application, after the image data set generation system acquires map data and models the static layer according to the map data to obtain a three-dimensional map model, it can construct a model of the dynamic layer based on the three-dimensional map model and dynamic parameters, thereby obtaining a first virtual reality model.
[0064] It is understandable that in the embodiments of the present application, the first virtual reality model can be considered to be obtained by adding a dynamic layer model to a static layer model (three-dimensional map model). Specifically, the image data set generation system can add dynamic factors including moving vehicles, walking people, changing light, rain, snow, wind and other climatic conditions to the three-dimensional map model, thereby realizing the construction of a dynamic layer model based on a static layer model.
[0065] It should be noted that in an embodiment of the present application, when constructing a dynamic layer model, the image data set generation system can also add different shooting parameters to the three-dimensional map model, so as to more accurately simulate the image acquisition process of different scenes and different shooting conditions, and finally complete the construction of the first virtual reality model.
[0066] Furthermore, in an embodiment of the present application, the dynamic parameters may include dynamic object parameters, dynamic environment parameters, and shooting parameters.
[0067] For example, in the present application, dynamic object parameters can be used to simulate and set people or objects in the real world whose states are changing, so that dynamically changing people and objects such as walking pedestrians and moving vehicles can be added to the constructed three-dimensional map model.
[0068] Exemplarily, in the present application, dynamic environmental parameters can be used to simulate and set the changing weather and environment in the real world, so that factors such as changing climate, light, etc. such as day and night, cloudy and rainy weather, snow, and the changing seasons can be added to the constructed three-dimensional map model.
[0069] Exemplarily, in the present application, the shooting parameters can be used to simulate and set different shooting devices and parameters used in the shooting process, so that factors of changing shooting conditions can be added to the constructed three-dimensional map model.
[0070] Specifically, in the present application, shooting parameters may include configuration parameters of different terminals, and may also include parameters used in the shooting process such as shutter, aperture, sensitivity, exposure value, focal length, etc. Furthermore, it may also include setting parameters for whether to turn on the flash, anti-shake and other functions.
[0071] Further, in the embodiment of the present application, when the image data set generation system models the dynamic layer based on the three-dimensional map model and the dynamic parameters, the three-dimensional map model can be trained using the dynamic object parameters, the dynamic environment parameters and the shooting parameters, so as to obtain the first virtual reality model. It can be seen that the addition of the dynamic object parameters, the dynamic environment parameters and the shooting parameters can enable the first virtual reality model to simulate the dynamic factors existing in the shooting process on the basis of simulating the real world.
[0072] It can be understood that in the embodiment of the present application, the first virtual reality model can be a model preliminarily constructed based on the three-dimensional map model. Whether the selection and setting of dynamic parameters are appropriate and whether the shooting simulation effect of the first virtual reality model is accurate still needs further testing.
[0073] Step 103: Use the first training data and the first image data to train the first virtual reality model to obtain a second virtual reality model; wherein the first image data is a real image captured by the first terminal based on the first training data.
[0074] In an embodiment of the present application, after completing the construction of the static layer model and the static layer model to obtain the first simulated reality model, the image data set generation system can first use the first training data and the first image data to train the first virtual reality model, so as to obtain the second virtual reality model.
[0075] It should be noted that, in the embodiment of the present application, the first image data may be a real image captured by the first terminal based on the first training data. That is, under the shooting conditions formed by the first training data, the first terminal performs image acquisition to obtain the first image data.
[0076] It is understandable that in the present application, for a set of training data, the first terminal can capture and obtain a corresponding unique frame of image, that is, unique first image data. In other words, any change in any data in the first training data will make the obtained first image data different.
[0077] Exemplarily, in an embodiment of the present application, the first terminal can be any terminal with shooting and storage functions, such as: a tablet computer, a mobile phone, an e-reader, a remote control, a personal computer (PC), a laptop computer, a vehicle-mounted device, an Internet TV, a wearable device, a personal digital assistant (PDA), a portable media player (PMP), a navigation device, and other terminals.
[0078] Further, in an embodiment of the present application, the first training data may determine all dynamic data when the first terminal collects the first image data. Specifically, in the present application, the first training data may include first object data, first environment data, and first shooting data.
[0079] It can be understood that in an embodiment of the present application, the first object data can represent a person or object whose state is changing when the first terminal collects the first image data, so that the first object data can be used to restore dynamically changing people and objects such as walking passers-by and moving vehicles in the process of collecting the first image data.
[0080] It can be understood that in the embodiments of the present application, the first environmental data can characterize the changing weather and environment when the first terminal collects the first image data, so that the changing climate, light and other factors such as day and night, sunny, rainy, snowy, and seasons during the process of collecting the first image data can be restored through the first environmental data.
[0081] It is understandable that in the embodiments of the present application, the first shooting data may represent the shooting conditions used by the first terminal when collecting the first image data, so that the shooting conditions set in the process of collecting the first image data can be restored through the first shooting data. The first shooting data may include the configuration parameters of the first terminal, and may also include parameters such as shutter, aperture, sensitivity, exposure value, focal length, etc. used by the first terminal in the process of shooting the first image data, and further, may also include setting parameters of whether to turn on the flash, anti-shake, etc.
[0082] Furthermore, in the embodiments of the present application, Figure 4 Schematic diagram of the implementation process of the method for generating image datasets Figure 2 ,like Figure 4 As shown, before the image dataset generation system trains the first virtual reality model using the first training data and the first image data to obtain the second virtual reality model, that is, before step 103, the method for generating the image dataset by the image dataset generation system may further include the following steps:
[0083] Step 106: When the first terminal collects the first image data, the first object data, the first environment data and the first shooting data are obtained.
[0084] Step 107: determine the first object data, the first environment data, and the first shooting data as first training data.
[0085] In an embodiment of the present application, when the first terminal collects the first image data, the image data set generation system can detect the first object data, the first environment data and the first shooting data, and then the first object data, the first environment data and the first shooting data obtained by the detection can be determined as the first training data.
[0086] It can be understood that in the embodiments of the present application, when the first terminal collects the first image data, the image data set generation system can detect dynamic people and objects appearing in the first image data, thereby obtaining the first object data; correspondingly, when the first terminal collects the first image data, the image data set generation system can also detect environmental and climate data such as light, wind and rain, thereby obtaining the first environmental data.
[0087] Furthermore, in an embodiment of the present application, the system for generating the image data set may determine the configuration parameters of the first terminal and the shooting parameters used by the first terminal when collecting the first image data, thereby obtaining the first shooting data.
[0088] That is, the system for generating the image data set can complete the detection and acquisition of the first training data including the first object data, the first environment data and the first shooting data when the first terminal collects the first image data.
[0089] Furthermore, in the embodiments of the present application, Figure 5 Schematic diagram of the implementation process of the method for generating image datasets Figure 3 ,like Figure 5 As shown, before the image dataset generation system trains the first virtual reality model using the first training data and the first image data to obtain the second virtual reality model, that is, before step 103, the method for generating the image dataset by the image dataset generation system may further include the following steps:
[0090] Step 108: Receive first training data sent by the first terminal; wherein the first training data includes first object data, first environment data, and first shooting data stored by the first terminal when collecting the first image data.
[0091] In an embodiment of the present application, when the first terminal collects the first image data, it can detect the corresponding first object data, the first environment data and the first shooting data, and then store the first object data, the first environment data and the first shooting data. Accordingly, the image data set generation system can receive the first training data sent by the first terminal, which includes the first object data, the first environment data and the first shooting data.
[0092] That is to say, the image data set generation system can establish a connection with the first terminal, so that before using the first training data and the first image data to train the first virtual reality model to obtain the second virtual reality model, the first training data sent by the first terminal can be directly received, wherein the first training data is detected and stored by the first terminal when the first image data is collected.
[0093] Further, in an embodiment of the present application, after acquiring the first training data and the first image data, the image data set generation system can use the one-to-one corresponding first training data and the first image data to train the first virtual reality model, thereby obtaining the second virtual reality model. Among them, the first simulated reality model and the second simulated reality model are both generation models that are established by the image data set generation system and combine the static layer and the dynamic layer. However, based on the different first image data obtained by the first terminal using different first shooting data, the image data set generation system further performs model training on the basis of the first simulated reality model. Therefore, compared with the first simulated reality model, the second simulated reality model obtained can be better adapted to different shooting conditions and obtain more realistic image data for different shooting data.
[0094] Step 104: train the second virtual reality model using the second training data and the second image data to obtain an image data generation model; wherein the second image data is a real image captured by a second terminal based on the second training data; and the second terminal is a terminal other than the first terminal.
[0095] In the real time of the present application, after the image data set generation system uses the first training data and the first image data to train the first virtual reality model to obtain the second virtual reality model, it can continue to use the second training data and the second image data to train the second virtual reality model, thereby obtaining the image data generation model.
[0096] It should be noted that, in the embodiment of the present application, the second image data may be a real image captured by the second terminal based on the second training data. That is, under the shooting conditions formed by the second training data, the second terminal performs image acquisition to obtain the second image data.
[0097] It is understandable that in the present application, for a set of training data, the second terminal can capture and obtain the corresponding second image data. In other words, the change of any two data in the second training data will make the obtained second image data different.
[0098] It should be noted that in the embodiment of the present application, the second terminal is at least one other terminal different from the first terminal. That is, the second image data is obtained by shooting by other terminals other than the first terminal based on the second training data. Among them, a second terminal can shoot a corresponding unique frame of image using a set of training data, that is, the unique second image data.
[0099] Exemplarily, in an embodiment of the present application, the second terminal can be any terminal with shooting and storage functions, such as: tablet computers, mobile phones, e-readers, remote controls, personal computers (PC), laptop computers, vehicle-mounted equipment, Internet TVs, wearable devices, personal digital assistants (PDA), portable media players (PMP), navigation devices and other terminals.
[0100] Further, in an embodiment of the present application, the second training data may determine all dynamic data when the second terminal collects the second image data. Specifically, in the present application, the second training data may include second object data, second environment data, and second shooting data.
[0101] It can be understood that in an embodiment of the present application, the second object data can represent a person or object whose state is changing when the second terminal collects the second image data, so that the second object data can be used to restore dynamically changing people and objects such as walking passers-by and moving vehicles in the process of collecting the second image data.
[0102] It can be understood that in the embodiments of the present application, the second environmental data can characterize the changing weather and environment when the second terminal collects the second image data, so that the changing climate, light and other factors such as day and night, sunny, rainy, snowy, and seasons during the process of collecting the second image data can be restored through the second environmental data.
[0103] It is understandable that in the embodiments of the present application, the second shooting data may represent the shooting conditions used by the second terminal when collecting the second image data, so that the shooting conditions set in the process of collecting the second image data can be restored through the second shooting data. Among them, the second shooting data may include the configuration parameters of the second terminal, and may also include parameters such as shutter, aperture, sensitivity, exposure value, focal length, etc. used by the second terminal in the process of shooting the second image data, and further, may also include setting parameters of whether to turn on the flash, anti-shake, etc.
[0104] Furthermore, in the embodiments of the present application, Figure 6 Schematic diagram of the implementation process of the method for generating image datasets Figure 4 ,like Figure 6 As shown, before the image data set generation system uses the second training data and the second image data to train the second virtual reality model to obtain the image data generation model, that is, before step 104, the method for generating the image data set by the image data set generation system may further include the following steps:
[0105] Step 109: When the second terminal collects the second image data, detect the second object data, the second environment data and the second shooting data.
[0106] Step 110: determine the second object data, the second environment data and the second shooting data as second training data.
[0107] In an embodiment of the present application, when the second terminal collects the second image data, the image data set generation system can detect the second object data, the second environment data and the second shooting data, and then the second object data, the second environment data and the second shooting data obtained by the detection can be determined as the second training data.
[0108] It can be understood that in the embodiments of the present application, when the second terminal collects the second image data, the image data set generation system can detect dynamic people and objects appearing in the second image data, thereby obtaining second object data; correspondingly, when the second terminal collects the second image data, the image data set generation system can also detect environmental and climate data such as light, wind and rain, thereby obtaining second environmental data.
[0109] Furthermore, in an embodiment of the present application, the system for generating the image data set may determine the configuration parameters of the second terminal and the shooting parameters used by the second terminal when collecting the second image data, thereby obtaining the second shooting data.
[0110] That is, the system for generating the image data set can complete the detection and acquisition of the second training data including the second object data, the second environment data and the second shooting data when the second terminal collects the second image data.
[0111] Furthermore, in the embodiments of the present application, Figure 7 Schematic diagram of the implementation process of the method for generating image datasets Figure 5 ,like Figure 7 As shown, before the image data set generation system uses the second training data and the second image data to train the second virtual reality model to obtain the image data generation model, that is, before step 104, the method for generating the image data set by the image data set generation system may further include the following steps:
[0112] Step 111: Receive second training data sent by the second terminal; wherein the second training data includes second object data, second environment data, and second shooting data stored by the second terminal when collecting second image data.
[0113] In an embodiment of the present application, when the second terminal collects the second image data, it can detect the corresponding second object data, second environment data and second shooting data, and then store the second object data, second environment data and second shooting data. Accordingly, the image data set generation system can receive the second training data sent by the second terminal, which includes the second object data, second environment data and second shooting data.
[0114] That is to say, the image data set generation system can establish a connection with the second terminal, so that before using the second training data and the second image data to train the second virtual reality model to obtain the image data generation model, the second training data sent by the second terminal can be directly received, wherein the second training data is detected and stored by the second terminal when collecting the second image data.
[0115] Furthermore, in an embodiment of the present application, after acquiring the second training data and the second image data, the image data set generation system can use the one-to-one corresponding second training data and the second image data to train the second virtual reality model, thereby obtaining an image data generation model.
[0116] It is understandable that in the present application. The first simulated reality model, the second simulated reality model and the image data generation model are all generation models established by the generation system of the image data set, combining the static layer and the dynamic layer. However, based on the different first image data obtained by shooting with different first shooting data by the first terminal, and the different second image data obtained by shooting with different second shooting data by the second terminal, the generation system of the image data set further performs model training on the basis of the first simulated reality model and the second simulated reality model. Therefore, the obtained image data generation model can be better applied to different shooting conditions and obtain more realistic image data for different shooting data compared with the first simulated reality model. Furthermore, the obtained image data generation model can be better applied to different shooting terminals and obtain more realistic image data for the configuration parameters of different shooting terminals compared with the second simulated reality model.
[0117] It can be understood that in the embodiments of the present application, the image data generation model is used to generate an image data set. That is to say, through the constructed image data generation model, the image data generation model can simulate infinite shooting scenes constituted by any shooting object, any shooting position, any shooting environment, any shooting terminal and any shooting parameters and other shooting conditions, thereby obtaining a large amount of accurate and rich image data.
[0118] Step 105: Generate an image data set through an image data generation model; wherein the image data set is used to test the shooting function.
[0119] In an embodiment of the present application, after the image data set generation system completes the training of the image data generation model using the first training data and the second training data respectively, the image data set generation system can use the image data generation model to generate the image data set, so that the generated image data set can be used to test the shooting function.
[0120] That is to say, in the embodiments of the present application, since the constructed image data generation model can simulate infinite shooting scenes constituted by any shooting conditions, the image data set generation system can be set according to the different shooting scenes and different shooting devices required, and then simulate the shooting based on the image data generation model to obtain the corresponding image data set.
[0121] It can be understood that in an embodiment of the present application, when the image data set generation system generates the image data set through the image data generation model, it can first set the third object data, the third environment data and the third shooting data; and then input the third object data, the third environment data and the third shooting data into the image data generation model, so as to obtain the image data set.
[0122] Exemplarily, in the present application, the third object data can be used to simulate and set people or objects in the real world whose states are changing, so that the images obtained in the simulated shooting can meet the needs of dynamically changing people and objects such as walking passers-by and moving vehicles.
[0123] Illustratively, in the present application, the third environmental data can be used to simulate and set the changing weather and environment in the real world, so that the images obtained in the simulated shooting can meet the requirements of changing climate, light and other factors such as day and night, cloudy and rainy, snowy, and the changing of seasons.
[0124] Exemplarily, in the present application, the third shooting data can be used to simulate and set different shooting devices and parameters used in the shooting process, so that the image obtained by the simulated shooting can meet the requirements of factors such as the shooting device and shooting parameters.
[0125] Specifically, in the present application, the third shooting data may include configuration parameters of different terminals, and may also include parameters used in the shooting process such as shutter, aperture, sensitivity, exposure value, focal length, etc. Further, it may also include setting parameters for whether to turn on the flash, anti-shake and other functions.
[0126] It can be understood that in the embodiments of the present application, after the image data set generation system generates the image data set through the image data generation model, the image data set can be used to test and optimize the shooting function. In other words, the image data set generated by the image data set generation system can be used to test the shooting function of any terminal, and because the image data generation model can obtain an accurate and rich image data set including unlimited image data by simulating unlimited shooting scenes composed of any shooting conditions, the image data set can be used to more efficiently and accurately complete the detection and optimization of the shooting function of the terminal.
[0127] In the embodiments of the present application, further, Figure 8 Schematic diagram of the implementation process of the method for generating image datasets Figure 6 ,like Figure 8 As shown, after the second virtual reality model is trained using the second training data and the second image data to obtain the image data generation model, that is, after step 104, the method for generating the image data set by the image data set generation system may further include the following steps:
[0128] Step 112: Update at least one of the map data, the dynamic parameters, the first training data, the first image data, the second training data, and the second image data to update the image data generation model.
[0129] In an embodiment of the present application, after the image data set generation system completes the training of the image data generation model using the first training data and the second training data respectively, the image data set generation system can also update the map data, dynamic parameters, the first training data and at least one of the first image data, the second training data and the second image data, thereby completing the update of the image data generation model.
[0130] It can be understood that in the embodiments of the present application, when the image data set generation system updates the map data, dynamic parameters, first training data and one or more of the first image data, second training data and second image data, it can perform update processing according to a preset cycle, or it can perform update processing in real time, and it can also perform update processing through received update instructions.
[0131] In summary, through the image data set generation method proposed in step 101 to step 112, the image data set generation system can realize the shooting and processing of infinite scenes in virtual reality by constructing an image data generation model, thereby generating an image data set that can be used to test the shooting function of the terminal. Among them, the image data set generation system can integrate technologies such as simultaneous localization and mapping (SLAM), virtual reality (VR) and machine learning (ML) to establish an image data generation model, so that various permutation and combination scenes can be created in the virtual world, and then unlimited image data can be obtained by simulating shooting.
[0132] In other words, the image data generation model constructed in this application can obtain various information in a constantly changing environment through machine learning, experience real heating behavior and actual weather conditions, understand the actual situation, generate realistic image data streams, and create various different environments.
[0133] Illustratively, the image data generation model constructed in the present application can simulate day, night, rain, snow and other climatic conditions in the virtual world, create blinding glare or limited vision at night and various road surfaces and terrains in different time periods, thereby obtaining image data sets under various scenes and conditions, and then when using the image data sets to test and optimize the shooting function, it can improve the test efficiency and enhance the optimization results.
[0134] It can be seen that the image data generation model can be used to quickly and efficiently input complete and comprehensive image data in the virtual world according to the needs of actual occasions, thereby improving the efficiency and quality of image data generation, saving time and improving image quality.
[0135] An embodiment of the present application provides a method for generating an image data set. The image data set generation system obtains map data, and models a static layer according to the map data to obtain a three-dimensional map model; models a dynamic layer based on the three-dimensional map model and dynamic parameters to obtain a first virtual reality model; trains the first virtual reality model using first training data and first image data to obtain a second virtual reality model; wherein the first image data is a real image captured by a first terminal based on the first training data; trains the second virtual reality model using second training data and second image data to obtain an image data generation model; wherein the second image data is a real image captured by a second terminal based on the second training data; the second terminal is a terminal other than the first terminal; generates an image data set through the image data generation model; wherein the image data set is used to test a shooting function. It can be seen that in the embodiments of the present application, the image data set generation system can perform static layer model and dynamic layer model training based on map data and dynamic parameters to obtain a first virtual reality model, and then use the first training data and the corresponding first image data, and the second training data and the corresponding second image data to perform model training, and then obtain an image data generation model, wherein the image data generation model can be used to simulate shooting of infinite scenes in the virtual world, so the image data model can be used to quickly generate a rich and highly applicable image data set for testing shooting functions. Therefore, when generating an image data set, the real-time and consistency of the image data can be guaranteed, and the generation efficiency and data quality of the image data set can be greatly improved.
[0136] Based on the above embodiment, in yet another embodiment of the present application, the image data set generation system uses the first training data and the first image data to train the first virtual reality model, and the method for obtaining the second virtual reality model may include the following steps:
[0137] Step 103a: using the first virtual reality model, determine a first simulated image corresponding to the first training data.
[0138] Step 103b: Compare the first image data and the first simulated image to generate a first precision of simulated shooting.
[0139] Step 103c: obtaining a second virtual reality model according to the first accuracy.
[0140] In an embodiment of the present application, after completing the construction of the static layer model and the static layer model and obtaining the first simulated reality model, the image data set generation system can use the first training data and the first image data to train the first virtual reality model. Specifically, the image data set generation system can first use the first virtual reality model to determine the first simulated image corresponding to the first training data, and then compare the first simulated image and the first image data to obtain a first accuracy of the simulated shooting, and then generate a second virtual reality model based on the first accuracy.
[0141] It should be noted that, in an embodiment of the present application, the system for generating an image data set inputs the first training data into the first virtual reality model, and can perform shooting processing in the virtual world based on the first training data to obtain a first simulated image, and the first simulated image is captured by simulated shooting.
[0142] It can be understood that, in the embodiments of the present application, both the first image data and the first simulated image are obtained by shooting based on the first training data. The difference is that the first image data is a real image shot in the real world, and the first simulated image is a test image shot in the virtual world. Therefore, the comparison result obtained by comparing the first image data and the first simulated image, that is, the first accuracy, can effectively reflect the accuracy of the first virtual reality model.
[0143] It can be understood that, in the embodiment of the present application, the first accuracy may be the similarity between the first image data and the first simulated image.
[0144] Further, in an embodiment of the present application, when the image data set generation system obtains the second virtual reality model according to the first accuracy, if the first accuracy is greater than or equal to the first preset accuracy threshold, the first virtual reality model is determined as the second virtual reality model; if the first accuracy is less than the first preset accuracy threshold, the first virtual reality model is corrected according to the first accuracy to obtain the second virtual reality model.
[0145] That is to say, in the present application, after comparing the first image data and the first simulated image and determining the first accuracy, the image data set generation system can use the first preset accuracy threshold to determine whether the first accuracy meets the expected standard. Specifically, if the first accuracy is greater than or equal to the first preset accuracy threshold, then it can be considered that the accuracy of the first virtual reality model is relatively high, so the first virtual reality model can be directly determined as the second virtual reality model; if the first accuracy is less than the first preset accuracy threshold, then it can be considered that the accuracy of the first virtual reality model is relatively low, so it is necessary to correct the first virtual reality model according to the first accuracy, and finally determine the corrected model as the second virtual reality model.
[0146] Exemplarily, in the present application, if the first accuracy is the similarity between the first image data and the first simulated image, then correspondingly, the first preset accuracy threshold may be a fixed value between 0 and 1, for example, the first preset accuracy threshold may be 90%.
[0147] It should be noted that, in the embodiment of the present application, when the image data set generation system corrects the first virtual reality model according to the first accuracy to obtain the second virtual reality model, the dynamic parameters can be first adjusted according to the first accuracy to obtain the adjusted parameters; then the first virtual reality model can be corrected based on the adjusted parameters until the first accuracy is greater than or equal to the first preset accuracy threshold.
[0148] That is to say, in the present application, when the image data set generation system amends the first virtual reality model, it mainly improves the applicability of the model by adjusting and optimizing the dynamic layer. Specifically, the image data set generation system adjusts the dynamic parameters for constructing the dynamic layer model, so as to obtain the adjusted parameters, and then uses the adjusted parameters to adjust the dynamic layer model to correct the first virtual reality model. When the first accuracy regenerated based on the first image data and the first training data is greater than or equal to the first preset accuracy threshold, the second virtual reality model can be obtained.
[0149] The method for obtaining the image data generation model by using the second training data and the second image data to train the second virtual reality model may include the following steps:
[0150] Step 104a: using the second virtual reality model, determine a second simulated image corresponding to the second training data.
[0151] Step 104b: Compare the first image data and the second simulated image to generate a second precision of simulated shooting.
[0152] Step 104c: obtaining an image data generation model according to the second precision.
[0153] In an embodiment of the present application, after the image data set generation system uses the first training data and the first image data to train the first virtual reality model to obtain the second virtual reality model, the image data set generation system may use the second training data and the second image data to train the second virtual reality model. Specifically, the image data set generation system may first use the second virtual reality model to determine the second simulated image corresponding to the second training data, and then compare the second simulated image and the second image data to obtain a second accuracy of the simulated shooting, and then generate a second virtual reality model according to the second accuracy.
[0154] It should be noted that, in an embodiment of the present application, the image data set generation system inputs the second training data into the second virtual reality model, and can perform shooting processing in the virtual world based on the second training data to obtain a second simulated image, and the second simulated image is captured by the simulated shooting.
[0155] It can be understood that in the embodiments of the present application, both the second image data and the second simulated image are obtained by shooting based on the second training data. The difference is that the second image data is a real image shot in the real world, and the second simulated image is a test image shot in the virtual world. Therefore, the comparison result obtained by comparing the second image data and the second simulated image, that is, the second accuracy, can effectively reflect the accuracy of the second virtual reality model.
[0156] It can be understood that, in the embodiment of the present application, the second accuracy may be the similarity between the second image data and the second simulated image.
[0157] Furthermore, in an embodiment of the present application, when the image data set generation system obtains the first image data generation model according to the second precision, if the second precision is greater than or equal to the second preset precision threshold, the second virtual reality model is determined as the image data generation model; if the second precision is less than the second preset precision threshold, the second virtual reality model is corrected according to the second precision to obtain the image data generation model.
[0158] That is to say, in the present application, after comparing the second image data and the second simulated image and determining the second accuracy, the image data set generation system can use the second preset accuracy threshold to determine whether the second accuracy meets the expected standard. Specifically, if the second accuracy is greater than or equal to the second preset accuracy threshold, then it can be considered that the second virtual reality model has a high degree of accuracy, and therefore the second virtual reality model can be directly determined as the image data generation model; if the second accuracy is less than the second preset accuracy threshold, then it can be considered that the second virtual reality model has a low degree of accuracy, and therefore the second virtual reality model needs to be corrected according to the second accuracy, and finally the corrected model is determined as the image data generation model.
[0159] Exemplarily, in the present application, if the second accuracy is the similarity between the second image data and the second simulated image, then correspondingly, the second preset accuracy threshold may be a fixed value between 0 and 1, for example, the second preset accuracy threshold may be 85%.
[0160] It can be understood that, in the present application, the first preset accuracy threshold and the second preset accuracy threshold may be the same or different.
[0161] It should be noted that, in the embodiment of the present application, when the image data set generation system corrects the second virtual reality model according to the second precision to obtain the image data generation model, the dynamic parameters can be first adjusted according to the second precision to obtain the adjusted parameters; then the first virtual reality model can be corrected based on the adjusted parameters until the first precision is greater than or equal to the first preset precision threshold, and the second precision is greater than or equal to the second preset precision threshold.
[0162] That is to say, in the present application, when the image data set generation system corrects the second virtual reality model, it mainly improves the applicability of the model by adjusting and optimizing the dynamic layer. Specifically, the image data set generation system adjusts the dynamic parameters for constructing the dynamic layer model, so as to obtain the adjusted parameters, and then uses the adjusted parameters to adjust the dynamic layer model to calibrate the first virtual reality model and the second virtual reality model. When the first accuracy regenerated based on the first image data and the first training data is greater than or equal to the first preset accuracy threshold, and the second accuracy regenerated based on the second image data and the second training data is greater than or equal to the second preset accuracy threshold, the image data generation model can be obtained.
[0163] An embodiment of the present application provides a method for generating an image data set. The image data set generation system obtains map data, and models a static layer according to the map data to obtain a three-dimensional map model; models a dynamic layer based on the three-dimensional map model and dynamic parameters to obtain a first virtual reality model; trains the first virtual reality model using first training data and first image data to obtain a second virtual reality model; wherein the first image data is a real image captured by a first terminal based on the first training data; trains the second virtual reality model using second training data and second image data to obtain an image data generation model; wherein the second image data is a real image captured by a second terminal based on the second training data; the second terminal is a terminal other than the first terminal; generates an image data set through the image data generation model; wherein the image data set is used to test a shooting function. It can be seen that in the embodiments of the present application, the image data set generation system can perform static layer model and dynamic layer model training based on map data and dynamic parameters to obtain a first virtual reality model, and then use the first training data and the corresponding first image data, and the second training data and the corresponding second image data to perform model training, and then obtain an image data generation model, wherein the image data generation model can be used to simulate shooting of infinite scenes in the virtual world, so the image data model can be used to quickly generate a rich and highly applicable image data set for testing shooting functions. Therefore, when generating an image data set, the real-time and consistency of the image data can be guaranteed, and the generation efficiency and data quality of the image data set can be greatly improved.
[0164] Based on the above embodiment, in another embodiment of the present application, Fig. 9 Schematic diagram of the implementation process of the method for generating image datasets Figure 7 ,like Fig. 9 As shown, the method for generating an image data set by the image data set generation system may further include the following steps:
[0165] Step 201: Obtain map data.
[0166] In an embodiment of the present application, a system for generating an image data set may first obtain map data, where the map data is real-world geographic location map data, and the map data may be obtained through geographic information systems, maps, and satellite image data.
[0167] Step 202: Model the static layer.
[0168] In an embodiment of the present application, the image data set generation system can model the static layer based on the map data, so as to complete the construction of a three-dimensional map model. The three-dimensional map model can reflect the real world and simulate the static objects in the real world to the greatest extent. Specifically, the three-dimensional map model is a three-dimensional digital map of the real world.
[0169] When constructing a three-dimensional map model, the image data set generation system can use simultaneous localization and map construction SLAM. Specifically, when constructing a three-dimensional map model, the image data set generation system can complete the modeling of the static layer based on the map data through computer vision and deep learning algorithms.
[0170] Step 203: Model the dynamic layer.
[0171] In an embodiment of the present application, the image data set generation system can construct a model of the dynamic layer based on the three-dimensional map model and dynamic parameters, so as to obtain a first virtual reality model. The first virtual reality model can be considered to be obtained by adding a dynamic layer model to a static layer model (three-dimensional map model). Specifically, the image data set generation system can add dynamic factors including moving vehicles, walking people, changing light, rain, snow, wind and other climatic conditions to the three-dimensional map model, thereby realizing the construction of a dynamic layer model based on a static layer model.
[0172] Step 204: input the first training data into the first virtual reality model to obtain a first simulated image.
[0173] Step 205 : Whether the similarity between the first simulated image and the first image data meets a first accuracy threshold, if so, execute step 206 , otherwise execute step 207 .
[0174] Step 206: Obtain a second virtual reality model.
[0175] If the similarity between the first simulated image and the first image data is greater than or equal to the first preset accuracy threshold, it can be considered that the first virtual reality model has a higher accuracy, and thus the first virtual reality model can be directly determined as the second virtual reality model.
[0176] Step 207: Modify the first virtual reality model.
[0177] If the similarity between the first simulated image and the first image data is less than the first preset accuracy threshold, it can be considered that the accuracy of the first virtual reality model is low, and therefore the first virtual reality model needs to be corrected according to the first accuracy, and finally the corrected model is determined as the second virtual reality model.
[0178] Step 208: input the second training data into the second virtual reality model to obtain a second simulated image.
[0179] Step 209 : Whether the similarity between the second simulated image and the second image data meets the second accuracy threshold; if so, execute step 2010 ; otherwise, execute step 2011 .
[0180] Step 2010: Obtain an image data generation model.
[0181] If the similarity between the second simulated image and the second image data is greater than or equal to the second preset accuracy threshold, it can be considered that the second virtual reality model has a high degree of accuracy, and thus the second virtual reality model can be directly determined as the image data generation model.
[0182] Step 2011: Modify the first virtual reality model and the second virtual reality model.
[0183] If the similarity between the second simulated image and the second image data is less than the second preset accuracy threshold, it can be considered that the accuracy of the second virtual reality model is low, and therefore the second virtual reality model needs to be corrected according to the second accuracy, and finally the corrected model is determined as the image data generation model.
[0184] When the image data set generation system corrects the second virtual reality model according to the second precision to obtain the image data generation model, the dynamic parameters can be first adjusted according to the second precision to obtain the adjusted parameters; then the first virtual reality model can be corrected based on the adjusted parameters until the first precision is greater than or equal to the first preset precision threshold, and the second precision is greater than or equal to the second preset precision threshold.
[0185] Step 2012: Generate an image data set using an image data generation model.
[0186] In an embodiment of the present application, after the image data generation model is trained using the first training data and the second training data respectively, the image data generation model can be used to generate the image data set, so that the generated image data set can be used to test the shooting function. The generation of the image data set can be completed by setting the third object data, the third environment data and the third shooting data.
[0187] Step 2013: Update the image data generation model.
[0188] In an embodiment of the present application, after the image data set generation system completes the training of the image data generation model using the first training data and the second training data respectively, the image data set generation system can also update the map data, dynamic parameters, the first training data and at least one of the first image data, the second training data and the second image data, thereby completing the update of the image data generation model.
[0189] The existing technology is unable to complete the generation of countless image data sets in the physical world, while the method for generating image data sets proposed in the present application can construct an image data generation model through virtual reality, and realize unlimited simulated shooting tests in the virtual world, so as to determine in the virtual world what are the differences in image data obtained by different terminals for different shooting scenes and shooting conditions, and then complete the test of the shooting function of the terminal, and make corresponding adjustments and optimizations to the shooting function of the terminal according to the test results.
[0190] It can be seen that in the present application, by constructing an image data generation model, an algorithm flow for a terminal to collect image data sets can be established in the virtual world, and different shooting terminals, different shooting scenes, different shooting conditions, different shooting environments, etc. can be simulated based on different object data, environmental data, and shooting data, thereby generating an unlimited amount of image data.
[0191] An embodiment of the present application provides a method for generating an image data set. The image data set generation system obtains map data, and models a static layer according to the map data to obtain a three-dimensional map model; models a dynamic layer based on the three-dimensional map model and dynamic parameters to obtain a first virtual reality model; trains the first virtual reality model using first training data and first image data to obtain a second virtual reality model; wherein the first image data is a real image captured by a first terminal based on the first training data; trains the second virtual reality model using second training data and second image data to obtain an image data generation model; wherein the second image data is a real image captured by a second terminal based on the second training data; the second terminal is a terminal other than the first terminal; generates an image data set through the image data generation model; wherein the image data set is used to test a shooting function. It can be seen that in the embodiments of the present application, the image data set generation system can perform static layer model and dynamic layer model training based on map data and dynamic parameters to obtain a first virtual reality model, and then use the first training data and the corresponding first image data, and the second training data and the corresponding second image data to perform model training, and then obtain an image data generation model, wherein the image data generation model can be used to simulate shooting of infinite scenes in the virtual world, so the image data model can be used to quickly generate a rich and highly applicable image data set for testing shooting functions. Therefore, when generating an image data set, the real-time and consistency of the image data can be guaranteed, and the generation efficiency and data quality of the image data set can be greatly improved.
[0192] Based on the above embodiment, in another embodiment of the present application, Fig.10 Schematic diagram of the composition structure of the system for generating image datasets Figure 1 ,like Fig.10 As shown, the image data set generation system 10 proposed in the embodiment of the present application may include: an acquisition unit 11, a construction unit 12, a training unit 13, a generation unit 14, a determination unit 15, a receiving unit 16, and an update unit 17.
[0193] The acquisition unit 11 is used to acquire map data;
[0194] The construction unit 12 is used to model the static layer according to the map data to obtain a three-dimensional map model; and to model the dynamic layer based on the three-dimensional map model and dynamic parameters to obtain a first virtual reality model;
[0195] The training unit 13 is used to train the first virtual reality model using first training data and first image data to obtain a second virtual reality model; wherein the first image data is a real image captured by the first terminal based on the first training data; and to train the second virtual reality model using second training data and second image data to obtain an image data generation model; wherein the second image data is a real image captured by the second terminal based on the second training data; and the second terminal is a terminal other than the first terminal;
[0196] The generating unit 14 is used to generate an image data set by using the image data generating model; wherein the image data set is used to test a shooting function.
[0197] Further, in an embodiment of the present application, the acquisition unit 11 is further configured to train the first virtual reality model using the first training data and the first image data, and before obtaining the second virtual reality model, acquire the first object data, the first environment data, and the first shooting data when the first terminal collects the first image data;
[0198] The determining unit 15 is configured to determine the first object data, the first environment data, and the first shooting data as the first training data.
[0199] Furthermore, in an embodiment of the present application, the receiving unit 16 is used to train the first virtual reality model using the first training data and the first image data, and receive the first training data sent by the first terminal before obtaining the second virtual reality model; wherein the first training data includes the first object data, first environment data, and first shooting data stored by the first terminal when collecting the first image data.
[0200] Furthermore, in an embodiment of the present application, the training unit 13 is specifically used to use the first virtual reality model to determine a first simulated image corresponding to the first training data; compare the first image data and the first simulated image to generate a first accuracy of simulated shooting; and obtain the second virtual reality model according to the first accuracy.
[0201] Furthermore, in an embodiment of the present application, the training unit 13 is further specifically configured to determine the first virtual reality model as the second virtual reality model if the first accuracy is greater than or equal to a first preset accuracy threshold; and to correct the first virtual reality model according to the first accuracy to obtain the second virtual reality model if the first accuracy is less than the first preset accuracy threshold.
[0202] Furthermore, in an embodiment of the present application, the training unit 13 is further specifically configured to adjust the dynamic parameters according to the first precision to obtain adjusted parameters; and correct the first virtual reality model based on the adjusted parameters until the first precision is greater than or equal to the first preset precision threshold.
[0203] Further, in an embodiment of the present application, the acquisition unit 11 is further configured to train the second virtual reality model using the second training data and the second image data, and before obtaining the image data generation model, when the second terminal collects the second image data, detect the second object data, the second environment data, and the second shooting data;
[0204] The determining unit 15 is further configured to determine the second object data, the second environment data, and the second shooting data as the second training data.
[0205] Furthermore, in an embodiment of the present application, the receiving unit 16 is also used to train the second virtual reality model using the second training data and the second image data, and receive the second training data sent by the second terminal before obtaining the image data generation model; wherein the second training data includes the second object data, second environment data and second shooting data stored by the second terminal when collecting the second image data.
[0206] Furthermore, in an embodiment of the present application, the training unit 13 is also specifically used to use the second virtual reality model to determine a second simulated image corresponding to the second training data; compare the first image data and the second simulated image to generate a second precision of simulated shooting; and obtain the image data generation model according to the second precision.
[0207] Furthermore, in an embodiment of the present application, the training unit 13 is further specifically configured to determine the second virtual reality model as the image data generation model if the second accuracy is greater than or equal to a second preset accuracy threshold; and if the second accuracy is less than the second preset accuracy threshold, correct the second virtual reality model according to the second accuracy to obtain the image data generation model.
[0208] Furthermore, in an embodiment of the present application, the training unit 13 is further specifically used to adjust the dynamic parameters according to the second precision to obtain adjusted parameters; and correct the first virtual reality model based on the adjusted parameters until the first precision is greater than or equal to the first preset precision threshold, and the second precision is greater than or equal to the second preset precision threshold.
[0209] Furthermore, in an embodiment of the present application, the generation unit 14 is specifically used to set third object data, third environment data and third shooting data; input the third object data, the third environment data and the third shooting data into the image data generation model to obtain the image data set.
[0210] Furthermore, in an embodiment of the present application, the dynamic parameters include dynamic object parameters, dynamic environment parameters and shooting parameters, and the construction unit 12 is specifically used to train the three-dimensional map model using the dynamic object parameters, the dynamic environment parameters and the shooting parameters to obtain the first virtual reality model.
[0211] Furthermore, in an embodiment of the present application, the construction unit 12 is also specifically used to mark different feature data in the map data to obtain marked data; and to construct a model of the static layer based on the marked data to obtain the three-dimensional map model.
[0212] Furthermore, in an embodiment of the present application, the updating unit 17 is used to train the second virtual reality model using the second training data and the second image data, and after obtaining the image data generation model, update at least one of the map data, the dynamic parameters, the first training data, the first image data, the second training data and the second image data to update the image data generation model.
[0213] In the embodiments of the present application, further, Fig.11 Schematic diagram of the composition structure of the system for generating image datasets Figure 2 ,like Fig.11 As shown, the image dataset generation system 10 proposed in the embodiment of the present application may also include a processor 18 and a memory 19 storing executable instructions of the processor 18. Furthermore, the image dataset generation system 10 may also include a communication interface 110 and a bus 111 for connecting the processor 18, the memory 19 and the communication interface 110.
[0214] In the embodiment of the present application, the processor 18 may be at least one of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a Central Processing Unit (CPU), a controller, a microcontroller, and a microprocessor. It is understandable that for different devices, the electronic device used to implement the function of the processor may also be other, and the embodiment of the present application is not specifically limited. The image data set generation system 10 may also include a memory 19, which may be connected to the processor 18, wherein the memory 19 is used to store executable program code, the program code includes computer operation instructions, and the memory 19 may include a high-speed RAM memory, and may also include a non-volatile memory, for example, at least two disk memories.
[0215] In the embodiment of the present application, the bus 111 is used to connect the communication interface 110, the processor 18 and the memory 19, as well as the mutual communication between these devices.
[0216] In the embodiment of the present application, the memory 19 is used to store instructions and data.
[0217] Furthermore, in an embodiment of the present application, the processor 18 is used to acquire map data, and model a static layer according to the map data to obtain a three-dimensional map model; model a dynamic layer based on the three-dimensional map model and dynamic parameters to obtain a first virtual reality model; train the first virtual reality model using first training data and first image data to obtain a second virtual reality model; wherein the first image data is a real image captured by the first terminal based on the first training data; train the second virtual reality model using second training data and second image data to obtain an image data generation model; wherein the second image data is a real image captured by the second terminal based on the second training data; the second terminal is a terminal other than the first terminal; an image data set is generated by the image data generation model; wherein the image data set is used to test the shooting function.
[0218] In practical applications, the memory 19 may be a volatile memory, such as a random access memory (RAM); or a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk (HDD) or a solid-state drive (SSD); or a combination of the above types of memory, and provide instructions and data to the processor 18.
[0219] In addition, each functional module in this embodiment can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or software functional modules.
[0220] If the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) or a processor to perform all or part of the steps of the method of this embodiment. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.
[0221] An embodiment of the present application provides a system for generating an image data set, which acquires map data, models a static layer according to the map data, and obtains a three-dimensional map model; models a dynamic layer based on the three-dimensional map model and dynamic parameters, and obtains a first virtual reality model; trains the first virtual reality model using first training data and first image data, and obtains a second virtual reality model; wherein the first image data is a real image captured by a first terminal based on the first training data; trains the second virtual reality model using second training data and second image data, and obtains an image data generation model; wherein the second image data is a real image captured by a second terminal based on the second training data; the second terminal is a terminal other than the first terminal; generates an image data set through the image data generation model; wherein the image data set is used to test a shooting function. It can be seen that in the embodiments of the present application, the image data set generation system can perform static layer model and dynamic layer model training based on map data and dynamic parameters to obtain a first virtual reality model, and then use the first training data and the corresponding first image data, and the second training data and the corresponding second image data to perform model training, and then obtain an image data generation model, wherein the image data generation model can be used to simulate shooting of infinite scenes in the virtual world, so the image data model can be used to quickly generate a rich and highly applicable image data set for testing shooting functions. Therefore, when generating an image data set, the real-time and consistency of the image data can be guaranteed, and the generation efficiency and data quality of the image data set can be greatly improved.
[0222] An embodiment of the present application provides a computer-readable storage medium having a program stored thereon, and when the program is executed by a processor, the method for generating an image data set as described above is implemented.
[0223] Specifically, the program instructions corresponding to the method for generating an image data set in this embodiment may be stored on a storage medium such as a CD, a hard disk, or a USB flash drive. When the program instructions corresponding to the method for generating an image data set in the storage medium are read or executed by an electronic device, the following steps are included:
[0224] Acquire map data, and model the static layer according to the map data to obtain a three-dimensional map model;
[0225] Modeling the dynamic layer based on the three-dimensional map model and the dynamic parameters to obtain a first virtual reality model;
[0226] The first virtual reality model is trained using the first training data and the first image data to obtain a second virtual reality model; wherein the first image data is a real image captured by the first terminal based on the first training data;
[0227] The second virtual reality model is trained using the second training data and the second image data to obtain an image data generation model; wherein the second image data is a real image captured by a second terminal based on the second training data; and the second terminal is a terminal other than the first terminal;
[0228] An image data set is generated by the image data generation model; wherein the image data set is used to test a shooting function.
[0229] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of hardware embodiments, software embodiments, or embodiments in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.
[0230] The present application is described with reference to implementation flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the process in the flowchart. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0231] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which is implemented in the implementation flow diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0232] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing the steps in the flowchart. Figure 1 A process or multiple processes and / or boxes Figure 1The steps for the functions specified in one or more boxes.
[0233] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application.
Claims
1. A method for generating an image data set, characterized in that: The method comprises: Acquire map data, and model the static layer according to the map data to obtain a three-dimensional map model; Modeling the dynamic layer based on the three-dimensional map model and the dynamic parameters to obtain a first virtual reality model; The first virtual reality model is trained using the first training data and the first image data to obtain a second virtual reality model; wherein the first image data is a real image captured by the first terminal based on the first training data; The second virtual reality model is trained using the second training data and the second image data to obtain an image data generation model; wherein the second image data is a real image captured by a second terminal based on the second training data; and the second terminal is a terminal other than the first terminal; An image data set is generated by the image data generation model; wherein the image data set is used to test a shooting function.
2. The method according to claim 1, characterized in that Before the first virtual reality model is trained by using the first training data and the first image data to obtain the second virtual reality model, the method further includes: When the first terminal collects the first image data, acquiring first object data, first environment data, and first shooting data; The first object data, the first environment data, and the first shooting data are determined as the first training data.
3. The method according to claim 1, characterized in that Before the first virtual reality model is trained by using the first training data and the first image data to obtain the second virtual reality model, the method further includes: The first training data sent by the first terminal is received; wherein the first training data includes first object data, first environment data, and first shooting data stored by the first terminal when the first image data is collected.
4. The method according to any one of claims 1 to 3, characterized in that: The method of training the first virtual reality model using the first training data and the first image data to obtain the second virtual reality model includes: Determine, using the first virtual reality model, a first simulated image corresponding to the first training data; Comparing the first image data with the first simulated image to generate a first precision of simulated shooting; The second virtual reality model is obtained according to the first accuracy.
5. The method according to claim 4, characterized in that The step of obtaining the second virtual reality model according to the first precision includes: If the first accuracy is greater than or equal to a first preset accuracy threshold, determining the first virtual reality model as the second virtual reality model; If the first precision is less than the first preset precision threshold, the first virtual reality model is corrected according to the first precision to obtain the second virtual reality model.
6. The method according to claim 5, characterized in that The step of correcting the first virtual reality model according to the first precision to obtain the second virtual reality model includes: Adjust the dynamic parameter according to the first precision to obtain an adjusted parameter; The first virtual reality model is modified based on the adjusted parameters until the first accuracy is greater than or equal to the first preset accuracy threshold.
7. The method according to claim 1, characterized in that Before the second virtual reality model is trained using the second training data and the second image data to obtain the image data generation model, the method further includes: When the second terminal collects the second image data, detecting second object data, second environment data and second shooting data; The second object data, the second environment data, and the second shooting data are determined as the second training data.
8. The method according to claim 1, characterized in that Before the second virtual reality model is trained using the second training data and the second image data to obtain the image data generation model, the method further includes: The second training data sent by the second terminal is received; wherein the second training data includes second object data, second environment data and second shooting data stored by the second terminal when the second image data is collected.
9. The method according to claim 1, 7 or 8, characterized in that: The step of training the second virtual reality model using the second training data and the second image data to obtain an image data generation model includes: Determining a second simulated image corresponding to the second training data using the second virtual reality model; Comparing the first image data with the second simulated image to generate a second precision of simulated shooting; The image data generation model is obtained according to the second precision.
10. The method according to claim 9, characterized in that The step of obtaining the image data generation model according to the second precision comprises: If the second accuracy is greater than or equal to a second preset accuracy threshold, determining the second virtual reality model as the image data generation model; If the second accuracy is less than the second preset accuracy threshold, the second virtual reality model is corrected according to the second accuracy to obtain the image data generation model.
11. The method according to claim 10, characterized in that The step of correcting the second virtual reality model according to the second precision to obtain the image data generation model includes: Adjust the dynamic parameter according to the second precision to obtain an adjusted parameter; The first virtual reality model is corrected based on the adjusted parameters until the first accuracy is greater than or equal to a first preset accuracy threshold, and the second accuracy is greater than or equal to the second preset accuracy threshold; wherein the first accuracy is generated by: using the first virtual reality model, determining a first simulated image corresponding to the first training data, comparing the first image data and the first simulated image, and generating the first accuracy of simulated shooting.
12. The method according to claim 1, characterized in that The step of generating an image data set by using the image data generation model comprises: Setting third object data, third environment data and third shooting data; The third object data, the third environment data and the third shooting data are input into the image data generation model to obtain the image data set.
13. The method according to claim 1, characterized in that The dynamic parameters include dynamic object parameters, dynamic environment parameters and shooting parameters, and the dynamic layer is modeled based on the three-dimensional map model and the dynamic parameters to obtain a first virtual reality model, including: The three-dimensional map model is trained using the dynamic object parameters, the dynamic environment parameters, and the shooting parameters to obtain the first virtual reality model.
14. The method according to claim 1, characterized in that The step of modeling the static layer according to the map data to obtain a three-dimensional map model includes: Marking different feature data in the map data to obtain marked data; The static layer model is constructed based on the labeled data to obtain the three-dimensional map model.
15. The method according to claim 1, characterized in that After the second virtual reality model is trained using the second training data and the second image data to obtain the image data generation model, the method further includes: At least one of the map data, the dynamic parameters, the first training data, the first image data, the second training data, and the second image data is updated to update the image data generation model.
16. A system for generating an image data set, characterized in that: The image data set generation system includes: an acquisition unit, a construction unit, a training unit, a generation unit, The acquisition unit is used to acquire map data; The construction unit is used to model the static layer according to the map data to obtain a three-dimensional map model; and to model the dynamic layer based on the three-dimensional map model and dynamic parameters to obtain a first virtual reality model; The training unit is used to train the first virtual reality model using first training data and first image data to obtain a second virtual reality model; wherein the first image data is a real image captured by the first terminal based on the first training data; and to train the second virtual reality model using second training data and second image data to obtain an image data generation model; wherein the second image data is a real image captured by the second terminal based on the second training data; and the second terminal is a terminal other than the first terminal; The generating unit is used to generate an image data set by using the image data generating model; wherein the image data set is used to test a shooting function.
17. A system for generating an image data set, characterized in that: The system for generating the image data set includes a processor and a memory storing instructions executable by the processor. When the instructions are executed by the processor, the method according to any one of claims 1 to 15 is implemented.
18. A computer-readable storage medium having a program stored thereon, applied to a system for generating an image data set, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 15 is implemented.
Citation Information
Patent Citations
Method and device for generating training data
CN109190674A
Image quality evaluation method and device based on artificial intelligence and computer equipment
CN110807757A
Image training data set generation method, device and equipment and medium
CN111047693A
Image data set processing method and system, storage medium, program and device
CN111311721A
Methods and systems for automatic generation of massive training data sets from 3D models for training deep learning networks
US10489683B1
