Simulation scene generation method and device, electronic equipment and computer program product
By extracting the outline information of the target object in the real scene target image and generating the target map, the problem of low similarity between the simulated scene and the real scene in the prior art is solved, and a higher similarity and morphological consistency are achieved.
Patent Information
- Application Number
- CN202411974851.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-30
- Publication Date
- 2025-05-06
AI Technical Summary
When generating a simulated scene, the existing style transfer method easily destroys the content of the target image, resulting in a low similarity between the simulated scene and the real scene.
By obtaining the target image of the real scene, extracting the outline information of each target object, determining the target map, and generating a simulated scene based on these maps to ensure that the shape and structure of the target object are consistent with the real scene.
The similarity between the simulation scene and the real scene is improved, ensuring that the shape and structure of the target object in the simulation scene are consistent with the real scene.
Smart Images
Figure CN119941888A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of image processing technology, and in particular, relates to a method, device, electronic device and computer program product for generating a simulation scene. Background Art
[0002] When generating a simulation scene corresponding to a real scene, it is usually necessary to obtain a target image corresponding to the real scene, and then perform style transfer on the target image, so as to obtain a simulation scene. The current common style transfer method usually performs transition processing on the content of the target image, resulting in the destruction of the content of the target image during the style transfer process. For example, in an intelligent driving test scenario, it is necessary to obtain a target image of real objects such as vehicles, pedestrians, buildings, and roadblocks, and then perform style transfer on the target image to obtain a simulation scene for intelligent driving testing. However, the current common style transfer method usually causes the shapes and details of simulated objects such as vehicles, pedestrians, buildings, and roadblocks in the simulation scene to be inconsistent with the shapes and details of real objects such as vehicles, pedestrians, buildings, and roadblocks in the target image, reducing the similarity between the generated simulation scene and the real scene. Summary of the invention
[0003] In view of this, the embodiments of the present application provide a method, device, electronic device and computer program product for generating a simulation scene, so as to solve the technical problem that the existing generated simulation scene has a low similarity with the real scene.
[0004] In a first aspect, an embodiment of the present application provides a method for generating a simulation scene, comprising:
[0005] Obtain the target image corresponding to the real scene;
[0006] Extracting contour information of each target object in the target image;
[0007] Determining a target map for each target object according to the contour information of each target object;
[0008] A simulation scene corresponding to the real scene is generated according to the target map of each target object.
[0009] Optionally, extracting the contour information of each target object in the target image includes:
[0010] Inputting the target image into a preset semantic segmentation model, and obtaining semantic information of each pixel of the target image output by the semantic segmentation model according to the target image, wherein the semantic information includes a first object category corresponding to the pixel;
[0011] According to the semantic information, the contour information of each target object in the target image is determined.
[0012] Optionally, determining the contour information of each target object in the target image according to the semantic information includes:
[0013] Determine, according to the semantic information of each pixel, a plurality of pixel combinations, each of which includes a plurality of pixels of the same first object category and adjacent to each other;
[0014] The contour information of each of the target objects is determined according to the position information of each of the pixel points in each of the pixel point combinations.
[0015] Optionally, determining a target map of each target object according to the contour information of each target object includes:
[0016] For each of the target objects, a second object category of the target object is determined according to the first object category of the pixel points in the pixel point combination corresponding to the target object, and according to the second object category, a number of candidate maps are determined from a preset map database, and according to the contour information of the target object, the similarity between the contour of the target object and each of the candidate maps is calculated, and according to the similarity between the contour of the target object and each of the candidate maps, the target map of the target object is determined from each of the candidate maps.
[0017] Optionally, the preset map database includes maps corresponding to the first object at each preset viewing angle and texture maps corresponding to the second object; wherein a first dependency coefficient of the first object on the perspective angle is greater than a second dependency coefficient of the second object on the perspective angle.
[0018] Optionally, generating a simulation scene corresponding to the real scene according to the target map of each target object includes:
[0019] For each target object, according to the contour information corresponding to the target object, the target map of the target object is added to the corresponding area in the simulation scene.
[0020] Optionally, after adding the target map of the target object to a corresponding area in the simulation scene according to the contour information corresponding to the target object, the method further includes:
[0021] The edge of the target map added to the corresponding area in the simulation scene is smoothed.
[0022] In a second aspect, an embodiment of the present application provides a device for generating a simulation scene, including:
[0023] An image acquisition unit, used to acquire a target image corresponding to a real scene;
[0024] A contour information extraction unit, used to extract contour information of each target object in the target image;
[0025] A map determining unit, used for determining a target map of each target object according to the contour information of each target object;
[0026] The simulation scene generation unit is used to generate a simulation scene corresponding to the real scene according to the target map of each target object.
[0027] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, each step in the method for generating a simulation scene as described in any one of the first aspects above is implemented.
[0028] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the method for generating a simulation scene as described in any one of the first aspects above are implemented.
[0029] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes each step in the method for generating a simulation scene as described in any one of the first aspects above.
[0030] The simulation scene generation method, device, electronic device and computer program product provided by the embodiments of the present application have the following beneficial effects:
[0031] In the method for generating a simulation scene provided in an embodiment of the present application, a target image corresponding to a real scene is first obtained, and then the contour information of each target object in the target image is extracted, and then the target map of each target object is determined according to the contour information of each target object, and finally the simulation scene corresponding to the real scene is generated according to the target map of each target object. The method of the present application extracts the contour information of each target object in the target image, and then generates the simulation scene corresponding to the real scene according to the contour information of each target object, so that the morphology and structure of the target object in the simulation scene are basically consistent with the morphology and structure of the target object in the real scene, thereby improving the similarity between the generated simulation scene and the real scene. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0033] Figure 1 A flowchart of a method for generating a simulation scene provided in an embodiment of the present application;
[0034] Figure 2 A schematic diagram of the structure of a device for generating a simulation scene provided in an embodiment of the present application;
[0035] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0036] It should be noted that the terms used in the embodiments of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application. In the description of the embodiments of the present application, unless otherwise specified, "multiple" refers to two or more than two, and "at least one", "one or more" refers to one, two or more. The terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, it is defined that the "first" and "second" features can explicitly or implicitly include one or more of the features.
[0037] References to "one embodiment" or "some embodiments" etc. described in this specification mean that a particular feature, structure or characteristic described in conjunction with the embodiment is included in one or more embodiments of the present application. Thus, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear at different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.
[0038] The execution subject of the method for generating a simulation scene provided in the embodiment of the present application may be an electronic device, which may execute each step of the method for generating a simulation scene provided in the embodiment of the present application. Specifically, the electronic device may include but is not limited to mobile phones, tablet computers, laptop computers, desktop computers and other devices.
[0039] The method for generating a simulation scene provided in the embodiment of the present application can be applied to the application scenario of generating simulation scenes corresponding to various real scenes. Specifically, when it is necessary to generate a simulation scene corresponding to a real scene, the user can perform each step of the method for generating a simulation scene provided in the embodiment of the present application through an electronic device, thereby generating a simulation scene corresponding to the real scene, and improving the similarity between the generated simulation scene and the real scene.
[0040] Exemplarily, the method for generating a simulation scene provided in an embodiment of the present application can be applied to a test scene of intelligent driving. In the test scene of intelligent driving, it is necessary to convert a real scene including real objects such as vehicles, pedestrians, buildings, trees, and roads into a simulation scene, so as to test the intelligent driving technology of the vehicle through the simulation scene. Based on this, in the test scene of intelligent driving, the various steps provided by the method for generating a simulation scene provided in an embodiment of the present application can be executed by an electronic device, so as to generate a simulation scene for performing an intelligent driving test, and the generated intelligent driving test simulation scene can have a high degree of similarity with the real scene.
[0041] See also Figure 1 , Figure 1 The present invention provides a flow chart of a method for generating a simulation scene according to an embodiment of the present invention. The method for generating a simulation scene may include S101 to S104, which are described in detail as follows:
[0042] In S101, a target image corresponding to a real scene is obtained.
[0043] In an embodiment of the present application, when it is necessary to obtain a simulated scene corresponding to a real scene, the electronic device may first obtain a target image corresponding to the real scene.
[0044] In one possible implementation, the electronic device may capture the real scene using a shooting device to obtain a target image corresponding to the real scene. In another possible implementation, the user may input the target image corresponding to the real scene into the electronic device so that the electronic device can obtain the target image corresponding to the real scene.
[0045] In S102, contour information of each target object in the target image is extracted.
[0046] In an embodiment of the present application, after acquiring a target image corresponding to a real scene, the electronic device may extract contour information of each target object included in the target image.
[0047] In practical applications, the electronic device can determine the target object according to the specific application scenario. For example, if the application scenario is an intelligent driving test scenario, the electronic device can determine real objects such as vehicles, pedestrians, buildings, trees, and roads in the target image as the target object. The specific method for determining the target object can be set according to actual needs and is not limited here.
[0048] In a possible implementation, the electronic device can extract the contour information of each target object in the target image through steps a and b. The details are as follows:
[0049] In step a, the target image is input into a preset semantic segmentation model, and the semantic information of each pixel point of the target image output by the semantic segmentation model based on the target image is obtained.
[0050] In this implementation, the electronic device can input the target image into a preset semantic segmentation model, and obtain the semantic information of each pixel of the target image output by the semantic segmentation model according to the target image. The semantic information includes the first object category corresponding to the pixel, and each pixel has its corresponding semantic information, that is, each pixel corresponds to a first object category.
[0051] For example, in the field of intelligent driving testing, the first object category may include but is not limited to vehicles, pedestrians, buildings, trees, roads, etc.
[0052] Exemplarily, the target image may include pixel a, pixel b, pixel c, pixel d and pixel e, etc. Based on this, the semantic information of pixel a may be a vehicle, the semantic information of pixel b may be a pedestrian, the semantic information of pixel c may be a building, the semantic information of pixel d may be a tree, and the semantic information of pixel e may be a road.
[0053] The semantic segmentation model may be pre-trained. The specific training method of the semantic segmentation model may be set according to actual needs and is not limited here.
[0054] In step b, the contour information of each target object in the target image is determined according to the semantic information.
[0055] In this implementation, after determining the semantic information of each pixel of the target image, the electronic device can determine the contour information of each target object in the target image according to the semantic information of each pixel of the target image. The contour information of the target object can be the outer contour line information of the target object. Specifically, the contour information of the target object can be the position coordinates of each pixel constituting the outer contour line of the target object.
[0056] Specifically, the electronic device may determine a plurality of pixel combinations based on the semantic information of each pixel in the target image, wherein each pixel combination includes a plurality of pixels of the same first object category and adjacent to each other. Specifically, the electronic device may determine the pixels of the same first object category and adjacent to each other as the same pixel combination, thereby determining a plurality of pixel combinations. Exemplarily, if the first object categories corresponding to pixel e, pixel f, pixel g, and pixel h are all vehicles, and pixel e is adjacent to pixel f, pixel f is adjacent to pixel g, and pixel g is adjacent to pixel h, then the electronic device may determine pixel e, pixel f, pixel g, and pixel h as the same pixel combination.
[0057] After determining a number of pixel point combinations, the electronic device may determine the contour information of each target object according to the position information of each pixel point in each pixel point combination.
[0058] In a possible implementation, the electronic device may also determine the contour information of each target object by using any one or more of the Canny edge detection algorithm, the Sobel edge detection algorithm, and the Prewitt edge detection algorithm.
[0059] In S103, a target map of each target object is determined according to the contour information of each target object.
[0060] In the embodiment of the present application, after determining the contour information of each target object, the electronic device may determine the target map of each target object according to the contour information of each target object.
[0061] Specifically, the electronic device may determine, for each target object, the second object category of the target object according to the first object category of the pixel points in the pixel point combination corresponding to the target object.
[0062] Specifically, the electronic device may determine the first object category of the pixel points in the pixel point combination corresponding to the target object as the second object category of the target object. Exemplarily, if the first object category of the pixel points in the pixel point combination corresponding to the target object is a vehicle, the second object category of the target object may be determined as a vehicle; if the first object category of the pixel points in the pixel point combination corresponding to the target object is a pedestrian, the second object category of the target object may be determined as a pedestrian.
[0063] After determining the second object category of the target object, the electronic device may determine a number of stickers to be selected from a preset sticker database according to the second object category of the target object. Specifically, the electronic device may determine all stickers in the preset sticker database whose object category is the second object category of the target object as stickers to be selected. Exemplarily, if the second object category of the target object is a vehicle, the electronic device may determine all stickers in the preset sticker database whose object category is a vehicle as stickers to be selected.
[0064] After determining several to-be-selected maps, the electronic device can calculate the similarity between the outline of the target object and each to-be-selected map based on the outline information of the target object. The specific method for calculating the similarity between the outline of the target object and each to-be-selected map can be set according to actual needs and is not limited here. Exemplarily, the electronic device can determine the similarity between the outline of the target object and each to-be-selected map by calculating the Hausdorff distance between the outline of the target object and each to-be-selected map; or, the electronic device can determine the similarity between the outline of the target object and each to-be-selected map by calculating the chamfer distance between the outline of the target object and each to-be-selected map; or, the electronic device can also determine the similarity between the outline of the target object and each to-be-selected map by a shape context matching method.
[0065] After calculating the similarity between the contour of the target object and each of the candidate maps, the electronic device can determine the target map of the target object from the candidate maps according to the similarity between the contour of the target object and each of the candidate maps. Specifically, the electronic device can determine the candidate map with the highest similarity to the contour of the target object as the target map of the target object.
[0066] In a possible implementation, the map database may include maps corresponding to a plurality of first objects at various preset viewing angles, and texture maps corresponding to a second object, wherein the difference between the first object and the second object is that a first dependence coefficient of the first object on the perspective angle is greater than a second dependence coefficient of the second object on the perspective angle.
[0067] Exemplarily, in the intelligent driving test scene, the first object may be an object such as a vehicle and a pedestrian, and the shape and posture of these first objects are strongly dependent on the perspective angle, that is, the first dependence coefficient of the first object on the perspective angle is greater than the preset dependence coefficient threshold. Therefore, when the second object category of the target object is the category corresponding to the first object (that is, when the second object category of the target object is an object such as a vehicle and a pedestrian), it is necessary to select a target map with a high similarity to the contour information of the target object in the map database according to the contour information of the target object, and ensure that the target map is geometrically consistent with the shape and perspective of the target object, thereby improving the similarity between the generated simulation scene and the real scene. Taking the target object as a vehicle as an example, if the vehicle is displayed frontally in the target image, the electronic device will select a front perspective image of the selected vehicle that is similar to the contour of the vehicle in the map database according to the contour information of the target object as the target map. Correspondingly, if the vehicle is displayed sideways in the target image, the electronic device will select a side perspective image of the selected vehicle that is similar to the contour of the vehicle in the map database according to the contour information of the target object as the target map.
[0068] Exemplarily, in an intelligent driving test scenario, the second object may be objects such as trees, roads, and buildings. The shapes and postures of these second objects are weakly dependent on the perspective angle, that is, the second dependence coefficient of the second object on the perspective angle is less than or equal to the preset dependence coefficient threshold. The appearance of these second objects is largely determined by the surface texture, and changes in the perspective angle have little effect on the appearance of the second object. Therefore, when the second object category of the target object is the category corresponding to the second object (that is, when the second object category of the target object is objects such as trees, roads, and buildings), a texture map with a high similarity to the contour information of the target object can be selected from the map database as the target map.
[0069] In actual applications, the user may mark each first object and each second object and input the mark into the electronic device, so that the electronic device can distinguish the first object from the second object.
[0070] In practical applications, the texture database can store 3D images of each first object, so that the electronic device can obtain the texture of the first object at different viewing angles from the texture database. In addition, the texture database can store texture images of each second object without storing 3D images of each second object.
[0071] In S104, a simulation scene corresponding to the real scene is generated according to the target map of each target object.
[0072] In the embodiment of the present application, after determining the target map of each target object, the electronic device can generate a simulation scene corresponding to the real scene according to the target map of each target object.
[0073] Specifically, the electronic device can, for each target object, add the target map of the target object to the corresponding area in the simulation scene according to the contour information corresponding to the target object. Specifically, the electronic device can, for each target object, determine the position information of the target map in the corresponding area in the simulation scene and the setting angle information of the target map in the simulation scene according to the contour information corresponding to the target object. After that, the electronic device can add the target map of the target object to the corresponding area in the simulation scene according to the position information of the target map in the corresponding area in the simulation scene and the setting angle information of the target map in the simulation scene.
[0074] In a possible implementation, after adding the target map of the target object to the corresponding area in the simulation scene, the electronic device may also correct the position of the target map added to the simulation scene. Specifically, the electronic device may determine whether the position of the target map added to the simulation scene is completely consistent with the position of the target object corresponding to the target map in the target image based on the semantic information of each pixel of the target image. If not, the electronic device may correct the position of the target map added to the simulation scene so that the position of the target map added to the simulation scene is completely consistent with the position of the target object corresponding to the target map in the target image.
[0075] In one possible implementation, after adding the target map of the target object to the corresponding area in the simulation scene, the electronic device may also smooth the edges of the target map added to the corresponding area in the simulation scene to eliminate the seam problem that may occur in the process of adding the target map to the simulation scene, thereby making the transition between the target map and the original area of the simulation scene more natural.
[0076] It can be seen from the above that in the method for generating a simulation scene provided in the embodiment of the present application, the target image corresponding to the real scene is first obtained, and then the contour information of each target object in the target image is extracted, and then the target map of each target object is determined according to the contour information of each target object, and finally the simulation scene corresponding to the real scene is generated according to the target map of each target object. The method of the present application extracts the contour information of each target object in the target image, and then generates the simulation scene corresponding to the real scene according to the contour information of each target object, so that the morphology and structure of the target object in the simulation scene are basically consistent with the morphology and structure of the target object in the real scene, thereby improving the similarity between the generated simulation scene and the real scene.
[0077] Specifically, since the method for generating the simulation scene of the present application determines the target map of each target object according to the extracted contour information of each target object, and since the map database includes the maps of the first object at different viewing angles, the target map of each target object determined to be obtained has a high similarity with the target object, and can be basically consistent in form and structure. However, some style transfer methods in the prior art usually over-process the target object, and do not take into account the perspective problem of the target object, which may cause the target object with a high dependence coefficient on the perspective angle to deviate in form and structure during the simulation process. In addition, since the method for generating the simulation scene of the present application needs to obtain the semantic information of each pixel (i.e., the first object category of each pixel) when extracting the contour information of each target object in the target image, the accuracy of the extracted contour information of each target object in the target image is high, thereby further improving the similarity between the generated simulation scene and the real scene.
[0078] Based on the simulation scene generation method provided in the above embodiment, the present application further provides a simulation scene generation device that implements the above method embodiment, please refer to Figure 2 , Figure 2 The schematic diagram of the structure of a device for generating a simulation scene provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the simulation scene generation device 20 may include: an image acquisition unit 21, a contour information extraction unit 22, a map determination unit 23 and a simulation scene generation unit 24. Among them:
[0079] The image acquisition unit 21 is used to acquire a target image corresponding to a real scene.
[0080] The contour information extraction unit 22 is used to extract the contour information of each target object in the target image.
[0081] The map determining unit 23 is used to determine the target map of each target object according to the contour information of each target object.
[0082] The simulation scene generation unit 24 is used to generate a simulation scene corresponding to the real scene according to the target map of each target object.
[0083] Optionally, the contour information extraction unit 22 is specifically used for:
[0084] Inputting the target image into a preset semantic segmentation model, and obtaining semantic information of each pixel of the target image output by the semantic segmentation model according to the target image, wherein the semantic information includes a first object category corresponding to the pixel;
[0085] According to the semantic information, the contour information of each target object in the target image is determined.
[0086] Optionally, the contour information extraction unit 22 is specifically used for:
[0087] Determine a plurality of pixel combinations according to the semantic information of each pixel, each pixel combination including a plurality of pixels that are of the same first object category and adjacent to each other;
[0088] According to the position information of each pixel point in each pixel point combination, the contour information of each target object is determined.
[0089] Optionally, the mapping determination unit 23 is specifically used for:
[0090] For each target object, a second object category of the target object is determined according to the first object category of the pixel points in the pixel point combination corresponding to the target object, and according to the second object category, a number of candidate maps are determined from a preset map database, and according to the contour information of the target object, the similarity between the contour of the target object and each candidate map is calculated, and according to the similarity between the contour of the target object and each candidate map, the target map of the target object is determined from each candidate map.
[0091] Optionally, the preset map database includes maps corresponding to the first object at each preset viewing angle and texture maps corresponding to the second object; wherein a first dependency coefficient of the first object on the perspective angle is greater than a second dependency coefficient of the second object on the perspective angle.
[0092] Optionally, the simulation scenario generating unit 24 is specifically used for:
[0093] For each target object, according to the contour information corresponding to the target object, the target map of the target object is added to the corresponding area in the simulation scene.
[0094] Optionally, the simulation scene generation unit 24 is specifically used to: smooth the edge of the target map added to the corresponding area in the simulation scene.
[0095] It should be noted that the information interaction, execution process and other contents between the above-mentioned units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be specifically referred to the method embodiment part and will not be repeated here.
[0096] See also Figure 3 , Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 3As shown, the electronic device 3 provided in this embodiment may include: a processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the processor 30, such as a program corresponding to the method for generating a simulation scene. When the processor 30 executes the computer program 32, the steps in the embodiment of the method for generating a simulation scene are implemented, such as Figure 1 Alternatively, when the processor 30 executes the computer program 32, the functions of each module / unit in the embodiment of the device for generating the simulation scene are realized, for example Figure 2 The functions of the units 21 to 24 are shown.
[0097] Exemplarily, the computer program 32 may be divided into one or more modules / units, one or more modules / units are stored in the memory 31 and executed by the processor 30 to complete the present application. One or more modules / units may be a series of computer program instruction segments that can complete specific functions, and the instruction segments are used to describe the execution process of the computer program 32 in the electronic device 3. For example, the computer program 32 may be divided into an image acquisition unit 21, a contour information extraction unit 22, a mapping determination unit 23, and a simulation scene generation unit 24. For the specific functions of each unit, please refer to Figure 2 The relevant descriptions in the corresponding embodiments are not repeated here.
[0098] Those skilled in the art will understand that Figure 3 This is only an example of the electronic device 3 and does not constitute a limitation on the electronic device 3 , which may include more or less components than those shown in the figure, or a combination of certain components, or different components.
[0099] The processor 30 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.
[0100] The memory 31 may be an internal storage unit of the electronic device 3, such as a hard disk or memory of the electronic device 3. The memory 31 may also be an external storage device of the electronic device 3, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, or a flash card, etc., equipped on the electronic device 3. Further, the memory 31 may also include both an internal storage unit and an external storage device of the electronic device 3. The memory 31 is used to store computer programs and other programs and data required by the electronic device. The memory 31 may also be used to temporarily store data that has been output or is to be output.
[0101] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units is used as an example for illustration. In actual applications, the above-mentioned function allocation can be completed by different functional units as needed, that is, the internal structure of the generation device of the simulation scene can be divided into different functional units to complete all or part of the functions described above. The functional units in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.
[0102] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.
[0103] An embodiment of the present application provides a computer program product. When the computer program product is executed on a terminal device, the terminal device implements the steps in the above-mentioned various method embodiments.
[0104] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0105] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0106] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.
Claims
1. A method for generating a simulation scene, characterized in that: include: Obtain the target image corresponding to the real scene; Extracting contour information of each target object in the target image; Determining a target map for each target object according to the contour information of each target object; A simulation scene corresponding to the real scene is generated according to the target map of each target object.
2. The method according to claim 1, characterized in that The extracting the contour information of each target object in the target image includes: Inputting the target image into a preset semantic segmentation model, and obtaining semantic information of each pixel of the target image output by the semantic segmentation model according to the target image, wherein the semantic information includes a first object category corresponding to the pixel; According to the semantic information, the contour information of each target object in the target image is determined.
3. The method according to claim 2, characterized in that Determining the contour information of each target object in the target image according to the semantic information includes: Determine, according to the semantic information of each pixel, a plurality of pixel combinations, each of which includes a plurality of pixels of the same first object category and adjacent to each other; The contour information of each of the target objects is determined according to the position information of each of the pixel points in each of the pixel point combinations.
4. The method according to claim 3, characterized in that Determining a target map of each target object according to the contour information of each target object includes: For each of the target objects, a second object category of the target object is determined according to the first object category of the pixel points in the pixel point combination corresponding to the target object, and according to the second object category, a number of candidate maps are determined from a preset map database, and according to the contour information of the target object, the similarity between the contour of the target object and each of the candidate maps is calculated, and according to the similarity between the contour of the target object and each of the candidate maps, the target map of the target object is determined from each of the candidate maps.
5. The method according to claim 4, characterized in that The preset map database includes maps corresponding to the first object at each preset viewing angle and texture maps corresponding to the second object; wherein a first dependency coefficient of the first object on the perspective angle is greater than a second dependency coefficient of the second object on the perspective angle.
6. The method according to any one of claims 1 to 5, characterized in that: Generating a simulation scene corresponding to the real scene according to the target map of each target object includes: For each target object, according to the contour information corresponding to the target object, the target map of the target object is added to the corresponding area in the simulation scene.
7. The method according to claim 6, characterized in that After adding the target map of the target object to the corresponding area in the simulation scene according to the contour information corresponding to the target object, the method further includes: The edge of the target map added to the corresponding area in the simulation scene is smoothed.
8. A device for generating a simulation scene, characterized in that: include: An image acquisition unit, used to acquire a target image corresponding to a real scene; A contour information extraction unit, used to extract contour information of each target object in the target image; A map determining unit, used for determining a target map of each target object according to the contour information of each target object; The simulation scene generation unit is used to generate a simulation scene corresponding to the real scene according to the target map of each target object.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, each step in the method for generating a simulation scene according to any one of claims 1 to 7 is implemented.
10. A computer program product, characterized in that When the computer program product is executed by a processor, each step in the method for generating a simulation scene according to any one of claims 1 to 7 is implemented.