Simulation image generation methods, apparatus, computer equipment and computer program products

Abstract

This application relates to the field of image processing technology, and provides a method, apparatus, computer device, and computer program product for generating simulated images. The method includes: constructing a three-dimensional scene based on pre-acquired real-world images and map data; integrating foreground materials into the three-dimensional scene; rendering the three-dimensional scene after integrating the foreground materials; and generating a simulated image for target detection. This simulated image can accurately simulate various scenes in the real environment, thereby helping pedestrian detection or recognition systems to be more accurate when processing actual images.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, and in particular relates to a method, apparatus, computer equipment and computer program product for generating simulated images. Background Technology

[0002] In recent years, with the development of computer vision technology, image and video processing technologies have been widely applied in many fields, especially target (such as pedestrian) detection and recognition, which has become one of the important research directions. Existing target detection methods mainly rely on two-dimensional image information, identifying targets through feature extraction, model training, and other methods. However, these methods show a significant decrease in recognition performance when the target is partially occluded (such as a pedestrian being partially occluded behind a billboard). Summary of the Invention

[0003] This application provides a method, apparatus, computer device, and computer program product for generating simulated images, which can solve the technical problem that the recognition effect will be significantly reduced and inaccurate when the target is partially occluded in existing target detection and recognition methods.

[0004] In a first aspect, embodiments of this application provide a method for generating simulated images, including:

[0005] A 3D scene is constructed based on pre-acquired real-world images and map data;

[0006] Integrate the foreground material into the 3D scene;

[0007] The 3D scene, after incorporating the foreground material, is rendered to generate a simulated image for target detection.

[0008] In one possible implementation of the first aspect, constructing a 3D scene based on pre-acquired real-world images and map data includes:

[0009] Acquire the real-scene image and determine the display range of the real-scene image;

[0010] An initial 3D scene corresponding to the display range is generated based on the map data;

[0011] Align the real-world image with the initial 3D scene to obtain the 3D scene.

[0012] In one possible implementation of the first aspect, aligning the real-world image and the initial 3D scene to obtain the 3D scene includes:

[0013] Obtain the first coordinates of the preset anchor point on the real-world image;

[0014] Determine the second coordinates of the preset anchor point in the initial 3D scene;

[0015] Based on the first coordinate and the second coordinate, the real-world image and the initial 3D scene are aligned to obtain the 3D scene.

[0016] In one possible implementation of the first aspect, integrating the foreground material into the 3D scene includes:

[0017] Import the foreground material into the 3D twin engine;

[0018] Adjust the coordinate system of the foreground material imported into the 3D twin engine so that the coordinate system of the foreground material is consistent with the world coordinate system of the 3D twin engine;

[0019] The foreground material is positioned at a preset location in the three-dimensional scene to obtain the three-dimensional scene integrated with the foreground material.

[0020] In one possible implementation of the first aspect, the process of integrating foreground material into the 3D scene further includes:

[0021] Behavioral parameters are set for the foreground material to generate dynamic foreground material; wherein, the behavioral parameters include any combination of the following: position, speed, trajectory, rotation parameters, interaction rules, and physical properties;

[0022] The step of rendering the 3D scene after incorporating the foreground material to generate a simulated image for object detection includes:

[0023] Update the state of the dynamic foreground materials in the 3D scene;

[0024] The objects to be rendered in the three-dimensional scene are subjected to geometric transformations and lighting and shadow processing to generate the simulation image; wherein, the objects include the dynamic foreground material.

[0025] In one possible implementation of the first aspect, the foreground material includes: obstacle material and non-obstacle material;

[0026] Generating the simulated image includes:

[0027] Based on the obstacle material and the target to be detected, a simulated image of partial occlusion is generated;

[0028] Alternatively, based on the non-obstacle material and the target to be detected, a simulated image containing multiple targets is generated; wherein the multiple targets include the target to be detected and the non-obstacle material;

[0029] Alternatively, based on the obstacle material, the non-obstacle material, and the target to be detected, a simulated image containing multiple targets with partial occlusion can be generated.

[0030] In one possible implementation of the first aspect, the foreground material includes at least one of the following: a 3D model foreground material, a network image foreground material, or a cutout transplanted foreground material.

[0031] Secondly, embodiments of this application provide a simulated image generation apparatus, comprising:

[0032] The building module is used to construct 3D scenes based on pre-acquired real-world images and map data;

[0033] The fusion module is used to integrate foreground materials into the 3D scene.

[0034] The rendering module is used to render the 3D scene after incorporating the foreground material, and generate a simulated image for target detection.

[0035] Thirdly, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the simulation image generation method described in any one of the first aspects above.

[0036] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the simulation image generation method described in any one of the first aspects above.

[0037] Fifthly, embodiments of this application provide a computer program product that, when run on a computer device, causes the computer device to execute the simulation image generation method described in any one of the first aspects.

[0038] The beneficial effects of this application's embodiments compared to existing technologies are: by integrating foreground materials into a 3D scene and utilizing 3D scene rendering technology, more realistic simulated images for user object detection are generated. These simulated images can better reflect the local occlusion of pedestrians in complex environments, thereby significantly improving the detection success rate of the algorithm in practical application scenarios.

[0039] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a schematic flowchart of a simulation image generation method provided in an embodiment of this application;

[0042] Figure 2 This is a schematic flowchart illustrating a method for generating a simulated image, provided in another embodiment of this application.

[0043] Figure 3 This is a schematic diagram of a simulated image of partial occlusion provided in an embodiment of this application;

[0044] Figure 4 This is a schematic diagram of a simulation image containing multiple targets provided in an embodiment of this application;

[0045] Figure 5 This is a schematic diagram of a simulated image containing multiple targets with partial occlusion provided in an embodiment of this application;

[0046] Figure 6 This is a schematic diagram of the structure of the simulation image generation device provided in the embodiments of this application;

[0047] Figure 7 This is a schematic diagram of the structure of the computer device provided in the embodiments of this application. Detailed Implementation

[0048] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0049] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0050] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0051] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0052] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0053] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0054] Figure 1 A schematic flowchart of a simulation image generation method provided in an embodiment of this application is shown.

[0055] S101 constructs a 3D scene based on pre-acquired real-world images and map data.

[0056] Among them, real-scene images are actual perceptual information obtained through multi-sensor devices, which can be real-scene photographs or real-scene videos from a fixed perspective. Real-scene images provide the visual foundation for 3D scenes, enabling the constructed 3D scenes to more closely resemble the real world.

[0057] The map data is 3D map data, which can be high-resolution / non-high-resolution tile maps or vector maps. Map data provides accurate geographic information, including the spatial location and attributes of roads, buildings, natural features, etc., and is the foundation for constructing a 3D scene that closely matches the real world. For example, if your real-world image is a street, map data can help you determine the street's width and the location of buildings.

[0058] S102, Integrate the foreground material into the three-dimensional scene.

[0059] Foreground elements are elements specifically added to a 3D scene to enhance its details or functionality. Foreground elements can be static (such as buildings, roads, and natural landscapes) or dynamic (such as moving pedestrians and vehicles) to simulate real-world objects, increasing the scene's realism and complexity. Foreground elements are typically stored in a resource library, which can be a digital collection of resources providing various 3D models, images, and other visual elements to support scene creation and simulation.

[0060] In this embodiment of the application, after the three-dimensional scene is constructed, the foreground material is loaded into the three-dimensional scene and coordinated with the original elements in the three-dimensional scene (such as background, road, pedestrian).

[0061] S103, render the three-dimensional scene after incorporating the foreground material to generate a simulated image for target detection.

[0062] In this embodiment of the application, all elements in the three-dimensional scene after incorporating foreground material are converted into two-dimensional images, which are the simulation images.

[0063] The simulated images are images generated using computer graphics techniques, mimicking scenes, objects, or phenomena in the real world. They can serve as datasets for training and testing object detection algorithms. Specifically:

[0064] Simulated images provide training data for machine learning models, especially when obtaining real-world data is costly or impractical. By introducing variations into simulated images, such as adding noise, changing object size, or rotating objects, the diversity of the dataset is enhanced, improving the model's generalization ability.

[0065] Simulated images are used to test the performance of object detection algorithms, such as their performance in tasks like pedestrian detection and vehicle recognition. They can also simulate various complex scenarios, such as occlusion, to improve the adaptability and robustness of the algorithms.

[0066] In this embodiment, by integrating foreground materials into a 3D scene and utilizing 3D scene rendering technology, more realistic simulated images for user target detection are generated. These simulated images can better reflect the local occlusion of pedestrians in complex environments, thereby significantly improving the detection success rate of the algorithm in real-world application scenarios.

[0067] Figure 2 A schematic flowchart of a simulation image generation method provided in another embodiment of this application is shown.

[0068] S201, acquire the real-scene image and determine the display range of the real-scene image.

[0069] The display range, also known as the imaging area, refers to the area in which the photosensitive element of a multi-source sensing device can effectively capture light.

[0070] S202, Generate an initial three-dimensional scene corresponding to the display range based on the map data.

[0071] In this embodiment, the aforementioned map data is imported into a 3D twin engine. Using the imported map data, an initial 3D scene corresponding to the display range of the real-world image is constructed in the 3D twin engine. This initial 3D scene includes elements such as roads and buildings, the positions and shapes of which are determined based on the map data.

[0072] Among them, 3D twin engines include, but are not limited to: UE4 / 5, Unity, OpenGL, and WebGL.

[0073] S203, Align the real-world image with the initial 3D scene to obtain the 3D scene.

[0074] In this embodiment, the initial 3D scene constructed above may not visually match the actual captured image (i.e., the real-world image). Therefore, it is necessary to align the real-world image with the initial 3D scene to create an accurate 3D scene that matches the real world.

[0075] In an optional embodiment, S203 includes: obtaining the first coordinates of a preset anchor point on the real-world image; determining the second coordinates of the preset anchor point in the initial three-dimensional scene; and aligning the real-world image and the initial three-dimensional scene based on the first coordinates and the second coordinates to obtain the three-dimensional scene.

[0076] The first coordinate can be a landmark on the real-world image, such as a utility pole, a large tree, or a building.

[0077] In this process, the position corresponding to the preset anchor point in the real-world image is found in the initial 3D scene, and this position is used as the second coordinate of the preset anchor point.

[0078] In this embodiment of the application, by aligning the first coordinates of the preset anchor points on the real scene image with the second coordinates of the preset anchor points in the initial three-dimensional scene, the traffic scene of the real scene image is aligned with the initial three-dimensional scene to obtain a three-dimensional scene.

[0079] The alignment methods mentioned above include:

[0080] Pixel-level alignment is achieved by adjusting the viewpoint and anchor points in the 3D scene, ensuring that the viewpoint of the 3D scene is completely consistent with that of the real-world image. The anchor points can be key locations, such as the horizon pixel position or the end-point pixel position of a road in the real-world image.

[0081] Alternatively, you can obtain the intrinsic and extrinsic parameters of a real camera or camcorder, such as resolution, focal length, and aperture, as well as external parameters like position, height, and angle. Then, you can set up a virtual camera in the 3D scene, making it use the same intrinsic and extrinsic parameters as a real camera, thus achieving automatic alignment.

[0082] This step ensures that the elements in the real-world image are spatially consistent with their corresponding elements in the 3D scene. This allows for a seamless visual integration when the real-world image is combined with the 3D scene, making the constructed 3D scene appear just like the real world.

[0083] S204, import the foreground material into the 3D twin engine.

[0084] One option is to drag and drop foreground materials into the window, or import them into the 3D twin engine via an interface.

[0085] S205, Adjust the coordinate system of the foreground material imported into the 3D twin engine so that the coordinate system of the foreground material is consistent with the world coordinate system of the 3D twin engine.

[0086] Since different foreground assets may use different coordinate systems, it may be necessary to adjust the coordinate system of the foreground assets to match the coordinate system of the 3D twin engine. For example, in Unity, it is usually necessary to adjust the coordinate system of the foreground assets from Z-forward to Y-up to conform to Unity's left-handed coordinate system.

[0087] S206, Position the foreground material at a preset position in the three-dimensional scene to obtain the three-dimensional scene integrated with the foreground material.

[0088] In a 3D twin engine, imported foreground materials need to be positioned accurately within the 3D scene. This may involve moving, rotating, and scaling the foreground materials in the 3D twin engine's scene editor to ensure they are correctly aligned with other elements in the 3D scene.

[0089] S207, Set behavior parameters for the foreground material to generate dynamic foreground material.

[0090] This involves setting behavioral parameters for foreground materials, which define the material's behavior and dynamic performance within the scene. These behavioral parameters can include position, velocity, trajectory, rotation parameters, interaction rules, and physical properties. The behavioral parameters can be any combination of the following: position, velocity, trajectory, rotation parameters, interaction rules, and physical properties.

[0091] For example, position parameters define the initial position of the foreground material in the scene; velocity and trajectory parameters define the movement path and speed of the foreground material; rotation parameters define the rotation behavior of the foreground material; interaction rules define how the foreground material responds to user input or other events; physical properties such as mass and friction are used for physical simulation.

[0092] For example, you can set rules for the appearance of foreground elements in the scene. These rules can be triggered based on time, location, or specific events. For instance, you can set foreground elements (pedestrians) to appear at an intersection at a specific time, or for vehicles to pass through the intersection when the green light is on. Alternatively, you can set motion trajectories for foreground elements. The motion trajectory can be a predefined sequence of path points or a dynamic path generated by an algorithm.

[0093] This allows setting behavior parameters for some or all of the foreground materials imported into the 3D twin engine to generate dynamic foreground materials. Thus, the 3D twin engine may contain both dynamic and static foreground materials.

[0094] S208, update the state of the dynamic foreground material in the three-dimensional scene; perform geometric transformation and lighting processing on the object to be rendered in the three-dimensional scene to generate the simulation image; wherein, the object includes the dynamic foreground material.

[0095] In step S207, behavioral parameters are set for the foreground material to generate dynamic foreground material. Control scripts also need to be written in the 3D twin engine to implement the behavior of the dynamic foreground material. These scripts will control the movement of the dynamic foreground material according to the defined behavioral parameters, including moving along a set path and responding to specific events or conditions. The position and state of the dynamic foreground material are updated in real time based on the control scripts and behavioral parameters, including updates to parameters such as position, pose, speed, and rotation, to reflect its dynamic behavior in the scene.

[0096] This process involves applying geometric transformations to all objects to be rendered in the 3D scene (including dynamic foreground materials, static foreground materials, and backgrounds). These transformations ensure that all elements in the 3D scene, including backgrounds, dynamic foreground materials, and static foreground materials, are correctly displayed in the rendered result. Even though static foreground materials do not change position over time, they still undergo these transformations to ensure correct display under different camera positions and viewpoints. Furthermore, geometric transformations also involve operations such as clipping and perspective correction to optimize rendering performance and visual effects.

[0097] This involves processing the lighting and shadows of all objects to be rendered in the 3D scene, including but not limited to: lighting and shading. Specifically, based on the light sources in the 3D scene and the material properties of each material, lighting effects and shading are calculated, including determining how the light source affects the color and brightness of the material, and how to generate shadows. In particular, it ensures that the lighting effects of dynamic foreground materials are updated in real time when their position and posture change.

[0098] In this embodiment, the dynamic updating of the foreground material's state ensures that the behavior and positional changes of the material in the 3D scene are reflected in the rendering result in real time. Geometric transformation and lighting processing are necessary rendering steps based on the state update to generate realistic simulation images. Their combined effect enables the generation or display of simulation images with realism and detail.

[0099] In one optional embodiment, after generating the simulation image, post-processing can be performed on the simulation image, such as tone mapping, color correction, anti-aliasing, etc., to improve the quality of the simulation image.

[0100] In one optional embodiment, the foreground material includes: obstacle material and non-obstacle material; generating the simulated image includes: generating a partially occluded simulated image based on the obstacle material and the target to be detected; or, generating a simulated image containing multiple targets based on the non-obstacle material and the target to be detected; wherein the multiple targets include the target to be detected and the non-obstacle material; or, generating a simulated image containing multiple targets with partial occlusion based on the obstacle material, the non-obstacle material, and the target to be detected.

[0101] In this embodiment, the process of integrating foreground materials into a 3D scene and generating a simulated image involves different types of foreground materials, including obstacle materials and non-obstacle materials. Obstacle materials can be open cardboard boxes, closed cardboard boxes, cones (road cones or traffic cones), or pillars (vertical obstacles, such as utility poles, streetlights, etc.). Non-obstacle materials can be pedestrian materials and vehicle materials. Pedestrian materials include pedestrians or construction workers facing different directions, etc., while vehicle materials include bicycles, electric vehicles, etc. The definition of non-obstacle materials can vary depending on the actual situation; for example, pedestrian materials and vehicle materials can also be obstacle materials. For instance, if an electric vehicle obstructs a pedestrian, then the electric vehicle is considered an obstacle material.

[0102] like Figure 3 As shown, a simulated image of partial occlusion is generated based on obstacle material A and the target to be detected B. Figure 4 As shown, a simulated image containing multiple targets is generated based on non-obstacle material C and the target to be detected B; wherein, the multiple targets include the target to be detected and non-obstacle material. Figure 5 As shown, a simulated image containing multiple targets with partial occlusion is generated based on obstacle material A, non-obstacle material C, and the target to be detected B. In this embodiment, the simulated image with partial occlusion can be used to simulate and improve the performance (accuracy, efficiency, and robustness) of the target detection algorithm when the target is partially occluded. The simulated image containing multiple targets can be used to test the algorithm's ability to detect and track multiple targets in multi-target scenes, and can also be used to simulate and improve the detection capability in complex scenes. The simulated image with partial occlusion and multiple targets is particularly useful for training and evaluating the performance of the target detection algorithm in complex scenes.

[0103] In the embodiments of this application, the aforementioned obstacle and non-obstacle materials can be used individually or in combination to create complex scenes. In a 3D scene, the position, size, and shape of these materials can be adjusted as needed to simulate different occlusion and obstacle conditions. For example, in training a pedestrian detection algorithm, the position and size of obstacles can be changed to simulate partially occluded pedestrians, thereby testing the algorithm's performance under these conditions. Furthermore, these materials can also be used to enhance the visual complexity of the scene, making it more closely resemble the diversity of the real world.

[0104] In one optional embodiment, the foreground material includes at least one of the following: a 3D model foreground material, a network image foreground material, or a cutout foreground material.

[0105] Among them, the 3D model foreground material can be a pre-created 3D model, which usually has detailed geometry and texture, and is used to create depth and a sense of space in the scene.

[0106] Foreground elements from online images can be photographs, illustrations, or any other form of image. They are typically used as backgrounds or overlaid on parts of a 3D scene to add visual detail and information.

[0107] Foreground cutout is a foreground object obtained by removing the background from an image using image editing techniques. Foreground cutout can be transferred or composited onto a new background to create composite images or scenes.

[0108] After incorporating these foreground elements into a 3D scene, the rendering pipeline converts all elements in the scene into images, generating the final simulated image. These images can be used to train and evaluate various computer vision algorithms, especially in object detection and recognition. This can improve the performance of algorithms in complex scenes and has broad application prospects in fields such as autonomous driving and security monitoring.

[0109] In one optional embodiment, after generating a large number of simulated images, the target detection model can be trained and optimized based on these images to obtain a more accurate model. For example, occluded targets in the image are labeled according to the degree of occlusion. If a pedestrian is partially occluded, it can be labeled "Occlusion degree: Partial occlusion". All objects can be labeled manually or automatically. The accuracy and consistency of the labels are checked to ensure that the quality of the labels meets the requirements of the training model. Based on the feedback from model training, it may be necessary to return to the image labeling stage for adjustments and improvements to obtain an optimized model. This optimized model is then used for target detection and recognition, improving the accuracy of target detection and recognition.

[0110] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0111] Corresponding to the simulation image generation method described in the above embodiments, Figure 6 A structural block diagram of the simulation image generation apparatus provided in the embodiments of this application is shown. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0112] Reference Figure 6 The simulated image generation device includes:

[0113] The building module is used to construct 3D scenes based on pre-acquired real-world images and map data;

[0114] The fusion module is used to integrate foreground materials into the 3D scene.

[0115] The rendering module is used to render the 3D scene after incorporating the foreground material, and generate a simulated image for target detection.

[0116] In one possible implementation, a building module is used for:

[0117] Acquire the real-scene image and determine the display range of the real-scene image;

[0118] An initial 3D scene corresponding to the display range is generated based on the map data;

[0119] Align the real-world image with the initial 3D scene to obtain the 3D scene.

[0120] In one possible implementation, a building module is used for:

[0121] Obtain the first coordinates of the preset anchor point on the real-world image;

[0122] Determine the second coordinates of the preset anchor point in the initial 3D scene;

[0123] Based on the first coordinate and the second coordinate, the real-world image and the initial 3D scene are aligned to obtain the 3D scene.

[0124] In one possible implementation, the fusion module is used for:

[0125] Import the foreground material into the 3D twin engine;

[0126] Adjust the coordinate system of the foreground material imported into the 3D twin engine so that the coordinate system of the foreground material is consistent with the world coordinate system of the 3D twin engine;

[0127] The foreground material is positioned at a preset location in the three-dimensional scene to obtain the three-dimensional scene integrated with the foreground material.

[0128] In one possible implementation, the simulated image generation device further includes a behavior parameter setting module for:

[0129] Behavioral parameters are set for the foreground material to generate dynamic foreground material; wherein, the behavioral parameters include any combination of the following: position, speed, trajectory, rotation parameters, interaction rules, and physical properties;

[0130] The rendering module is used for:

[0131] Update the state of the dynamic foreground materials in the 3D scene;

[0132] The objects to be rendered in the three-dimensional scene are subjected to geometric transformations and lighting and shadow processing to generate the simulation image; wherein, the objects include the dynamic foreground material.

[0133] In one possible implementation, the foreground material includes: obstacle material and non-obstacle material; the rendering module is used for:

[0134] Based on the obstacle material and the target to be detected, a simulated image of partial occlusion is generated;

[0135] Alternatively, based on the non-obstacle material and the target to be detected, a simulated image containing multiple targets is generated; wherein the multiple targets include the target to be detected and the non-obstacle material;

[0136] Alternatively, based on the obstacle material, the non-obstacle material, and the target to be detected, a simulated image containing multiple targets with partial occlusion can be generated.

[0137] In one possible implementation, the foreground material includes at least one of the following: 3D model foreground material, network image foreground material, and cutout transplanted foreground material.

[0138] It should be noted that the information interaction and execution process between the above modules are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0139] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0140] This application also provides a computer device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor executes the computer program to implement the steps in any of the above method embodiments.

[0141] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0142] This application provides a computer program product that, when run on a mobile terminal, enables the mobile terminal to implement the steps described in the above-described method embodiments.

[0143] Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Figure 7 As shown, the computer device of this embodiment includes: at least one processor 20 ( Figure 7 (Only one is shown in the diagram), memory 21, and computer program 22 stored in said memory 21 and executable on said at least one processor 20, wherein said processor 20 executes said computer program 22 to implement the steps in any of the above embodiments of the simulation image generation method.

[0144] The computer device may include, but is not limited to, a processor 20 and a memory 21. Those skilled in the art will understand that... Figure 7 The examples of computer devices are merely examples and do not constitute a limitation on computer devices. They may include more or fewer components than shown in the illustration, or combinations of certain components, or different components. For example, they may also include input / output devices, network access devices, etc.

[0145] The processor 20 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0146] In some embodiments, the memory 21 may be an internal storage unit of the computer device, such as a hard disk or memory. In other embodiments, the memory 21 may be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc. Furthermore, the memory 21 may include both internal and external storage units of the computer device. The memory 21 is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory 21 can also be used to temporarily store data that has been output or will be output.

[0147] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a device / computer equipment, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0148] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0149] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0150] In the embodiments provided in this application, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings or direct couplings or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0151] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0152] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

[0153] The relevant user personal information that may be involved in the various embodiments of this application is processed in strict accordance with the requirements of laws and regulations, following the principles of legality, legitimacy, and necessity, based on the reasonable purpose of the business scenario, and includes personal information that users actively provide or that is generated as a result of using the product / service, as well as personal information obtained with user authorization.

[0154] The personal information of users processed by the applicant will vary depending on the specific product / service scenario and will be based on the specific scenario in which the user uses the product / service. This may involve the user's account information, device information, driving information, vehicle information, or other related information. The applicant will treat the user's personal information and its processing with a high degree of diligence.

[0155] The applicant attaches great importance to the security of users' personal information and has taken reasonable and feasible security protection measures that comply with industry standards to protect users' information and prevent unauthorized access, disclosure, use, modification, damage or loss of personal information.

Claims

1. A method for generating simulated images, characterized in that, include: A 3D scene is constructed based on pre-acquired real-world images and map data; Integrate the foreground material into the 3D scene; The 3D scene, after incorporating the foreground material, is rendered to generate a simulated image for target detection.

2. The simulation image generation method as described in claim 1, characterized in that, The construction of a 3D scene based on pre-acquired real-world images and map data includes: Acquire the real-scene image and determine the display range of the real-scene image; An initial 3D scene corresponding to the display range is generated based on the map data; Align the real-world image with the initial 3D scene to obtain the 3D scene.

3. The simulation image generation method as described in claim 2, characterized in that, The step of aligning the real-world image with the initial 3D scene to obtain the 3D scene includes: Obtain the first coordinates of the preset anchor point on the real-scene image; Determine the second coordinates of the preset anchor point in the initial 3D scene; Based on the first coordinate and the second coordinate, the real-world image and the initial 3D scene are aligned to obtain the 3D scene.

4. The simulation image generation method according to any one of claims 1 to 3, characterized in that, The process of integrating foreground materials into the 3D scene includes: Import the foreground material into the 3D twin engine; Adjust the coordinate system of the foreground material imported into the 3D twin engine so that the coordinate system of the foreground material is consistent with the world coordinate system of the 3D twin engine; The foreground material is positioned at a preset location in the three-dimensional scene to obtain the three-dimensional scene integrated with the foreground material.

5. The simulation image generation method as described in claim 4, characterized in that, The process of integrating foreground materials into the 3D scene further includes: Behavioral parameters are set for the foreground material to generate dynamic foreground material; wherein, the behavioral parameters include any combination of the following: position, speed, trajectory, rotation parameters, interaction rules, and physical properties; The step of rendering the 3D scene after incorporating the foreground material to generate a simulated image for object detection includes: Update the state of the dynamic foreground materials in the 3D scene; The objects to be rendered in the three-dimensional scene are subjected to geometric transformations and lighting and shadow processing to generate the simulation image; wherein, the objects include the dynamic foreground material.

6. The simulation image generation method as described in claim 4, characterized in that, The foreground materials include: obstacle materials and non-obstacle materials; Generating the simulated image includes: Based on the obstacle material and the target to be detected, a simulated image of partial occlusion is generated; Alternatively, based on the non-obstacle material and the target to be detected, a simulated image containing multiple targets is generated; wherein the multiple targets include the target to be detected and the non-obstacle material; Alternatively, based on the obstacle material, the non-obstacle material, and the target to be detected, a simulated image containing multiple targets with partial occlusion can be generated.

7. The simulation image generation method as described in claim 4, characterized in that, The foreground material includes at least one of the following: 3D model foreground material, network image foreground material, and cutout transplanted foreground material.

8. A simulated image generation device, characterized in that, include: The building module is used to construct 3D scenes based on pre-acquired real-world images and map data; The fusion module is used to integrate foreground materials into the 3D scene. The rendering module is used to render the 3D scene after incorporating the foreground material, and generate a simulated image for target detection.

9. A computer device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method as claimed in any one of claims 1 to 7.

10. A computer program product, characterized in that, When the computer program product is run on a computer device, it causes the computer device to perform the method as described in any one of claims 1 to 7.

Images