Synthetic image generation method, BEV model training method and target perception method

Abstract

The present application relates to the field of image processing technology, and specifically provides a synthetic image generation method, a BEV model training method and a target perception method, aiming to solve the existing problem that it is impossible to conveniently and effectively obtain images under different camera parameters. To this end, the synthetic image generation method of the present application includes: based on a pre-constructed virtual three-dimensional scene, randomly generating multiple virtual cameras, each of the multiple virtual cameras corresponds to a set of camera parameters; randomly combining multiple virtual cameras to obtain multiple virtual camera groups; using multiple virtual camera groups to respectively obtain two-dimensional images of the virtual three-dimensional scene to obtain multiple synthetic image data sets, each of the multiple synthetic image data sets includes multiple synthetic images, and the synthetic images are annotated with the camera parameters of the corresponding virtual camera. This method can effectively and conveniently obtain a large number of synthetic images under different camera parameters, can effectively simulate the camera deployment conditions in more scenarios, and is conducive to obtaining a generalized synthetic image data set.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and specifically provides a synthetic image generation method, a BEV model training method and a target perception method. Background Art

[0002] In the field of autonomous driving, the BEV (Bird's Eye View) model can be used to perceive targets in driving scenes. However, the effect of the BEV model is easily affected by the camera parameters of the on-board camera. When the internal and external parameters of the camera of a new model are significantly different from those of the old model, the effect of the BEV model will also vary greatly, and the BEV model on the old model cannot be directly migrated to the new model.

[0003] In order to improve the migration capability of the BEV model across vehicle models, it is necessary to provide a large number of images with different camera parameters for different vehicle models as training samples. In this regard, one situation in the related technology is to collect multi-view images for different vehicle models through a real data acquisition vehicle and configure it with a camera with specific camera parameters, and manually annotate the collected multi-view images. However, this method consumes a lot of manpower and material resources, and the time cost is also very high, and it cannot be scaled. In another case, a large number of images with different camera parameters are obtained through geometric transformation or generative methods, but the geometric transformation method cannot handle the problem of large-angle camera parameter changes, and the generative method cannot guarantee the temporal consistency of multi-view data. Summary of the invention

[0004] The present application aims to solve the above technical problem, that is, to solve the problem that it is currently impossible to conveniently and effectively obtain images under different camera parameters.

[0005] In a first aspect, the present application provides a method for generating a composite image, comprising:

[0006] Based on a pre-constructed virtual three-dimensional scene, a plurality of virtual cameras are randomly generated, each of the plurality of virtual cameras corresponding to a set of camera parameters;

[0007] Randomly combining the multiple virtual cameras to obtain multiple virtual camera groups;

[0008] The two-dimensional images of the virtual three-dimensional scene are respectively acquired by using the multiple virtual camera groups to obtain multiple synthetic image data sets, each of which includes multiple synthetic images, and the synthetic images are annotated with camera parameters of the corresponding virtual cameras.

[0009] In some embodiments, randomly generating multiple virtual cameras based on a pre-built virtual three-dimensional scene includes:

[0010] Based on the pre-constructed virtual three-dimensional scene, randomly generate a plurality of camera extrinsic parameters and a plurality of camera intrinsic parameters;

[0011] The multiple camera extrinsic parameters and the multiple camera intrinsic parameters are randomly combined in pairs to obtain multiple groups of camera parameters, and the multiple virtual cameras are generated based on the multiple groups of camera parameters.

[0012] In some embodiments, after randomly generating a plurality of camera extrinsics based on the pre-constructed virtual three-dimensional scene, the method further includes:

[0013] Determine a target camera extrinsic parameter that meets a first preset condition from the multiple camera extrinsic parameters, so as to obtain the multiple groups of camera parameters based on the target camera extrinsic parameter and the multiple camera intrinsic parameters;

[0014] The first preset condition includes at least one of the following: the camera setting position corresponding to the camera extrinsic parameter is within a target angle interval of a preset orientation, and the camera setting position corresponding to the camera extrinsic parameter is within a preset height interval.

[0015] In some embodiments, after randomly combining the multiple virtual cameras to obtain multiple virtual camera groups, the method further includes:

[0016] Filtering out a virtual camera group satisfying a second preset condition from the multiple virtual camera groups, and using the filtered virtual camera group to acquire a two-dimensional image of the virtual three-dimensional scene to obtain the multiple synthetic image data sets;

[0017] The second preset condition includes at least one of the following: the field of view of the virtual camera group satisfies the preset field of view, and the overlap angle of the field of view angles of any two virtual cameras in the virtual camera group satisfies the preset overlap range.

[0018] In a second aspect, the present application provides a BEV model training method, which comprises:

[0019] Acquire multiple synthetic image data sets, each of which includes multiple synthetic images; each of the multiple synthetic images is pre-labeled with a true value;

[0020] Training the BEV model based on the multiple synthetic image data sets to obtain a trained BEV model;

[0021] The multiple synthetic image data sets are obtained by using any of the synthetic image generation methods described above, and at least some of the multiple synthetic image data sets correspond to different virtual camera groups.

[0022] In some embodiments, the training of the BEV model based on the multiple synthetic image data sets to obtain a trained BEV model includes:

[0023] Inputting the plurality of synthetic image data sets into the BEV model;

[0024] For each synthetic image data set, the BEV model is used to transform the multiple synthetic images to a preset plane based on the camera parameters of the multiple synthetic images, and feature extraction and feature fusion are performed on the converted multiple synthetic images to obtain a BEV feature map, and a perception result is obtained based on the BEV feature map;

[0025] The BEV model is iteratively trained based on the perception results and the annotated true values ​​to obtain a trained BEV model.

[0026] In a third aspect, the present application provides a target perception method, which includes:

[0027] Acquire multiple images of a target scene, wherein the multiple images are images of the target scene at different viewing angles;

[0028] The multiple images are input into a trained BEV model to obtain a target perception result; wherein the trained BEV model is obtained based on any of the BEV model training methods described above.

[0029] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any of the methods described above is implemented.

[0030] In a fifth aspect, the present application provides a smart device, comprising:

[0031] at least one processor;

[0032] and, a memory communicatively coupled to the at least one processor;

[0033] The memory stores a computer program, and when the computer program is executed by the at least one processor, any of the above-mentioned methods is implemented.

[0034] In the case of adopting the above technical solution, the present application can randomly generate multiple virtual cameras based on a pre-constructed virtual three-dimensional scene, and each of the multiple virtual cameras corresponds to a set of camera parameters; randomly combine the multiple virtual cameras to obtain multiple virtual camera groups; use the multiple virtual camera groups to respectively obtain two-dimensional images of the virtual three-dimensional scene to obtain multiple synthetic image data sets, and the multiple synthetic image data sets each include multiple synthetic images, and the synthetic images are annotated with the camera parameters of the corresponding virtual cameras. This method can effectively and conveniently obtain a large number of synthetic images under different camera parameters by randomly generating multiple virtual cameras. By obtaining multiple virtual camera groups based on the random combination of multiple virtual cameras, the camera deployment conditions in more scenes can be effectively simulated, which is conducive to obtaining a generalized synthetic image data set, and can save costs and avoid the problem of inconsistent timing of multi-view images.

[0035] In addition, training the BEV model based on multiple synthetic image data sets obtained by the synthetic image generation method provided in this application is conducive to improving the migration capability of the BEV model across vehicle models and improving the effect of the BEV model. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The preferred embodiments of the present application are described below in conjunction with the accompanying drawings, in which:

[0037] Figure 1 It is a schematic diagram of a synthetic image generation method provided in an embodiment of the present application;

[0038] Figure 2 is a schematic flow chart of a synthetic image generation method provided by another embodiment of the present application;

[0039] Figure 3 is a flow chart of a synthetic image generation method provided by another embodiment of the present application;

[0040] Figure 4 This is a flow chart of a BEV model training method provided by an embodiment of the present application;

[0041] Figure 5 It is a flowchart of a target perception method provided by an embodiment of the present application;

[0042] Figure 6 It is a schematic diagram of the structure of the smart device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0043] Some embodiments of the present application are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present application and are not intended to limit the protection scope of the present application.

[0044] See also Figure 1 As shown, Figure 1 : is a schematic flow chart of a synthetic image generation method provided in an embodiment of the present application, which may include:

[0045] Step S11: Based on the pre-constructed virtual three-dimensional scene, a plurality of virtual cameras are randomly generated, and each of the plurality of virtual cameras corresponds to a set of camera parameters;

[0046] Step S12: randomly combining multiple virtual cameras to obtain multiple virtual camera groups;

[0047] Step S13: using a plurality of virtual camera groups to respectively acquire two-dimensional images of the virtual three-dimensional scene, to obtain a plurality of synthetic image data sets, each of the plurality of synthetic image data sets comprising a plurality of synthetic images, and the synthetic images are annotated with camera parameters of the corresponding virtual cameras.

[0048] In some embodiments, step S11 may specifically include:

[0049] Based on the pre-built virtual 3D scene, multiple camera extrinsics and multiple camera intrinsics are randomly generated;

[0050] Multiple camera extrinsic parameters and multiple camera intrinsic parameters are randomly combined in pairs to obtain multiple sets of camera parameters, and multiple virtual cameras are generated based on the multiple sets of camera parameters.

[0051] In some embodiments, a required virtual three-dimensional scene may be constructed based on actual needs. For example, a virtual three-dimensional scene of a driving scene may be constructed.

[0052] In the present application, a conventional random algorithm in the art can be used to randomly generate multiple camera extrinsic parameters and multiple camera intrinsic parameters. Camera extrinsic parameters refer to parameters used to describe the position and posture of a camera in a world coordinate system. In some embodiments, the camera extrinsic parameters may include at least one of a rotation matrix and a translation vector. The rotation matrix represents the rotation of the camera in space, and the translation vector represents the position of the camera in the world coordinate system. The camera extrinsic parameters can be used to determine the angle and position from which to observe a three-dimensional scene. Camera intrinsic parameters refer to parameters used to represent the internal mapping relationship of a camera. In some embodiments, the camera intrinsic parameters may include at least one of a focal length, a distortion coefficient, and a pixel aspect ratio. The camera intrinsic parameters can be used to describe how a camera maps points in a three-dimensional world to a two-dimensional image plane, and optical properties such as distortion are taken into account.

[0053] In an embodiment of the present application, at least some of the multiple virtual cameras have different camera parameters, so as to facilitate acquisition of images under a variety of different camera parameters and obtain information of the virtual three-dimensional scene under multiple perspectives based on the images.

[0054] In some embodiments, in step S12, multiple virtual cameras are randomly combined to obtain multiple virtual camera groups. The multiple virtual camera groups may include different numbers of virtual cameras or the same number of virtual cameras.

[0055] In some embodiments, step S13 may be specifically as follows: for each virtual camera group in the multiple virtual camera groups, using a rendering engine, render a virtual three-dimensional scene according to the camera parameters of each virtual camera in the current virtual camera group, and obtain a two-dimensional image corresponding to each virtual camera, that is, a synthetic image. Each virtual camera group may obtain at least one synthetic image data set, each synthetic image data set may include multiple synthetic images, and the synthetic image may be annotated with the camera parameters of the corresponding virtual camera, and the corresponding virtual camera is the virtual camera used to obtain the current synthetic image.

[0056] The rendering engine can simulate real visual effects by calculating the propagation and interaction of light in a virtual 3D scene. After setting the camera parameters, the rendering engine will calculate the color and brightness of each pixel based on these parameters to obtain a synthetic image.

[0057] The above is a synthetic image generation method provided in an embodiment of the present application. By randomly generating multiple virtual cameras, the method can effectively and conveniently obtain a large number of synthetic images under different camera parameters. By obtaining multiple virtual camera groups based on random combinations of multiple virtual cameras, the camera deployment conditions in more scenarios can be effectively simulated, which is conducive to obtaining a generalized synthetic image data set, and can save costs and avoid the problem of inconsistent timing of multi-view images.

[0058] In some embodiments, after step S11, the randomly generated multiple virtual cameras may be filtered according to the scene requirements to select virtual cameras that meet the scene requirements. For example, when the scene requirement is to set multiple vehicle-mounted virtual cameras, virtual cameras whose heights corresponding to the setting positions do not meet the vehicle height range may be filtered out from the multiple virtual cameras based on the camera extrinsics, or virtual cameras whose setting directions do not meet the preset directions may be filtered out from the multiple virtual cameras based on the camera extrinsics.

[0059] In other embodiments, when multiple camera extrinsic parameters and multiple camera intrinsic parameters are randomly generated and a virtual camera is obtained based on the multiple camera extrinsic parameters and the multiple camera intrinsic parameters, in order to meet scene requirements and save processes, the randomly generated camera extrinsic parameters can also be filtered after the multiple camera extrinsic parameters are randomly generated to effectively obtain a virtual camera that meets the scene requirements. Please refer to the description of the following embodiments for details.

[0060] See also Figure 2 As shown, Figure 2 FIG. 1 is a flow chart of a method for generating a composite image provided by another embodiment of the present application, which may include:

[0061] Step S21: randomly generating a plurality of camera extrinsic parameters and a plurality of camera intrinsic parameters based on a pre-constructed virtual three-dimensional scene;

[0062] Step S22: determining a target camera extrinsic parameter that meets a first preset condition from a plurality of camera extrinsic parameters;

[0063] Step S23: randomly combining the target camera external parameters and the multiple camera internal parameters in pairs to obtain multiple sets of camera parameters, and generating multiple virtual cameras based on the multiple sets of camera parameters;

[0064] Step S24: randomly combining multiple virtual cameras to obtain multiple virtual camera groups;

[0065] Step S25: using multiple virtual camera groups to respectively acquire two-dimensional images of the virtual three-dimensional scene, to obtain multiple synthetic image data sets, each of the multiple synthetic image data sets including multiple synthetic images, and the synthetic images are annotated with camera parameters of the corresponding virtual cameras.

[0066] Among them, steps S24 and S25 can be implemented in the same manner as steps S12 and S13, and will not be described here for the sake of brevity. For details, please refer to the above description.

[0067] In some embodiments, step S21 may specifically be to use a conventional random algorithm in the art to randomly generate multiple camera extrinsic parameters and multiple camera intrinsic parameters. The camera extrinsic parameters may include at least one of a rotation matrix and a translation vector, and the camera intrinsic parameters may include at least one of a focal length, a distortion coefficient, and a pixel aspect ratio.

[0068] It should be noted that step S22 may be performed after a plurality of camera extrinsic parameters are randomly generated in step S21. Here, it is exemplarily shown that step S22 is performed after a plurality of camera extrinsic parameters and a plurality of camera intrinsic parameters are randomly generated in step S21.

[0069] The first preset condition can be flexibly set based on scene requirements. In some embodiments, the first preset condition includes at least one of the following: the camera setting position corresponding to the camera extrinsic parameter is in a target angle interval of a preset orientation, and the camera setting position corresponding to the camera extrinsic parameter is in a preset height interval.

[0070] In some embodiments, when applied to a vehicle-mounted scene, the preset directions may include the front, rear, left and right sides of the vehicle, etc., and the target angle interval may be flexibly set based on demand, for example, it may be an angle interval of 10° to the corresponding direction. The preset height interval may be an interval corresponding to the height of the bottom of the vehicle from the ground as the minimum threshold and the height of the top of the vehicle from the ground as the maximum threshold. In other embodiments, the first preset condition may also include virtual cameras located at two relative directions of the vehicle, and the height difference satisfies a preset range. If the heights of the two virtual cameras in relative directions need to be basically the same, the preset range may be set to be greater than or equal to zero and less than the maximum allowable height difference.

[0071] In an embodiment of the present application, by determining the target camera extrinsic parameters that meet preset conditions from multiple camera extrinsic parameters, and then randomly generating a virtual camera based on the determined target camera extrinsic parameters, the process can be saved and a virtual camera that meets the scene requirements can be effectively obtained, which is facilitating the subsequent acquisition of a synthetic image that meets the scene requirements.

[0072] In some embodiments, there may be multiple target camera extrinsic parameters, and step S23 may specifically be to randomly combine multiple target camera extrinsic parameters and multiple camera intrinsic parameters in pairs to obtain multiple sets of camera parameters, and generate multiple virtual cameras based on the multiple sets of camera parameters.

[0073] The above is a synthetic image generation method provided by another embodiment of the present application, which can achieve the same Figure 1 The same beneficial effects as the corresponding embodiments are achieved, and by filtering the randomly generated camera extrinsic parameters, the process can be saved and a virtual camera that meets the scene requirements can be effectively obtained, which is conducive to obtaining a synthetic image that meets the scene requirements.

[0074] In some embodiments, in order to obtain a synthetic image that meets the field of view angle requirement, the obtained multiple virtual camera groups may be filtered to select a virtual camera group that meets the requirement. For details, please refer to the description of the following embodiments.

[0075] It should be noted that this embodiment can be based on the above Figure 1 or Figure 2 The corresponding embodiment is implemented here based on Figure 1 The corresponding embodiment is described by taking the implementation as an example. Figure 3 As shown, Figure 3 FIG. 1 is a flow chart of a synthetic image generation method provided by another embodiment of the present application, which may include:

[0076] Step S31: Based on the pre-constructed virtual three-dimensional scene, a plurality of virtual cameras are randomly generated, and each of the plurality of virtual cameras corresponds to a set of camera parameters;

[0077] Step S32: randomly combining multiple virtual cameras to obtain multiple virtual camera groups;

[0078] Step S33: selecting a virtual camera group that meets a second preset condition from the multiple virtual camera groups;

[0079] Step S34: using the screened virtual camera group to obtain a two-dimensional image of the virtual three-dimensional scene, and obtaining a plurality of synthetic image data sets, each of which includes a plurality of synthetic images, and the synthetic images are annotated with camera parameters of the corresponding virtual camera.

[0080] Among them, steps S31 and S32 can be implemented in the same manner as S11 and S12 respectively, and step S34 can be implemented in the same manner as S13 after replacing the multiple virtual camera groups in S13 with the screened virtual camera groups.

[0081] In an embodiment of the present application, the second preset condition can be flexibly set based on the actual field of view requirement. In some embodiments, the second preset condition includes at least one of the following: the field of view range of the virtual camera group meets the preset field of view range, and the field of view angle overlap angle of any two virtual cameras in the virtual camera group meets the preset overlap range.

[0082] In some embodiments, the preset field of view range can be greater than or equal to 360°. When the field of view range of the virtual camera group meets the preset field of view range, the total field of view angle of multiple virtual cameras in the current virtual camera group can at least cover the surrounding 360° area, which is conducive to the subsequent acquisition of a surround-view synthetic image of the virtual three-dimensional scene.

[0083] In some embodiments, the preset overlap range can be greater than zero degrees and less than or equal to a set value, where the set value can be set as required, for example, it can be 30°. By obtaining a virtual camera group that satisfies the overlap angle of the field of view of the two virtual cameras greater than zero degrees, the overlapping part of the field of view angle in the two adjacent synthetic images subsequently obtained is the key area for image stitching and fusion. By ensuring a certain overlap angle, the two images can be smoothly transitioned when stitched, reducing stitching gaps and misalignment. In addition, multi-angle observation of the same area can also be achieved, which is conducive to obtaining more complete information about the overlapping area. In some embodiments, in order to avoid too much redundant information, a relatively small set value can be set, such as a set value that can be less than or equal to 60°.

[0084] The above is a method for generating a composite image provided by another embodiment of the present application, which can achieve the same Figure 1The same beneficial effects as the corresponding embodiments are achieved. At the same time, by screening out a virtual camera group that meets the second preset condition from multiple virtual camera groups, and using the screened virtual camera group to obtain a two-dimensional image of the virtual three-dimensional scene, it is beneficial to obtain a synthetic image that meets the field of view angle requirements.

[0085] Another aspect of the present application provides a BEV model training method, see Figure 4 As shown, it may include:

[0086] Step S41: acquiring a plurality of synthetic image data sets, each of the plurality of synthetic image data sets comprising a plurality of synthetic images; the plurality of synthetic images are all pre-labeled with true values;

[0087] Step S42: training the BEV model based on multiple synthetic image data sets to obtain a trained BEV model;

[0088] Wherein, a plurality of synthetic image data sets are obtained by using the synthetic image generation method provided by any of the above embodiments, and at least part of the plurality of synthetic image data sets correspond to different virtual camera groups.

[0089] By adopting the synthetic image generation method provided in the embodiment of the present application to obtain multiple synthetic image data sets and use them for training the BEV model, it is possible to effectively simulate the camera deployment conditions in more scenarios and obtain a generalized synthetic image data set, which is beneficial to obtaining a BEV model with cross-camera internal and external parameters, effectively improving the training effect of the BEV model and its migration capability across vehicle models.

[0090] The BEV (Bird's Eye View) model can convert images from different perspectives into images from a unified bird's eye view and use them for prediction tasks. The true value of the synthetic image can be pre-labeled based on the type of prediction task. In some embodiments, when the prediction task is target recognition, the type, position, bounding box and other information of the target object in the synthetic image can be pre-labeled; when the prediction task is classification, the type of each object in the synthetic image can be pre-labeled.

[0091] The perception result is the result corresponding to the model prediction task. For example, when the prediction task is target recognition, the perception result can be the predicted bounding box corresponding to the target object.

[0092] In some embodiments, step S42 may specifically include:

[0093] Input multiple synthetic image datasets into the BEV model;

[0094] For each synthetic image data set, the BEV model is used to transform multiple synthetic images to a preset plane based on their respective camera parameters, and feature extraction and feature fusion are performed on the converted multiple synthetic images to obtain a BEV feature map, and the perception result is obtained based on the BEV feature map;

[0095] The BEV model is iteratively trained based on the perception results and the annotated true values ​​to obtain a trained BEV model.

[0096] The preset plane may be a BEV plane.

[0097] Another aspect of the present application also provides a target perception method, see Figure 5 As shown, Figure 5 It is a flowchart of a target perception method provided by an embodiment of the present application, which may include:

[0098] Step S51: acquiring multiple images of a target scene, where the multiple images are images of the target scene at different viewing angles;

[0099] Step S52: input multiple images into the trained BEV model to obtain target perception results;

[0100] Among them, the trained BEV model is obtained based on the BEV model training method described in any of the above embodiments.

[0101] The method adopts the BEV model training method based on any of the above embodiments to obtain a trained BEV model for target perception, which is beneficial to improving the accuracy of perception.

[0102] In some embodiments, the target perception method provided in the embodiments of the present application can be applied to the detection and identification of target objects in a target scene, and the corresponding target perception results may include the position, category, semantic information of the target object, etc.

[0103] It is understood by those skilled in the art that all or part of the processes in the method for implementing the above-mentioned embodiments of the present application can also be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device, medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory, random access memory, electric carrier signal, telecommunication signal and software distribution medium, etc. that can carry the computer program code.

[0104] Another aspect of the present application further provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the method described in any of the above embodiments can be implemented. The computer-readable storage medium can be a storage device formed by various electronic devices. Optionally, the computer-readable storage medium in the embodiment of the present application is a non-temporary computer-readable storage medium.

[0105] In another aspect of the present application, a smart device is provided.

[0106] In an embodiment of an intelligent device according to the present application, the intelligent device may include at least one processor; and a memory connected to the at least one processor in communication; wherein a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the method described in any of the above embodiments is implemented. The intelligent device described in the present application may include a driving device, a smart car, a robot, and the like. Figure 6 , Figure 6 FIG. 6 exemplarily shows that the memory 61 and the processor 62 are communicatively connected via a bus.

[0107] In some embodiments of the present application, the smart device may further include at least one sensor for sensing information. The sensor is communicatively connected to any type of processor mentioned in the present application. Optionally, the smart device may further include an autonomous driving system for guiding the smart device to drive itself or assist in driving. The processor communicates with the sensor and / or the autonomous driving system to complete the method described in any of the above embodiments.

[0108] The relevant user personal information that may be involved in the various embodiments of this application is strictly in accordance with the requirements of laws and regulations, following the principles of legality, legitimacy and necessity, based on the reasonable purposes of business scenarios, to process the personal information that users actively provide during the use of products / services or generated due to the use of products / services, as well as the personal information obtained with the user's authorization.

[0109] The user personal information processed by this application will vary depending on the specific product / service scenario, and shall be based on the specific scenario in which the user uses the product / service, and may involve the user's account information, device information, driving information, vehicle information or other related information. This application will treat the user's personal information and its processing with a high degree of diligence.

[0110] This application attaches great importance to the security of user personal information and has taken reasonable and feasible security protection measures that meet industry standards to protect user information and prevent personal information from being accessed, disclosed, used, modified, damaged or lost without authorization.

[0111] So far, the technical solutions of the present application have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present application is obviously not limited to these specific embodiments. Without departing from the principles of the present application, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present application.

Claims

1. A method for generating a synthetic image, characterized in that: include: Based on a pre-constructed virtual three-dimensional scene, a plurality of virtual cameras are randomly generated, each of the plurality of virtual cameras corresponding to a set of camera parameters; Randomly combining the multiple virtual cameras to obtain multiple virtual camera groups; Filtering out a virtual camera group that meets a second preset condition from the multiple virtual camera groups; wherein the second preset condition includes at least one of the following: the field of view range of the virtual camera group meets a preset field of view range, and the overlap angle of the field of view angles of any two virtual cameras in the virtual camera group meets a preset overlap range; The two-dimensional images of the virtual three-dimensional scene are respectively acquired by using the screened virtual camera groups to obtain a plurality of synthetic image data sets, each of which includes a plurality of synthetic images, and the synthetic images are annotated with camera parameters of the corresponding virtual cameras.

2. The method according to claim 1, characterized in that The randomly generating multiple virtual cameras based on the pre-built virtual three-dimensional scene includes: Based on the pre-constructed virtual three-dimensional scene, randomly generate a plurality of camera extrinsic parameters and a plurality of camera intrinsic parameters; The multiple camera extrinsic parameters and the multiple camera intrinsic parameters are randomly combined in pairs to obtain multiple groups of camera parameters, and the multiple virtual cameras are generated based on the multiple groups of camera parameters.

3. The method according to claim 2, characterized in that After randomly generating a plurality of camera extrinsics based on the pre-constructed virtual three-dimensional scene, the method further includes: Determine a target camera extrinsic parameter that meets a first preset condition from the multiple camera extrinsic parameters, so as to obtain the multiple groups of camera parameters based on the target camera extrinsic parameter and the multiple camera intrinsic parameters; The first preset condition includes at least one of the following: the camera setting position corresponding to the camera extrinsic parameter is within a target angle interval of a preset orientation, and the camera setting position corresponding to the camera extrinsic parameter is within a preset height interval.

4. A BEV model training method, characterized in that: include: Acquire multiple synthetic image data sets, each of which includes multiple synthetic images; each of the multiple synthetic images is pre-labeled with a true value; Training the BEV model based on the multiple synthetic image data sets to obtain a trained BEV model; The multiple synthetic image data sets are obtained by using the synthetic image generation method described in any one of claims 1 to 3, and at least some of the multiple synthetic image data sets correspond to different virtual camera groups.

5. The method according to claim 4, characterized in that The step of training the BEV model based on the multiple synthetic image data sets to obtain a trained BEV model includes: Inputting the plurality of synthetic image data sets into the BEV model; For each synthetic image data set, the BEV model is used to transform the multiple synthetic images to a preset plane based on the respective camera parameters of the multiple synthetic images, and feature extraction and feature fusion are performed on the converted multiple synthetic images to obtain a BEV feature map, and a perception result is obtained based on the BEV feature map; The BEV model is iteratively trained based on the perception results and the annotated true values ​​to obtain a trained BEV model.

6. A target perception method, characterized in that: include: Acquire multiple images of a target scene, wherein the multiple images are images of the target scene at different viewing angles; The multiple images are input into a trained BEV model to obtain a target perception result; wherein the trained BEV model is obtained based on the BEV model training method described in any one of claims 4 or 5.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

8. A smart device, characterized in that: include: at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores a computer program, and when the computer program is executed by the at least one processor, the method according to any one of claims 1 to 6 is implemented.

Images