Image generation method and device, storage medium and electronic device

Abstract

The application discloses an image generation method and device, a storage medium and an electronic device, and relates to the technical field of image generation. The method comprises the following steps: acquiring a first background image corresponding to a target road region in a pixel coordinate system, wherein the first background image does not contain lane lines; converting the first background image into a second background image in a world coordinate system; generating a first lane line in the second background image, and converting the first lane line in the world coordinate system into a second lane line in the pixel coordinate system; and fusing the first background image and the second lane line to obtain a target image. The technical scheme has the advantages that the problem that the generated image lacks authenticity caused by generating an image with lane lines through simulation software is solved.

Description

Technical Field

[0001] This invention relates to the field of communications, and more specifically, to an image generation method and apparatus, a storage medium, and an electronic device. Background Technology

[0002] Algorithms targeting intersections or roadsides typically rely heavily on visual image data. Whether for training or testing, simply collecting data through cameras is inefficient and may not capture specific scene samples. While simulation software can simulate traffic scenes to simulate some data, it has limitations such as high rendering capabilities required for the simulation environment, making it difficult to simulate realistic image data. Due to the complexity and variability of real-world environments, generating complete scene images entirely through algorithms is even more challenging.

[0003] Regarding the issue of generating images with lane lines using simulation software, which results in images lacking realism, no effective solution has yet been proposed.

[0004] Therefore, it is necessary to improve the relevant technology to overcome the aforementioned defects. Summary of the Invention

[0005] This invention provides an image generation method and apparatus, storage medium and electronic device to at least solve the problem that images with lane lines generated by simulation software lack realism.

[0006] According to one aspect of the present invention, an image generation method is provided, comprising: acquiring a first background image corresponding to a target road region in a pixel coordinate system, wherein the first background image does not contain lane lines; converting the first background image into a second background image in a world coordinate system; generating a first lane line in the second background image, and converting the first lane line in the world coordinate system into a second lane line in the pixel coordinate system; and fusing the first background image and the second lane line to obtain a target image.

[0007] In an exemplary embodiment, obtaining a first background image corresponding to a target road region in a pixel coordinate system includes: acquiring an image of the target road region to obtain an original image of the target road region; and removing lane lines from the original image to obtain the first background image.

[0008] In an exemplary embodiment, removing lane lines from the original image to obtain the first background image includes: determining a first set of pixels corresponding to the lane lines in the original image; determining a first pixel value corresponding to each pixel in the first set of pixels using an image inpainting algorithm; wherein the first pixel value is the pixel value corresponding to the road surface in the original image; and setting the value of each pixel in the first set of pixels to the corresponding first pixel value to obtain the first background image.

[0009] In an exemplary embodiment, converting the first background image into a second background image in the world coordinate system includes: determining a homography matrix, wherein the homography matrix is ​​used to indicate the transformation relationship of each pixel point of the first background image from the pixel coordinate system to the world coordinate system; and converting the first background image in the pixel coordinate system into the second background image in the world coordinate system through the homography matrix.

[0010] In an exemplary embodiment, determining the homography matrix includes: self-calibrating the first background image to obtain the homography matrix, wherein, when the first background image is an image obtained by removing lane lines from the original image, self-calibrating the first background image includes: self-calibrating the first background image with reference to camera parameters; the camera parameters are parameters determined by first data and second data, the first data being data corresponding to target information in the original image, and the second data being data corresponding to the target information in a standard database; the target information includes at least one of the following: lane lines, lane width, and road surface design.

[0011] In an exemplary embodiment, generating a first lane line in the second background image includes: generating the first lane line in the second background image based on at least one reference information from lane line standards and road design specifications; wherein the road design specifications are used to indicate the design specifications of different lane lines under different road scenarios.

[0012] In an exemplary embodiment, generating the first lane line in the second background image based on at least one reference information from lane line standards and road design specifications includes: determining a second set of pixels and a second pixel value corresponding to each pixel in the second background image based on lane line standards; or determining a second set of pixels and a second pixel value corresponding to each pixel in the second pixel set based on lane line standards and road design specifications; wherein, if the value of each pixel in the second set of pixels in the second background image is the corresponding second pixel value, the second background image contains the first lane line.

[0013] In an exemplary embodiment, converting the first lane line in the world coordinate system into a second lane line in the pixel coordinate system includes: performing an inverse transformation on the first pixel matrix corresponding to the first lane line using the homography matrix to obtain a second pixel matrix corresponding to the second lane line; wherein, the first pixel matrix is ​​used to indicate the second set of second pixel points corresponding to the first lane line in the second background image and the second pixel value of each pixel point in the second pixel point set; the second pixel matrix is ​​used to indicate the third set of third pixel points corresponding to the second lane line in the first background image and the third pixel value of each pixel point in the third pixel point set.

[0014] In an exemplary embodiment, fusing the first background image and the second lane line to obtain a target image includes: obtaining a second pixel matrix corresponding to the second lane line, wherein the second pixel matrix is ​​used to indicate the third pixel point set corresponding to the second lane line in the first background image and the third pixel value of each pixel point in the third pixel point set; and replacing the pixel value of each pixel point in the third pixel point set in the first background image with the corresponding third pixel value.

[0015] According to another aspect of the present invention, an image generation apparatus is also provided, comprising: an acquisition module, configured to acquire a first background image corresponding to a target road region in a pixel coordinate system, wherein the first background image does not contain lane lines; a first processing module, configured to convert the first background image into a second background image in a world coordinate system; a second processing module, configured to generate a first lane line in the second background image and convert the first lane line in the world coordinate system into a second lane line in the pixel coordinate system; and a fusion module, configured to fuse the first background image and the second lane line to obtain a target image.

[0016] According to another aspect of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, wherein the computer program is configured to execute the above-described image generation method at runtime.

[0017] According to another aspect of the present invention, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the image generation method described above through the computer program.

[0018] This invention obtains a first background image corresponding to a target road area in a pixel coordinate system, converts the first background image into a second background image in a world coordinate system, generates a first lane line in the second background image, converts the first lane line into a second lane line in a pixel coordinate system, and merges the first background image and the second lane line to obtain the target image. Since lane line generation for a traffic scene is performed using existing images without simulation software, the generated image is more realistic, solving the problem of unrealistic images generated by simulation software. Furthermore, this method can freely simulate any traffic line layout, increasing the diversity of images under the same background. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with the description thereof, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0020] Figure 1 This is a hardware structure block diagram of a computer terminal for the image generation method according to an embodiment of the present invention;

[0021] Figure 2 This is a flowchart of an image generation method according to an embodiment of the present invention;

[0022] Figure 3 This is a schematic diagram of the original image according to an embodiment of the present invention;

[0023] Figure 4 This is a schematic diagram of the scene where the original image is located according to an embodiment of the present invention;

[0024] Figure 5 This is a schematic diagram of a first background image according to an embodiment of the present invention;

[0025] Figure 6 This is a schematic diagram of a second background image according to an embodiment of the present invention;

[0026] Figure 7 This is a schematic diagram of a reference image according to an embodiment of the present invention;

[0027] Figure 8 This is a schematic diagram of a target image according to an embodiment of the present invention;

[0028] Figure 9 This is a flowchart of another image generation method according to an embodiment of the present invention;

[0029] Figure 10 This is a structural block diagram of an image generation apparatus according to an embodiment of the present invention. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] The methods and embodiments provided in this application can be executed on a computer terminal or similar computing device. Taking running on a computer terminal as an example, Figure 1 This is a hardware structure block diagram of a computer terminal for the image generation method according to an embodiment of the present invention. For example... Figure 1 As shown, a computer terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor unit (MPU) or a programmable logic device (PLD)) and a memory 104 for storing data are also shown. In one exemplary embodiment, the computer terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the computer terminal described above. For example, the computer terminal may also include components that are more complex than those described above. Figure 1 The more or fewer components shown, or having the same Figure 1 Equivalent functions or ratios shown Figure 1 The functions shown have more different configurations.

[0033] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the image generation method in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to a computer terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0034] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider for the computer terminal. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0035] Specifically, this embodiment provides an image generation method. Figure 2 This is a flowchart of an image generation method according to an embodiment of the present invention, the process including the following steps:

[0036] Step S202: Obtain the first background image corresponding to the target road area in the pixel coordinate system, wherein the first background image does not contain lane lines;

[0037] It should be noted that the pixel coordinate system is a coordinate system corresponding to the camera's shooting angle of the first background image.

[0038] As an alternative example, the first background image can be an image captured by a camera when there are no lane markings in the target road area.

[0039] In an exemplary embodiment, step S202 described above can be implemented by the following steps S11-S12:

[0040] Step S11: Acquire images of the target road area to obtain the original image of the target road area;

[0041] As an alternative example, the original image can be as follows: Figure 3As shown; the road scene where the original image is located can be as follows: Figure 4 As shown.

[0042] Step S12: Remove lane lines from the original image to obtain the first background image.

[0043] As an alternative example, the first background image after removing lane lines can be as follows: Figure 5 As shown.

[0044] In this embodiment, the diversity of background images is improved because the first background image can be obtained from an existing image of lane lines.

[0045] In an exemplary embodiment, step S12 above can be implemented by the following steps S121-S123:

[0046] Step S121: Determine the set of first pixels corresponding to the second lane line in the original image;

[0047] As an alternative example, the first set of pixels in the original image can be determined by manual labeling, or the second lane line in the original image can be automatically identified by a lane line segmentation algorithm to determine the first set of pixels.

[0048] Step S122: Determine the first pixel value corresponding to each pixel in the first pixel set using an image restoration algorithm; wherein, the first pixel value is the pixel value corresponding to the road surface in the original image;

[0049] As an optional example, the first pixel value corresponding to each pixel in the first pixel set can be determined from the road surface background in the original image. Alternatively, the pixel values ​​of the pixels in the pixel set corresponding to the road surface in the original image can be weighted and summed to obtain the first pixel value corresponding to each pixel in the first pixel set.

[0050] Step S123: Set the value of each pixel in the first pixel set to the first pixel value to obtain the first background image.

[0051] In this embodiment, the second lane line in the original image can be quickly and accurately eliminated through the above steps S121-S123.

[0052] Step S204: Convert the first background image into a second background image in the world coordinate system;

[0053] It should be noted that the world coordinate system is a coordinate system corresponding to the perspective of the bird's-eye view. In other words, the second background image is a bird's-eye view.

[0054] As an alternative example, the second background image can be as follows: Figure 6 As shown.

[0055] It should be noted that converting the first background image into a second background image from a bird's-eye view is to restore the image from the camera's perspective to the real-world coordinate system, thereby facilitating the generation of new traffic lines based on known national traffic line (including lane lines) standards and improving the realism of the generated traffic lines.

[0056] In an exemplary embodiment, step S204 above can be implemented by the following steps S21-S22:

[0057] Step S21: Determine the homography matrix, wherein the homography matrix is ​​used to indicate the transformation relationship of each pixel point in the first background image from the pixel coordinate system to the world coordinate system;

[0058] In other words, the homography matrix can be used to transform the first background image from the camera's perspective into a second background image from a bird's-eye view.

[0059] In one exemplary embodiment, the first background image can be self-calibrated to obtain a homography matrix.

[0060] As an optional example, if the first background image is an image obtained by removing the second lane lines from the original image, the first background image can be self-calibrated with reference to camera parameters. The camera parameters are parameters determined by first data and second data, where the first data is the data corresponding to the target information in the original image, and the second data is the data corresponding to the target information in a standard database; the target information includes at least one of the following: lane lines, lane width, and road surface design.

[0061] In other words, the self-calibration method can obtain the relationship between the first background image and the bird's-eye view. During self-calibration, the camera parameters are estimated by referring to the actual line width of traffic lines and the lane / road surface design relationship in national standards, thereby obtaining the homography matrix, which can then transform the scene image after removing the markings into a bird's-eye view.

[0062] Step S22: Convert the first background image in the pixel coordinate system into a second background image in the world coordinate system using the homography matrix.

[0063] In this embodiment, the homography matrix can make the resulting second background image more realistic.

[0064] Step S206: Generate a first lane line in the second background image, and convert the first lane line in the world coordinate system into a second lane line in the pixel coordinate system;

[0065] As an optional example, lane marking standards include: lane marking style, lane marking length, width, size, etc.

[0066] As an alternative example, the image generated from the second background image after the first lane line is created can be as follows: Figure 7 As shown.

[0067] In an exemplary embodiment, the generation of the first lane line in the second background image can be achieved by generating the first lane line in the second background image based on at least one reference information from lane line standards and road design specifications; wherein the road design specifications are used to indicate the design specifications of different lane lines under different road scenarios.

[0068] Optionally, the first lane lines can be generated manually from the second background image according to lane line standards, thereby making the generated first lane lines more diverse.

[0069] Optionally, the first lane lines can be automatically generated in the second background image based on lane line standards and road design specifications, thereby improving the generation speed.

[0070] In one exemplary embodiment, the first lane line is generated in the second background image based on at least one reference information from lane line standards and road design specifications, including the following methods one and two:

[0071] Method 1: Determine the second set of pixels and the second pixel value corresponding to each pixel in the second set of pixels in the second background image based on the lane line standard;

[0072] As an alternative example, via method one, a second set of pixels and the second pixel value corresponding to each pixel in the second set of pixels can be manually determined in the second background image according to the lane line standard, so as to generate a first lane line in the second background image.

[0073] Method 2: Based on lane line standards and road design specifications, determine the second pixel set and the second pixel value corresponding to each pixel in the second background image;

[0074] It should be noted that the algorithm for automatically generating the first lane line based on lane line standards and road design specifications can be a deep learning-based generation algorithm. The generated lane lines should be as close as possible to real-world lane lines. Furthermore, human intervention can be incorporated to explore more possibilities.

[0075] It should be noted that when the value of each pixel in the second pixel set of the second background image is the corresponding second pixel value, the second background image contains the first lane line.

[0076] It should be noted that after setting the value of each pixel in the second pixel set of the second background image to the corresponding second pixel value, the following can be obtained: Figure 7 The reference image shown.

[0077] In an exemplary embodiment, the first lane line in the world coordinate system can be converted into a second lane line in the pixel coordinate system by performing an inverse transformation on the first pixel matrix corresponding to the first lane line using the homography matrix to obtain the second pixel matrix corresponding to the second lane line.

[0078] It should be noted that the first pixel matrix is ​​used to indicate the second set of pixels corresponding to the first lane line in the second background image and the second pixel value of each pixel in the second set of pixels; the second pixel matrix is ​​used to indicate the third set of pixels corresponding to the second lane line in the first background image and the third pixel value of each pixel in the third set of pixels.

[0079] It should be noted that both the first and second lane lines are represented as pixel matrices, and each element in the pixel matrix corresponds to a position in the coordinate system.

[0080] Step S208: The first background image and the second lane line are fused to obtain the target image.

[0081] As an optional example, the target image can be as follows: Figure 8 As shown.

[0082] As an optional example, step S208 above can be implemented in the following way: obtaining the second pixel matrix corresponding to the second lane line, wherein the second pixel matrix is ​​used to indicate the third pixel point set corresponding to the second lane line in the first background image and the third pixel value of each pixel point in the third pixel point set; replacing the pixel value of each pixel point in the third pixel point set in the first background image with the corresponding third pixel value.

[0083] As an alternative example, a reference image can be obtained after generating the first lane line in the second background image, and the reference image can be inversely transformed using the homography matrix to obtain the target image.

[0084] As an alternative example, lane lines can also be generated directly from the first background image to obtain an image after the lane lines have been generated.

[0085] Through the above steps, a first background image corresponding to the target road area in pixel coordinates is obtained. This first background image is then converted to a second background image in world coordinates. Based on lane line standards, a first lane line is generated in the second background image. This first lane line is then converted to a second lane line in pixel coordinates. Finally, the first background image and the second lane line are merged to obtain the target image. Since lane line generation for traffic scenes is performed using existing images without simulation software, the generated images are more realistic, solving the problem of unrealistic images generated using simulation software. Furthermore, this method can freely simulate any traffic line layout, increasing the diversity of images under the same background.

[0086] Obviously, the embodiments described above are merely some embodiments of the present invention, and not all embodiments. To better understand the above image generation method, the process is described below with reference to embodiments, but this is not intended to limit the technical solutions of the embodiments of the present invention. Specifically:

[0087] In one exemplary embodiment, this embodiment aims to generate a new image with any different lane lines on an existing image, using the same background. Figure 9 This is a flowchart of another image generation method according to an embodiment of the present invention, specifically comprising the following steps:

[0088] Step S902: Input an image;

[0089] Specifically, the input image is as follows: Figure 3 As shown.

[0090] Step S904: Eliminate the original markings;

[0091] Specifically, based on the input image, original lane marking removal is performed. Original lane marking removal involves erasing the lane lines from the existing image and automatically filling the background using image inpainting methods. First, the pixels containing the lane markings (i.e., the lane lines) need to be determined. Then, an image inpainting algorithm is used to fill in the original lane marking pixel positions, obtaining the scene image after lane marking removal (e.g., ...). Figure 5 The pixels where the markings are located can be determined through manual marking or automatic identification by marking segmentation algorithms. The image restoration algorithm can be a traditional method or a deep learning-based method.

[0092] Step S906: Obtain the transformation relationship of the bird's-eye view;

[0093] Specifically, based on the input image, the bird's-eye view transformation relationship is obtained. This transformation aims to restore the camera image to a real-world coordinate system, facilitating the generation of new road markings according to known national standards. A self-calibration method can be used to obtain the relationship between the current image and the bird's-eye view. During self-calibration, camera parameters are estimated by referencing national standards such as the actual line width of the road markings and lane / road surface design relationships, thereby obtaining the homography matrix.

[0094] Step S908: Transformation of the scene's bird's-eye view;

[0095] Specifically, homography can be used to transform a scene image after removing gradations into a bird's-eye view, for example... Figure 6 .

[0096] Step S910: New markings are generated;

[0097] Specifically, new road markings can be generated based on the transformed bird's-eye view. Because the bird's-eye view is scaled to the real world, new road markings can be generated according to standards such as road design specifications and road marking specifications (lane line standards). This allows for the generation of various road markings with different layouts and textures. A simple example is... Figure 7 As shown. One possible method is manual drawing, another is automatic drawing within a defined area based on possible lane design logic, or generation through deep learning methods. The generation algorithm should aim to produce features that closely resemble real-world lane markings, and can be combined with human intervention to obtain more possibilities.

[0098] Step S912: Inverse transformation of the new marking line;

[0099] Specifically, based on the homography matrix obtained from the previous bird's-eye view transformation, an inverse transformation is performed on the image after the datum lines are generated, which can transform the image of the newly generated datum lines back to the original image coordinate system.

[0100] Step S914: Combine the new marking line with the background image;

[0101] Specifically, by combining the newly transformed markings with the background image after removing the original markings, a scene with the new markings against the same background can be obtained, for example... Figure 8 As shown.

[0102] Step S916: Obtain the new lane line image.

[0103] By following the steps above, any target on a ground plane with a small elevation can be simulated.

[0104] It should be noted that the technical solution of this application embodiment is a purely visual solution, generating new images based solely on existing scene images without relying on any simulation software. Furthermore, the generated images closely resemble real images; only the markings are generated, while the background remains entirely unchanged from the original image. In a bird's-eye view, the markings generated according to national standards are almost identical in size to the real images. The generation algorithm also closely approximates the real image in terms of texture and color. Moreover, any marking layout can be freely simulated, such as rare marking layouts or special marking scenarios.

[0105] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0106] This embodiment also provides an image generation apparatus for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, hardware implementations, or a combination of software and hardware, are also possible and contemplated.

[0107] Figure 10 This is a structural block diagram of an image generation apparatus according to an embodiment of the present invention, the apparatus comprising:

[0108] The acquisition module 1002 is used to acquire a first background image corresponding to the target road area in the pixel coordinate system, wherein the first background image does not contain lane lines;

[0109] The first processing module 1004 is used to convert the first background image into a second background image in the world coordinate system;

[0110] The second processing module 1006 is used to generate a first lane line in the second background image and convert the first lane line in the world coordinate system into a second lane line in the pixel coordinate system.

[0111] The fusion module 1008 is used to fuse the first background image and the second lane line to obtain the target image.

[0112] The above modules acquire a first background image corresponding to the target road area in pixel coordinates, convert the first background image into a second background image in world coordinates, generate a first lane line in the second background image, convert the first lane line into a second lane line in pixel coordinates, and then fuse the first background image and the second lane line to obtain the target image. Since lane line generation for traffic scenes is performed using existing images without simulation software, the generated images are more realistic, solving the problem of unrealistic images generated by simulation software. Furthermore, this method can freely simulate any traffic line layout, increasing the diversity of images under the same background.

[0113] In an exemplary embodiment, the acquisition module 1002 is further configured to acquire images of the target road area to obtain an original image of the target road area; and remove lane lines from the original image to obtain the first background image.

[0114] In an exemplary embodiment, the acquisition module 1002 is further configured to remove lane lines in the original image by: determining a first set of pixels corresponding to the lane lines in the original image; determining a first pixel value corresponding to each pixel in the first set of pixels by an image restoration algorithm; wherein the first pixel value is the pixel value corresponding to the road surface in the original image; and setting the value of each pixel in the first set of pixels to the first pixel value to obtain the first background image.

[0115] In an exemplary embodiment, the first processing module 1004 is further configured to determine a homography matrix, wherein the homography matrix is ​​used to indicate the transformation relationship of each pixel point in the first background image from the pixel coordinate system to the world coordinate system; and to convert the first background image in the pixel coordinate system into a second background image in the world coordinate system through the homography matrix.

[0116] In an exemplary embodiment, the first processing module 1004 is further configured to perform self-calibration on the first background image to obtain the homography matrix. Specifically, when the first background image is an image obtained by removing lane lines from the original image, the self-calibration of the first background image includes: self-calibrating the first background image with reference to camera parameters; the camera parameters are parameters determined by first data and second data, where the first data is data corresponding to target information in the original image, and the second data is data corresponding to the target information in a standard database; the target information includes at least one of the following: lane lines, lane width, and road surface design.

[0117] In an exemplary embodiment, the second processing module 1006 is further configured to generate the first lane line in the second background image based on at least one reference information from lane line standards and road design specifications; wherein the road design specifications are at least used to indicate the design specifications of different lane lines under different road scenarios.

[0118] In an exemplary embodiment, the second processing module 1006 is further configured to determine, in the second background image, a second set of pixels and a second pixel value corresponding to each pixel in the second set of pixels according to lane line standards; or to determine, in the second background image, a second set of pixels and a second pixel value corresponding to each pixel in the second set of pixels according to lane line standards and road design specifications; wherein, if the value of each pixel in the second set of pixels in the second background image is the corresponding second pixel value, the second background image contains the first lane line.

[0119] In an exemplary embodiment, the second processing module 1006 is further configured to perform an inverse transformation on the first pixel matrix corresponding to the first lane line using the homography matrix to obtain a second pixel matrix corresponding to the second lane line; wherein, the first pixel matrix is ​​used to indicate the second set of pixel points corresponding to the first lane line in the second background image and the second pixel value of each pixel point in the second set of pixel points; the second pixel matrix is ​​used to indicate the third set of pixel points corresponding to the second lane line in the first background image and the third pixel value of each pixel point in the third set of pixel points.

[0120] In an exemplary embodiment, the fusion module 1008 is further configured to obtain a second pixel matrix corresponding to the second lane line, wherein the second pixel matrix is ​​used to indicate the third pixel point set corresponding to the second lane line in the first background image and the third pixel value of each pixel point in the third pixel point set; and to replace the pixel value of each pixel point in the third pixel point set in the first background image with the corresponding third pixel value.

[0121] Embodiments of the present invention also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to perform the steps in any of the above method embodiments when executed.

[0122] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0123] S1, Obtain the first background image corresponding to the target road area in the pixel coordinate system, wherein the first background image does not contain lane lines;

[0124] S2, convert the first background image into a second background image in the world coordinate system;

[0125] S3, Generate a first lane line in the second background image, and convert the first lane line in the world coordinate system into a second lane line in the pixel coordinate system;

[0126] S4, the first background image and the second lane line are fused to obtain the target image.

[0127] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0128] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0129] Embodiments of the present invention also provide an electronic device including a memory and a processor, the memory storing a computer program and the processor being configured to run the computer program to perform the steps in any of the above method embodiments.

[0130] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0131] S1, Obtain the first background image corresponding to the target road area in the pixel coordinate system, wherein the first background image does not contain lane lines;

[0132] S2, convert the first background image into a second background image in the world coordinate system;

[0133] S3, Generate a first lane line in the second background image, and convert the first lane line in the world coordinate system into a second lane line in the pixel coordinate system;

[0134] S4, the first background image and the second lane line are fused to obtain the target image.

[0135] In one exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0136] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0137] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0138] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An image generation method, characterized in that, include: Obtain the first background image corresponding to the target road area in the pixel coordinate system, wherein the first background image does not contain lane lines; Convert the first background image into a second background image in the world coordinate system; A first lane line is generated in the second background image, and the first lane line in the world coordinate system is converted into a second lane line in the pixel coordinate system; The first background image and the second lane line are fused together to obtain the target image.

2. The method according to claim 1, characterized in that, Obtain the first background image corresponding to the target road area in pixel coordinates, including: Image acquisition is performed on the target road area to obtain the original image of the target road area; The lane lines in the original image are removed to obtain the first background image.

3. The method according to claim 2, characterized in that, The first background image is obtained by removing lane lines from the original image, including: Determine the first set of pixels corresponding to the lane lines in the original image; The first pixel value corresponding to each pixel in the first pixel set is determined by an image restoration algorithm; wherein, the first pixel value is the pixel value corresponding to the road surface in the original image; The value of each pixel in the first pixel set is set to the corresponding first pixel value to obtain the first background image.

4. The method according to claim 1, characterized in that, Converting the first background image into a second background image in world coordinates includes: Determine a homography matrix, wherein the homography matrix is ​​used to indicate the transformation relationship of each pixel point in the first background image from the pixel coordinate system to the world coordinate system; and use the homography matrix to transform the first background image in the pixel coordinate system into a second background image in the world coordinate system.

5. The method according to claim 4, characterized in that, Determine the homography matrix, including: The first background image is self-calibrated to obtain the homography matrix. When the first background image is obtained by removing lane lines from the original image, the self-calibration of the first background image includes: self-calibrating the first background image with reference to camera parameters; the camera parameters are parameters determined by first data and second data, where the first data is the data corresponding to the target information in the original image, and the second data is the data corresponding to the target information in a standard database; the target information includes at least one of the following: lane lines, lane width, and road surface design.

6. The method according to claim 1, characterized in that, Generating a first lane line in the second background image includes: The first lane line is generated in the second background image based on at least one reference information from lane line standards and road design specifications; wherein... The road design specifications are used to indicate at least the design specifications for different lane lines in different road scenarios.

7. The method according to claim 6, wherein generating the first lane line in the second background image based on at least one reference information from lane line standards and road design specifications comprises: Based on the lane line standard, determine the second set of pixels and the second pixel value corresponding to each pixel in the second background image; or Based on lane line standards and road design specifications, a second set of pixels and the second pixel value corresponding to each pixel in the second background image are determined. Wherein, if the value of each pixel in the second pixel set of the second background image is the corresponding second pixel value, the second background image contains the first lane line.

8. The method according to claim 4, characterized in that, Converting the first lane line in the world coordinate system to the second lane line in the pixel coordinate system includes: The first pixel matrix corresponding to the first lane line is inversely transformed using the homography matrix to obtain the second pixel matrix corresponding to the second lane line; Wherein, the first pixel matrix is ​​used to indicate the second set of pixel points corresponding to the first lane line in the second background image and the second pixel value of each pixel point in the second set of pixel points; the second pixel matrix is ​​used to indicate the third set of pixel points corresponding to the second lane line in the first background image and the third pixel value of each pixel point in the third set of pixel points.

9. The method according to claim 1, characterized in that, The first background image and the second lane line are fused to obtain the target image, including: Obtain the second pixel matrix corresponding to the second lane line, wherein the second pixel matrix is ​​used to indicate the third pixel point set corresponding to the second lane line in the first background image and the third pixel value of each pixel point in the third pixel point set; Replace the pixel value of each pixel in the third pixel set in the first background image with the corresponding third pixel value.

10. An image generation apparatus, characterized in that, include: The acquisition module is used to acquire a first background image corresponding to the target road area in the pixel coordinate system, wherein the first background image does not contain lane lines; The first processing module is used to convert the first background image into a second background image in the world coordinate system; The second processing module is used to generate a first lane line in the second background image and convert the first lane line in the world coordinate system into a second lane line in the pixel coordinate system. The fusion module is used to fuse the first background image and the second lane line to obtain the target image.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program, when executed, performs the method described in any one of claims 1 to 9.

12. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method described in any one of claims 1 to 9 through the computer program.

Images