Image generation method, model training method, device, equipment and storage medium
By establishing a three-dimensional ground model, drawing traffic markings and fusion background maps, and generating diverse inverse perspective images, the problem of difficulty in collecting inverse perspective images is solved, and sufficient training samples are provided, which reduces costs and improves accuracy.
Patent Information
- Application Number
- CN202411047393.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-01
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-08-01
AI Technical Summary
The difficulty in collecting inverse perspective transformed images leads to insufficient data volume, especially in autonomous driving or assisted driving. The model training samples for lane line detection are demanding and the collection cost is high.
By establishing a three-dimensional ground model with preset size, drawing traffic markers, fusing background images, and generating inverse perspective images using a simulated four-way fisheye camera, using the inverse perspective fusion algorithm to splice into a diverse inverse perspective images as a model training sample .
A large number of inverse perspective images are achieved, which meets the needs of diversity, solves the problem of insufficient data, reduces manual labeling costs, and improves the accuracy of labeling.
Smart Images

Figure CN118570409B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing technology, and in particular to an image generation method, a model training method, a device, a equipment and a storage medium. Background Art
[0002] Inverse Perspective Mapping (IPM) refers to the angle transformation in order to eliminate the perspective effect of the image taken by the camera. For example, in autonomous driving or assisted driving, lane line detection is very important. In the image taken by the forward-looking camera, due to the existence of the perspective effect, the originally parallel objects intersect in the image. The IPM transformation can be used to eliminate this perspective effect. The IPM image obtained by IPM is sometimes also called a "bird's-eye view". IPM images are difficult to collect and the collection cost is high, but in some cases, a large number of diverse IPM images are required. For example, a model for recognizing IPM images requires a large number of IPM images as training samples. Summary of the invention
[0003] In view of this, the present invention provides an image generation method, a model training method, an apparatus, a device and a storage medium to solve the problem of insufficient data volume caused by the difficulty in collecting IPM pictures.
[0004] In a first aspect, the present invention provides an image generation method, the method comprising:
[0005] Create a three-dimensional ground model of preset size;
[0006] Drawing traffic markings on the ground of the three-dimensional ground model, wherein the traffic markings are marks drawn on the ground, including parking space lines, lane lines, parking space marks and / or driving instruction marks;
[0007] fusing a background image onto the three-dimensional ground model on which traffic markings have been drawn;
[0008] Using a simulated four-way fisheye camera to generate a four-way fisheye picture based on the three-dimensional ground model that has been fused with the background image, the simulated four-way fisheye camera is used to simulate the four-way fisheye cameras on the front, back, left and right sides of the vehicle;
[0009] The four fisheye images are spliced into an inverse perspective image based on an inverse perspective fusion algorithm.
[0010] In an optional implementation, drawing traffic markings on the ground of the three-dimensional ground model includes:
[0011] Setting a parking space entrance center point in the ground plan of the three-dimensional ground model;
[0012] A parking space line is drawn on the ground plan according to the set parking space rotation angle, parking space size, and the center point of the parking space entrance.
[0013] In an optional implementation, the drawing of traffic markings on the ground of the three-dimensional ground model further includes:
[0014] Determine, in the ground plan view, a first drawing area for the lane line and / or a second drawing area for the driving indication mark according to the center point of the parking space entrance;
[0015] Determine the center of the lane line in the first drawing area, and draw the lane line according to the determined center of the lane line; and / or determine the center of the driving instruction sign in the second drawing area, and draw the driving instruction sign in the second drawing area according to the determined center of the driving instruction sign.
[0016] In an optional implementation, the drawing of traffic markings on the ground of the three-dimensional ground model further includes:
[0017] Determining a drawing center point of the parking space mark according to the parking space entrance center point, the parking space size and the parking space rotation angle;
[0018] The parking space mark is drawn according to the drawing center point of the parking space mark.
[0019] In an optional implementation, the using a simulated four-way fisheye camera to generate a four-way fisheye picture based on the three-dimensional ground model fused with the background image includes:
[0020] Get the distance between the set wheel stopper and the parking space line;
[0021] Determining a drawing center point of the wheel blocking rod according to the distance, the parking space size and the parking space rotation angle;
[0022] Obtain a three-dimensional wheel blocking rod model;
[0023] Based on the drawing center point of the wheel blocking rod, the three-dimensional wheel blocking rod model is fused with the three-dimensional ground model that has been fused with the background image to obtain a fused three-dimensional model;
[0024] The four-way fisheye image is generated based on the three-dimensional model.
[0025] In a second aspect, the present invention provides a model training method, comprising:
[0026] Obtaining a reverse perspective image generated by the image generation method of the first aspect or any corresponding embodiment thereof, and obtaining drawing parameters and / or categories corresponding to traffic markings in the reverse perspective image;
[0027] The reverse perspective image is used as a training sample, and the drawing parameters and / or categories are used as label values to train the model.
[0028] In a third aspect, the present invention provides an image generating device, the device comprising:
[0029] A three-dimensional model building module is used to build a three-dimensional ground model of a preset size;
[0030] A drawing module, used for drawing traffic markings on the ground of the three-dimensional ground model, wherein the traffic markings are marks drawn on the ground, including parking space lines, lane lines, parking space marks and / or driving instruction marks;
[0031] A background fusion module, used for fusing the background image onto the three-dimensional ground model on which the traffic markings have been drawn;
[0032] A picture generation module, used to generate a four-way fisheye picture based on the three-dimensional ground model integrated with the background image using a simulated four-way fisheye camera, wherein the simulated four-way fisheye camera is used to simulate four-way fisheye cameras on the front, back, left and right sides of the vehicle;
[0033] The splicing module is used to splice the four fisheye images into an inverse perspective image based on an inverse perspective fusion algorithm.
[0034] In a fourth aspect, the present invention provides a model training device, comprising:
[0035] An acquisition module, used to acquire a reverse perspective image generated by the image generation method of the first aspect or any corresponding embodiment thereof, and to acquire drawing parameters and / or categories corresponding to traffic markings in the reverse perspective image;
[0036] The model training module is used to train the model by taking the reverse perspective image as a training sample and taking the drawing parameters and / or categories as label values.
[0037] In a fifth aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, computer instructions being stored in the memory, and the processor executing the image generation method of the first aspect or any corresponding embodiment thereof, or executing the model training method of the second aspect by executing the computer instructions.
[0038] In a sixth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the image generation method of the first aspect or any corresponding embodiment thereof, or to execute the model training method of the second aspect.
[0039] In a seventh aspect, the present invention provides a computer program product, comprising computer instructions, wherein the computer instructions are used to enable a computer to execute the image generation method of the first aspect or any corresponding embodiment thereof, or to execute the model training method of the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0041] Figure 1 is a flowchart of an image generating method according to an embodiment of the present invention;
[0042] Figure 2 is a schematic diagram of drawing a plurality of parking space lines based on a plurality of parking space entrance center points according to an embodiment of the present invention;
[0043] Figure 3 is a schematic diagram of the center of a driving indication mark determined in a drivable area after the parking space line is drawn according to an embodiment of the present invention;
[0044] Figure 4 is a schematic diagram of drawing a driving indication mark in a drivable area after drawing a parking space line according to an embodiment of the present invention;
[0045] Figure 5 is a schematic diagram of a parking space number drawing center point determined after the parking space line is drawn according to an embodiment of the present invention;
[0046] Figure 6 is a schematic diagram of drawing a parking space use mark in a parking space after drawing a parking space line according to an embodiment of the present invention;
[0047] Figure 7 is a schematic diagram of integrating a background image after drawing parking space lines and traffic markings according to an embodiment of the present invention;
[0048] Figure 8 is a schematic diagram of a three-dimensional ground model fused with a three-dimensional wheel blocking rod model and projected onto the ground according to an embodiment of the present invention;
[0049] Fig. 9 is one of the schematic diagrams of generating four-way fisheye images according to an embodiment of the present invention;
[0050] Fig.10 is a second schematic diagram of generating four-way fisheye images according to an embodiment of the present invention;
[0051] Fig.11 is a third schematic diagram of generating four-way fisheye images according to an embodiment of the present invention;
[0052] Fig.12 is a fourth schematic diagram of generating four-way fisheye images according to an embodiment of the present invention;
[0053] Fig.13 According to an embodiment of the present invention, Figure 9-12 Schematic diagram of an inverse perspective image obtained by splicing four fisheye images shown;
[0054] Fig.14 is a structural block diagram of an image generating device according to an embodiment of the present invention;
[0055] Fig.15 is a structural block diagram of a model training device according to an embodiment of the present invention;
[0056] Fig.16 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0057] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.
[0058] The embodiment of the present invention provides an image generation method, which can generate a large number of reverse perspective pictures to meet the demand for a large number of diverse reverse perspective pictures.
[0059] According to an embodiment of the present invention, an embodiment of an image generating method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of executable computer instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0060] In this embodiment, an image generation method is provided, which can be used in computer devices such as terminals and servers. Figure 1 is a flowchart of an image generating method according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:
[0061] Step S101, establishing a three-dimensional ground model of a preset size.
[0062] For example, the 3D ground model can be a 30m×30m 3D simulation model. The 3D ground model is based on the 3D wireframe model, and by adding relevant information such as faces, surface features, and edge connection directions, it realizes the definition and description of the 3D surface based on the face. This model can meet the needs of applications such as face-to-face intersection, line-to-face elimination, and light and dark color maps. Specifically, the 3D ground model uses the face domain surrounded by directed edges to define the surface of the shape, and the shape is defined by a collection of faces.
[0063] Step S102, drawing traffic markings on the ground of the three-dimensional ground model. Traffic markings are signs drawn on the ground, including parking space lines, lane lines, parking space signs and / or driving instruction signs.
[0064] In the embodiment of the present invention, when drawing traffic markings on the ground of a three-dimensional ground model, the model ground needs to be converted into a plane image according to the size ratio, that is, a ground plane map, and then the traffic markings are drawn according to the pixel size.
[0065] For example, if the pixel size is pixel=1.875cm, converting the plane=30mx30m size into pixels will give the image resolution resolution(1600,1600).
[0066] Among them, the parking space line is the line that defines the parking space, which is generally a closed rectangle, a parallelogram, two parallel lines in front and back plus a short line connecting the two ends of the two lines. The parking space identification includes the parking space number and the parking space use identification (such as disabled parking space, family parking space, special parking space, etc.). The driving direction sign includes the guide arrow (or driving guidance arrow), such as the straight direction arrow, left turn direction arrow, right turn direction arrow, U-turn direction arrow, etc.
[0067] Each time a reverse perspective image is generated, only a part of the above-mentioned traffic markings can be drawn, or all of them can be drawn, and each time a picture is generated, the specific shape and position of the same traffic marking can be different, thereby improving the diversity of the generated pictures.
[0068] The following will explain in detail how to draw traffic markings on the ground of a three-dimensional ground model with examples, which will not be described in detail here.
[0069] Step S103, integrating the background image onto the three-dimensional ground model on which the traffic markings have been drawn.
[0070] Step S104, using a simulated four-way fisheye camera, generating a four-way fisheye picture based on the three-dimensional ground model that has been fused with the background image, wherein the simulated four-way fisheye camera is used to simulate four-way fisheye cameras on the front, back, left, and right sides of the vehicle.
[0071] A simulated fisheye camera is a camera that calculates a fisheye image based on given camera intrinsic and extrinsic parameters and field of view. The camera intrinsic and extrinsic parameters and field of view here can be consistent with those of an actual fisheye camera. Different given camera intrinsic and extrinsic parameters and / or field of view will generate different fisheye images. Therefore, different fisheye images can be generated by adjusting the camera intrinsic and extrinsic parameters and / or field of view of the simulated fisheye camera.
[0072] Step S105, splicing the four fisheye images into an inverse perspective image based on an inverse perspective fusion algorithm. Specifically, the four fisheye images can be converted into inverse perspective images respectively using an inverse perspective algorithm, and then spliced. During the conversion process, an inverse perspective calculation formula can be used to calculate the conversion. For details, please refer to the relevant technology. The image generation method provided in this embodiment can generate a large number of various inverse perspective images to meet the rich and diverse requirements of inverse perspective images.
[0073] The following describes in detail how to draw traffic markings on the ground of a three-dimensional ground model with examples.
[0074] In some optional specific implementations, step S102, i.e. drawing traffic markings on the ground of the three-dimensional ground model, includes:
[0075] Step S1021, setting a parking space entrance center point (also referred to as a parking space entrance line center or a parking space entrance line center point) in the ground plan of the three-dimensional ground model;
[0076] Specifically, a coordinate system can be established for a three-dimensional ground model, in which the x-axis and y-axis planes are on the ground, that is, the x-axis and y-axis planes are on the ground plan, the x-axis is on the right of the coordinate system, the y-axis is in front, and the z-axis is above. The coordinate center point can be set at the center of the ground. According to the needs of image generation, the parking space entrance center point is set on the x-axis and y-axis planes. Only one parking space entrance center point can be set or multiple parking space entrance center points can be set. If only one is set, then one or more parking space entrance center points can be determined based on the parking space entrance center point and according to the parking space size. If multiple parking space entrance center points are set, the distance between the parking space entrance center points also needs to meet the parking space size limit.
[0077] Step S1022: Draw a parking space line on the ground plan according to the set parking space rotation angle, parking space size, and parking space entrance center point.
[0078] There are rectangular parking spaces and oblique parking spaces (the shape of the parking space is a parallelogram). The rotation angle of the rectangular parking space is 90 degrees, and the rotation angle of the oblique parking space is an acute angle. The length of the parking space (the length of the long side) is recorded as , width (short side dimension) is recorded as , in meters. When drawing parking space lines on the ground plan, the parking space size needs to be converted into the number of pixels according to the image resolution. The coordinates of the center point of the parking space entrance are 、The parking space rotation angle is , then the four vertices of the parking space are , , , The parking space line drawn based on the four vertices is as follows Figure 2 Of course, the four vertices of the parking space can also be , , , , or , , , , or , , , .
[0079] Specifically, after the coordinates of the four vertices of the parking space are calculated, the coordinate values need to be converted into pixel coordinate values, and then the parking space line is drawn on the ground plan.
[0080] Figure 2 The figure shows an example of drawing multiple parking space lines based on multiple parking space entrance center points. Figure 2 The coordinates of the four vertices of the vertical parking space are , , , . Figure 2 The coordinates of the four vertices of the middle and side parking spaces are , , , .
[0081] In the embodiment of the present invention, the traffic markings related to parking spaces are drawn on the ground of the three-dimensional ground model. In other optional embodiments, the traffic markings related to lanes can also be drawn.
[0082] In some optional specific implementations, step S102, i.e. drawing traffic markings on the ground of the three-dimensional ground model, further includes:
[0083] Step S1023, determining a first drawing area of lane lines and / or a second drawing area of driving indication marks in the ground plan according to the center point of the parking space entrance;
[0084] Step S1024, determining the center of the lane line in the first drawing area, and drawing the lane line according to the determined center of the lane line; and / or determining the center of the driving instruction mark in the second drawing area, and drawing the driving instruction mark in the second drawing area according to the determined center of the driving instruction mark.
[0085] In the embodiment of the present invention, after the parking space line is drawn, lane lines and / or driving indication marks (guiding arrows) may be added in the drivable area based on the center point of the parking space entrance.
[0086] Specifically, when drawing the driving sign, the center point of the parking space entrance and the set effective range (for example, the effective range is within 10m of the center range) can be used to determine the second drawing area, and then the center position coordinates of the driving sign can be randomly generated in the second drawing area. , the center position coordinates The pixel coordinates converted to the image are , The origin of the physical coordinate system established for the 3D ground model is at the center of the ground, while the origin of the image coordinate system is at the upper left corner of the image. resolution[0] is the number of pixels on the x-axis of the image, and resolution[1] is the number of pixels on the y-axis of the image.
[0087] Figure 3 The figure shows an example of the center of the driving sign determined in the drivable area after the parking space lines are drawn.
[0088] Figure 4 The figure shows an example of drawing a driving indication mark in a drivable area after drawing the parking space lines.
[0089] In some other optional specific implementations, step S102, i.e. drawing traffic markings on the ground of the three-dimensional ground model, further includes:
[0090] Step S1025, determining the drawing center point of the parking space mark according to the parking space entrance center point, parking space size and parking space rotation angle; the parking space mark includes the parking space number and / or the parking space use mark, and the parking space use mark can be, for example, a family parking space mark, a disabled parking space mark, etc.
[0091] Step S1026, drawing the parking space mark according to the drawing center point of the parking space mark.
[0092] Specifically, the drawing center point of the parking space mark, for example, the drawing center point of the parking space number, may coincide with the center point of the parking space entrance (eg Figure 5 The center point of the parking space number in the vertical parking space), so the coordinates of the center point of the parking space number are , , convert the coordinate value into pixel coordinate value , The center point of the parking space number may not coincide with the center point of the parking space entrance, for example, outside the parking space entrance. Figure 5 The center point of the parking space number for a medium-sloped parking space is just outside the parking space entrance. Of course, the center point of the parking space number can also be inside the parking space.
[0093] Figure 5 The red dot in the figure is the center point of the parking space number.
[0094] In addition, the drawing center point of the parking space mark, such as the drawing center point of the parking space use mark, may also be at a certain distance from the center point of the parking space entrance. For example, the horizontal distance between the drawing center point of the parking space use mark and the center point of the parking space entrance is For rectangular side parking spaces and vertical parking spaces, the coordinates of the center point of the parking space mark are:
[0095] ;
[0096] or, .
[0097] for Figure 2-4 For the oblique parking space shown, the coordinates of the center point of the parking space logo are:
[0098] ;
[0099] or, ;
[0100] or, ;
[0101] or, .
[0102] Convert the coordinate value of the center point of the parking space logo into pixel coordinate value , .
[0103] Figure 6 The figure shows an example of drawing a parking space use mark in a parking space after drawing the parking space line.
[0104] In other optional specific implementations, instead of first determining the drawing center point of the parking space mark, a corner point of the parking space mark may be determined, and then the parking space mark is drawn based on the corner point.
[0105] Figure 7 Shown is a schematic diagram of the background image after the parking space lines are drawn and the traffic markings (parking space numbers, arrows and disabled parking space signs) are drawn.
[0106] In some optional specific implementations, step S104, i.e., using a simulated four-way fisheye camera to generate four-way fisheye images based on the three-dimensional ground model that has been fused with the background image, includes:
[0107] Step S1041, obtaining the distance between the set wheel blocking rod and the parking space line; for example, the parking space line is generally a parking space line close to the front position or the rear position of the vehicle.
[0108] Step S1042, determining the drawing center point of the wheel blocking rod according to the distance, parking space size and parking space rotation angle;
[0109] Specifically, if the horizontal distance between the wheel stopper and the parking space line (the parking space line at the rear of the vehicle) is set to rear_distance=1m, for a rectangular vertical parking space, the coordinates of the drawing center point of the wheel stopper are:
[0110] ;
[0111] or, ;
[0112] For a rectangular side parking space, the coordinates of the center point of the wheel stopper are:
[0113] ;
[0114] or, ;
[0115] or, ;
[0116] or, ;
[0117] for Figure 2-4 For the inclined parking space shown, the coordinates of the center point of the wheel blocking rod are:
[0118] ;
[0119] or, ;
[0120] or, ;
[0121] or, ;
[0122] The coordinate value of the drawing center point of the above wheel blocking rod is converted into a pixel coordinate value , .
[0123] Step S1043, obtaining a three-dimensional wheel blocking rod model;
[0124] Step S1044, based on the drawing center point of the wheel blocking rod, the three-dimensional wheel blocking rod model is fused with the three-dimensional ground model that has been fused with the background image to obtain a fused three-dimensional model;
[0125] Figure 8 The figure shows an example of a three-dimensional ground model fused with a three-dimensional wheel stopper model and projected onto the ground (the background is not shown in the figure).
[0126] Step S1045, generating four fisheye images based on the three-dimensional model.
[0127] Specifically, the simulated four-way fisheye camera used to generate the four-way fisheye image can be implemented based on carlas-sim. Fig. 9 , Fig.10 , Fig.11 and Fig.12 , respectively simulating the images captured by the fisheye camera in front of the car, the images captured by the fisheye camera on the left side of the car, the images captured by the fisheye camera on the back side of the car, and the images captured by the fisheye camera on the right side of the car.
[0128] Based on the inverse perspective fusion algorithm Figure 9-12 The reverse perspective image obtained by splicing the four fisheye images shown in Fig.13 shown.
[0129] An embodiment of the present invention provides a model training method, comprising:
[0130] Step 1: obtaining the reverse perspective image generated by the above-mentioned image generation method embodiment, and obtaining the drawing parameters and / or categories corresponding to the traffic markings in the reverse perspective image;
[0131] Specifically, the drawing parameters may include size, coordinate position, etc., and may also include parking space number. The category may be the category of traffic markings, such as parking space lines, lane lines, parking space signs, and driving signs. The category may be the category of guide arrows, such as straight-ahead arrows, left-turn arrows, right-turn arrows, and U-turn arrows. The category may be lane line signs, such as solid lines and dashed lines. The category may also be parking space sign categories, such as family parking spaces, disabled parking spaces, and special parking spaces.
[0132] Drawing parameters and / or categories are obtained according to the needs of model training annotation, that is, whatever annotation values are needed during model training is obtained.
[0133] Step 2: Use the reverse perspective image as a training sample and the drawing parameters and / or categories as annotation values to train a model. The model may be a traffic line recognition model, a positioning model, and the like.
[0134] In the embodiment of the present invention, the above-mentioned image generation method embodiment is used to generate the reverse perspective image required for model training, which solves the problem of insufficient reverse perspective images due to the difficulty in obtaining reverse perspective images, and then insufficient model training samples. In addition, the relevant parameters when the image is generated can be directly used as the annotation value, which not only saves the manual annotation cost, but also has high annotation accuracy.
[0135] In this embodiment, an image generation device is also provided, which is used to implement the above-mentioned image generation method embodiment and preferred implementation mode, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.
[0136] This embodiment provides an image generating device, such as Fig.14 As shown, including:
[0137] A three-dimensional model building module 1101 is used to build a three-dimensional ground model of a preset size;
[0138] A drawing module 1102 is used to draw traffic markings on the ground of the three-dimensional ground model, wherein the traffic markings are marks drawn on the ground, including parking space lines, lane lines, parking space marks and / or driving instruction marks;
[0139] Background fusion module 1103, used for fusing the background image onto the three-dimensional ground model on which the traffic markings have been drawn;
[0140] The image generation module 1104 is used to generate a four-way fisheye image based on the three-dimensional ground model integrated with the background image using a simulated four-way fisheye camera, wherein the simulated four-way fisheye camera is used to simulate four-way fisheye cameras on the front, back, left and right sides of the vehicle;
[0141] The stitching module 1105 is used to stitch the four fisheye images into an inverse perspective image based on an inverse perspective fusion algorithm.
[0142] In some optional implementations, the drawing module 1102 includes:
[0143] A parking space entrance center point setting unit, used to set the parking space entrance center point in the ground plan of the three-dimensional ground model;
[0144] The first drawing unit is used to draw a parking space line on the ground plan according to a set parking space rotation angle, parking space size, and a parking space entrance center point.
[0145] In some optional implementations, the drawing module 1102 further includes:
[0146] A first determining unit is used to determine a first drawing area of the lane line and / or a second drawing area of the driving indication mark in the ground plan according to the center point of the parking space entrance;
[0147] The second drawing unit is used to determine the center of the lane line in the first drawing area and draw the lane line according to the determined center of the lane line; and / or determine the center of the driving indication sign in the second drawing area and draw the driving indication sign in the second drawing area according to the determined center of the driving indication sign.
[0148] In some optional implementations, the drawing module 1102 further includes:
[0149] A second determining unit, configured to determine a drawing center point of the parking space mark according to the parking space entrance center point, the parking space size and the parking space rotation angle;
[0150] The third drawing unit is used to draw the parking space mark according to the drawing center point of the parking space mark.
[0151] In some optional implementations, the picture generation module includes:
[0152] A first acquisition unit is used to acquire the distance between the set wheel blocking rod and the parking space line;
[0153] A third determining unit, configured to determine a drawing center point of the wheel blocking rod according to the distance, the parking space size and the parking space rotation angle;
[0154] A second acquisition unit is used to acquire a three-dimensional wheel blocking rod model;
[0155] a fusion unit, configured to fuse the three-dimensional wheel blocking rod model with the three-dimensional ground model fused with the background image based on the drawing center point of the wheel blocking rod, so as to obtain a fused three-dimensional model;
[0156] A generating unit is used to generate the four fisheye images based on the three-dimensional model.
[0157] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.
[0158] The image generating device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.
[0159] In this embodiment, a model training device is also provided, which is used to implement the above-mentioned model training method embodiment and preferred implementation mode, which have been described and will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware is also possible and conceived.
[0160] The present invention provides a model training device. Fig.15 ,include:
[0161] The acquisition module 1201 is used to acquire the reverse perspective image generated by the above-mentioned image generation method embodiment, and acquire the drawing parameters and / or categories corresponding to the traffic markings in the reverse perspective image;
[0162] The model training module 1202 is used to train the model by using the reverse perspective image as a training sample and the rendering parameters and / or categories as label values.
[0163] The model training device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.
[0164] The embodiment of the present invention also provides a computer device having the above Fig.14 The image generating device shown or the above Fig.15 The model training apparatus shown.
[0165] See also Fig.16 , Fig.16 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Fig.16As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig.16 A processor 10 is taken as an example.
[0166] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.
[0167] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.
[0168] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0169] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.
[0170] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Fig.16 The example of connecting through bus is taken in the following.
[0171] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device may be a touch screen.
[0172] The computer device also includes a communication interface, which is used for the computer device to communicate with other devices or a communication network.
[0173] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.
[0174] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.
[0175] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.
Claims
1. A method for generating an image, It is characterized in that The method comprises: Create a three-dimensional ground model of preset size; Drawing traffic markings on the ground of the three-dimensional ground model, wherein the traffic markings are marks drawn on the ground, including parking space lines, lane lines, parking space marks and / or driving instruction marks; fusing a background image onto the three-dimensional ground model on which traffic markings have been drawn; Using a simulated four-way fisheye camera to generate a four-way fisheye picture based on the three-dimensional ground model that has been fused with the background image, the simulated four-way fisheye camera is used to simulate the four-way fisheye cameras on the front, back, left and right sides of the vehicle; splicing the four fisheye images into an inverse perspective image based on an inverse perspective fusion algorithm; Drawing traffic markings on the ground of the three-dimensional ground model includes: Setting a parking space entrance center point in the ground plan of the three-dimensional ground model; Draw a parking space line on the ground plan according to the set parking space rotation angle, parking space size, and the center point of the parking space entrance; The method of drawing traffic markings on the ground of the three-dimensional ground model further includes: Determine, in the ground plan view, a first drawing area for the lane line and / or a second drawing area for the driving indication mark according to the center point of the parking space entrance; Determine the center of the lane line in the first drawing area, and draw the lane line according to the determined center of the lane line; and / or determine the center of the driving indication mark in the second drawing area, and draw the driving indication mark in the second drawing area according to the determined center of the driving indication mark; The method of drawing traffic markings on the ground of the three-dimensional ground model further includes: Determining a drawing center point of the parking space mark according to the parking space entrance center point, the parking space size and the parking space rotation angle; Drawing the parking space mark according to the drawing center point of the parking space mark; The method of using a simulated four-way fisheye camera to generate a four-way fisheye picture based on the three-dimensional ground model integrated with the background image comprises: Get the distance between the set wheel stopper and the parking space line; Determining the drawing center point of the wheel blocking rod according to the distance, parking space size and parking space rotation angle; Obtain a three-dimensional wheel blocking rod model; Based on the drawing center point of the wheel blocking rod, the three-dimensional wheel blocking rod model is fused with the three-dimensional ground model that has been fused with the background image to obtain a fused three-dimensional model; The four-way fisheye image is generated based on the three-dimensional model.
2. A model training method, It is characterized in that include: Obtaining the reverse perspective image generated by the image generation method according to claim 1, and obtaining the drawing parameters and / or categories corresponding to the traffic markings in the reverse perspective image; The reverse perspective image is used as a training sample, and the drawing parameters and / or categories are used as label values to train the model.
3. An image generating device, It is characterized in that The device comprises: A three-dimensional model building module is used to build a three-dimensional ground model of a preset size; A drawing module, used for drawing traffic markings on the ground of the three-dimensional ground model, wherein the traffic markings are marks drawn on the ground, including parking space lines, lane lines, parking space marks and / or driving instruction marks; A background fusion module, used for fusing the background image onto the three-dimensional ground model on which the traffic markings have been drawn; A picture generation module, used to generate a four-way fisheye picture based on the three-dimensional ground model integrated with the background image using a simulated four-way fisheye camera, wherein the simulated four-way fisheye camera is used to simulate four-way fisheye cameras on the front, back, left and right sides of the vehicle; A splicing module, used for splicing the four fisheye images into an inverse perspective image based on an inverse perspective fusion algorithm; The drawing module comprises: A parking space entrance center point setting unit, used to set the parking space entrance center point in the ground plan of the three-dimensional ground model; A first drawing unit, configured to draw a parking space line on the ground plan according to a set parking space rotation angle, parking space size, and a parking space entrance center point; The drawing module also includes: A first determining unit is used to determine a first drawing area of the lane line and / or a second drawing area of the driving indication mark in the ground plan according to the center point of the parking space entrance; A second drawing unit is used to determine the center of the lane line in the first drawing area, and draw the lane line according to the determined center of the lane line; and / or determine the center of the driving indication mark in the second drawing area, and draw the driving indication mark in the second drawing area according to the determined center of the driving indication mark; The drawing module also includes: A second determining unit, configured to determine a drawing center point of the parking space mark according to the parking space entrance center point, the parking space size and the parking space rotation angle; A third drawing unit, configured to draw the parking space mark according to a drawing center point of the parking space mark; The picture generation module includes: A first acquisition unit is used to acquire the distance between the set wheel blocking rod and the parking space line; A third determining unit, configured to determine a drawing center point of the wheel blocking rod according to the distance, the parking space size and the parking space rotation angle; A second acquisition unit is used to acquire a three-dimensional wheel blocking rod model; a fusion unit, configured to fuse the three-dimensional wheel blocking rod model with the three-dimensional ground model fused with the background image based on the drawing center point of the wheel blocking rod, so as to obtain a fused three-dimensional model; A generating unit is used to generate the four fisheye images based on the three-dimensional model.
4. A computer device, It is characterized in that include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the image generation method according to claim 1 or the model training method according to claim 2 by executing the computer instructions.
5. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the image generation method described in claim 1 or the model training method described in claim 2.
6. A computer program product, It is characterized in that It includes computer instructions, and the computer instructions are used to enable a computer to execute the image generation method described in claim 1 or the model training method described in claim 2.
Citation Information
Patent Citations
Vehicle end real-time parking space detection method and device
CN114140429A
Fisheye image sample generation method and device, computer equipment and storage medium
CN117011474A
