Local map construction method and device, electronic equipment and storage medium

Abstract

The invention discloses a local map construction method and device, electronic equipment and a storage medium, and relates to the technical field of vehicles, a first top view and a second top view which are adjacent are selected from four top views, and an overlapping area between the first top view and the second top view is determined; determining a to-be-spliced region in the overlapping region; according to the distance from each pixel point in the to-be-spliced area to the boundary line of the first area and the distance from each pixel point in the to-be-spliced area to the boundary line of the second area, determining the pixel value of each pixel point in the to-be-spliced area in the first top view and the second top view, and splicing the to-be-spliced area; and repeating the image splicing process until all the top views are spliced to obtain a spliced local map. When the overlapped parts of the four paths of top views are fused and spliced, each pixel point in the to-be-spliced area is determined through two adjacent top views, so that each pixel point is influenced by the two top views, and the obtained fused image is smoother and more natural.

Description

Technical Field

[0001] The present disclosure relates to the field of vehicle technology, and in particular to a method and device for constructing a local map, an electronic device, and a storage medium. Background Art

[0002] Surround map construction refers to the use of fisheye cameras installed in the front, back, left and right directions of the vehicle to collect images of the vehicle's surroundings, and use image stitching technology to construct a bird's-eye view that can cover the vehicle's surroundings in a panoramic (360°) manner. In the surround map construction algorithm, four-way image stitching is a key step in determining the quality of surround map construction.

[0003] Currently, when a local map covering the area around the vehicle is constructed through a pre-prototype of surround views around the vehicle, it is difficult to align the four-way surround views, that is, one side of the surround view is aligned and the other side is not aligned, resulting in an excessively unnatural coverage area and artifacts. Summary of the invention

[0004] The present disclosure provides a method, device, electronic device and storage medium for constructing a local map, the main purpose of which is to solve the problem of excessive coverage of unnatural areas and the appearance of artifacts.

[0005] According to a first aspect of the present disclosure, a method for constructing a local map is provided, comprising:

[0006] Selecting an adjacent first top view and a second top view from the four top views, and determining an overlapping area between the first top view and the second top view according to a preset coordinate threshold; wherein in the overlapping area, there is a first area without image information in the first top view and a second area without image information in the second top view, respectively;

[0007] Determine a region to be spliced ​​in the overlapping region; wherein the region to be spliced ​​is smaller than the overlapping region, and the region to be spliced ​​does not contain the first region and the second region;

[0008] Determine the pixel value of each pixel in the area to be stitched in the first top view and the second top view according to the distance from each pixel in the area to be stitched to the boundary line of the first area and the distance to the boundary line of the second area, and stitch the area to be stitched;

[0009] Repeat the above image stitching process until all the top views are stitched together to obtain a stitched local map.

[0010] Optionally, determining the pixel value of each pixel in the area to be stitched in the first top view and the second top view according to the distance of each pixel in the area to be stitched from the boundary line of the first area and the distance to the boundary line of the second area, and stitching the area to be stitched includes:

[0011] Calculate a first distance weight corresponding to the distance of the first area and a second distance weight corresponding to the distance of the second area according to the distance of each pixel point in the area to be stitched from the boundary line of the first area and the distance to the boundary line of the second area;

[0012] The area to be stitched is stitched according to the first distance weight, the second distance weight, the first top view and the second top view of the pixel points at different positions; wherein the first distance weight is the proportion of the pixel values ​​of the pixel points in the first top view in the area to be stitched, and the second distance weight is the proportion of the pixel values ​​of the pixel points in the second top view in the area to be stitched.

[0013] Optionally, calculating, based on the distances of each pixel point in the area to be stitched from the boundary line of the first area and the distances to the boundary line of the second area, a first distance weight corresponding to the distance of the first area and a second distance weight corresponding to the distance of the second area includes:

[0014] Select a target pixel point in the area to be stitched, and determine a first distance from the target pixel point to the first area and a second distance from the target pixel point to the second area;

[0015] Adding the first distance to the second distance to obtain a sum of the distances;

[0016] Respectively determine the proportion of the first distance in the sum of the distances and the proportion of the second distance in the sum of the distances to obtain the first distance weight and the second distance weight;

[0017] Repeat the above steps until the first distance weights and the second distance weights of all the pixels in the area to be stitched are determined.

[0018] Optionally, stitching the to-be-stitched area according to the first distance weight, the second distance weight of the pixels at different positions, and the first top view and the second top view includes:

[0019] Calculate the product of a pixel value of a pixel point corresponding to a first position in the first top view and the first distance weight to obtain a first pixel value;

[0020] Calculate the product of a pixel value of a pixel point corresponding to the first position in the second top view and the second distance weight to obtain a second pixel value;

[0021] The first pixel value and the second pixel value are added to obtain a target pixel value corresponding to the pixel at the first position in the area to be stitched; the above steps are repeated until the pixel values ​​of all the pixels in the area to be stitched are determined and stitching is terminated.

[0022] Optionally, before selecting the adjacent first top view and the second top view from the four top views, the method further includes:

[0023] Collecting a preset number of first training fisheye images based on a preset number of fisheye cameras; wherein the arrangement positions of the first calibration plates in different first training fisheye images are different;

[0024] Based on the preset number of first training fisheye images, the internal parameters of each of the fisheye cameras are calibrated to obtain the internal parameters of the fisheye camera; wherein the first training fisheye image is only used for parameter calibration of the fisheye camera that captures the image.

[0025] Optionally, before selecting the adjacent first top view and the second top view from the four top views, the method further includes:

[0026] Based on each of the fisheye cameras, a second training fisheye image including a second calibration plate is captured; wherein the second calibration plate is placed in advance at a preset distance from the vehicle, and one fisheye camera corresponds to one second calibration plate;

[0027] Dedistorting each of the second training fisheye images based on internal camera parameters of each of the fisheye cameras to obtain a dedistorted second training fisheye image;

[0028] Determine respectively in each of the dedistorted second training fisheye images the actual coordinates and expected coordinates of a preset number of corner points of each of the second calibration plates; wherein the expected coordinates are the coordinates of the corner points of the second calibration plate in the dedistorted second training fisheye images after perspective transformation, and one corner point corresponds to one actual coordinate and one expected coordinate;

[0029] A transformation matrix is ​​obtained by performing calculations based on the actual coordinates and the expected coordinates.

[0030] Optionally, before selecting the adjacent first top view and the second top view and determining the overlapping area between the first top view and the second top view according to a preset coordinate threshold, the method further includes:

[0031] Collecting the fisheye images to be stitched based on the fisheye cameras, and dedistorting the fisheye images to be stitched based on the internal parameters of the fisheye cameras to obtain the dedistorted fisheye images to be stitched;

[0032] A perspective transformation is performed on each of the dedistorted fisheye images to be spliced ​​and the transformation matrix to obtain a top view corresponding to each of the fisheye images to be spliced.

[0033] According to a second aspect of the present disclosure, a device for constructing a local map is provided, comprising:

[0034] A first determining unit is used to select a first top view and a second top view that are adjacent to each other from the four top views, and determine an overlapping area between the first top view and the second top view according to a preset coordinate threshold; wherein the overlapping area includes a first area without image information in the first top view and a second area without image information in the second top view;

[0035] A second determining unit is used to determine a to-be-joined area within the overlapping area; wherein the to-be-joined area is smaller than the overlapping area, and the to-be-joined area does not contain the first area and the second area;

[0036] a stitching unit, configured to determine the pixel value of each pixel in the area to be stitched in the first top view and the second top view according to the distance of each pixel in the area to be stitched from the boundary line of the first area and the distance to the boundary line of the second area, and to stitch the area to be stitched;

[0037] The stitching unit is further used to repeat the above image stitching process until all the top views are stitched together to obtain a stitched local map.

[0038] Optionally, the splicing unit includes:

[0039] A calculation module, used for calculating a first distance weight corresponding to the distance of the first area and a second distance weight corresponding to the distance of the second area according to the distance of each pixel point in the area to be stitched from the boundary line of the first area and the distance to the boundary line of the second area;

[0040] A stitching module is used to stitch the area to be stitched according to the first distance weight, the second distance weight, the first top view and the second top view of the pixel points at different positions; wherein the first distance weight is the proportion of the pixel values ​​of the pixel points in the first top view in the area to be stitched, and the second distance weight is the proportion of the pixel values ​​of the pixel points in the second top view in the area to be stitched.

[0041] Optionally, the calculation module is further used for:

[0042] Select a target pixel point in the area to be stitched, and determine a first distance from the target pixel point to the first area and a second distance from the target pixel point to the second area;

[0043] Adding the first distance to the second distance to obtain a sum of the distances;

[0044] Respectively determine the proportion of the first distance in the sum of the distances and the proportion of the second distance in the sum of the distances to obtain the first distance weight and the second distance weight;

[0045] Repeat the above steps until the first distance weights and the second distance weights of all the pixels in the area to be stitched are determined.

[0046] Optionally, the splicing module is also used for:

[0047] Calculate the product of a pixel value of a pixel point corresponding to a first position in the first top view and the first distance weight to obtain a first pixel value;

[0048] Calculate the product of a pixel value of a pixel point corresponding to the first position in the second top view and the second distance weight to obtain a second pixel value;

[0049] The first pixel value and the second pixel value are added to obtain a target pixel value corresponding to the pixel at the first position in the area to be stitched; the above steps are repeated until the pixel values ​​of all the pixels in the area to be stitched are determined and stitching is terminated.

[0050] Optionally, the device further comprises:

[0051] The acquisition unit is used to acquire a preset number of first training fisheye images based on a preset number of fisheye cameras before the first determination unit selects the adjacent first top view and the second top view from the four top views; wherein,

[0052] The arrangement positions of the first calibration plates in different first training fisheye images are different;

[0053] The calibration unit is used to calibrate the internal parameters of each of the fisheye cameras based on the preset number of first training fisheye images to obtain the internal parameters of the fisheye camera; wherein the first training fisheye image is only used for parameter calibration of the fisheye camera that captures the image.

[0054] Optionally, the device further comprises:

[0055] a shooting unit, configured to shoot a second training fisheye image including a second calibration plate based on each of the fisheye cameras before the first determination unit selects the adjacent first top view and second top view from the four-way top view; wherein the second calibration plate is placed in advance at a preset distance from the vehicle, and one fisheye camera corresponds to one second calibration plate;

[0056] The processing unit is further used to dedistort each of the second training fisheye images based on the internal camera parameters of each of the fisheye cameras to obtain a dedistorted second training fisheye image;

[0057] a third determining unit, configured to respectively determine the actual coordinates and expected coordinates of a preset number of corner points of each of the second calibration plates in each of the dedistorted second training fisheye images; wherein the expected coordinates are the coordinates of the corner points of the second calibration plate in the dedistorted second training fisheye images after perspective transformation, and one corner point corresponds to one actual coordinate and one expected coordinate;

[0058] The calculation unit is used to calculate according to the actual coordinates and the expected coordinates to obtain a transformation matrix.

[0059] Optionally, the device further comprises:

[0060] an acquisition unit, configured to acquire fisheye images to be stitched based on each of the fisheye cameras before selecting the adjacent first top view and the second top view in the first determination unit, and determining an overlapping area between the first top view and the second top view according to a preset coordinate threshold, and dedistort each of the fisheye images to be stitched based on an internal camera parameter of each of the fisheye cameras to obtain the dedistorted fisheye images to be stitched;

[0061] The perspective transformation unit is used to perform perspective transformation according to each of the dedistorted fisheye images to be spliced ​​and the transformation matrix to obtain a top view corresponding to each of the fisheye images to be spliced.

[0062] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0063] at least one processor; and

[0064] a memory communicatively connected to the at least one processor; wherein,

[0065] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect.

[0066] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the first aspect.

[0067] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the method as described in the first aspect above.

[0068] The method, device, electronic device and storage medium for constructing a local map provided by the present disclosure mainly include the following technical solutions: selecting adjacent first and second top views from four top views, and determining the overlapping area between the first and second top views according to a preset coordinate threshold; determining the area to be spliced ​​in the overlapping area; wherein the area to be spliced ​​is smaller than the overlapping area, and the area to be spliced ​​does not contain the first and second areas; determining the pixel values ​​of each pixel in the area to be spliced ​​in the first and second top views according to the distance from each pixel in the area to be spliced ​​to the boundary line of the first area and the distance to the boundary line of the second area, and splicing the area to be spliced; repeating the above image splicing process until all top views are spliced ​​to obtain a spliced ​​local map. Compared with the related art, when the embodiment of the present application fuses and splices the overlapping parts of the four top views, each pixel in the area to be spliced ​​is jointly determined by two adjacent top views, so that each pixel is affected by the two top views, so that the obtained fused image is smoother and more natural.

[0069] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0071] Figure 1 A schematic diagram of a flow chart of a method for constructing a local map provided in an embodiment of the present disclosure;

[0072] Figure 2 A schematic diagram of a top view of a vehicle provided in an embodiment of the present application;

[0073] Figure 3 A schematic diagram of a flow chart of a method for constructing a local map provided in an embodiment of the present disclosure;

[0074] Figure 4 A schematic diagram of a flow chart of a method for constructing a local map provided in an embodiment of the present disclosure;

[0075] Figure 5 A schematic diagram of joint calibration of a fisheye camera provided in an embodiment of the present application;

[0076] Figure 6 A schematic diagram of the structure of a local map construction device provided in an embodiment of the present disclosure;

[0077] Figure 7 A schematic diagram of the structure of a local map construction device provided in an embodiment of the present disclosure;

[0078] Figure 8 A schematic block diagram of an exemplary electronic device provided for an embodiment of the present disclosure. DETAILED DESCRIPTION

[0079] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0080] The following describes a method, device, electronic device, and storage medium for constructing a local map according to an embodiment of the present disclosure with reference to the accompanying drawings.

[0081] Figure 1 A schematic flow chart of a method for constructing a local map provided in an embodiment of the present disclosure.

[0082] like Figure 1 As shown, the method comprises the following steps:

[0083] Step 101, select an adjacent first top view and a second top view, and determine the overlapping area between the first top view and the second top view according to a preset coordinate threshold; wherein, in the overlapping area, there is a first area without image information in the first top view and a second area without image information in the second top view.

[0084] In one implementable method of the embodiment of the present application, the overhead view includes a front overhead view, a left overhead view, a rear overhead view, and a right overhead view of the vehicle; in actual applications, the number of overhead views that need to be spliced ​​can be determined according to actual needs, such as splicing two adjacent overhead views or three adjacent overhead views, etc. In the embodiment of the present application, four overhead views are used as an example for explanation. It should be noted that this narrative method is not a specific limitation on the number of overhead views, and the embodiment of the present application does not limit this; the four overhead views are respectively obtained by visual transformation of four fisheye images, and the four fisheye images are obtained by taking fisheye cameras pre-arranged around the vehicle (i.e., front, rear, left, and right directions). The embodiment of the present application does not limit the number of arranged fisheye cameras.

[0085] In one possible implementation of the present application, please refer to Figure 2 , Figure 2 A schematic diagram of a vehicle's top view splicing provided in an embodiment of the present application; in the embodiment of the present application, the first top view is a front top view, and the second top view is a left top view for illustration, but it should be noted that this narrative method is only an exemplary description, and is not a specific limitation on the order of top view splicing. There is an overlapping area between the front top view and the left top view. When determining the overlapping area, it can be determined based on the coordinate threshold of the pixel points in the four-way top view, such as determining that the coordinates (50, x) and (x, 50) are in the range of overlapping areas, that is, the range where the x coordinate is less than 50 and the y coordinate is less than 50 is the overlapping area; it should be noted that this narrative method is only an exemplary description, and is not a limitation on the specific coordinate threshold. In actual use, the range size of the four-way top view and the pixel quality of the image are required to determine the pixel threshold, and the embodiment of the present application does not limit this.

[0086] In one possible implementation of the present application, please continue to refer to Figure 2 Since there is distortion in the fisheye image taken by the fisheye camera, it is necessary to perform a dedistortion operation and capture the desired image size from the dedistorted fisheye image. There will be some image positions without image information in the dedistorted fisheye image. Figure 2 As shown in the area, specifically, according to the different sizes of the captured images, the size of the first area without image information is also different. It is necessary to set a preset coordinate threshold according to the sizes of the first area and the second area so that the determined overlapping area can include all the areas without image information. It should be noted that the first position and the second position are only used to distinguish the areas without image information in different overhead views, and are not a limitation on the order of images.

[0087] In one implementable method of an embodiment of the present application, the preset coordinate threshold is an empirical value. In actual applications, it can be set according to the following steps: Continuing to use the front view and the left view as examples, after fixing the shooting angle of the fisheye camera, take a front view and a left view respectively, and determine the fusion area in the front view and the left view, determine the coordinate values ​​of each pixel point on the boundary of the fusion area, and then determine a horizontal coordinate boundary and a vertical coordinate boundary; these two coordinate boundaries can be determined as the preset coordinate thresholds; it should be noted that this narrative method is only an exemplary description, and is not a specific limitation on the setting of a specific preset coordinate threshold, and the embodiment of the present application does not limit this.

[0088] In one possible implementation of the embodiment of the present application, the coordinates of the pixel values ​​of each overhead view in the embodiment of the present application are arranged from the upper left corner of the image to the lower right corner of the image.

[0089] Step 102, determining a region to be spliced ​​in the overlapping region; wherein the region to be spliced ​​is smaller than the overlapping region, and the region to be spliced ​​does not contain the first region and the second region.

[0090] In one possible implementation of the embodiment of the present application, there are two methods for determining the area to be spliced. The first method: Please continue to refer to Figure 2 ,like Figure 2 As shown, the hypotenuse of the first area of ​​the front top view is determined to be AB, and the hypotenuse of the second area of ​​the left top view is determined to be CD, then the area to be stitched can be determined to be the area surrounded by EABFDC, that is, the image area outside the first area and the second area is determined to be the area to be stitched; the second method: the area to be stitched is determined to be the area surrounded by EAFCE. Since the area surrounded by ABF is close to the first area, and the area surrounded by CFD is close to the second area, close to the edge of the image, the distortion of this part of the fisheye image is more serious, resulting in low image quality, so this part of the content is directly discarded; since there is no image information of the first area in the front top view, but there is image information of this part in the left top view, this part of the image is directly filled with the image in the left top view, and the second area in the left top view is similar, and the part of the image information in the front top view is directly taken.

[0091] The embodiment of the present application is described by taking the area to be spliced ​​as the area surrounded by EAFCE as an example, but this description is only an exemplary description and is not a limitation on the specific splicing method.

[0092] Step 103, according to the distance from each pixel point in the area to be stitched to the boundary line of the first area, and the distance to the boundary line of the second area, determine the pixel value of each pixel point in the area to be stitched in the first top view and the second top view, and stitch the area to be stitched.

[0093] In one implementable method of an embodiment of the present application, when determining the pixel value of each pixel point in the area to be stitched according to the distance of each pixel point to the boundary line of the first area and the distance to the boundary line of the second area, there are the following two methods: first, according to the distance, determine the pixel value of the pixel point in the area to be stitched as the pixel value of the point in the top view with a closer distance; second, according to the weight ratio of the distance, respectively take the pixel values ​​with corresponding weights in the first area and the second area, and add them to obtain the pixel value of the pixel point in the area to be stitched; specifically, the embodiment of the present application is not limited to this.

[0094] Step 104, repeat the above image stitching process until all the top views are stitched together to obtain a stitched local map.

[0095] According to the stitching algorithm of step 104, after all the pixel values ​​of the pixel points in the area to be stitched are determined, the overlapping area of ​​the second top view and the adjacent third top view is stitched (such as the left top view and the rear top view); until all four top views are stitched together, a stitched local map of the vehicle can be obtained.

[0096] The method for constructing a local map provided by the present disclosure mainly includes the following technical solutions: selecting adjacent first and second top views from four top views, and determining the overlapping area between the first and second top views according to a preset coordinate threshold; determining the area to be spliced ​​in the overlapping area; wherein the area to be spliced ​​is smaller than the overlapping area, and the area to be spliced ​​does not contain the first and second areas; determining the pixel value of each pixel in the area to be spliced ​​in the first and second top views according to the distance from each pixel in the area to be spliced ​​to the boundary line of the first area, and the distance to the boundary line of the second area, and splicing the area to be spliced; repeating the above image splicing process until all the top views are spliced ​​to obtain a spliced ​​local map. Compared with the related art, when the embodiment of the present application fuses and splices the overlapping parts of the four top views, each pixel in the area to be spliced ​​is jointly determined by two adjacent top views, so that each pixel is affected by the two top views, and thus the obtained fused image is smoother and more natural.

[0097] In one possible implementation of the embodiment of the present application, when executing step 103 to splice the band splicing area, the splicing may be performed according to the following steps:

[0098] Calculate a first distance weight corresponding to the distance of the first area and a second distance weight corresponding to the distance of the second area according to the distance of each pixel point in the area to be stitched from the boundary line of the first area and the distance to the boundary line of the second area;

[0099] Please continue reading Figure 2 , in the area EAFCE to be spliced, the distances from the pixel points corresponding to all coordinate points to the AB side and the CD side are calculated respectively, and the weights are calculated according to the distances. For example, the distance from pixel point 1 to the AB side is 4, and the distance to the CD side is 6, then the first distance weight from the AB side is 0.4, and the second distance weight from the CD side is 0.6; the distance from pixel point 2 to the AB side is 5, and the distance to the CD side is 5, then the first distance weight from the AB side is 0.5, and the second distance weight from the CD side is 0.5; it should be noted that this narrative method is only an exemplary description, and is not a specific limitation on specific distances and distance weights, and the embodiments of the present application do not limit this.

[0100] The area to be stitched is stitched according to the first distance weight, the second distance weight, the first top view, and the second top view of the pixel points at different positions; wherein the first distance weight is the ratio of the pixel values ​​of the pixel points in the first top view to the value ratio of the pixel values ​​of the pixel points in the second top view to the value ratio of the pixel values ​​of the pixel points in the second top view to the value ratio of the pixel values ​​of the pixel points in the second top view;

[0101] In one possible implementation of the embodiment of the present application, a pixel point in the fisheye image corresponds to a pixel value, and the first distance weight is used to determine the pixel value ratio of the pixel point at the position in the first top view, and the second distance weight is used to determine the pixel value ratio of the pixel point at the position in the second top view. Please continue to refer to Figure 2 ,like Figure 2As shown, at the position of the same coordinate value, there are three corresponding pixel points: the pixel point corresponding to the coordinate value in the first top view, the pixel point corresponding to the coordinate value in the second top view, and the pixel point corresponding to the coordinate value in the to-be-joined area; the pixel value of the pixel point corresponding to the coordinate value in the to-be-joined area is the sum of the pixel value of the pixel point with the coordinate value in the first top view multiplied by the first distance weight, and the pixel value of the pixel point with the coordinate value in the second top view multiplied by the second distance weight; for example, if the first distance weight is 0.4, the second distance weight is 0.6, the pixel value of the pixel point with the coordinate value in the first top view is 186, and the pixel value of the pixel point with the coordinate value in the second top view is 186. The pixel value of the pixel point at this position in the two top views is 185, so the pixel value of the pixel point at this coordinate value in the area to be stitched is 186 multiplied by 0.4+185 multiplied by 0.6, which is 185; if the first distance weight is 0.5, the second distance weight is 0.5, the pixel value of the pixel point at this coordinate value in the first top view is 200, and the pixel value of the pixel point at this coordinate value in the second top view is 210, then the pixel value of the pixel point at this coordinate value in the area to be stitched is 205; it should be noted that this narrative method is only an exemplary description and is not a limitation on specific data, and the embodiments of the present application are not limited to this.

[0102] In one possible implementation of the embodiment of the present application, when the to-be-joined area is joined in step 104, the following steps may be referred to; Figure 3 , Figure 3 A schematic diagram of a flow chart of a method for constructing a local map provided in an embodiment of the present disclosure includes:

[0103] Step 201 : Calculate the product of a pixel value of a pixel point corresponding to a first position in the first top view and the first distance weight to obtain a first pixel value.

[0104] In one possible implementation of the embodiment of the present application, the same position corresponds to a pixel point in the first top view, the second top view, and the area to be stitched, respectively; following the above application embodiment, the first distance weight is the weight ratio of the distance between the pixel point at the first position and the first area. For example, the first distance weight is 0.7, and the pixel value of the pixel point corresponding to the first position in the first top view is 75, then the first pixel value is the product of 75 and 0.7, that is, 52.5. In practical applications, the numerical value of the pixel value needs to be an integer, so the first pixel value is 53;

[0105] It should be noted that this description is only an exemplary description and is not a specific limitation on the specific first distance weight, the first pixel value and the pixel value corresponding to the pixel point.

[0106] Step 202: Calculate the product of the pixel value of the pixel point corresponding to the first position in the second top view and the second distance weight to obtain a second pixel value.

[0107] The calculation process of the second pixel value is the same as the determination method of the first pixel value, so the detailed steps for determining the second pixel value please refer to step 201, and the embodiments of the present application will not be described one by one here.

[0108] Step 203, adding the first pixel value to the second pixel value to obtain a target pixel value corresponding to the pixel at the first position in the area to be stitched; repeating the above steps until the pixel values ​​of all the pixels in the area to be stitched are determined and stitching is terminated.

[0109] The first pixel value is added to the second pixel value, and the target pixel value obtained by the addition is the pixel value of the pixel point corresponding to the first position in the area to be stitched; for example, the first pixel value is 40 and the second pixel value is 38, then the target pixel value is 78, that is, the pixel value of the pixel point at the first position in the area to be stitched is 78; the process of calculating the pixel value is repeated until the pixel values ​​of all the pixel points in the area to be stitched are calculated.

[0110] In one implementation of the embodiment of the present application, when executing step 103 to calculate the first distance weight and the second distance weight, the following steps may be referred to:

[0111] Select a target pixel point in the area to be stitched, and determine a first distance from the target pixel point to the first area and a second distance from the target pixel point to the second area;

[0112] Adding the first distance to the second distance to obtain a sum of the distances;

[0113] Respectively determine the proportion of the first distance in the sum of the distances and the proportion of the second distance in the sum of the distances to obtain the first distance weight and the second distance weight;

[0114] Repeat the above steps until the first distance weights and the second distance weights of all the pixels in the area to be stitched are determined.

[0115] In one possible implementation of the embodiment of the present application, since the image information captured by the fisheye camera is distorted, the fisheye image needs to be dedistorted. However, when dedistorting, the dedistortion operation needs to be performed according to the internal parameters of the fisheye camera. Therefore, the internal parameters of the fisheye camera need to be calibrated in advance. The fisheye camera can be calibrated according to the following steps:

[0116] Collecting a preset number of first training fisheye images based on a preset number of fisheye cameras; wherein the arrangement positions of the first calibration plates in different first training fisheye images are different;

[0117] When dedistorting a fisheye image taken by the same fisheye camera, only the internal parameters of the same fisheye camera can be used for dedistortion. Therefore, in practical applications, as many fisheye cameras as are used, the internal parameters of the cameras need to be calibrated. The embodiment of the present application does not limit the preset number of fisheye cameras. The internal parameters of the camera include the focal length, optical center, radial distortion coefficient, etc. Specifically, the embodiment of the present application does not limit the internal parameters of the camera.

[0118] When calibrating the internal parameters of the camera, a variety of methods can be used for calibration. The embodiment of the present application uses Zhang's calibration method to calibrate the internal parameters of the camera. It should be noted that this description method is not a specific limitation on the specific calibration method, and the embodiment of the present application does not limit it.

[0119] According to the image information captured by each fisheye camera, when shooting, the calibration plate is placed at different positions in the image, and multiple shots are taken to obtain a preset number of first training fisheye images. For example, each fisheye camera takes 20 first training fisheye images. Specifically, the number of shots of the image information can be determined based on empirical values, and the embodiments of the present application are not limited to this.

[0120] Based on the preset number of first training fisheye images, internal parameters of each of the fisheye cameras are calibrated to obtain internal parameters of the fisheye camera; wherein the first training fisheye image is only used for parameter calibration of the fisheye camera that captures the image;

[0121] In one implementable method of the embodiment of the present application, after the internal parameter calibration of each fisheye camera is completed, after the fisheye image is captured by the fisheye camera, the fisheye image captured by the fisheye camera is dedistorted according to the internal parameters of each fisheye camera to obtain a dedistorted fisheye image; the fisheye image output by the fisheye camera is the dedistorted fisheye image.

[0122] In one possible implementation of the embodiment of the present application, the construction of the vehicle local map requires converting the independent four-way fisheye images to the same coordinates for fusion, that is, it is necessary to obtain the posture relationship between the four fisheye cameras through the four-way fisheye joint calibration, and then fuse the captured four-way images into a global surround map; please refer to Figure 4 , Figure 4 A schematic diagram of a flow chart of a method for constructing a local map provided in an embodiment of the present disclosure includes:

[0123] Step 301 : each of the fisheye cameras captures a second training fisheye image containing a second calibration plate; wherein the second calibration plate is placed in advance at a preset distance from the vehicle, and one fisheye camera corresponds to one second calibration plate.

[0124] See also Figure 5 , Figure 5 A schematic diagram of a joint calibration of a fisheye camera provided by an embodiment of the present application is shown as follows: Figure 5 As shown, a second calibration plate is placed around the vehicle (i.e., front, rear, left and right), respectively, wherein the four second calibration plates are at the same distance from the vehicle body. In practical applications, the placement of the second calibration plate directly affects the effect of the perspective transformation. For example, if the second calibration plate is placed close to the vehicle body, the effect of the perspective transformation of the image within the close range of the vehicle body is better. If the second calibration plate is placed far from the vehicle body, the effect of the perspective transformation of the image within the long range of the vehicle body is better. In practical applications, the distance between the second calibration plate and the vehicle body can be determined according to actual needs, and the embodiments of the present application do not limit this.

[0125] When constructing a four-way top view, the top view range of the top view can be determined in advance, such as a top view within a range of 3 meters around the vehicle body or a top view within a range of 2.5 meters. The embodiment of the present application takes the top view range of 3 meters as an example for explanation, but it should be noted that this description method is not a specific limitation on the specific top view range. After determining the top view range, according to the 1:1 scale relationship between the real world and the four-way top view, that is, 1 cm in the display world corresponds to one pixel point in the four-way top view, after placing the calibration plate, the coordinate values ​​of the four corner points of each calibration plate in the four-way top view can be determined.

[0126] Step 302 : Dedistort each of the second training fisheye images based on the internal parameters of each of the fisheye cameras to obtain a dedistorted second training fisheye image.

[0127] After four fisheye images are taken based on each fisheye image (i.e., the front fisheye image, the left fisheye image, the rear fisheye image, and the right fisheye image), the image is dedistorted according to the internal parameters of each camera to obtain a second training fisheye image; in actual applications, the internal parameters of each fisheye camera are stored in each fisheye camera, so after taking the fisheye image, the output fisheye image is the dedistorted fisheye image.

[0128] Step 303, respectively determine the actual coordinates and expected coordinates of a preset number of corner points of each of the second calibration plates in each of the dedistorted second training fisheye images; wherein the expected coordinates are the coordinates of the corner points of the second calibration plate in the dedistorted second training fisheye image after perspective transformation, and one corner point corresponds to one actual coordinate and one expected coordinate.

[0129] In one implementable method of the embodiment of the present application, the preset number can be set according to an empirical value, and can be set to one or two, etc. The embodiment of the present application is illustrated using the preset number of 4 as an example, but it should be noted that this narrative method is not a specific limitation on the preset number.

[0130] In an implementable manner of the embodiment of the present application, the coordinate values ​​of the four corner points of each calibration plate are determined in each fisheye image respectively, and the actual coordinate value is the position of the pixel point corresponding to the corner point in the fisheye image; the expected coordinates are the coordinate values ​​of the pixels corresponding to the corner points of the calibration plate in the four-way overhead view; in an implementable manner of the embodiment of the present application, when determining the expected coordinates of the corner points, since the size of the calibration plate, the distance between the calibration plate and the vehicle, and the size range of the four fisheye images are known, the distances of the four corners of the calibration plate from the boundaries in the four-way overhead view can be determined, and since one pixel point in the four-way overhead view is equivalent to one centimeter in the real world, the coordinate values ​​of the four corner points of the calibration plate can be determined.

[0131] Step 304: Calculate based on the actual coordinates and the expected coordinates to obtain a transformation matrix.

[0132] In one implementable method of an embodiment of the present application, after determining the actual coordinates and expected coordinates of each corner point, the transformation matrix required for the perspective transformation can be calculated based on the actual coordinates and expected coordinates of each corner point through an open source computer vision library (OpoenCV); for the specific method of calculating the transformation matrix, please refer to any implementation method in the prior art, and the embodiments of the present application will not be described one by one here.

[0133] In one implementable manner of the embodiment of the present application, after obtaining the transformation matrix, it can be applied to the actual perspective transformation process: based on the fisheye cameras to collect the fisheye images to be stitched, each of the fisheye images to be stitched is dedistorted based on the internal camera parameters of each of the fisheye cameras to obtain the dedistorted fisheye images to be stitched; based on each of the dedistorted fisheye images to be stitched and the transformation matrix, a perspective transformation is performed to obtain the top view corresponding to each of the fisheye images to be stitched.

[0134] Projection transformation refers to the process of transforming one image into another image when the same camera is used to shoot the same plane at two positions A and B, respectively. In this case, position A is the position where the fisheye camera is installed on the car body, and position B is a position above the car parallel to the ground. We need to use perspective transformation to transform the fisheye image into a bird's-eye view, that is, transform the image taken by the fisheye camera installed on the car body into the image taken by a camera above the car parallel to the ground.

[0135] Corresponding to the above-mentioned local map construction method, the present invention also provides a local map construction device. Since the device embodiment of the present invention corresponds to the above-mentioned method embodiment, details not disclosed in the device embodiment can be referred to the above-mentioned method embodiment, and will not be repeated in the present invention.

[0136] Figure 6 A schematic diagram of a local map construction device provided in an embodiment of the present disclosure is shown in FIG. Figure 6 As shown, including:

[0137] The first determining unit 41 is used to select a first top view and a second top view that are adjacent to each other in the four top views, and determine an overlapping area between the first top view and the second top view according to a preset coordinate threshold; wherein, in the overlapping area, there is a first area without image information in the first top view and a second area without image information in the second top view, respectively;

[0138] A second determining unit 42 is used to determine a to-be-joined area within the overlapping area; wherein the to-be-joined area is smaller than the overlapping area, and the to-be-joined area does not contain the first area and the second area;

[0139] A stitching unit 43 is used to determine the pixel value of each pixel in the area to be stitched in the first top view and the second top view according to the distance between each pixel in the area to be stitched and the boundary line of the first area and the distance to the boundary line of the second area, and stitch the area to be stitched;

[0140] The stitching unit 43 is further used to repeat the above image stitching process until all the top views are stitched together to obtain a stitched local map.

[0141] The local map construction device provided by the present disclosure has the following main technical solutions: selecting adjacent first and second top views from four top views, and determining the overlapping area between the first and second top views according to a preset coordinate threshold; determining the area to be spliced ​​in the overlapping area; wherein the area to be spliced ​​is smaller than the overlapping area, and the area to be spliced ​​does not contain the first and second areas; determining the pixel value of each pixel in the area to be spliced ​​in the first and second top views according to the distance from each pixel in the area to be spliced ​​to the boundary line of the first area, and the distance to the boundary line of the second area, and splicing the area to be spliced; repeating the above image splicing process until all the top views are spliced ​​to obtain a spliced ​​local map. Compared with the related art, when the embodiment of the present application fuses and splices the overlapping parts of the four top views, each pixel in the area to be spliced ​​is jointly determined by two adjacent top views, so that each pixel is affected by the two top views, and thus the obtained fused image is smoother and more natural.

[0142] Furthermore, in a possible implementation of this embodiment, as Figure 7 As shown, the splicing unit 43 includes:

[0143] A calculation module 431 is used to calculate a first distance weight corresponding to the distance of the first area and a second distance weight corresponding to the distance of the second area according to the distance from each pixel point in the area to be stitched to the boundary line of the first area and the distance to the boundary line of the second area;

[0144] The stitching module 432 is used to stitch the area to be stitched according to the first distance weight, the second distance weight, the first top view and the second top view of the pixel points at different positions; wherein the first distance weight is the proportion of the pixel values ​​of the pixel points in the first top view in the area to be stitched, and the second distance weight is the proportion of the pixel values ​​of the pixel points in the second top view in the area to be stitched.

[0145] Furthermore, in a possible implementation of this embodiment, the calculation module 431 is further configured to:

[0146] Select a target pixel point in the area to be stitched, and determine a first distance from the target pixel point to the first area and a second distance from the target pixel point to the second area;

[0147] Adding the first distance to the second distance to obtain a sum of the distances;

[0148] Respectively determine the proportion of the first distance in the sum of the distances and the proportion of the second distance in the sum of the distances to obtain the first distance weight and the second distance weight;

[0149] Repeat the above steps until the first distance weights and the second distance weights of all the pixels in the area to be stitched are determined.

[0150] Furthermore, in a possible implementation of this embodiment, as Figure 7 As shown, the splicing module 432 is also used for:

[0151] Calculate the product of a pixel value of a pixel point corresponding to a first position in the first top view and the first distance weight to obtain a first pixel value;

[0152] Calculate the product of a pixel value of a pixel point corresponding to the first position in the second top view and the second distance weight to obtain a second pixel value;

[0153] The first pixel value and the second pixel value are added to obtain a target pixel value corresponding to the pixel at the first position in the area to be stitched; the above steps are repeated until the pixel values ​​of all the pixels in the area to be stitched are determined and stitching is terminated.

[0154] Furthermore, in a possible implementation of this embodiment, as Figure 7 As shown, the device also includes:

[0155] The acquisition unit 44 is used to acquire a preset number of first training fisheye images based on a preset number of fisheye cameras before the first determination unit 41 selects adjacent first top views and second top views from the four top views; wherein the arrangement positions of the first calibration plates in different first training fisheye images are different;

[0156] The calibration unit 45 is used to calibrate the internal parameters of each of the fisheye cameras based on the preset number of first training fisheye images to obtain the internal parameters of the fisheye camera; wherein the first training fisheye image is only used for parameter calibration of the fisheye camera that captures the image.

[0157] Furthermore, in a possible implementation of this embodiment, as Figure 7 As shown, the device also includes:

[0158] The shooting unit 46 is used to shoot a second training fisheye image including a second calibration plate based on each of the fisheye cameras before the first determining unit 41 selects the adjacent first top view and the second top view from the four-way top view; wherein the second calibration plate is placed in advance at a preset distance from the vehicle, and one fisheye camera corresponds to one second calibration plate;

[0159] The processing unit 47 is further configured to dedistort each of the second training fisheye images based on the internal camera parameters of each of the fisheye cameras to obtain a dedistorted second training fisheye image;

[0160] The third determining unit 48 is used to determine the actual coordinates and expected coordinates of the preset number of corner points of each of the second calibration plates in each of the dedistorted second training fisheye images, respectively; wherein the expected coordinates are the coordinates of the corner points of the second calibration plate in the dedistorted second training fisheye images after perspective transformation, and one corner point corresponds to one actual coordinate and one expected coordinate;

[0161] The calculation unit 49 is used to perform calculations according to the actual coordinates and the expected coordinates to obtain a transformation matrix.

[0162] Furthermore, in a possible implementation of this embodiment, as Figure 7 As shown, the device also includes:

[0163] The acquisition unit 410 is used for selecting the adjacent first top view and the second top view in the first calculation unit 41, and before determining the overlapping area between the first top view and the second top view according to a preset coordinate threshold, acquiring the fisheye images to be stitched based on the fisheye cameras, and dedistorting the fisheye images to be stitched based on the internal camera parameters of the fisheye cameras to obtain the dedistorted fisheye images to be stitched;

[0164] The perspective transformation unit 411 is used to perform perspective transformation according to each of the dedistorted fisheye images to be spliced ​​and the transformation matrix to obtain a top view corresponding to each of the fisheye images to be spliced.

[0165] It should be noted that the above explanation of the method embodiment is also applicable to the device of this embodiment, and the principle is the same, which is not limited in this embodiment.

[0166] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0167] Figure 8A schematic block diagram of an example electronic device 500 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0168] like Figure 8 As shown, the device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 502 or a computer program loaded from a storage unit 508 to a RAM (Random Access Memory) 503. In the RAM 503, various programs and data required for the operation of the device 500 can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An I / O (Input / Output) interface 505 is also connected to the bus 504.

[0169] A number of components in the device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0170] The computing unit 501 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a CPU (Central Processing Unit), a GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, a DSP (Digital Signal Processor), and any appropriate processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as the construction method of the local map. For example, in some embodiments, the construction method of the local map may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to execute the aforementioned local map construction method in any other appropriate manner (eg, by means of firmware).

[0171] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Standard Products), SOCs (System On Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor that may be a dedicated or general-purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0172] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0173] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0174] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball), through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0175] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

[0176] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0177] It should be noted that artificial intelligence is a discipline that studies how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.), and includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, as well as machine learning / deep learning, big data processing technology, knowledge graph technology, and other major directions.

[0178] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0179] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for constructing a local map, characterized in that: include: Selecting an adjacent first top view and a second top view, and determining an overlapping area between the first top view and the second top view according to a preset coordinate threshold; wherein in the overlapping area, there is a first area without image information in the first top view and a second area without image information in the second top view, respectively; Determine a region to be spliced ​​in the overlapping region; wherein the region to be spliced ​​is smaller than the overlapping region, and the region to be spliced ​​does not contain the first region and the second region; Determine the pixel value of each pixel in the area to be stitched in the first top view and the second top view according to the distance from each pixel in the area to be stitched to the boundary line of the first area and the distance to the boundary line of the second area, and stitch the area to be stitched; Repeat the above image stitching process until all the top views are stitched together to obtain a stitched local map.

2. The method according to claim 1, characterized in that Determining the pixel value of each pixel in the area to be stitched in the first top view and the second top view according to the distance of each pixel in the area to be stitched from the boundary line of the first area and the distance to the boundary line of the second area, stitching the area to be stitched includes: Calculate a first distance weight corresponding to the distance of the first area and a second distance weight corresponding to the distance of the second area according to the distance of each pixel point in the area to be stitched from the boundary line of the first area and the distance to the boundary line of the second area; The area to be stitched is stitched according to the first distance weight, the second distance weight, the first top view and the second top view of the pixel points at different positions; wherein the first distance weight is the proportion of the pixel values ​​of the pixel points in the first top view in the area to be stitched, and the second distance weight is the proportion of the pixel values ​​of the pixel points in the second top view in the area to be stitched.

3. The method according to claim 2, characterized in that Calculating the first distance weight corresponding to the distance of the first area and the second distance weight corresponding to the distance of the second area according to the distance of each pixel point in the area to be stitched from the boundary line of the first area and the distance to the boundary line of the second area includes: Select a target pixel point in the area to be stitched, and determine a first distance from the target pixel point to the first area and a second distance from the target pixel point to the second area; Adding the first distance to the second distance to obtain a sum of the distances; Respectively determine the proportion of the first distance in the sum of the distances and the proportion of the second distance in the sum of the distances to obtain the first distance weight and the second distance weight; Repeat the above steps until the first distance weights and the second distance weights of all the pixels in the area to be stitched are determined.

4. The method according to claim 2, characterized in that: The stitching of the to-be-stitched area according to the first distance weight of the pixels at different positions, the second distance weight, the first top view, and the second top view comprises: Calculate the product of a pixel value of a pixel point corresponding to a first position in the first top view and the first distance weight to obtain a first pixel value; Calculate the product of a pixel value of a pixel point corresponding to the first position in the second top view and the second distance weight to obtain a second pixel value; The first pixel value and the second pixel value are added to obtain a target pixel value corresponding to the pixel at the first position in the area to be stitched; the above steps are repeated until the pixel values ​​of all the pixels in the area to be stitched are determined and stitching is terminated.

5. The method according to claim 1, characterized in that Before selecting the adjacent first top view and second top view, the method includes: Collecting a preset number of first training fisheye images based on a preset number of fisheye cameras; wherein the arrangement positions of the first calibration plates in different first training fisheye images are different; Based on the preset number of first training fisheye images, the internal parameters of each of the fisheye cameras are calibrated to obtain the internal parameters of the fisheye camera; wherein the first training fisheye image is only used for parameter calibration of the fisheye camera that captures the image.

6. The method according to claim 5, characterized in that Before selecting the adjacent first top view and second top view, the method includes: Based on each of the fisheye cameras, a second training fisheye image including a second calibration plate is captured; wherein the second calibration plate is placed in advance at a preset distance from the vehicle, and one fisheye camera corresponds to one second calibration plate; Dedistorting each of the second training fisheye images based on internal camera parameters of each of the fisheye cameras to obtain a dedistorted second training fisheye image; Determine respectively in each of the dedistorted second training fisheye images the actual coordinates and expected coordinates of a preset number of corner points of each of the second calibration plates; wherein the expected coordinates are the coordinates of the corner points of the second calibration plate in the dedistorted second training fisheye images after perspective transformation, and one corner point corresponds to one actual coordinate and one expected coordinate; A transformation matrix is ​​obtained by performing calculations based on the actual coordinates and the expected coordinates.

7. The method according to claim 6, characterized in that Before selecting the adjacent first top view and the second top view and determining the overlapping area between the first top view and the second top view according to a preset coordinate threshold, the method includes: Based on each of the fisheye cameras, the fisheye images to be stitched are collected, and each of the fisheye images to be stitched is dedistorted based on the internal parameters of each of the fisheye cameras to obtain the dedistorted fisheye images to be stitched. A perspective transformation is performed based on each of the dedistorted fisheye images to be stitched and the transformation matrix to obtain a top view corresponding to each of the fisheye images to be stitched.

8. A device for constructing a local map, characterized in that: include: A first determining unit is used to select an adjacent first top view and a second top view, and determine an overlapping area between the first top view and the second top view according to a preset coordinate threshold; wherein, in the overlapping area, there is a first area without image information in the first top view and a second area without image information in the second top view, respectively; A second determining unit is used to determine a to-be-joined area within the overlapping area; wherein the to-be-joined area is smaller than the overlapping area, and the to-be-joined area does not contain the first area and the second area; a stitching unit, configured to determine each pixel point in the area to be stitched in the first top view and the second top view according to the distance between each pixel point in the area to be stitched and the boundary line of the first area and the boundary line of the second area, and to stitch the area to be stitched; The stitching unit is further used to repeat the above image stitching process until all the top views are stitched together to obtain a stitched local map.

9. A vehicle, characterized in that: The vehicle comprises the local map construction device as claimed in claim 8.

10. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

11. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-7.

12. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images