Method, device, equipment and storage medium for restoring space elements around vehicle

Abstract

The present application relates to a method, apparatus, device, and storage medium for restoring spatial elements surrounding a vehicle. The method acquires images from cameras in multiple directions of the vehicle, scales the vehicle's top view according to a preset scaling ratio, and generates a vehicle scaled schematic diagram. Each image is converted into a trapezoidal image. Using a clockwise or counterclockwise reference direction, each trapezoidal image is sequentially placed along the reference direction to fill the peripheral area of ​​the vehicle scaled schematic diagram, generating a vehicle spatial restoration diagram. Because the vehicle spatial restoration diagram includes image information captured by cameras in multiple directions, the vehicle spatial restoration diagram restores the elements surrounding the vehicle. Drivers viewing the vehicle spatial restoration diagram on the in-vehicle display can grasp the objects surrounding the vehicle in real time without frequently adjusting their line of sight.

Description

Technical Field

[0001] The present application relates to the field of automatic navigation technology, and in particular to a method, device, equipment and storage medium for restoring spatial elements around a vehicle. Background Art

[0002] When driving, drivers need to use the mirrors in all directions to observe the objects around them in real time. This forces the driver to frequently adjust their line of sight. Long driving periods can easily increase driver fatigue, reduce driver concentration, and easily lead to traffic accidents. Summary of the Invention

[0003] To solve or partially solve the problems existing in the related art, the present application provides a method, device, equipment and storage medium for restoring the spatial elements around a vehicle, which can restore the spatial elements around the vehicle. The first aspect of the present application provides a method for restoring the spatial elements around a vehicle, comprising:

[0004] Collecting images captured by cameras in multiple directions of the vehicle, wherein the images carry spatial elements around the vehicle;

[0005] Scaling the top view of the vehicle by a preset scale according to a preset scale to obtain a scaled schematic diagram of the vehicle;

[0006] Performing graphic conversion on the space elements around the vehicle carried by each of the images to obtain a plurality of trapezoidal images;

[0007] Taking the clockwise or counterclockwise direction as a reference direction, each of the trapezoidal images is sequentially filled into the peripheral area of ​​the vehicle zoom diagram along the reference direction to generate a vehicle space restoration diagram.

[0008] Preferably, the process of collecting images captured by cameras in multiple directions of the vehicle, wherein the images carry space elements surrounding the vehicle, further comprises:

[0009] identifying an upper edge and a lower edge in each of the images;

[0010] Setting a filling area, wherein the filling area includes a central area and a peripheral area located outside the central area;

[0011] scaling the top view of the vehicle according to the area parameters of the central area to obtain a scaled schematic diagram of the vehicle, and filling the central area with the scaled schematic diagram of the vehicle;

[0012] Performing a stretching graphic conversion on each of the images according to the area parameters of the central area and the area parameters of the peripheral area to obtain a plurality of trapezoidal images;

[0013] Taking the clockwise or counterclockwise direction as the reference direction, each trapezoidal image is sequentially filled into the peripheral area along the reference direction to generate a vehicle space restoration diagram, wherein the upper side lines of each trapezoidal image coincide with the side length lines of the peripheral area in a one-to-one correspondence, and the lower side lines of each trapezoidal image coincide with the side length lines of the central area in a one-to-one correspondence.

[0014] Preferably, the identifying the upper edge and the lower edge in each of the images includes:

[0015] Using the top edge of the image as the upper edge;

[0016] The contour lines of each element in each of the images are identified by using a binarization function; the bottom area of ​​each of the images is located, and the bottom edge line is identified in the bottom area.

[0017] Preferably, the locating the bottom area of ​​each of the images and identifying the bottom line in the bottom area includes:

[0018] Identifying an elliptical contour line in the bottom area, calculating a midpoint of the elliptical contour line, and matching a midline passing through the midpoint and along the length direction of the image;

[0019] The midpoint is moved downward along the midline according to a preset step size to obtain a target point, and a vertical line passing through the target point and perpendicular to the midline is matched as the lower edge line.

[0020] Preferably, the elliptical contour line is the elliptical contour line closest to the bottom edge of the bottom area.

[0021] Preferably, the collecting of images captured by cameras in multiple directions of the vehicle includes:

[0022] Collect images captured by cameras in the front, rear, left, and right directions of the vehicle; or

[0023] Collect images captured by cameras in six directions: front, rear, upper left, lower left, upper right or lower right of the vehicle.

[0024] A second aspect of the present application provides a device for restoring elements of a space surrounding a vehicle, comprising:

[0025] An acquisition module is used to acquire images captured by cameras in multiple directions of the vehicle, wherein the images carry spatial elements surrounding the vehicle;

[0026] a zoom module, configured to zoom the top view of the vehicle by a preset ratio according to a preset zoom ratio to obtain a zoomed schematic diagram of the vehicle;

[0027] a conversion module, configured to perform graphic conversion on the space elements surrounding the vehicle carried in each of the images to obtain a plurality of trapezoidal images;

[0028] The generating module is used to fill each of the trapezoidal images into the peripheral area of ​​the vehicle zoom diagram in sequence along the reference direction in a clockwise or counterclockwise direction to generate a vehicle space restoration diagram.

[0029] Preferably, the device further comprises:

[0030] A recognition module, configured to recognize an upper edge and a lower edge in each of the images;

[0031] A creation module is used to set a filling area, wherein the filling area includes a central area and a peripheral area located outside the central area;

[0032] a zooming module, configured to zoom the top view of the vehicle according to the area parameters of the central area to obtain a zoomed schematic diagram of the vehicle, and fill the central area with the zoomed schematic diagram of the vehicle;

[0033] a conversion module, configured to perform a stretching graphic conversion on each of the images according to the area parameters of the central area and the area parameters of the peripheral area, to obtain a plurality of trapezoidal images;

[0034] A generation module is used to fill the peripheral area with each trapezoidal image in sequence along the reference direction, using a clockwise or counterclockwise direction as a reference direction, to generate a vehicle space restoration map, wherein the upper side lines of each trapezoidal image correspond one-to-one to coincide with the side length lines of the peripheral area, and the lower side lines of each trapezoidal image correspond one-to-one to coincide with the side length lines of the central area.

[0035] A third aspect of the present application provides an electronic device, including:

[0036] processor; and

[0037] A memory having executable codes stored thereon, which, when executed by the processor, causes the processor to execute the above-mentioned method for restoring space elements surrounding the vehicle.

[0038] The fourth aspect of the present application provides a computer-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor executes the method for restoring the space elements surrounding the vehicle as described above.

[0039] The technical solution provided by this application may have the following beneficial effects:

[0040] The technical solution of this application obtains images from cameras in multiple directions of the vehicle, scales the top view of the vehicle according to a preset scaling ratio, and generates a vehicle zoom diagram. Each image is converted into a trapezoidal image. Using a clockwise or counterclockwise reference direction, each trapezoidal image is sequentially applied to the periphery of the vehicle zoom diagram along the reference direction to generate a vehicle spatial restoration map. Because the vehicle spatial restoration map includes image information captured by cameras in multiple directions, it restores the elements surrounding the vehicle. Drivers viewing the vehicle spatial restoration map on the in-vehicle display can grasp the objects surrounding the vehicle in real time without frequently adjusting their line of sight.

[0041] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The above and other objects, features and advantages of the present application will become more apparent by describing in more detail exemplary embodiments of the present application in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the present application.

[0043] Figure 1 1 is a flow chart of a method for restoring space elements around a vehicle according to an embodiment of the present application;

[0044] Figure 2 is a flow chart of a method for restoring space elements around a vehicle according to another embodiment of the present application;

[0045] Figure 3 Schematic diagram of the structure of the space element restoration device around a vehicle shown in an embodiment of the present application;

[0046] Figure 4 1 is a schematic structural diagram of a device for restoring elements of the space surrounding a vehicle according to another embodiment of the present application;

[0047] Figure 5 is a schematic structural diagram of a vehicle map and a vehicle peripheral area shown in an embodiment of the present application;

[0048] Figure 6 is a simulation schematic diagram of identifying the lower edge of an image shown in an embodiment of the present application;

[0049] Figure 7 is a schematic structural diagram of the filling area shown in an embodiment of the present application;

[0050] Figure 8 is a schematic diagram of a simulation of image stretching transformation shown in an embodiment of the present application;

[0051] Figure 9is a simulation schematic diagram of generating a restoration image shown in an embodiment of the present application;

[0052] Figure 10 It is a structural diagram of an electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION

[0053] The following describes embodiments of the present application in more detail with reference to the accompanying drawings. Although the accompanying drawings illustrate embodiments of the present application, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. Rather, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0054] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. As used in this application and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0055] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0056] Currently, in related technologies, when driving a vehicle, the driver needs to use the mirrors in all directions of the vehicle to observe the objects around the vehicle in real time, which forces the driver to frequently adjust the direction of his or her sight, easily causing the driver to feel tired and reducing the driver's attention.

[0057] In response to the above problems, the present application provides a method, device, equipment and storage medium for restoring the spatial elements around a vehicle, which can restore the spatial elements around the vehicle.

[0058] The technical solution of this application is described in detail below with reference to the accompanying drawings.

[0059] Figure 1 A flow chart of a method for restoring space elements around a vehicle in an embodiment of the present application is shown.

[0060] See also Figure 1A method for restoring space elements around a vehicle comprises the following steps:

[0061] Step S11: collecting images captured by cameras in multiple directions of the vehicle, wherein the images carry spatial elements surrounding the vehicle.

[0062] It should be noted that the direction of the vehicle can be divided into four directions, namely the front (the direction corresponding to the front of the vehicle), the rear (the direction corresponding to the rear of the vehicle), the left (the direction corresponding to the left door) and the right (the direction corresponding to the right door). Cameras are installed in all four directions to capture image data in the current direction.

[0063] It should also be noted that due to the different lengths of different vehicle models, in order to restore the elements surrounding the current vehicle as much as possible, for longer vehicles, two additional directions can be added to the aforementioned directions. That is, the vehicle's direction can be divided into six directions: front (the direction corresponding to the front of the vehicle), rear (the direction corresponding to the rear of the vehicle), upper left (the direction corresponding to the upper left door), lower left (the direction corresponding to the lower left door), upper right (the direction corresponding to the upper right door), and lower right (the direction corresponding to the lower right door).

[0064] Step S12: scaling the top view of the vehicle by a preset scaling ratio to obtain a scaled schematic view of the vehicle.

[0065] It should be noted that obtaining a top view of the vehicle is limited by the display size of the display screen in the vehicle's cab. Therefore, after obtaining the top view of the vehicle, it is necessary to scale the top view of the vehicle according to a preset scaling ratio to obtain a scaled diagram of the vehicle. The scaled diagram of the vehicle is the scaled top view of the vehicle. The preset scaling ratio can be set based on the display size of the display screen, or it can be manually set to complete the overall scaling of the vehicle top view.

[0066] Step S13: convert the space elements around the vehicle carried in each image to obtain a plurality of trapezoidal images.

[0067] It should be noted that the images captured by the camera installed on the vehicle are all rectangular images, and each image needs to be converted into a trapezoidal image by performing a graphic conversion.

[0068] Step S14: Using the clockwise or counterclockwise direction as a reference direction, the trapezoidal images are sequentially filled into the peripheral area of ​​the vehicle zoom diagram along the reference direction to generate a vehicle space restoration diagram.

[0069] like Figure 5 As shown, Figure 5The solid line indicates the area surrounding the zoomed-in vehicle diagram, while the dashed line indicates the periphery of the diagram. After converting each image into a trapezoidal image, the image is laid out in a clockwise or counterclockwise direction, with the zoomed-in vehicle diagram as the center. Because the images are pre-converted into trapezoidal images, the edges of adjacent trapezoidal images overlap when filling the periphery of the diagram, eliminating any unfilled areas and enhancing the visual quality of the vehicle spatial reconstruction.

[0070] To better understand the filling process, assume that the front, rear, left, and right trapezoidal images of the current vehicle are captured, and the clockwise direction is selected as the reference direction. When filling the peripheral area of ​​the vehicle image, the peripheral area of ​​the vehicle zoom diagram is filled in the order of the front, right, rear, and left trapezoidal images to generate a vehicle space restoration map. Similarly, assume that the front, rear, upper left, lower left, upper right, and lower right trapezoidal images of the current vehicle are captured, and the counterclockwise direction is selected as the reference direction. When filling the peripheral area of ​​the vehicle zoom diagram, the peripheral area of ​​the vehicle zoom diagram is filled in the order of the front, upper left, lower left, rear, upper right, and lower right trapezoidal images to generate a vehicle space restoration map. Since the vehicle space restoration map contains image information collected by cameras in multiple directions, the vehicle space restoration map restores the elements around the vehicle. The driver can grasp the objects around the current vehicle in real time without frequently adjusting the direction of sight.

[0071] Figure 2 A flow chart of a method for restoring space elements around a vehicle in another embodiment of the present application is shown. Figure 2 Relative to Figure 1 , which describes the technical solution of this application in more detail.

[0072] See also Figure 2 A method for restoring space elements around a vehicle comprises the following steps:

[0073] Step S21: collecting images captured by cameras in multiple directions of the vehicle, wherein the images carry spatial elements surrounding the vehicle.

[0074] It should be noted that the direction of the vehicle can be divided into four directions, namely the front (the direction corresponding to the front of the vehicle), the rear (the direction corresponding to the rear of the vehicle), the left (the direction corresponding to the left door) and the right (the direction corresponding to the right door). Cameras will be installed in all four directions, and the cameras are used to capture image data in the current direction. For vehicles with longer bodies, two additional directions can be added to the above directions. That is, the direction of the vehicle can be divided into six directions, namely the front (the direction corresponding to the front of the vehicle), the rear (the direction corresponding to the rear of the vehicle), the upper left (the direction corresponding to the upper left door), the lower left (the direction corresponding to the lower left door), the upper right (the direction corresponding to the upper right door) and the lower right (the direction corresponding to the lower right door).

[0075] Step S22: Identify the upper and lower edges of each image.

[0076] It should be noted that since the God's perspective cannot see objects located below the chassis of the current vehicle, and the image captured by the camera includes objects below the chassis, in order to prevent the driver from having a visual illusion when watching the restored image from the God's perspective. The elements below the chassis in the image need to be covered. First, the upper and lower edges of each image are identified. Since the image is a rectangular image before conversion, the top edge of the image is used as the upper edge of the image. As for the lower edge of the image (the lower edge is the chassis line of the current vehicle), the image is processed using a binarization function to identify the contour lines of each element in the image and locate it at the bottom area of ​​the image. The bottom area is the area close to the bottom edge of the image, and the lower edge is identified in the bottom area.

[0077] To identify the lower edge in the bottom area, refer to the following method: Figure 6 , since the contour lines of each element in the image have been identified during the binarization operation, it is necessary to find the elliptical contour line (i.e. Figure 6 If multiple elliptical contour lines are identified, the elliptical contour line closest to the bottom edge of the bottom area is selected as the target processing object, the midpoint O of the elliptical contour line is calculated, and the center line AB passing through the midpoint O and along the length direction of the image is matched. The midpoint O is moved downward along the direction of the center line AB according to a preset step size to obtain the target point O', and the perpendicular line CD passing through the target point O' and perpendicular to the center line AB is matched, and the perpendicular line CD is used as the lower edge line.

[0078] Because the camera is mounted on top of the vehicle, the tire contours of the vehicle can be found by identifying the elliptical contours. The target point O' is obtained by moving the midpoint O by a preset step size. A vertical line CD passing through the target point O' and perpendicular to the midline AB is then matched. This vertical line CD is the corresponding lower edge of the image (i.e., the chassis line of the vehicle). The preset step size is a manually set value that can be specifically set based on the tire size of the vehicle and is not specifically limited in this embodiment.

[0079] Step S23: Setting a filling area, where the filling area includes a central area and a peripheral area located outside the central area.

[0080] It should be noted that Figure 7 The filling area 30 is shown in the embodiment. The filling area 30 is an axisymmetric area structure. The filling area 30 includes a central area 31 and a peripheral area 32. The peripheral area 32 is located in the central area 31. The central area 31 is constructed based on four coordinate points (such as Figure 8 Coordinate point 1, coordinate point 2, coordinate point 3 and coordinate point 4 shown in FIG), and the peripheral area 32 is constructed based on 8 coordinate points (such as Figure 7 The central area 31 and the peripheral area 32 both have area parameters. The area parameters of the central area 31 are the lengths of the sides of the central area 31 (the lengths of the sides can be calculated using the two coordinate points, the same below). The area parameters of the peripheral area 32 are the lengths of the sides of the peripheral area 32.

[0081] Step S24 : scaling the top view of the vehicle according to the regional parameters of the central area to obtain a scaled schematic diagram of the vehicle, and filling the central area with the scaled schematic diagram of the vehicle.

[0082] It should be noted that, based on the regional parameters of central region 31, a corresponding preset scaling ratio can be obtained, enabling scaling of the vehicle's top view to produce a scaled vehicle schematic. Because the scaled vehicle schematic is scaled according to the regional parameters of central region 31, the scaled vehicle schematic fills the entire central region 31. The purpose of providing fill region 30 is to set a preset scaling ratio, scaling the vehicle's top view according to the preset scaling ratio so that the scaled vehicle schematic fills the entire central region 31.

[0083] Step S25: Perform stretching graphic conversion on each image according to the area parameters of the central area and the area parameters of the peripheral area to obtain a plurality of trapezoidal images.

[0084] It should be noted that, according to the area parameters of the central area 31 and the area parameters of the peripheral area 32, each image can be stretched and converted. Figure 8 The specific stretching and conversion process is as follows: Assuming that the image GHLN is image data captured by the camera in front of the vehicle, the image GHLN needs to be filled in front of the vehicle image, and the line segment IJ is determined as the lower edge of the image GHLN by identification, and the line segment GH is determined as the upper edge of the image GHLN. By calculating the coordinate values ​​of coordinate point 1, coordinate point 2, coordinate point 5 and coordinate point 6, the image GHLN can be stretched and converted into a trapezoidal image G'H'L'N', with the line segment G'H' as the upper edge of the trapezoidal image G'H'L'N' and the line segment I'J' as the lower edge of the trapezoidal image G'H'L'N'. Referring to the above method, the stretching and conversion of images captured by cameras in other directions can be completed.

[0085] Step S26: Using the clockwise or counterclockwise direction as the reference direction, fill the peripheral area with each trapezoidal image in sequence along the reference direction to generate a vehicle space restoration diagram, wherein the upper side lines of each trapezoidal image correspond one-to-one to coincide with the side length lines of the peripheral area, and the lower side lines of each trapezoidal image correspond one-to-one to coincide with the side length lines of the central area.

[0086] Please note that Figure 8 and Figure 9 After each image stretching graphic is converted into a trapezoidal image, the clockwise or counterclockwise direction is used as the reference direction, and each trapezoidal image is sequentially filled into the peripheral area 32 along the reference direction to generate a vehicle space restoration map, wherein the upper side lines of each trapezoidal image correspond one-to-one with the side length lines of the peripheral area (such as the upper side line G'H' of the trapezoidal image G'H'L'N' coincides with the side length line of coordinate points 5 and 6), and the lower side lines of each trapezoidal image correspond one-to-one with the side length lines of the central area (such as the lower side line I'J' of the trapezoidal image G'H'L'N' coincides with the side length line of coordinate points 1 and 2). Since the vehicle space restoration map contains image information collected by cameras in all directions, the vehicle space restoration map restores the elements around the vehicle. The driver can view the vehicle space restoration map on the in-vehicle display screen in real time without frequently adjusting the direction of sight and can grasp the objects around the current vehicle.

[0087] Taking into account the length of the vehicle body, for vehicles with a longer body length, the vehicle direction will be adjusted from the original four directions to six directions, that is, the vehicle direction can be divided into six directions. These six directions are front (the direction corresponding to the front of the vehicle), rear (the direction corresponding to the rear of the vehicle), upper left (the direction corresponding to the upper left door), lower left (the direction corresponding to the lower left door), upper right (the direction corresponding to the upper right door) and lower right (the direction corresponding to the lower right door).

[0088] Therefore, when stretching and converting the images in the six directions and filling them, the stretching and filling methods for the front and rear images are consistent with those described above. For the upper left image, lower left image, upper right image, and lower right image, taking the upper left image and lower left image as examples, the upper left image and lower left image are first stretched and converted into an upper left trapezoidal image and a lower left trapezoidal image. Considering the wide-angle camera, when filling the upper left trapezoidal image and the lower left trapezoidal image, the identical elements of the upper left trapezoidal image and the lower left trapezoidal image need to be cropped. Finally, the cropped upper left trapezoidal image and the lower left trapezoidal image are filled into the peripheral area 32. This prevents overlapping elements in the upper left trapezoidal image and the lower left trapezoidal image, which could cause visual confusion for the driver when viewing the vehicle space reconstruction diagram.

[0089] Figure 3 The schematic diagram of the structure of a device for restoring space elements around a vehicle according to this embodiment is shown. The device can execute the method for restoring space elements around a vehicle according to any of the above embodiments.

[0090] See also Figure 3 A device 70 for restoring space elements around a vehicle includes a collection module 710 , a scaling module 720 , a conversion module 730 and a generation module 740 .

[0091] The acquisition module 710 is used to acquire images captured by cameras in multiple directions of the vehicle, wherein the images carry spatial elements surrounding the vehicle.

[0092] It should be noted that the direction of the vehicle can be divided into four directions, namely the front (the direction corresponding to the front of the vehicle), the rear (the direction corresponding to the rear of the vehicle), the left (the direction corresponding to the left door) and the right (the direction corresponding to the right door). Cameras will be installed in all four directions, and the cameras are used to capture image data in the current direction. For vehicles with longer bodies, two additional directions can be added to the above directions. That is, the direction of the vehicle can be divided into six directions, namely the front (the direction corresponding to the front of the vehicle), the rear (the direction corresponding to the rear of the vehicle), the upper left (the direction corresponding to the upper left door), the lower left (the direction corresponding to the lower left door), the upper right (the direction corresponding to the upper right door) and the lower right (the direction corresponding to the lower right door).

[0093] The scaling module 720 is used to scale the top view of the vehicle by a preset scale according to a preset scale to obtain a scaled schematic diagram of the vehicle.

[0094] It should be noted that obtaining a top view of the vehicle is limited by the display size of the display screen in the vehicle's cab. Therefore, after obtaining the top view of the vehicle, it is necessary to scale the top view of the vehicle according to a preset scaling ratio to obtain a scaled diagram of the vehicle. The scaled diagram of the vehicle is the scaled top view of the vehicle. The preset scaling ratio can be set based on the display size of the display screen, or it can be manually set to complete the overall scaling of the vehicle top view.

[0095] The conversion module 730 is used to perform graphic conversion on the space elements around the vehicle carried by each image to obtain a plurality of trapezoidal images.

[0096] It should be noted that the images captured by the camera installed on the vehicle are all rectangular images, and each image needs to be converted into a trapezoidal image by performing a graphic conversion.

[0097] The generation module 740 is used to fill each trapezoidal image into the peripheral area of ​​the vehicle zoom diagram in sequence along the reference direction, using the clockwise or counterclockwise direction as the reference direction, to generate a vehicle space restoration diagram.

[0098] Assuming the captured vehicle's front, rear, left, and right trapezoidal images are in place, and a clockwise direction is selected as the reference direction, the outer area of ​​the zoomed-in vehicle diagram is filled in sequentially, following the front, right, rear, and left trapezoidal images to generate a restored vehicle spatial image. The filling process for the image data in the six directions is similar to the above method and will not be repeated here.

[0099] The vehicle surrounding space element restoration device 70 of the present application uses an acquisition module 710 to collect images captured by cameras in multiple directions of the vehicle, a zoom module 720 to zoom the top view of the vehicle to obtain a zoomed schematic diagram of the vehicle, a conversion module 730 to perform graphic conversion on each image to obtain a plurality of trapezoidal images, and a generation module 740 to sequentially fill each trapezoidal image into the peripheral area of ​​the zoomed schematic diagram along the reference direction, using a clockwise or counterclockwise direction as a reference direction, to generate a vehicle space restoration map. Because the vehicle space restoration map includes image information captured by cameras in various directions, the vehicle space restoration map restores the elements surrounding the vehicle. The driver can view the vehicle space restoration map on the in-vehicle display screen in real time without having to frequently adjust the direction of sight.

[0100] Figure 4 A schematic structural diagram of a vehicle surrounding space element restoration device according to another embodiment of the present application is shown. The device can execute the vehicle surrounding space element restoration method according to any of the aforementioned embodiments.

[0101] See also Figure 4 A device 80 for restoring space elements around a vehicle includes an acquisition module 810, an identification module 820, a creation module 830, a scaling module 840, a conversion module 850 and a generation module 860.

[0102] The acquisition module 810 can be seen in Figure 3 The description is not repeated here.

[0103] The recognition module 820 is used to recognize the upper edge and the lower edge in each image.

[0104] It should be noted that since the God's perspective cannot see objects located below the chassis of the current vehicle, and the image captured by the camera includes objects below the chassis, in order to prevent the driver from having a visual illusion when watching the vehicle space restoration image from the God's perspective. The elements below the chassis in the image need to be covered. First, the upper and lower edges of each image are identified. Since the image is a rectangular image before conversion, the top edge of the image is used as the upper edge of the image. As for the lower edge of the image (the lower edge is the chassis line of the current vehicle), the image is processed using a binarization function to identify the contour lines of each element in the image and locate it at the bottom area of ​​the image. The bottom area is the area close to the bottom edge of the image, and the lower edge is identified in the bottom area.

[0105] The creation module 830 is used to set a filling area, where the filling area includes a central area and a peripheral area located outside the central area.

[0106] It should be noted that Figure 7 The filling area 30 is shown in the embodiment. The filling area 30 is an axisymmetric area structure. The filling area 30 includes a central area 31 and a peripheral area 32. The peripheral area 32 is located in the central area 31. The central area 31 is constructed based on four coordinate points (such as Figure 8 Coordinate point 1, coordinate point 2, coordinate point 3 and coordinate point 4 shown in FIG), and the peripheral area 32 is constructed based on 8 coordinate points (such as Figure 7 The central area 31 and the peripheral area 32 both have area parameters. The area parameters of the central area 31 are the lengths of the sides of the central area 31 (the lengths of the sides can be calculated using the two coordinate points, the same below). The area parameters of the peripheral area 32 are the lengths of the sides of the peripheral area 32.

[0107] The scaling module 840 is used to scale the top view of the vehicle according to the regional parameters of the central area to obtain a scaled schematic diagram of the vehicle, and fill the central area with the scaled schematic diagram of the vehicle.

[0108] It should be noted that, based on the regional parameters of central region 31, a corresponding preset scaling ratio can be obtained, enabling scaling of the vehicle's top view to produce a scaled vehicle schematic. Because the scaled vehicle schematic is scaled according to the regional parameters of central region 31, the scaled vehicle schematic fills the entire central region 31. The purpose of providing fill region 30 is to set a preset scaling ratio, scaling the vehicle's top view according to the preset scaling ratio so that the scaled vehicle schematic fills the entire central region 31.

[0109] The conversion module 850 is used to perform a stretching graphic conversion on each image according to the area parameters of the central area and the area parameters of the peripheral area to obtain a plurality of trapezoidal images.

[0110] It should be noted that, according to the area parameters of the central area 31 and the area parameters of the peripheral area 32, each image can be stretched and transformed. Figure 8 The specific stretching and conversion process is as follows: Assuming that the image GHLN is image data captured by the camera in front of the vehicle, the image GHLN needs to be filled in front of the vehicle image, and the line segment IJ is determined as the lower edge of the image GHLN by identification, and the line segment GH is determined as the upper edge of the image GHLN. By calculating the coordinate values ​​of coordinate point 1, coordinate point 2, coordinate point 5 and coordinate point 6, the image GHLN can be stretched and converted into a trapezoidal image G'H'L'N', with the line segment G'H' as the upper edge of the trapezoidal image G'H'L'N' and the line segment I'J' as the lower edge of the trapezoidal image G'H'L'N'. Referring to the above method, the stretching and conversion of images captured by cameras in other directions can be completed.

[0111] The generation module 860 is used to fill the peripheral area with each trapezoidal image in sequence along the reference direction with a clockwise or counterclockwise direction as the reference direction to generate a vehicle space restoration diagram, wherein the upper side lines of each trapezoidal image correspond one-to-one with the side length lines of the peripheral area, and the lower side lines of each trapezoidal image correspond one-to-one with the side length lines of the central area.

[0112] Please note that Figure 8 and Figure 9After each image is stretched and transformed into a trapezoidal image, the clockwise or counterclockwise direction is used as the reference direction, and each trapezoidal image is sequentially filled into the peripheral area 32 along the reference direction to generate a restoration image, wherein the upper side lines of each trapezoidal image correspond one-to-one with the side length lines of the peripheral area (such as the upper side line G'H' of the trapezoidal image G'H'L'N' coincides with the side length line of coordinate points 5 and 6), and the lower side lines of each trapezoidal image correspond one-to-one with the side length lines of the central area (such as the lower side line I'J' of the trapezoidal image G'H'L'N' coincides with the side length line of coordinate points 1 and 2). Since the restoration image contains image information collected by cameras in all directions, the vehicle space restoration image restores the elements around the vehicle. The driver can view the vehicle space restoration image on the in-vehicle display screen in real time without frequently adjusting the direction of sight and can grasp the objects around the current vehicle.

[0113] See also Figure 10 , the electronic device 900 includes a processor 910 and a memory 920 .

[0114] The processor 910 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0115] The memory 920 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. ROM may store static data or instructions required by the processor 910 or other modules of the computer. The permanent storage device may be a readable and writable storage device. The permanent storage device may be a non-volatile storage device that does not lose stored instructions and data even when the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (such as a magnetic or optical disk, flash memory) as the permanent storage device. In other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as dynamic random access memory. The system memory may store some or all instructions and data required by the processor during operation. In addition, the memory 920 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (such as DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and magnetic disks and / or optical disks may also be used. The memory 920 stores executable codes. When the executable codes are processed by the processor 910 , the processor 910 may execute part or all of the above-mentioned methods.

[0116] In addition, the method according to the present application may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing some or all of the steps in the above method of the present application.

[0117] Alternatively, the present application can also be implemented as a computer-readable storage medium (or non-transitory machine-readable storage medium or machine-readable storage medium) on which executable code (or computer program or computer instruction code) is stored. When the executable code (or computer program or computer instruction code) is executed by a processor of an electronic device (or server, etc.), the processor executes part or all of the steps of the above-mentioned method according to the present application.

[0118] The embodiments of the present application have been described above. The above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or improvements to the technology in the market, or to enable other persons skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for restoring space elements around a vehicle, characterized in that: include: Collecting images captured by cameras in multiple directions of the vehicle, wherein the images carry spatial elements around the vehicle; Scaling the top view of the vehicle by a preset scale according to a preset scale to obtain a scaled schematic diagram of the vehicle; Performing graphic conversion on the space elements around the vehicle carried by each of the images to obtain a plurality of trapezoidal images; Taking a clockwise or counterclockwise direction as a reference direction, sequentially filling each of the trapezoidal images into a peripheral area of ​​the vehicle zoom diagram along the reference direction to generate a vehicle space restoration diagram; The method further includes collecting images captured by cameras in multiple directions of the vehicle, wherein the images carry spatial elements surrounding the vehicle, and further includes: Identify the upper and lower edges of each image; wherein the lower edge determination process includes: Using a binarization function to identify the contours of each element in each of the images; locating the bottom area of ​​each of the images; Identify an elliptical contour line in the bottom area, and calculate a midpoint of the elliptical contour line; Matching a midline passing through the midpoint and along the length direction of the image; Move the midpoint downward along the midline according to a preset step size to obtain a target point; Matching a vertical line passing through the target point and perpendicular to the midline as the lower edge line; Setting a filling area, wherein the filling area includes a central area and a peripheral area located outside the central area; scaling the top view of the vehicle according to the area parameters of the central area to obtain a scaled schematic diagram of the vehicle, and filling the central area with the scaled schematic diagram of the vehicle; Performing a stretching graphic conversion on each of the images according to the area parameters of the central area and the area parameters of the peripheral area to obtain a plurality of trapezoidal images; Taking the clockwise or counterclockwise direction as the reference direction, each trapezoidal image is sequentially filled into the peripheral area along the reference direction to generate a vehicle space restoration diagram, wherein the upper side lines of each trapezoidal image coincide with the side length lines of the peripheral area in a one-to-one correspondence, and the lower side lines of each trapezoidal image coincide with the side length lines of the central area in a one-to-one correspondence.

2. The method for restoring space elements around a vehicle according to claim 1, characterized in that: The identifying of the upper edge and the lower edge in each of the images includes: The top edge of the image is used as the upper edge.

3. The method for restoring space elements around a vehicle according to claim 1, characterized in that: The elliptical contour line is the elliptical contour line closest to the bottom edge of the bottom area.

4. The method for restoring space elements around a vehicle according to any one of claims 1 to 3, characterized in that: The image captured by the vehicle's cameras in multiple directions is collected, including: Collect images captured by cameras in the front, rear, left, and right directions of the vehicle; or Collect images captured by cameras in six directions: front, rear, upper left, lower left, upper right or lower right of the vehicle.

5. A device for restoring elements of the space around a vehicle, characterized in that: include: An acquisition module is used to acquire images captured by cameras in multiple directions of the vehicle, wherein the images carry spatial elements surrounding the vehicle; a zoom module, configured to zoom the top view of the vehicle by a preset ratio according to a preset zoom ratio to obtain a zoomed schematic diagram of the vehicle; a conversion module, configured to perform graphic conversion on the space elements surrounding the vehicle carried in each of the images to obtain a plurality of trapezoidal images; a generating module, configured to sequentially fill each of the trapezoidal images into a peripheral area of ​​the vehicle zoom diagram along a clockwise or counterclockwise direction as a reference direction, thereby generating a vehicle space restoration diagram; The method further includes collecting images captured by cameras in multiple directions of the vehicle, wherein the images carry spatial elements surrounding the vehicle, and further includes: Identify the upper and lower edges of each image; wherein the lower edge determination process includes: Using a binarization function to identify the contours of each element in each of the images; locating the bottom area of ​​each of the images; Identify an elliptical contour line in the bottom area, and calculate a midpoint of the elliptical contour line; Matching a midline passing through the midpoint and along the length direction of the image; Move the midpoint downward along the midline according to a preset step size to obtain a target point; Matching a vertical line passing through the target point and perpendicular to the midline as the lower edge line; Setting a filling area, wherein the filling area includes a central area and a peripheral area located outside the central area; scaling the top view of the vehicle according to the area parameters of the central area to obtain a scaled schematic diagram of the vehicle, and filling the central area with the scaled schematic diagram of the vehicle; Performing a stretching graphic conversion on each of the images according to the area parameters of the central area and the area parameters of the peripheral area to obtain a plurality of trapezoidal images; Taking the clockwise or counterclockwise direction as the reference direction, each trapezoidal image is sequentially filled into the peripheral area along the reference direction to generate a vehicle space restoration diagram, wherein the upper side lines of each trapezoidal image coincide with the side length lines of the peripheral area in a one-to-one correspondence, and the lower side lines of each trapezoidal image coincide with the side length lines of the central area in a one-to-one correspondence.

6. An electronic device, characterized in that: include: processor; as well as A memory having executable codes stored thereon, which, when executed by the processor, causes the processor to execute the vehicle surrounding space element restoration method according to any one of claims 1 to 4.

7. A computer-readable storage medium having executable code stored thereon, which, when executed by a processor of an electronic device, causes the processor to execute the vehicle surrounding space element restoration method according to any one of claims 1 to 4.

Images