Image processing method and device

Abstract

The present invention provides an image processing method and device, the method comprising: obtaining a first bird's-eye view of a vehicle in a current scene; in the first bird's-eye view, the pixel accuracy at pixels at different pixel distances from the vehicle is the same; performing nonlinear transformation on the pixel positions of different pixels in the first bird's-eye view to obtain a second bird's-eye view; in the second bird's-eye view, the average pixel accuracy of the second area is less than the average pixel accuracy of the third area, and the pixel distance between any pixel point in the second area and the vehicle is less than the pixel distance between any pixel point in the third area and the vehicle; performing image processing on the second bird's-eye view. The present invention processes images based on nonlinear transformation, improves the clarity of the area near the vehicle, expands the maximum perception range, and is conducive to providing the safety of the vehicle's automatic driving process.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and in particular relates to an image processing method, device, electronic equipment and computer-readable storage medium. Background Art

[0002] At present, for some autonomous driving vehicles, the bird's eye view (BEV) paradigm is mainly used for autonomous driving perception. In the autonomous driving perception process of the BEV paradigm, a pixelated bird's eye view is generally used as the perception space, that is, the BEV space. After the perception image around the vehicle body is obtained by the 2D camera arranged around the vehicle body, the perception image can be converted to the corresponding perception space through image processing, that is, the perception image obtained by the 2D camera is converted to the BEV space.

[0003] However, during the research process, the inventors found from the perception images converted to the BEV space that for the perception range of a certain distance around the vehicle body, the resolution of the perception image in the BEV space is low when the perception range requirements are met, and the perception range is too small when the image resolution requirements are met.

[0004] For the autonomous driving perception task of the BEV paradigm, the low resolution of the perception image or the small perception range in the BEV space will affect the image recognition during the vehicle driving process and reduce the safety of the vehicle driving process. Summary of the invention

[0005] Based on this, the present invention provides an image processing method to solve the problem in the related art that it is difficult to balance the accuracy and perception range of perceived images in the BEV space, thereby ensuring vehicle driving safety.

[0006] The present invention also provides an image processing device to ensure the implementation and application of the above method in practice.

[0007] In a first aspect, an embodiment of the present invention provides an image processing method, the method comprising the following steps:

[0008] Acquire a first bird's-eye view of the vehicle in the current scene; in the first bird's-eye view, the pixel accuracy of pixels at different pixel distances from the vehicle is the same;

[0009] Performing nonlinear transformation on pixel positions of different pixel points in the first bird's-eye view to obtain a second bird's-eye view; in the second bird's-eye view, an average pixel accuracy of the second region is less than an average pixel accuracy of the third region, and a pixel distance between any pixel point in the second region and the vehicle is less than a pixel distance between any pixel point in the third region and the vehicle;

[0010] Perform image processing on the second bird's-eye view image.

[0011] In a second aspect, an embodiment of the present invention further provides an image processing device, the device comprising:

[0012] An acquisition module, used for acquiring a first bird's-eye view of the vehicle in the current scene; in the first bird's-eye view, the pixel accuracy of the pixel points at different pixel distances from the vehicle is the same;

[0013] a nonlinear transformation module, configured to perform nonlinear transformation on pixel positions of different pixel points in the first bird's-eye view to obtain a second bird's-eye view; in the second bird's-eye view, an average pixel accuracy of the second region is less than an average pixel accuracy of the third region, and a pixel distance between any pixel point in the second region and the vehicle is less than a pixel distance between any pixel point in the third region and the vehicle;

[0014] A processing module is used to perform image processing on the second bird's-eye view.

[0015] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the image processing method.

[0016] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, which, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to perform the image processing method.

[0017] In the autonomous driving perception process of the BEV paradigm, it is difficult for the perception image in the BEV space to take into account both accuracy and perception range. The relevant technology can achieve high perception image accuracy and small perception range or low perception image accuracy and large perception range. In an embodiment of the present invention, a first bird's-eye view of the vehicle in the current scene is obtained, and the first bird's-eye view is a bird's-eye view of the vehicle in the current scene. In the first bird's-eye view, the pixel accuracy at the pixel points at different pixel distances from the vehicle is the same, that is, in the first bird's-eye view, each pixel corresponds to the same size of perception range in the current scene. Then, the pixel positions of different pixel points in the first bird's-eye view are nonlinearly transformed to transform the first bird's-eye view into a second bird's-eye view. In the second bird's-eye view, the average pixel accuracy of the second area closer to the vehicle is less than the average pixel accuracy of the third area farther from the vehicle, that is, in the second bird's-eye view, the closer the position to the vehicle is, the higher the pixel accuracy is, so as to identify the characteristics of the vehicle's surrounding environment, and the farther the position from the vehicle is, the lower the pixel accuracy is, which is only used to determine whether there are other vehicles or objects. Finally, the second bird's-eye view can be used for subsequent image processing.

[0018] By adopting the embodiments of the present invention, the pixel accuracy of the area near the vehicle is improved, which is beneficial to image recognition. By reducing the pixel accuracy of the area far from the vehicle, the perception range is improved, and both the accuracy and perception range of the perceived image are taken into account, which can enhance the safety of the vehicle's automatic driving process. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0020] Figure 1 It is a schematic diagram of the steps of BEV paradigm perception in the related technology provided by an embodiment of the present invention;

[0021] Figure 2 It is a flowchart of a BEV paradigm perception task in the related technology provided by an embodiment of the present invention;

[0022] Figure 3 It is a schematic diagram of the mapping relationship from 2D coordinates to 3D coordinates in BEV paradigm perception in the related technology provided by an embodiment of the present invention;

[0023] Figure 4 It is a schematic diagram of the mapping effect of mapping the surrounding scene of the vehicle in the BEV space in the related technology provided by the embodiment of the present invention;

[0024] Figure 5 It is a flowchart of the steps of a field image processing method provided by an embodiment of the present invention;

[0025] Figure 6 It is a flowchart of specific steps of an image processing method provided by an embodiment of the present invention;

[0026] Figure 7 It is a schematic diagram of a mapping effect of using a piecewise function to perform nonlinear transformation to map a vehicle scene in a BEV space provided by an embodiment of the present invention;

[0027] Figure 8 It is another schematic diagram of mapping effect of using piecewise function to perform nonlinear transformation to map a vehicle scene in a BEV space provided by an embodiment of the present invention;

[0028] Fig. 9 is a schematic diagram of a function curve of a trigonometric function used in performing a nonlinear transformation provided by an embodiment of the present invention;

[0029] Fig.10is a schematic diagram of a function curve of another trigonometric function used in performing nonlinear transformation provided by an embodiment of the present invention;

[0030] Fig.11 It is a schematic diagram of a partial function curve of an inverse proportional function used when performing a nonlinear transformation provided by an embodiment of the present invention;

[0031] Fig.12 is a schematic diagram of another part of a function curve of an inverse proportional function used when performing a nonlinear transformation provided by an embodiment of the present invention;

[0032] Fig.13 It is a schematic diagram of a partial function curve of an exponential function used when performing a nonlinear transformation provided by an embodiment of the present invention;

[0033] Fig.14 is a schematic diagram of another part of a function curve of an exponential function used when performing a nonlinear transformation provided by an embodiment of the present invention;

[0034] Fig.15 is a schematic diagram of a function curve of a trigonometric function used in performing an inverse transformation provided by an embodiment of the present invention;

[0035] Fig.16 A flowchart of an improved BEV paradigm perception task provided by an embodiment of the present invention;

[0036] Fig.17 is a block diagram of an image processing device provided by an embodiment of the present invention;

[0037] Fig.18 is a logic block diagram of an electronic device according to an embodiment of the present invention;

[0038] Fig.19 It is a logic block diagram of an electronic device according to another embodiment of the present invention. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] The present invention can be used in many general or special computing device environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multi-processor devices, distributed computing environments including any of the above devices or equipment, etc.

[0041] The present invention may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0042] To facilitate understanding of the embodiments of the present application, the relevant technologies are briefly described as follows:

[0043] Reference Figure 1 In the perception task of the BEV paradigm, BEV query vectors (BEV queries) are mainly used to capture BEV features. Then BEV queries query the spatial space and temporal space in turn, so that the spatiotemporal information can be aggregated in the BEV features. Finally, the object information in the vehicle's surrounding environment is judged based on the BEV features.

[0044] Specifically, refer to Figure 2 ,The perception tasks of the BEV paradigm include: 1) lifting the BEV plane coordinates to obtain a number of 3D position coordinates; 2) projecting the 3D position coordinates onto the camera plane to obtain a number of corresponding camera pixel coordinates; 3) performing feature sampling on the camera pixel coordinates to obtain BEV features; 4) executing backend tasks based on the BEV features and outputting the results.

[0045] In the BEV perception paradigm, a key issue is how to transform images captured by a 2D camera into the BEV space. Taking the deformable attention of BEV perception (BEV Former) as an example, other BEV perceptions also use a similar method to map the BEV space to the 2D camera.

[0046] First, a set of learnable parameters of dimension H×W×C is defined as BEV query vectors (BEV queries) to capture BEV features. Each query vector query at the (x, y) position in BEV queries is only responsible for characterizing its corresponding small area. BEV queries aggregate spatiotemporal information in BEV query features by querying the spatial space and temporal space in turn. These BEV features can support a variety of autonomous driving perception tasks including 3D object detection and map semantic segmentation.

[0047] In order to aggregate spatial information, the spatial attention mechanism is used, that is, the mechanism by which BEV queries extract the required spatial features from multi-camera features through attention. In BEV Former, a sparse attention mechanism called Deformable Attention is used. Deformable Attention means that each BEV query interacts with part of the image area.

[0048] After using Deformable Attention, for each BEV feature at the (x, y) position, we can calculate its corresponding real-world coordinates (x', y'). Then, the BEV query is "lifted" at the real-world coordinates to obtain multiple 3D points. Figure 3 , the projection points of 3D points on the camera view plane are obtained through the camera internal and external parameters. For surround view cameras, each BEV query generally has valid projection points on 1-2 camera view planes. Finally, these projection points are used as reference points, feature sampling is performed around them, and then weighted sampling features are used for update, thus completing feature aggregation in spatial space.

[0049] Due to factors such as video memory size and computing power limitations, the number of pixels in the BEV space is severely limited. For example, for an RTX3090 with 24G video memory, the number is generally 200 pixels × 200 pixels.

[0050] Taking the 200-pixel × 200-pixel BEV space as an example, the conversion process can be understood as: mapping the 100-meter × 100-meter perception range around the vehicle into a 200-pixel × 200-pixel gridded space, and then raising each grid to different heights to obtain a series of 3D spatial positions. Based on the camera’s internal and external parameters, the 3D spatial positions are projected onto the camera’s 2D pixel plane to obtain 2D coordinates. The mapping relationship between the pixel positions on the 2D pixel plane and the real-world coordinates is then obtained.

[0051] For a 200 pixel × 200 pixel BEV space, refer to Figure 4 If the range around the vehicle is 100 meters by 100 meters, each pixel represents 0.5 meters, that is, the pixel accuracy is 0.5 meters / pixel, which is difficult to meet the requirements for tasks such as lane line recognition. If the pixel accuracy is increased to 0.1 meters / pixel, that is, each pixel represents 0.1 meters, although the accuracy requirements can be met, the maximum perception range is only 20 meters, which means that only a range of 10 meters around the vehicle can be perceived, which is also difficult to meet the requirements for judging objects around the vehicle for autonomous driving.

[0052] But for vehicles, different pixel accuracy requirements are required for objects at different distances. For objects far from the vehicle, our goal is to identify whether the object exists, even if the pixel accuracy is low and the error is large, it is acceptable. But for objects or lane lines close to the vehicle, we need higher accuracy. 0.5 meters / pixel is not enough to identify vehicles or lane lines in the adjacent lanes.

[0053] To this end, an embodiment of the present invention provides an image processing method, which can be applied to vehicle autonomous driving scenarios, namely, a bird's-eye view is captured by a camera installed on the vehicle, and a nonlinear transformation is performed on the bird's-eye view to achieve a balance between the pixel accuracy and the perception range of the bird's-eye view, thereby identifying the environment at different distances from the vehicle during the autonomous driving process to improve the safety of autonomous driving.

[0054] refer to Figure 5 , which shows a flowchart of the steps of an image processing method. Specifically, the method may include:

[0055] Step 201: Obtain a first bird's-eye view of the vehicle in the current scene; in the first bird's-eye view, the pixel accuracy at pixel points at different pixel distances from the vehicle is the same.

[0056] In the embodiments of the present application, the current scene refers to the real environment in which the vehicle is located, and more specifically, refers to the real environment within a certain range that the vehicle needs to perceive. The first bird's-eye view refers to a bird's-eye view obtained by fusing 2D images collected by multiple 2D cameras on the vehicle, that is, an image of the current scene around the vehicle is displayed in a three-dimensional manner. The 2D camera can be a commonly used surround fisheye camera, etc., and the embodiments of the present application do not limit this.

[0057] It should be noted here that the bird's-eye view mentioned in the embodiments of the present application is not a picture directly observed by the human eye. When the BEV query queries the spatial space and the tempoal space in turn, and then aggregates the queried features, a feature vector is obtained, and the bird's-eye view is composed of these feature vectors. Correspondingly, the pixels of the bird's-eye view are not the grayscale / RGB values ​​of conventional pictures, but the feature vectors of the area where the pixels are located. In the first bird's-eye view, there are both vehicle images and current scene images, and the objects in the real scene correspond one-to-one to the objects displayed in the current scene image. The vehicle occupies a certain area in the first bird's-eye view, and it is generally not simply considered as a point. At this time, the pixel distance between the pixel point and the vehicle refers to the shortest pixel distance between the pixel point in the current scene image and the edge contour of the vehicle image. Of course, the vehicle can also be considered as a point. At this time, the pixel distance between other pixels and the vehicle can be directly determined.

[0058] In the embodiment of the present application, the unit of pixel accuracy is: meter / pixel, which reflects the actual range of a pixel in the image corresponding to the real environment. The pixel accuracy at the pixel point reflects the actual distance of the pixel point in the first bird's-eye view corresponding to the current environment.

[0059] For a 200-pixel × 200-pixel BEV space, to perceive a 100-meter × 100-meter range around the vehicle, the pixel accuracy is 0.5 meters per pixel, that is, one pixel corresponds to 0.5 meters in the current scene. To perceive a 200-meter × 200-meter range around the vehicle, the pixel accuracy is 1 meter per pixel, that is, one pixel corresponds to 1 meter in the current scene.

[0060] The first bird's-eye view is obtained by processing images collected by a 2D camera installed on the vehicle. Affected by the camera parameters, the pixel accuracy of the first bird's-eye view at different pixel points in the BEV space is the same, and the image accuracy and perception range are unbalanced, which makes it difficult to meet the requirements of autonomous driving.

[0061] Step 202: Perform a nonlinear transformation on the pixel positions of different pixel points in the first bird's-eye view to obtain a second bird's-eye view; in the second bird's-eye view, the average pixel accuracy of the second area is less than the average pixel accuracy of the third area, and the pixel distance between any pixel point in the second area and the vehicle is less than the pixel distance between any pixel point in the third area and the vehicle.

[0062] The pixel position refers to the relative position of a pixel point in a certain space, which can be expressed by constructing a coordinate system and then using coordinates. In the embodiment of the present application, the pixel positions of different pixel points in the first bird's-eye view refer to the BEV plane coordinates of different pixel points in the first bird's-eye view mapped in the BEV space. For a BEV space of 200 pixels × 200 pixels, it refers to all coordinates in the range of (0~200, 0~200).

[0063] The nonlinear transformation of the pixel position can be understood as the coordinate transformation of the BEV plane coordinates according to a certain nonlinear transformation relationship. The pixel position of each pixel point in the first bird's-eye view is nonlinearly transformed, that is, the coordinate transformation of all BEV plane coordinates in the BEV space is performed according to a certain nonlinear transformation relationship.

[0064] In the second bird's-eye view obtained by nonlinear transformation, the pixel precision at different pixel points may be different. The pixel precision at the pixel point closer to the vehicle is smaller, and the pixel precision at the pixel point farther from the vehicle is larger.

[0065] Assume that the environment image around the vehicle in the second bird's-eye view is divided into different regions, wherein the second region includes some pixels with relatively small pixel distances to the vehicle, and the third region includes some pixels with relatively large pixel distances to the vehicle. The average pixel accuracy of the pixels in the second region is less than the average pixel accuracy of the pixels in the third region, which means that the closer the region is to the vehicle, the smaller the pixel accuracy is, and the clearer the image is, while the farther the region is from the vehicle, the larger the pixel accuracy is, and the image is more blurred.

[0066] In the embodiment of the present application, pixel accuracy is used to illustrate the difference between the first bird's-eye view and the second bird's-eye view, but it does not mean that the vehicle needs to obtain the pixel accuracy of the corresponding image.

[0067] In addition, as another way of explanation, the first bird's-eye view and the second bird's-eye view may also be distinguished by standards such as resolution and clarity. For example, in the first bird's-eye view, the clarity at different pixel points is the same, and in the second bird's-eye view, the closer the part is to the vehicle, the higher the clarity, and the farther the part is from the vehicle, the lower the clarity. After obtaining the concept of the embodiment of the present invention, the technicians in this field may use other simple standards to distinguish the first bird's-eye view and the second bird's-eye view, which should also be within the scope of protection of this application.

[0068] Step 203: perform image processing on the second bird's-eye view.

[0069] In the embodiment of the present application, the second bird's-eye view is used to perform subsequent image processing to complete the autonomous driving perception process of the BEV paradigm. The implementation of this part of the content can adopt relevant existing technologies, and the embodiment of the present application does not specifically limit the subsequent image processing process.

[0070] Reference Figure 6 , which shows a flowchart of specific steps of an image processing method provided by an embodiment of the present invention. Specifically, the method may include:

[0071] Step 301: Obtain a first bird's-eye view of the vehicle in the current scene; in the first bird's-eye view, the pixel accuracy of pixels at different pixel distances from the vehicle is the same.

[0072] For step 301 , reference may be made to the relevant description of step 201 above, which will not be repeated here.

[0073] Step 302: Perform nonlinear transformation on the pixel positions of different pixel points in the first bird's-eye view according to a preset nonlinear transformation function to obtain a second bird's-eye view.

[0074] In the second bird's-eye view, an average pixel accuracy of the second area is less than an average pixel accuracy of the third area, and a pixel distance between any pixel point in the second area and the vehicle is less than a pixel distance between any pixel point in the third area and the vehicle.

[0075] Among them, the horizontal coordinate of the function curve of the nonlinear transformation function is the relative pixel distance between the first pixel point in the first bird's-eye view and the vehicle, the vertical coordinate of the function curve of the nonlinear transformation function is the relative pixel distance between the pixel position corresponding to the first pixel point in the second bird's-eye view and the vehicle, and the inflection point of the function curve of the nonlinear transformation function is the pixel point where the vehicle is located in the first bird's-eye view.

[0076] In the embodiment of the present application, a nonlinear transformation function is used to perform a nonlinear transformation on the pixel positions of different pixel points in the first bird's-eye view to obtain a second bird's-eye view.

[0077] The nonlinear transformation function is a function form different from the linear transformation function. The characteristic of the nonlinear transformation function is that the slope of the corresponding function curve is changing, that is, the function curve of the nonlinear transformation function is not a complete straight line.

[0078] The inflection point of the function curve of the nonlinear transformation function refers to the point where the direction of the function curve is changed. In the embodiment of the present application, the closer the pixel is to the vehicle, the lower the pixel accuracy is, and the farther the pixel is from the vehicle, the greater the pixel accuracy is. That is, on one side of the vehicle, the closer the distance is to the vehicle, the smaller the pixel accuracy is, and on the other side opposite to the vehicle, the closer the distance is to the vehicle, the smaller the pixel accuracy is.

[0079] For example, observing the slope of the nonlinear transformation function from the 100th pixel on the front side of the vehicle to the 100th pixel on the rear side of the vehicle, it should be: in the interval from the 100th pixel on the front side of the vehicle to the body contour on the front side of the vehicle, the slope gradually decreases, and in the interval from the body contour on the rear side of the vehicle to the 100th pixel on the rear side of the vehicle, the slope gradually increases. This is reflected in the function curve. The direction of the function curve changes at the pixel point where the vehicle is located. Therefore, the pixel point where the vehicle is located can be regarded as the inflection point of the function curve of the nonlinear transformation function.

[0080] In an embodiment of the present application, the input of the nonlinear transformation function is the pixel position of the pixel point in the first bird's-eye view, that is, the BEV plane coordinates of the first bird's-eye view in the BEV space, and the output of the nonlinear transformation function is the pixel position of the pixel point in the second bird's-eye view, that is, the extended BEV plane coordinates of the second bird's-eye view in the BEV space.

[0081] Specifically, when performing nonlinear transformation, the horizontal coordinate of the BEV plane coordinate is input into a preset nonlinear transformation function, and the transformed horizontal coordinate is output, and the transformed horizontal coordinate is used as the horizontal coordinate of the extended BEV plane coordinate. The vertical coordinate of the BEV plane coordinate is input into a preset nonlinear transformation function, and the transformed vertical coordinate is output, and the transformed vertical coordinate is used as the vertical coordinate of the extended BEV plane coordinate.

[0082] The nonlinear transformation function may include piecewise function, trigonometric function, exponential function, etc. In the embodiment of the present application, the protection point is mainly to transform the first bird's-eye view into a second bird's-eye view that meets the requirements through a nonlinear transformation relationship, and the function form of the nonlinear function is not specifically limited.

[0083] Optionally, the nonlinear transformation function is a piecewise function.

[0084] The characteristic of piecewise function is that its domain can be divided into several intervals, and in each interval, the slope of the function curve is unique. Therefore, the nonlinear transformation function can be designed using piecewise function. In the range close to the vehicle, the slope of the function curve is small, and in the range far from the vehicle, the slope of the function curve is large.

[0085] Optionally, within the second area, the pixel precision at each pixel point is the same; within the third area, the pixel precision at each pixel point is the same.

[0086] By adopting the concept of the above-mentioned piecewise function, the BEV space can be divided into at least two areas, namely, a second area closer to the vehicle and a third area farther from the vehicle. Accordingly, the piecewise function is divided into a first segment function corresponding to the second area and a second segment function corresponding to the third area, and the slopes of the first segment function and the second segment function are both unique, and the slope of the first segment function is smaller than the slope of the second segment function.

[0087] The idea of ​​designing a piecewise function after dividing the BEV space into two areas is relatively simple to implement and requires little work.

[0088] Optionally, the BEV space may be divided into more than two regions by adopting the concept of the piecewise function. For example, the embodiment of the present application divides the BEV space into three regions.

[0089] Specifically, refer to Figure 7In the figure, O represents the pixel where the vehicle is located, and the BEV space of 200 pixels × 200 pixels is divided into three intervals, namely (0~50, 0~50), (50~100, 50~100) and (100~200, 100~200). Among them, the interval (0~50, 0~50) corresponds to the range of (0~20 meters, 0~20 meters) around the vehicle in the current scene, and each pixel represents 0.4 meters; the interval (50~100, 50~100) corresponds to the range of (20 meters~50 meters, 20 meters~50 meters) around the vehicle in the current scene, and each pixel represents 0.6 meters; the interval (100~200, 100~200) corresponds to the range of (50 meters~200 meters, 50 meters~200 meters) around the vehicle in the current scene, and each pixel represents 1.5 meters.

[0090] It should be noted that in the embodiment of the present application, the BEV plane coordinates are expressed as (X_bev_pix, Y_bev_pix), where X_bev_pix and Y_bev_pix are both integers. The BEV plane coordinates are expressed according to the BEV space. Figure 7 , Figure 7 The midpoint O is the center of the BEV space, and in the embodiment of the present application, it represents the pixel point where the vehicle is located, and its BEV plane coordinates are (100, 100). The actual world coordinates are (X_world, Y_world), and the units of X_world and Y_world are both meters.

[0091] According to the above correspondence, the designed piecewise function can be:

[0092]

[0093] The BEV plane coordinates of the first bird's-eye view in the BEV space are input into the above-mentioned piecewise function, and the expanded BEV plane coordinates are output, so as to obtain the second bird's-eye view in the BEV space.

[0094] At this time, the mapping relationship between the pixel coordinates of the second bird's-eye view image and the actual world coordinates can be expressed in the form of the piecewise function. For example, the mapping relationship between the abscissa of the pixel coordinates of the second bird's-eye view image and the abscissa of the actual world coordinates can be expressed as:

[0095] if -25<(X_bev_pix-100)<=0:

[0096] X_world=(X_bev_pix-100)*0.4

[0097] else if-50<(X_bev_pix-100)<=-25:

[0098] X_world=10+(X_bev_pix-100)*0.6

[0099] else if-100<=(X_bev_pix-100)<=-50:

[0100] X_world=50+(X_bev_pix-100)*1.5

[0101] else if0<(X_bev_pix-100)<=25:

[0102] X_world=(X_bev_pix-100)*0.4

[0103] else if25<(X_bev_pix-100)<=50:

[0104] X_world=-10+(X_bev_pix-100)*0.6

[0105] else if50<(X_bev_pix-100)<=100:

[0106] X_world=-50+(X_bev_pix-100)*1.5

[0107] The mapping relationship between the ordinate of the pixel coordinates of the second bird's-eye view image and the ordinate of the actual world coordinates can also be expressed in the same way, and it is only necessary to replace "X" in the above mapping relationship with "Y".

[0108] After adopting the above piecewise function, compared with before, the pixel accuracy within 20 meters around the vehicle is changed from 0.5 meters / pixel to 0.4 meters / pixel, with higher clarity; the pixel accuracy within the range of 20 to 50 meters is changed from 0.5 meters / pixel to 0.6 meters / pixel, with slightly reduced clarity; the pixel accuracy within the range of 50 to 100 meters is 1.6 meters / pixel, and the perception range is expanded from 100 meters around the vehicle to 200 meters around the vehicle, and the perception distance is greatly improved.

[0109] It should be noted that after the BEV space is divided into multiple areas, the slope of the piecewise function corresponding to each area can be adjusted according to actual conditions, and the embodiment of the present application does not specifically limit the slope of the piecewise function.

[0110] Optionally, the piecewise function can be adjusted according to the actual situation. For example, if it is not necessary to expand the sensing range and only the pixel accuracy of the area near the vehicle needs to be improved, a piecewise function that meets the 200 pixel × 200 pixel BEV space and the 100 meter × 100 meter sensing area can be designed. Figure 8 , the following piecewise function can be designed:

[0111]

[0112] By using the above piecewise function, the pixel accuracy of the area within 10 meters around the vehicle can be reduced to 0.2 meters / pixel, the pixel accuracy of the area between 10 meters and 25 meters can be reduced to 0.3 meters / pixel, and the pixel accuracy of the area between 25 meters and 50 meters can be reduced to 0.75 meters / pixel. After using this piecewise function, the closer the location to the vehicle is in the 100m×100m sensing area, the higher the clarity is, which is convenient for identifying the specific features of objects, such as lane markings, etc. The clarity of the location farther away from the vehicle can also ensure that other objects can be identified without affecting the normal sensing task.

[0113] Of course, the above two specific piecewise functions are only examples, and the embodiments of the present application do not limit the specific form of the piecewise functions.

[0114] Optionally, in the second area, the smaller the pixel distance between the pixel point and the vehicle, the smaller the pixel accuracy at the corresponding pixel point, and the larger the pixel distance between the pixel point and the vehicle, the larger the pixel accuracy at the corresponding pixel point.

[0115] In the third area, the smaller the pixel distance between the pixel point and the vehicle, the smaller the pixel accuracy at the corresponding pixel point, and the larger the pixel distance between the pixel point and the vehicle, the larger the pixel accuracy at the corresponding pixel point.

[0116] It should be noted here that when the piecewise function is used for nonlinear transformation, although the piecewise function is simpler, since the function curve of the piecewise function is not smooth, the transformed second bird's-eye view may undergo a sudden change at the pixel point where the slope of the piecewise function changes, resulting in tiny faults in the image.

[0117] Taking into account the above-mentioned problems that may exist in the piecewise function, in the embodiment of the present application, it is also considered to use a smooth nonlinear transformation function to perform nonlinear transformation, so as to make the transformed second bird's-eye view better.

[0118] Optionally, the nonlinear transformation function is a trigonometric function, an inverse proportional function or an exponential function.

[0119] The slope of the tangent function in the trigonometric function shows a trend of first monotonically decreasing and then monotonically increasing within a certain interval, and can be used to design nonlinear transformation functions.

[0120] For example, taking a BEV space of 200 pixels × 200 pixels and a sensing range of 200 meters × 200 meters as an example, the embodiment of the present application provides a specific tangent function, and its functional relationship is:

[0121] F(x)=100tan[(x-100)×π / 400]

[0122] The function curve of the tangent function is as follows Fig. 9 As shown, after the nonlinear transformation using the tangent function, the mapping relationship between the pixel coordinates of the second bird's-eye view in the BEV space and the actual world coordinates in the current environment is expressed as:

[0123] X_world=100×tan[(X_bev_pix-100)×π / 400]

[0124] Y_world=100×tan[(Y_bev_pix-100)×π / 400]

[0125] Reference Fig. 9 , it can be seen that the BEV plane coordinates of 0 to 200 pixels are mapped to the current scene of -100 meters to 100 meters, that is, the perception range of 200 meters. The intersection of the function curve and the x-axis is the inflection point of the function curve, indicating the pixel point where the vehicle is located. "-" is used to indicate the direction. If the front side of the vehicle is the positive direction of the y-axis and the right side of the vehicle is the positive direction of the x-axis, then "-" can represent the rear or left side of the vehicle. For example, X_world = -100 meters, which represents the area 100 meters to the left of the vehicle.

[0126] The slope of the function curve can reflect the pixel accuracy, that is, the slope of the function curve at a certain point can represent the pixel accuracy of the corresponding pixel point. From the function curve, the slope first gradually decreases and then gradually increases, indicating that the closer the area is to the vehicle, the smaller the pixel accuracy is, the smaller the actual distance corresponding to the unit pixel is, and the higher the resolution is; the farther the area is from the vehicle, the greater the pixel accuracy is, the greater the actual distance corresponding to the unit pixel is, and the lower the resolution is.

[0127] By using the above tangent function, the perception range can be expanded to 200 meters × 200 meters. In addition, the pixel accuracy in the area close to the vehicle is small, which is also convenient for the implementation of perception tasks such as lane line recognition.

[0128] Of course, the coefficients of the above tangent function can also be adjusted and then normalized to adjust the unevenness of the function distribution. For example, the above tangent function is adjusted to:

[0129] F(x)=(100 / 1.63)tan[(x-100)×1.3×3.14159265 / 400]

[0130] The function curve of the tangent function is as follows Fig.10 As shown, compared Fig.10 and Fig. 9 , it can be seen that the closer the distance to the inflection point, Fig.10The smaller the slope of the function curve is, the smaller the pixel accuracy is in the area closer to the vehicle, which is more conducive to analyzing the environment in the area near the vehicle.

[0131] In the embodiment of the present application, the nonlinear transformation function is designed using trigonometric functions, which is not limited to the above-mentioned tangent function. By utilizing the conversion relationship between trigonometric functions, function forms such as sine function and cosine function can be adopted. The embodiment of the present application does not specifically limit the function form of trigonometric functions.

[0132] The nonlinear transformation function can also be designed in the form of an inverse proportional function, and the typical inverse proportional function is 1 / x. From the function curve of the inverse proportional function in the first quadrant, the slope of some intervals is large, the slope of some intervals is small, and the function curve is continuous. Using the inverse proportional function, the function can be adjusted appropriately to the following form:

[0133]

[0134] The function curve of the inverse proportional function in [0, 100) is as follows Fig.11 As shown, the function curve in the interval [100, 200] is as follows Fig.12 As shown in the figure. From the slope of the function curve, it can be seen that the closer the position is to the vehicle, the smaller the pixel accuracy is, and the farther the position is from the vehicle, the larger the pixel accuracy is. After adopting the above inverse proportional function, the perception range can be expanded to 200 meters × 200 meters, and the pixel accuracy of the area close to the vehicle is small, which is convenient for the implementation of perception tasks such as lane line recognition.

[0135] Nonlinear transformation functions can also be designed in the form of exponential functions. The typical exponential function is e x The exponential function also meets the characteristics of the above nonlinear transformation. By adjusting the exponential function, it can be adjusted to the following form:

[0136]

[0137] The function curve of the exponential function in [0, 100) is as follows Fig.13 As shown, the function curve in the interval [100, 200] is as follows Fig.14 As shown in the figure. From the slope of the function curve, it can be seen that the closer the position is to the vehicle, the smaller the pixel accuracy is, and the farther the position is from the vehicle, the larger the pixel accuracy is. After adopting the above inverse proportional function, the perception range can be expanded to 200 meters × 200 meters, and the pixel accuracy of the area close to the vehicle is small, which is convenient for the implementation of perception tasks such as lane line recognition.

[0138] The embodiments of the present application provide specific examples for trigonometric functions, inverse proportional functions, and exponential functions. After understanding the inventive concept of the present application, those skilled in the art can independently design the form of the nonlinear transformation function. The embodiments of the present application do not limit the specific form of the nonlinear transformation function.

[0139] Step 303: The pixel position of each pixel point in the second bird's-eye view is a two-dimensional coordinate. According to a set step size, a vertical coordinate is assigned to the pixel position of each pixel point in the second bird's-eye view in a direction perpendicular to the second bird's-eye view to obtain multiple three-dimensional coordinates.

[0140] After the nonlinear transformation, the pixel position of the second bird's-eye view includes a horizontal coordinate and a vertical coordinate, and the unit is meter. On this basis, the "lift" operation is adopted to assign the vertical coordinate to the pixel position of each pixel point in the second bird's-eye view in the direction perpendicular to the second bird's-eye view, that is, the direction perpendicular to the ground where the vehicle is located in the current scene, according to the set step size, so as to obtain multiple three-dimensional coordinates.

[0141] For example, the coordinate (15,10) is raised in steps of 0.25 to obtain a series of three-dimensional coordinates such as (15,10,0.25), (15,10,0.5), (15,10,0.75), (15,10,1.0), (15,10,1.25), (15,10,1.5), (15,10,1.75), and (15,10,2.0).

[0142] Step 304: Project the multiple three-dimensional coordinates onto a preset camera plane to obtain camera plane pixel positions.

[0143] Projection refers to mapping points in the real environment onto the pixel plane of the camera to obtain the pixel coordinates of the object on the pixel plane.

[0144] In the embodiment of the present application, the multiple three-dimensional coordinates obtained in step 303 are projected onto the camera plane of a 2D camera installed on the vehicle to obtain the camera plane pixel coordinates.

[0145] Taking (15,10,1.5) as an example, projection means mapping the three-dimensional point 15 meters to the left of the vehicle, 10 meters in front of the vehicle, and 1.5 meters high on the vertical axis onto the camera plane preset on the vehicle, such as obtaining the camera plane pixel coordinates (674,328).

[0146] After lifting and projecting the second bird's-eye view, a camera view of the current scene where the vehicle is located can be obtained. This view can be used to display to the driver to help the driver observe the environment around the vehicle. It can also provide data support for tasks such as lane line recognition and obstacle recognition during autonomous driving.

[0147] Step 305: perform feature sampling on the pixel positions of the camera plane to obtain sampling results.

[0148] Feature sampling refers to extracting the data of the pixels around a certain pixel. Taking (674,328) as an example, feature sampling can be to extract the feature data of the pixels within the 20×20 pixel range around the pixel through a neural network. It should be noted that the feature data is not the brightness data of these pixels.

[0149] After feature sampling, we can obtain the two-dimensional features within the range of 20×20 corresponding to the actual position of the three-dimensional point (15, 10, 1.5). This two-dimensional feature can be used for judgment or processing of subsequent perception tasks.

[0150] Step 306: Execute the backend task according to the sampling result to obtain the backend processing result;

[0151] The back-end tasks refer to tasks such as lane line recognition and obstacle recognition.

[0152] Taking lane line recognition as an example, the shape, width and other data of the lane line are judged using the results of feature sampling.

[0153] Step 307: Inversely transform the backend processing result in accordance with a method of performing nonlinear transformation on the pixel positions of different pixel points in the first bird's-eye view to obtain a true processing result in the current scene.

[0154] In the embodiment of the present application, the inverse transformation refers to constructing an inverse function of the nonlinear transformation function used when performing a nonlinear transformation from the first bird's-eye view to the second bird's-eye view, and performing a nonlinear transformation on the result of the backend task.

[0155] Taking the above tangent function F(x) = (100 / 1.63)tan[(x-100)×1.3×3.14159265 / 400] as an example, its inverse function is: F(x) = arctan(x / 100)×400 / 3.14159265+100, and the corresponding function curve is as follows Fig.15 shown.

[0156] In the embodiment of the present application, the inverse transformation is mainly performed to convert the result determined by the back-end task into the real distance in the real scene. For example, in the lane line recognition task, the width of the lane line around the vehicle is identified as A meters. Since the nonlinear transformation is performed in step 302, this value cannot be directly used as the real distance in the current scene. Therefore, the value is substituted into the corresponding inverse function, and the real lane line distance in the current scene is output as B meters.

[0157] It should be noted that, in practice, this inverse transformation process can be automatically learned through machine learning, neural networks and other methods to reduce the workload.

[0158] Fig.16 The flowchart of the BEV paradigm perception task after adopting the embodiment of the present invention is shown in FIG. After adopting the embodiment of the present invention, the BEV paradigm perception task includes: 1) performing nonlinear transformation on the BEV plane coordinates to obtain extended BEV coordinates; 2) lifting the extended BEV plane coordinates to obtain a number of 3D position coordinates; 3) projecting the 3D position coordinates on the camera plane to obtain a number of corresponding camera pixel coordinates; 4) performing feature sampling on the camera pixel coordinates to obtain BEV features; 5) executing backend tasks according to the BEV features and outputting backend processing results; 6) performing inverse transformation on the coordinates in the backend processing results to obtain the real results in the current scene.

[0159] The image processing method of the embodiment of the present application has the advantages of ensuring the pixel accuracy of the area near the vehicle and expanding the maximum range that the vehicle can perceive. At the same time, the processing method is more compatible with the 2D camera commonly installed on the vehicle in terms of performance. The near scene occupies a larger proportion of the image area, and the distant scene occupies a smaller proportion of the image area, which greatly reduces the loss in the BEV space conversion process and improves the efficiency of storage space utilization.

[0160] Fig.17 is a block diagram of an image processing device provided by an embodiment of the present invention, such as Fig.17 As shown, the device may include:

[0161] The acquisition module 401 is used to acquire a first bird's-eye view of the vehicle in the current scene; in the first bird's-eye view, the pixel accuracy of the pixel points at different pixel distances from the vehicle is the same.

[0162] The extraction module 402 is used to perform a nonlinear transformation on the pixel positions of different pixel points in the first bird's-eye view to obtain a second bird's-eye view; in the second bird's-eye view, the average pixel accuracy of the second area is less than the average pixel accuracy of the third area, and the pixel distance between any pixel point in the second area and the vehicle is less than the pixel distance between any pixel point in the third area and the vehicle.

[0163] The fusion module 403 is used to perform image processing on the second bird's-eye view.

[0164] Optionally, within the second area, the pixel precision at each pixel point is the same; within the third area, the pixel precision at each pixel point is the same.

[0165] Optionally, in the second area, the smaller the pixel distance between the pixel point and the vehicle, the smaller the pixel precision at the corresponding pixel point, and the larger the pixel distance between the pixel point and the vehicle, the larger the pixel precision at the corresponding pixel point;

[0166] In the third area, the smaller the pixel distance between the pixel point and the vehicle, the smaller the pixel accuracy at the corresponding pixel point, and the larger the pixel distance between the pixel point and the vehicle, the larger the pixel accuracy at the corresponding pixel point.

[0167] Optionally, the nonlinear transformation module includes:

[0168] The nonlinear transformation submodule is used to perform nonlinear transformation on the pixel positions of different pixel points in the first bird's-eye view according to a preset nonlinear transformation function to obtain a second bird's-eye view.

[0169] Among them, the horizontal coordinate of the function curve of the nonlinear transformation function is the relative pixel distance between the first pixel point in the first bird's-eye view and the vehicle, the vertical coordinate of the function curve of the nonlinear transformation function is the relative actual distance between the actual position corresponding to the first pixel point in the current scene and the vehicle, and the inflection point of the function curve of the nonlinear transformation function is the pixel point where the vehicle is located in the first bird's-eye view.

[0170] Optionally, the nonlinear transformation function is a piecewise function.

[0171] Optionally, the nonlinear transformation function is a trigonometric function, an inverse proportional function or an exponential function.

[0172] Optionally, the processing module includes:

[0173] A lifting module, used for assigning vertical coordinates to the pixel positions of each pixel point in the second bird's-eye view in a direction perpendicular to the second bird's-eye view according to a set step length, so as to obtain a plurality of three-dimensional coordinates; the pixel position of each pixel point in the second bird's-eye view is a two-dimensional coordinate;

[0174] The projection module is used to project the multiple three-dimensional coordinates onto a preset camera plane to obtain the camera plane pixel position.

[0175] Optionally, the processing module further includes:

[0176] The sampling module is used to perform feature sampling on the pixel positions of the camera plane to obtain sampling results.

[0177] Optionally, the processing module further includes:

[0178] A backend task module, used to execute backend tasks according to the sampling results to obtain backend processing results;

[0179] The inverse transformation module is used to perform an inverse transformation on the back-end processing result in a manner of performing a nonlinear transformation on the pixel positions of different pixel points in the first bird's-eye view, so as to obtain a true processing result in the current scene.

[0180] To summarize, an image processing device provided in an embodiment of the present application converts a first bird's-eye view into a second bird's-eye view through a nonlinear transformation method, thereby improving the pixel accuracy of the area near the vehicle, which is beneficial to image recognition. By reducing the pixel accuracy of the area far away from the vehicle, the perception range is improved, and both the accuracy and perception range of the perceived image are taken into account, thereby enhancing the safety of the vehicle's automatic driving process.

[0181] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0182] Fig.18 is a block diagram of an electronic device 600 according to an exemplary embodiment. For example, the electronic device 600 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0183] Reference Fig.18 , the electronic device 600 may include one or more of the following components: a processing component 602 , a memory 604 , a power component 606 , a multimedia component 608 , an audio component 610 , an input / output (I / O) interface 612 , a sensor component 614 , and a communication component 616 .

[0184] The processing component 602 generally controls the overall operation of the electronic device 600, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 602 may include one or more processors 620 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 602 may include one or more modules to facilitate the interaction between the processing component 602 and other components. For example, the processing component 602 may include a multimedia module to facilitate the interaction between the multimedia component 608 and the processing component 602.

[0185] The memory 604 is used to store various types of data to support the operation of the electronic device 600. Examples of such data include instructions for any application or method operating on the electronic device 600, contact data, phone book data, messages, pictures, multimedia, etc. The memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0186] The power supply component 606 provides power to the various components of the electronic device 600. The power supply component 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 600.

[0187] The multimedia component 608 includes a screen that provides an output interface between the electronic device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor can not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 608 includes a front camera and / or a rear camera. When the electronic device 600 is in an operating mode, such as a shooting mode or a multimedia mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0188] The audio component 610 is used to output and / or input audio signals. For example, the audio component 610 includes a microphone (MIC), and when the electronic device 600 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is used to receive an external audio signal. The received audio signal can be further stored in the memory 604 or sent via the communication component 616. In some embodiments, the audio component 610 also includes a speaker for outputting audio signals.

[0189] I / O interface 612 provides an interface between processing component 602 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0190] The sensor assembly 614 includes one or more sensors for providing various aspects of status assessment for the electronic device 600. For example, the sensor assembly 614 can detect the open / closed state of the electronic device 600, the relative positioning of the components, such as the display and keypad of the electronic device 600, and the sensor assembly 614 can also detect the position change of the electronic device 600 or a component of the electronic device 600, the presence or absence of contact between the user and the electronic device 600, the orientation or acceleration / deceleration of the electronic device 600, and the temperature change of the electronic device 600. The sensor assembly 614 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 614 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 614 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0191] The communication component 616 is used to facilitate wired or wireless communication between the electronic device 600 and other devices. The electronic device 600 can access a wireless network based on a communication standard, such as WiFi, a carrier network (such as 2G, 3G, 4G or 5G), or a combination thereof. In an exemplary embodiment, the communication component 616 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 616 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0192] In an exemplary embodiment, the electronic device 600 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to implement an image processing method provided in an embodiment of the present invention.

[0193] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 604 including instructions, and the instructions can be executed by a processor 620 of an electronic device 600 to perform the above method. For example, the non-transitory storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0194] Fig.19is a block diagram of an electronic device 700 according to an exemplary embodiment. For example, the electronic device 700 may be a server. Fig.19 The electronic device 700 includes a processing component 722, which further includes one or more processors, and a memory resource represented by a memory 732 for storing instructions executable by the processing component 722, such as an application. The application stored in the memory 732 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 722 is configured to execute instructions to perform an image processing method provided by an embodiment of the present invention.

[0195] The electronic device 700 may also include a power supply component 726 configured to perform power management of the electronic device 700, a wired or wireless network interface 750 configured to connect the electronic device 700 to a network, and an input / output (I / O) interface 758. The electronic device 700 may operate based on an operating system stored in the memory 732, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™ or the like.

[0196] An embodiment of the present invention further provides a computer program product, including a computer program, and the computer program implements the image processing method when executed by a processor.

[0197] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0198] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0199] The above is a detailed introduction to an image processing method, device, electronic device, computer-readable storage medium and computer program product provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for a person skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. An image processing method, characterized in that: The method comprises the following steps: Acquire a first bird's-eye view of the vehicle in the current scene; in the first bird's-eye view, the pixel accuracy of pixels at different pixel distances from the vehicle is the same; Performing nonlinear transformation on pixel positions of different pixel points in the first bird's-eye view to obtain a second bird's-eye view; in the second bird's-eye view, an average pixel accuracy of the second region is greater than an average pixel accuracy of the third region, and a pixel distance between any pixel point in the second region and the vehicle is less than a pixel distance between any pixel point in the third region and the vehicle; performing image processing on the second bird's-eye view; The step of performing nonlinear transformation on the pixel positions of different pixel points in the first bird's-eye view to obtain the second bird's-eye view includes: According to a preset nonlinear transformation function, nonlinearly transform the pixel positions of different pixel points in the first bird's-eye view to obtain a second bird's-eye view; Among them, the horizontal coordinate of the function curve of the nonlinear transformation function is the relative pixel distance between the first pixel point in the first bird's-eye view and the vehicle, the vertical coordinate of the function curve of the nonlinear transformation function is the relative pixel distance between the pixel position corresponding to the first pixel point in the second bird's-eye view and the vehicle, and the inflection point of the function curve of the nonlinear transformation function is the pixel point where the vehicle is located in the first bird's-eye view.

2. The method according to claim 1, characterized in that: The nonlinear transformation function is a piecewise function.

3. The method according to claim 1, characterized in that: The nonlinear transformation function is one of a trigonometric function, an inverse proportional function or an exponential function.

4. The method according to claim 1, characterized in that: In the second area, the pixel precision at each pixel point is the same; in the third area, the pixel precision at each pixel point is the same.

5. The method according to claim 1, characterized in that: In the second area, the smaller the pixel distance between the pixel point and the vehicle, the greater the pixel precision at the corresponding pixel point, and the larger the pixel distance between the pixel point and the vehicle, the smaller the pixel precision at the corresponding pixel point; In the third area, the smaller the pixel distance between the pixel point and the vehicle, the greater the pixel accuracy at the corresponding pixel point, and the larger the pixel distance between the pixel point and the vehicle, the smaller the pixel accuracy at the corresponding pixel point.

6. The method according to claim 1, characterized in that: The pixel position of each pixel point in the second bird's-eye view is a two-dimensional coordinate, and the image processing of the second bird's-eye view includes: According to the set step length, in a direction perpendicular to the second bird's-eye view, a vertical coordinate is assigned to a pixel position of each pixel point in the second bird's-eye view to obtain a plurality of three-dimensional coordinates; The multiple three-dimensional coordinates are projected onto a preset camera plane to obtain camera plane pixel positions.

7. The method according to claim 6, characterized in that: The performing image processing on the second bird's-eye view further includes: Feature sampling is performed on the pixel position of the camera plane to obtain a sampling result.

8. The method according to claim 7, characterized in that: The performing image processing on the second bird's-eye view further includes: Execute the backend task according to the sampling result to obtain the backend processing result; The back-end processing result is inversely transformed by performing nonlinear transformation on the pixel positions of different pixel points in the first bird's-eye view to obtain a true processing result in the current scene.

9. An image processing device, characterized in that: The device comprises: An acquisition module, used for acquiring a first bird's-eye view of the vehicle in the current scene; in the first bird's-eye view, the pixel accuracy of the pixel points at different pixel distances from the vehicle is the same; a nonlinear transformation module, configured to perform nonlinear transformation on pixel positions of different pixel points in the first bird's-eye view to obtain a second bird's-eye view; in the second bird's-eye view, an average pixel accuracy of the second region is greater than an average pixel accuracy of the third region, and a pixel distance between any pixel point in the second region and the vehicle is less than a pixel distance between any pixel point in the third region and the vehicle; A processing module, used for performing image processing on the second bird's-eye view; Wherein, the nonlinear transformation module comprises: A nonlinear transformation submodule, configured to perform nonlinear transformation on the pixel positions of different pixel points in the first bird's-eye view according to a preset nonlinear transformation function to obtain a second bird's-eye view; Among them, the horizontal coordinate of the function curve of the nonlinear transformation function is the relative pixel distance between the first pixel point in the first bird's-eye view and the vehicle, the vertical coordinate of the function curve of the nonlinear transformation function is the relative actual distance between the actual position corresponding to the first pixel point in the current scene and the vehicle, and the inflection point of the function curve of the nonlinear transformation function is the pixel point where the vehicle is located in the first bird's-eye view.

10. The device according to claim 9, characterized in that: The nonlinear transformation function is a piecewise function.

11. The device according to claim 9, characterized in that: The nonlinear transformation function is one of a trigonometric function, an inverse proportional function or an exponential function.

12. The device according to claim 9, characterized in that: In the second area, the pixel precision at each pixel point is the same; in the third area, the pixel precision at each pixel point is the same.

13. The device according to claim 9, characterized in that: In the second area, the smaller the pixel distance between the pixel point and the vehicle, the greater the pixel precision at the corresponding pixel point, and the larger the pixel distance between the pixel point and the vehicle, the smaller the pixel precision at the corresponding pixel point; In the third area, the smaller the pixel distance between the pixel point and the vehicle, the greater the pixel accuracy at the corresponding pixel point, and the larger the pixel distance between the pixel point and the vehicle, the smaller the pixel accuracy at the corresponding pixel point.

14. The device according to claim 9, characterized in that: The processing module comprises: A lifting module, used for assigning vertical coordinates to the pixel positions of each pixel point in the second bird's-eye view in a direction perpendicular to the second bird's-eye view according to a set step length, so as to obtain a plurality of three-dimensional coordinates; the pixel position of each pixel point in the second bird's-eye view is a two-dimensional coordinate; The projection module is used to project the multiple three-dimensional coordinates onto a preset camera plane to obtain the camera plane pixel position.

15. The device according to claim 14, characterized in that: The processing module also includes: The sampling module is used to perform feature sampling on the pixel positions of the camera plane to obtain sampling results.

16. The device according to claim 15, characterized in that: The processing module also includes: A backend task module, used to execute backend tasks according to the sampling results to obtain backend processing results; The inverse transformation module is used to perform an inverse transformation on the back-end processing result in a manner of performing a nonlinear transformation on the pixel positions of different pixel points in the first bird's-eye view, so as to obtain a true processing result in the current scene.

17. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the method according to any one of claims 1 to 8.

18. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the method as claimed in any one of claims 1 to 8.

Images