Image processing method, device and equipment

Abstract

The invention discloses an image processing method, device and equipment, and is applied to the technical field of image processing. The method comprises the following steps: acquiring a first image acquired by a camera; according to the non-correction area of the first image, a second image is constructed, the second image comprises the non-correction area and a correction area, and the position of the non-correction area in the second image is the same as the position of the non-correction area in the first image; according to the position mapping relation information, the first pixel point position of a first pixel point in the correction area in the first image is determined, and the position mapping relation information comprises the mapping relation between the pixel point position of the pixel point in the camera collection image and the pixel point position in the image constructed based on the camera collection image; and according to the pixel value of the first pixel point position, adjusting the pixel value of the first pixel point in the pixel point position in the correction area to obtain a third image. The method can solve the problem of low correction efficiency of the distortion area of the image.

Description

Technical Field

[0001] This application belongs to the fields of image processing and intelligent driving technology, and in particular relates to an image processing method, apparatus and device. Background Technology

[0002] In the field of intelligent driving, to obtain a wider range of information about the surrounding environment, vehicles are usually equipped with cameras with a large field of view, such as wide-angle lenses and ultra-wide-angle lenses. Although these cameras with a large field of view can effectively expand the imaging range, their optical structure can cause geometric distortion and spatial distortion in the resulting images. The distorted areas refer to the areas where the edges of objects that should be straight appear as curved arcs in the image.

[0003] In related technologies, multi-stage geometric correction methods are typically used to correct distorted regions in order to ensure image quality after imaging. However, this method requires the use of multiple complex mathematical models to correct distorted regions, which increases the correction time and results in low efficiency in correcting distorted regions of the image. Summary of the Invention

[0004] This application provides an image processing method, apparatus, and device that can solve the problem of low efficiency in correcting distorted areas of images.

[0005] In a first aspect, embodiments of this application provide an image processing method, which includes: acquiring a first image captured by a camera; constructing a second image based on an uncorrected region of the first image, the second image including an uncorrected region and a corrected region, the image size of the second image being the same as the image size of the first image, and the position of the uncorrected region in the second image being the same as the position of the uncorrected region in the first image; determining the position of a first pixel in the corrected region in the first image based on position mapping information, the position mapping information including a mapping relationship between the pixel position in the image captured by the camera and the pixel position in the image constructed based on the image captured by the camera; and adjusting the pixel value of the first pixel in the corrected region based on the pixel value of the first pixel position to obtain a third image.

[0006] In some possible implementations of this application, the image processing method further includes: acquiring a first sample image captured by a sample camera, the first sample image including a first sample correction region, the image size of the first sample image being the same as the image size of the first image, and the position of the first sample correction region in the first sample image being the same as the position of the correction region in the first image; acquiring a second sample image based on the first sample correction region and the viewpoint conversion parameters corresponding to the first sample correction region, the first sample correction region corresponding to the uncorrected region in the second sample image; and determining position mapping relationship information based on the position of the first sample pixel in the first sample image and the position of the second sample pixel in the second sample image of the sample pixel in the first sample correction region.

[0007] In some possible implementations of this application, the viewpoint transformation parameters include rotation transformation information and spatial transformation relationship, wherein the spatial transformation relationship is the spatial transformation relationship between the first space corresponding to the first sample image and the second space corresponding to the sample camera; obtaining the second sample image based on the first sample correction area and the viewpoint transformation parameters corresponding to the first sample correction area includes: determining, based on the spatial transformation relationship, a first transformed pixel position set corresponding to the pixel position set of the first sample correction area in the second space; determining, based on the rotation transformation information, a first transformed pixel position set corresponding to the first transformed pixel position set of the first sample correction area in the second space; determining, based on the spatial transformation relationship, a second transformed pixel position set corresponding to the first transformed pixel position set in the first space; and generating the second sample image based on the second transformed pixel position set.

[0008] In some possible implementations of this application, determining position mapping information based on the first sample pixel position in the first sample image and the second sample pixel position in the second sample image of the sample pixel in the first sample correction region includes: determining the third sample pixel position in the third sample image of the sample pixel in the first sample correction region based on the second sample pixel position in the second sample image of the sample pixel in the first sample correction region and perspective transformation parameters, wherein the third sample image is a distortion-free reference image corresponding to the first sample image; and determining position mapping information based on the first sample pixel position in the first sample image and the third sample pixel position in the third sample image of the sample pixel in the first sample correction region.

[0009] In some possible implementations of the embodiments of this application, constructing a second image based on the uncorrected area of ​​the first image includes: creating a blank image with the same image size as the first image; filling the corresponding positions of the uncorrected areas in the first image with the pixel values ​​of the uncorrected areas in the first image to obtain the second image.

[0010] In some possible implementations of this application, adjusting the pixel value of the first pixel at the position of the first pixel in the correction area according to the pixel value of the first pixel at the first pixel position to obtain a third image includes: filling the pixel position of the first pixel in the correction area according to the pixel value of the first pixel at the first pixel position using a preset pixel filling method to obtain a third image; wherein, the preset pixel filling method includes at least one of the following: bilinear interpolation and hardware texture interpolation.

[0011] In some possible implementations of the embodiments of this application, the camera is installed in the vehicle; the image processing method further includes: determining the driving scene in which the vehicle is located based on the first image; and determining the reference correction area associated with the driving scene as the correction area based on the association between the reference driving scene and the reference correction area.

[0012] Secondly, embodiments of this application provide an image processing apparatus, comprising: a first acquisition module for acquiring a first image captured by a camera; a first construction module for constructing a second image based on an uncorrected region of the first image, the second image including an uncorrected region and a corrected region, the image size of the second image being the same as the image size of the first image, and the position of the uncorrected region in the second image being the same as the position of the uncorrected region in the first image; a first determination module for determining the position of a first pixel in the corrected region in the first image based on position mapping relationship information, the position mapping relationship information including a mapping relationship between the pixel position in the image captured by the camera and the pixel position in the image constructed based on the image captured by the camera; and a first adjustment module for adjusting the pixel value of the first pixel in the corrected region based on the pixel value of the first pixel position to obtain a third image.

[0013] Thirdly, embodiments of this application provide a computer device, the computer device including: a processor and a memory storing computer program instructions; the processor executes the computer program instructions to implement the image processing method as described in any of the first aspects.

[0014] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the image processing method as described in any of the first aspects.

[0015] Fifthly, embodiments of this application provide a computer program product, which includes a computer program or instructions that, when executed by a processor, implement the image processing method as described in any of the first aspects.

[0016] The image processing method, apparatus, computer device, computer-readable storage medium, and computer program product of this application embodiments directly construct a second image with the same image size as the first image and consistent with the position of the non-corrected region. This allows the image correction process of the first image to directly determine the pixel position of the pixel in the first image within the corrected region using position mapping information. This eliminates the need to process the entire first image and avoids the process of establishing a distortion model and multiple iterative optimizations found in related technologies. By adjusting pixel values ​​based on position mapping information and remapping pixels only in the corrected region, targeted processing of the distorted region is achieved. This simplifies the correction process, reduces the correction time for the distorted region, significantly improves the correction efficiency of the distorted region, and simultaneously ensures the image quality of the corrected third image. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating some embodiments of the image processing method provided in this application is shown; Figure 2 The diagram illustrates a flowchart of a method for determining position mapping information in an image processing method provided in some embodiments of this application. Figure 3 A flowchart illustrating a specific implementation of step 160 provided in some embodiments of this application is shown; Figure 4 A flowchart illustrating a specific implementation of step 170 provided in some embodiments of this application is shown; Figure 5 The present application provides schematic diagrams of the structure of an image processing apparatus according to some embodiments. Figure 6 A schematic diagram of the structure of a computer device provided in some embodiments of this application is shown. Detailed Implementation

[0019] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0020] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0021] It should be noted that the acquisition, storage, use, and processing of data in this application embodiment all comply with the relevant provisions of national laws and regulations.

[0022] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0023] Before describing the technical solutions provided in the embodiments of this application, in order to facilitate understanding of the embodiments of this application, this application first provides a detailed description of the relevant technologies involved: In intelligent driving scenarios such as autonomous driving, parking, surround view, and blind spot detection, vehicles typically deploy large field-of-view (FOV) cameras—for example, fisheye lenses with a FOV ≥ 180°—to achieve the widest possible environmental perception range. While the imaging model of fisheye lenses allows them to capture ultra-wide angles of view, it also introduces severe nonlinear geometric distortion. Specifically, this manifests as significant spatial stretching and distortion in image edge regions; object edges that should be straight appear as curved arcs in the image. This distortion directly disrupts the scene's geometry, negatively impacting the geometric stability of perception tasks such as object detection, lane or curb recognition, and free space estimation.

[0024] To address the aforementioned issues, distortion correction of fisheye images is necessary, mapping them from a nonlinear imaging model to an approximately linear imaging plane. Related technologies typically employ multi-stage geometric correction methods to correct distorted regions. For example, preliminary distortion removal is first performed based on models such as isometric or isometric projection, converting the original fisheye image into an intermediate image with approximate perspective. Subsequently, perspective homography transformation is applied to the target plane to achieve geometric alignment between the image and the vehicle coordinate system. In the entire image processing chain, the original image usually undergoes 2 to 3 resampling operations. However, each stage of interpolation smooths high-frequency details in the image, and the sequential 2 to 3 resampling operations are particularly effective in the already severely stretched edge regions, easily causing texture blurring and jagged lines, leading to a decrease in image sharpness. Furthermore, multi-stage processing means multiple read / write operations and cache cycles for the entire image, increasing end-to-end processing latency and system power consumption, and increasing the correction time for distorted regions, resulting in low efficiency in image distortion correction.

[0025] To address the problems in the aforementioned related technologies, embodiments of this application provide an image processing method, apparatus, and device. The following description, in conjunction with the appendix, illustrates these methods. Figure 1 To be continued Figure 6 The image processing method provided in this application will be described in detail through specific embodiments and application scenarios.

[0026] Figure 1 A schematic flowchart of an image processing method provided by some embodiments of this application is shown. For example... Figure 1 As shown, the image processing method may include steps 110 to 140.

[0027] Step 110: Acquire a first image captured by the camera; Step 120: Construct a second image based on the uncorrected area of ​​the first image. The second image includes an uncorrected area and a corrected area. The image size of the second image is the same as that of the first image. The position of the uncorrected area in the second image is the same as its position in the first image; Step 130: Determine the position of the first pixel in the first image within the corrected area based on position mapping information. The position mapping information includes the mapping relationship between the pixel position in the camera-captured image and the pixel position in the image constructed based on the camera-captured image; Step 140: Adjust the pixel value of the first pixel in the corrected area based on the pixel value of the first pixel position to obtain a third image.

[0028] Therefore, by directly constructing a second image with the same image size as the first image and the same position of the non-corrected region, the image correction process for the first image only requires directly determining the pixel position of the pixel in the first image within the corrected region using the position mapping information. This eliminates the need to process the entire first image and avoids the process of establishing a distortion model and multiple iterative optimizations found in related technologies. This pixel value adjustment based on position mapping information, which only remaps pixels in the corrected region, achieves targeted processing of the distorted region. By simplifying the correction process and reducing the correction time for the distorted region, it significantly improves the correction efficiency of the distorted region while ensuring the image quality of the corrected third image.

[0029] The steps described above are explained in detail below.

[0030] First, regarding step 110, the camera in this embodiment has a wide field-of-view lens with a field of view greater than or equal to 60 degrees. Specifically, the camera may include, but is not limited to, at least one of the following: a wide-angle lens with a field of view between 60 and 90 degrees, an ultra-wide-angle lens with a field of view between 100 and 180 degrees, or a fisheye lens with a field of view greater than or equal to 180 degrees. The first image captured by the camera refers to the raw image data captured by the wide field-of-view lens on the device (such as a vehicle). The first image includes a corrected area composed of distorted areas at the image edges and an uncorrected area composed of undistorted areas at the image center. It is understood that the size of the corrected area in the first image is determined based on the camera's field of view and optical characteristics, as well as user or application requirements; that is, the user can adjust the boundaries defining the distorted and undistorted areas according to the requirements for the final image quality.

[0031] In this context, the uncorrected region refers to the image area in the first image that does not require geometric distortion correction or content modification. For example, in images captured by wide-angle or fisheye cameras, the distortion in the central part of the image is usually small and can be classified as an uncorrected region, the image content of which can be directly preserved. The corrected region refers to the part of the first image located at the edge where there is obvious geometric distortion.

[0032] In one example, the step of acquiring the first image captured by the camera can be triggered by user interaction, such as viewing playback of video recorded by the camera or browsing images already captured by the camera. This saves on the continuous computational overhead of image processing methods, as the target image captured by the camera is only corrected when the user needs to view it.

[0033] In another example, the vehicle's onboard system is in a mode requiring real-time display. Each screen refresh or the arrival of a new image frame acts as a trigger signal, directly and in real-time retrieving the latest frame of image data from the camera hardware's image data buffer and using it as the current first image. This enables low-latency real-time visual feedback, providing the user with a distortion-free real-time view.

[0034] Secondly, regarding step 120, the image size refers to the pixel width and pixel height of the image. For example, an image with a resolution of 1920×1080 has a width of 1920 pixels and a height of 1080 pixels. Clearly defining the correction area that needs correction and the non-correction area that does not require processing can effectively avoid performing invalid operations on the non-correction area, thus improving image processing efficiency. At the same time, ensuring that the size of the second image is consistent with the first image and the position of the non-correction area can ensure the continuity of the overall image structure and the accuracy of subsequent pixel mapping.

[0035] In some embodiments of this application, step 120 may include: creating a blank image with the same image size as the first image; filling the corresponding positions of the blank image with the pixel values ​​of the uncorrected regions in the first image according to their positions in the first image, to obtain the second image.

[0036] The blank image refers to an image in which all pixels have default values ​​in the initial state (such as black, white or transparent with all RGB channels at 0). The pixel width and height of the blank image are exactly the same as those of the first image.

[0037] For example, firstly, a blank image with the same size as the first image is created, and the coordinate range of the uncorrected region in the first image is determined. Secondly, through a pixel copying operation, the pixel value of each pixel in the uncorrected region of the first image is written to the corresponding pixel position in the blank image. Subsequently, after completing the pixel filling of the uncorrected region, the area in the blank image filled with pixel values ​​is the uncorrected region, and the area without filled pixel values ​​is the corrected region, thus obtaining the second image.

[0038] Therefore, by creating a blank image with the same image size as the first image, it can be ensured that the overall structure of the second image is completely matched with that of the first image, avoiding image correction errors caused by image size deviation. In the process of determining the second image, for the uncorrected areas in the second image, the pixel values ​​of the uncorrected areas in the first image can be directly reused, which can avoid processing image areas that do not need to be processed, preserve the effective information of the original image to the greatest extent, and ensure the integrity of the content of the uncorrected areas.

[0039] Furthermore, regarding step 130, the position mapping information is a look-up table (LUT). This look-up table defines the mapping relationship from the pixel position in the correction area of ​​the second image to the pixel position (source position) in the first image. This position mapping information can be pre-calculated using a camera calibration algorithm based on the camera's intrinsic parameters (focal length, principal point coordinates) and extrinsic parameters (mounting position, attitude) to create a pixel position mapping table between the original image and the target image. The first pixel refers to any pixel in the correction area of ​​the second image whose original position needs to be determined. The position of the first pixel refers to the specific coordinates, in pixels, of the first pixel in the first image.

[0040] For example, each first pixel in the correction area of ​​the second image is traversed; for each first pixel, a preset position mapping relationship is queried to determine which pixel position in the first image each first pixel should obtain a pixel value from.

[0041] In some embodiments of this application, before performing step 130, the image processing method may further include a step of determining position mapping relationship information. Based on this, such as... Figure 2 As shown, the above image processing method may further include steps 150 to 170.

[0042] Step 150: Obtain a first sample image captured by the sample camera. The first sample image includes a first sample correction region. The image size of the first sample image is the same as that of the first image. The position of the first sample correction region in the first sample image is the same as that of the correction region in the first image.

[0043] The sample camera refers to the camera used to acquire sample images. Its hardware parameters, such as focal length, sensor size, and installation location, are identical to those of the camera that actually acquired the first image. It is used to establish a general mapping rule through sample data. The first sample image refers to the original sample image acquired by the sample camera, serving as sample data for constructing the positional mapping relationship. Its image size is the same as the first image. The first sample correction region refers to the area in the first sample image that requires viewpoint transformation or pixel mapping. The position of the first sample correction region in the first sample image completely corresponds to the position of the correction region in the first image.

[0044] In one example, the sample camera can be an actual vehicle-mounted camera, i.e., the camera mentioned in step 110 above. In this case, the sample camera is the hardware that actually acquires the first image, and its lens distortion, sensor size, focal length, installation angle, and other parameters are all real-world parameters of the camera, eliminating the need for additional parameter matching calibration. Specifically, the sample camera is directly controlled to acquire sample images, for example, by photographing a preset calibration board or fixed scene when the vehicle is stationary, generating the first sample image. This eliminates errors caused by differences in parameters between the sample camera and the actual camera at the source, ensuring, to the greatest extent possible, that the subsequent position mapping relationship perfectly matches the actual vehicle-mounted scene.

[0045] In another example, the sample camera can be the same model or specification camera as the one actually installed on the vehicle in step 110, with parameters completely identical. Parameters may include, but are not limited to: lens type, resolution, focal length, sensor model, intrinsic parameters (focal length, principal point coordinates), and extrinsic parameters (installation position, attitude). For example, the camera parameters in step 110 can be recorded first, and then cameras with the same parameters can be selected as sample cameras. In this way, parameter consistency ensures the consistency between sample data and actual data, avoiding mapping deviations caused by parameter differences.

[0046] Step 160: Obtain the second sample image based on the first sample correction area and the viewpoint conversion parameters corresponding to the first sample correction area. The first sample correction area corresponds to the uncorrected area in the second sample image.

[0047] The perspective transformation parameters refer to a set of parameters used to describe the perspective transformation of an image. These parameters describe the geometric correspondence between image planes from two different perspectives. Specifically, they may include a rotation transformation matrix, which can be pre-calculated through camera calibration or a geometric transformation model. This matrix is ​​used to transform the first sample correction area into an image area under the target perspective. The second sample image refers to the new, perspective-corrected image obtained by applying the perspective transformation parameters to the first sample correction area of ​​the first sample image. After perspective transformation, the correction area in the first sample image corresponds to the uncorrected area in the second sample image, i.e., it appears as a normal image without perspective distortion.

[0048] For example, the first sample correction region in the first sample image is transformed according to the viewpoint transformation parameters. Other regions in the first sample image except for the first sample correction region are kept unchanged, and the first sample correction region after viewpoint transformation is used as the uncorrected region of the second sample image. The two are combined to form a second sample image with the same image size as the first sample image.

[0049] Step 170: Determine the position mapping relationship information based on the position of the first sample pixel in the first sample image and the position of the second sample pixel in the second sample image of the sample pixel in the first sample correction region.

[0050] Here, a sample pixel refers to any pixel within the first sample correction area. The position of the first sample pixel refers to its coordinates in the first sample image. The position of the second sample pixel refers to the corresponding coordinates of the same sample pixel in the second sample image after viewpoint transformation. The position mapping information refers to a general mapping table obtained by fitting the position mapping relationship of the sample pixels, used to describe the position mapping relationship between pixels in the original image and pixels in the target image.

[0051] For example, all sample pixels in the first sample correction region are traversed, and the first sample pixel position (x1, y1) of each sample pixel in the first sample image and its corresponding second sample pixel position (x2, y2) in the second sample image are recorded, forming multiple pairs of position data. The position correspondence of all sample pixels is stored as a lookup table, directly recording the mapping relationship between (x1, y1) and (x2, y2).

[0052] Therefore, this sample-based, offline computation method can solidify complex viewpoint correction relationships, which are specific to certain cameras and scenes, into mapping relationships. In practical applications, there is no need for time-consuming real-time geometric calculations. Image correction can be quickly completed by querying the pre-generated position mapping information, which greatly improves image processing efficiency and ensures the real-time speed of image correction.

[0053] In some embodiments of this application, the viewpoint transformation parameters may include rotation transformation information and spatial transformation relationships. The rotation transformation information defines a rotation transformation matrix about the Y-axis, used to simulate the rotation of the sample camera in physical space. For example, if the first sample image includes a left sample correction region located on the left side of the first sample image and a right sample correction region located on the right side of the first sample image, for the left sample correction region, in order to correct its content (such as a vehicle on the left) to be presented in the center of the image, the rotation transformation corresponds to simulating rotating the sample camera to the right by a preset angle (such as 60°) about its Y-axis. For the right sample correction region, in order to correct its content (such as a vehicle on the right) to be presented in the center of the image, the rotation transformation corresponds to simulating rotating the sample camera to the left by a preset angle (such as 60°) about its Y-axis. The spatial transformation relationship is the spatial transformation relationship between the first space corresponding to the first sample image and the second space corresponding to the sample camera; the first space refers to the image pixel coordinate system, which is in pixels, and the coordinates (i, j) can represent the row and column positions of the pixel in the image. The second space refers to the camera coordinate system corresponding to the sample camera. The second space is a three-dimensional space used to describe the three-dimensional position and orientation of a pixel relative to the camera's optical center. Based on this, such as... Figure 3 As shown, step 160 above may also include steps 1601 to 1604.

[0054] Step 1601: Based on the spatial transformation relationship, determine the first transformed pixel position set corresponding to the pixel position set of the first sample correction region in the second space.

[0055] For example, firstly, the pixel coordinates (i, j) in the pixel position set of the first sample correction region are normalized to the range [-0.5, 0.5]; then, the normalized relative coordinates are multiplied by 2 to transform to the interval [-1, 1], and the initial vector in the camera coordinate system is calculated in combination with the focal length; finally, the first transformed pixel position set (x, j) is obtained by calculating the magnitude of the vector and normalizing it. cam y cam , z cam In this way, the coordinate mapping from the image pixel space to the camera's three-dimensional space is completed.

[0056] Step 1602: Based on the rotation transformation information, determine the set of first transformed pixel positions in the second space that corresponds to the set of first transformed pixel positions in the first sample correction region.

[0057] For example, a matrix multiplication operation is performed between the rotation matrix corresponding to the rotation transformation information and the set of first transformed pixel positions to obtain a new set of 3D coordinates after rotation, which is also the set of first transformed pixel positions. Specifically, the first transformed pixel positions (x, y, ...) in the set of first transformed pixel positions...world y world , z world ) can be expressed by the following formula (1): Among them, cos rot The cosine of the rotation angle; sin rot is the cosine of the rotation angle; rot is the rotation angle, representing the angle of rotation around the Y-axis.

[0058] Step 1603: Based on the spatial transformation relationship, determine the second set of transformed pixel positions in the first space that corresponds to the first set of transformed pixel positions.

[0059] For example, the first set of transformed pixel positions after rotation is remapped back to the image pixel coordinate system using spatial transformation relationships to obtain a new set of two-dimensional pixel coordinates, which is the second set of transformed pixel positions.

[0060] For example, for a fisheye image, first (x world y world , z world Transform to a spherical coordinate system to obtain the angular information of the direction. Specifically, the angle between the direction vector and the positive direction of the Z-axis can be calculated using the following formula (2), which is the polar angle φ.

[0061] Furthermore, the angle between the projection of the direction vector onto the XY plane and the positive X-axis can be calculated using the following formula (3), i.e., the azimuth angle. .

[0062] Then, using the projection model corresponding to the sample camera, the spherical angular coordinates ( The coordinates (x, y) are mapped to the pixel coordinates of the image plane and then mapped back to the coordinates (x, y) of the original fisheye image through the fisheye projection model.

[0063] The equivalent focal length f of a fisheye projection fisheye It can be determined by the following formula (4).

[0064] Where fov is the maximum field of view designed for this fisheye lens, and radius is the distance from the image center to the farthest effective pixel.

[0065] The distance r of a pixel from the center point in a fisheye image can be determined by the following formula (5).

[0066] The x-coordinate of the second transformed pixel position corresponding to the first transformed pixel position in the first transformed pixel position set in the first space can be determined by the following formula (6).

[0067] Where, x center Let x be the x-coordinate of the optical center of the fisheye image.

[0068] The y-coordinate of the second transformed pixel position in the first space corresponding to the first transformed pixel position in the first transformed pixel position set can be determined by the following formula (7).

[0069] Among them, y center y is the coordinate of the optical center of the fisheye image.

[0070] Step 1604: Generate a second sample image based on the second set of transformed pixel locations.

[0071] For example, after obtaining the set of second transformed pixel locations, the pixel values ​​of these second transformed pixel locations can be determined from the first sample image by a pixel value sampling algorithm (e.g., nearest neighbor interpolation or bilinear interpolation), and then filled into the target pixel locations of the second sample image to generate a complete second sample image with the viewpoint corrected.

[0072] Therefore, by projecting the pixel positions of the first sample correction area into three-dimensional space, performing a physically meaningful rotation simulation, and then projecting it back into image space, a change in the viewing angle can be achieved, rather than simply removing distortion. This solves the problem of infinite magnification in edge regions in related image correction methods, improving the correction effect on distorted areas of the image.

[0073] In some embodiments of this application, such as Figure 4 As shown, step 170 above may specifically include steps 1701 and 1702.

[0074] Step 1701: Based on the position of the sample pixel in the first sample correction region in the second sample image and the perspective transformation parameters, determine the position of the third sample pixel in the third sample image. The third sample image is a distortion-free reference image corresponding to the first sample image.

[0075] The third sample image is a reference image, created through a mesh obtained under strict calibration or an ideal perspective view generated by 3D rendering, ensuring it is completely distortion-free and has correct perspective. The perspective transformation parameters include a 3×3 homography matrix representing the projection mapping between the two 2D image planes, used for geometric alignment. For example, the perspective transformation parameters can be calculated by image registration of the second and third sample images. The third sample pixel position is the corresponding pixel position of the sample pixel in the third sample image, representing the final reference position obtained after viewpoint transformation and perspective correction.

[0076] For example, using perspective transformation parameters, coordinate transformation is performed on the position of each second sample pixel, mapping the sample pixel from the coordinate system of the second sample image to the coordinate system of the third sample image, and calculating the precise position of each sample pixel in the third sample image, i.e., the position of the third sample pixel.

[0077] Step 1702: Determine the position mapping relationship information based on the position of the first sample pixel in the first sample image and the position of the third sample pixel in the third sample image of the sample pixel in the first sample correction region.

[0078] Among them, the position mapping relationship information represents the mapping relationship between (x1, y1) and (x3, y3), that is, the mapping rule between the pixel position of the original distorted image and the pixel position of the undistorted reference image.

[0079] For example, all sample pixels within the first sample correction area are traversed, and the mapping relationship between each group of (x1, y1) and (x3, y3) is recorded to form a dataset. All mapping relationships between (x1, y1) and (x3, y3) are stored as a lookup table.

[0080] Therefore, the third sample image, serving as a distortion-free reference, provides an ideal target for pixel positions, avoiding the slight distortions that may remain if relying solely on viewpoint transformation, and improving the accuracy of position mapping. This method, which combines viewpoint transformation with perspective correction to ultimately form a direct mapping from the original distorted image to the distortion-free reference image, retains the advantage of viewpoint transformation in avoiding edge magnification, while ensuring the standardization of the correction results through the reference image. This effectively improves the correction efficiency for distorted areas of the image while guaranteeing the quality of image correction.

[0081] In some embodiments of this application, before performing step 170 above, the method further includes determining the position of the second sample pixel in the second sample image. Based on this, the above image processing method may further include: determining the position of the second sample pixel in the second sample image based on the position of the first sample pixel in the first sample image and the viewpoint conversion parameter corresponding to the first sample correction area.

[0082] For example, for each sample pixel within the first sample correction area, the coordinates of each sample pixel are used as input parameters to a mathematical function. This function is substituted into the rotation transformation matrix defined by the viewpoint transformation parameters to calculate and output the new pixel position of that sample pixel in the second sample image, which is the second sample pixel position. This calculation process is performed one by one for each sample pixel within the first sample correction area.

[0083] Therefore, by obtaining the position of the second sample pixel through coordinate transformation based on the viewpoint transformation parameters, the final position mapping relationship is ensured to be accurate, unique and reproducible, effectively avoiding the errors and uncertainties that may be brought about by image feature matching.

[0084] Then, in step 140, the pixel value refers to the color or brightness information corresponding to a pixel. For example, in a grayscale image, it can be represented by a value of 0-255 (0 for black, 255 for white); in a color image, it can be represented by RGB three-channel values ​​(e.g., each channel value in (R, G, B) is 0-255). The third image refers to the final target image obtained after adjusting the pixel values ​​of the corrected area. The uncorrected area of ​​the third image is consistent with the first image, and the corrected area can be obtained by correcting the pixel values ​​of the corrected area of ​​the first image.

[0085] In some embodiments of this application, step 140 may specifically include: filling the pixel position of the first pixel in the correction area according to the pixel value of the first pixel at the first pixel position using a preset pixel filling method to obtain a third image; wherein the preset pixel filling method includes at least one of the following: bilinear interpolation and hardware texture interpolation.

[0086] Among them, the preset pixel fill method refers to a pre-defined algorithm used to calculate and assign the pixel value of the corresponding position in the first image to the pixel point in the correction area of ​​the second image, so that the pixel value in the correction area is more in line with the actual scene requirements and reduces distortion. Bilinear interpolation is a pixel interpolation algorithm that determines the pixel value of the target pixel by calculating the weighted average of the four nearest neighbor pixels around the target pixel. Hardware texture interpolation refers to an interpolation method that relies on hardware (such as the texture processing unit of the GPU) to implement the interpolation method. It accelerates the calculation of the pixel value of the target pixel through hardware and can be used in graphics rendering or real-time image processing scenarios.

[0087] In one example, first determine the four integer coordinate neighboring points around the first pixel position, and read the pixel values ​​of these four integer coordinate neighboring points in the first image; then calculate the horizontal and vertical distances from the first pixel to the four integer coordinate neighboring points, and assign weights according to the distances; then perform a weighted summation of the pixel values ​​of the four integer coordinate neighboring points to obtain the fill pixel value of the first pixel, and assign it to the pixel position corresponding to the correction area.

[0088] In another example, the first image is sent as a texture image to the GPU's texture memory; an instruction is sent to the GPU to specify the texture coordinates of the first pixel in the correction area, i.e., the position of the first pixel; the GPU automatically calculates the target pixel value through its built-in interpolation circuit (such as bilinear interpolation logic or trilinear interpolation logic) and writes it directly to the corresponding pixel position in the correction area.

[0089] Therefore, bilinear interpolation avoids problems such as jagged edges and color blocks caused by simple pixel duplication, resulting in a more natural image transition and superior visual effect in the corrected area. Hardware texture interpolation utilizes dedicated hardware units for parallel computation, significantly reducing pixel filling time compared to software-implemented interpolation algorithms. Both pixel filling methods are calculated based on the original pixel information of the first image, ensuring the correlation between the pixel values ​​in the corrected area and the original scene, reducing image distortion caused by missing or incorrectly filled pixels, and improving the overall realism of the third image.

[0090] In some embodiments of this application, the position mapping information generated in step 130 is a pixel position mapping table that completely corresponds to the pixels in the correction area. Thus, during the pixel position determination process in step 130, it is only necessary to traverse the pixel position of each pixel in the pixel position mapping table in the first image to complete the pixel position mapping, thereby effectively improving the efficiency and real-time performance of image correction.

[0091] In some embodiments of this application, after performing step 140, the image processing method may further include an incremental update step of the correction region and the position mapping relationship information. Based on this, the image processing method may further include: determining a first target region in the third image based on the third image and the first image, wherein the first target region is a region in the first image that has distortion but has not been corrected in the current image processing; merging the position mapping relationship information and the position mapping relationship information corresponding to the first target region to obtain merged position mapping relationship information; updating the sum of the regions of the first target region and the correction region to the correction region; and updating the merged position mapping relationship information to the position mapping relationship information.

[0092] The first target region refers to the region discovered in the third image after this correction process, which originally had distortion in the first image but was not covered by the current correction region. Merging the positional mapping information corresponding to the first target region with the existing positional mapping information means integrating the positional mapping information corresponding to the newly discovered first target region with the existing positional mapping information to form a new positional mapping information with a wider coverage.

[0093] For example, the presence of a first target region in the third image can be determined by comparing the straightness of local features between the third and first images, or by using a distortion assessment algorithm. For instance, if a straight line in a certain image region is still curved in the third image, that image region can be identified as the first target region. The geometric extent of the correction region is then updated to the union of the original correction region and the first target region, and the position mapping information corresponding to the first target region is merged with the existing position mapping information. Subsequently, the correction region in step 120 is updated to the newly determined correction region including the first target region, and the position mapping information in step 130 is updated to the merged position mapping information. Then, steps 120 to 140 are re-executed.

[0094] In some other embodiments of this application, after performing step 140, the image processing method may further include the steps of deleting and correcting the region and updating the position mapping relationship information. Based on this, the image processing method may further include: determining a second target region in the third image based on the third image and the first image, wherein the second target region is a region that does not have distortion in the first image but is corrected in the current image processing; deleting the position mapping relationship entry corresponding to the second target region in the position mapping relationship information to obtain the adjusted position mapping relationship information; deleting the corrected region in the first target region to obtain the deleted corrected region and updating it as the corrected region; and updating the adjusted position mapping relationship information as the position mapping relationship information.

[0095] The second target region refers to the area found in the third image after this correction process. This area did not show significant distortion in the first image, but was incorrectly subjected to viewpoint transformation by the current correction process. This is because the correction area was defined too large, covering normal areas that should not have been processed. A position mapping relationship entry refers to each input pixel position-output pixel position mapping relationship in the position mapping relationship information based on the lookup table; each entry can be called a position mapping relationship.

[0096] For example, referring to the first target region identification method described above, if a region that originally appeared normally in the first image now shows an unnatural deformation in the third image, that region can be identified as the second target region. After determining the second target region, all position mapping relationship entries whose output pixel positions fall within the second target region are located and removed from the position mapping relationship information. Subsequently, the correction region in step 120 is updated to the newly determined correction region obtained by subtracting the second target region from the original correction region, and the position mapping relationship information in step 130 is updated to the position mapping relationship information of the deleted position mapping relationship entries. Then, steps 120 to 140 are executed again.

[0097] This effectively prevents unnecessary and potentially side-effect-producing processing of normal areas that do not require correction, ensuring the overall quality of the final image. Furthermore, by reducing unnecessary correction areas, the number of pixels that need to be traversed and processed subsequently is reduced, thereby further improving the operating efficiency of the correction algorithm.

[0098] In some embodiments of this application, the camera is installed in a vehicle; the above image processing method may further include: determining the driving scene in which the vehicle is located based on the first image; and determining the reference correction area associated with the driving scene as the correction area based on the correlation between the reference driving scene and the reference correction area.

[0099] Here, "driving scenario" refers to the external environmental conditions and / or the vehicle's own operating state. "Reference driving scenario" refers to a predefined series of driving situations requiring image correction, such as, but not limited to: left turn, right turn, reversing, left turn on a highway ramp, and overtaking on a highway ramp. "Reference correction area" is an image region pre-bound to the reference driving scenario and requiring correction. For example, the reference correction area associated with a left turn scenario is the left side of the image, used to eliminate distortion in the left front blind spot; the reference correction area associated with a reversing scenario is a wide-angle area behind the image, used to obtain a distortion-free view behind the vehicle.

[0100] In one example, the driving scenario in which the vehicle is located can be determined through real-time scene recognition of the first image captured by the camera. For example, a deep learning model (such as a convolutional neural network CNN) can be used to perform semantic classification on the first image to determine the driving scenario in which the vehicle is located.

[0101] In another example, the driving scenario in which the vehicle is located can be determined logically based on signals provided by the vehicle's sensors corresponding to the time window of the first image acquisition. For example, if the turn signal is on to the left and the steering wheel angle is greater than a certain threshold, the driving scenario can be determined to be a left turn.

[0102] Subsequently, after determining the driving scenario in which the vehicle is located, the identified current driving scenario is compared with the list of reference driving scenarios corresponding to the association between reference driving scenarios and reference correction areas. Once a match is found, the reference correction area corresponding to the current driving scenario is automatically queried and activated based on the association between the reference driving scenario and the reference correction area, and this reference correction area is used as the correction area that needs to be processed now.

[0103] Therefore, by perceiving the scene of vehicle driving, the correction of distortion areas in a fixed large area of ​​the entire image can be transformed into correction of only the most critical and smaller distortion areas in the current driving scene. This significantly reduces the number of pixels that need to be traversed and processed, reduces the computational load, reduces the time for image correction, and can provide users with the optimal corrected image of the corresponding correction area adapted to the current driving scene more quickly.

[0104] The entire process described above does not require manual intervention from the driver to select a correction mode. It can autonomously and dynamically adjust the visual assistance strategy according to changes in the driving environment, effectively improving the user experience.

[0105] Based on the image processing method provided in the above embodiments, this application also provides specific implementations of the image processing apparatus. Please refer to the following embodiments.

[0106] See Figure 5 The image processing apparatus 200 provided in this application embodiment includes: a first acquisition module 201, a first construction module 202, a first determination module 203, and a first adjustment module 204.

[0107] The first acquisition module 201 is used to acquire a first image captured by a camera; the first construction module 202 is used to construct a second image based on the uncorrected area of ​​the first image, the second image including an uncorrected area and a corrected area, the image size of the second image being the same as the image size of the first image, and the position of the uncorrected area in the second image being the same as the position of the uncorrected area in the first image; the first determination module 203 is used to determine the position of the first pixel in the first image of the corrected area based on position mapping relationship information, the position mapping relationship information including the pixel position in the image captured by the camera and the pixel position in the image constructed based on the image captured by the camera; the first adjustment module 204 is used to adjust the pixel value of the first pixel in the corrected area based on the pixel value of the first pixel position to obtain a third image.

[0108] Therefore, by directly constructing a second image with the same image size and the same non-corrected region position as the first image through the first construction module 202, during the image correction process of the first image acquired by the first acquisition module 201, it is only necessary to directly determine the pixel position of the pixel in the first image within the first image using the position mapping relationship information through the first determination module 203. This eliminates the need to process the entire first image and avoids the process of establishing a distortion model and multiple iterative optimizations found in related technologies. This pixel value adjustment based on position mapping relationship information through the first adjustment module 204, which only performs pixel remapping in the correction region, achieves targeted processing of the distorted region. By simplifying the correction process and reducing the correction time for the distorted region, it significantly improves the correction efficiency of the distorted region while ensuring the image quality of the corrected third image.

[0109] In some embodiments of this application, the image processing apparatus further includes: a second acquisition module, a third acquisition module, and a second determination module.

[0110] The second acquisition module is used to acquire a first sample image captured by the sample camera. The first sample image includes a first sample correction area. The image size of the first sample image is the same as that of the first image. The position of the first sample correction area in the first sample image is the same as that of the correction area in the first image. The third acquisition module is used to acquire a second sample image based on the first sample correction area and the viewpoint conversion parameters corresponding to the first sample correction area. The first sample correction area corresponds to the non-corrected area in the second sample image. The second determination module is used to determine the position mapping relationship information based on the position of the first sample pixel in the first sample image and the position of the second sample pixel in the second sample image of the sample pixel in the first sample correction area.

[0111] In some embodiments of this application, the viewpoint transformation parameters include rotation transformation information and spatial transformation relationship, wherein the spatial transformation relationship is the spatial transformation relationship between the first space corresponding to the first sample image and the second space corresponding to the sample camera; the third acquisition module can be specifically used to determine, according to the spatial transformation relationship, a first transformed pixel position set corresponding to the pixel position set of the first sample correction area in the second space; determine, according to the rotation transformation information, a first transformed pixel position set corresponding to the first transformed pixel position set of the first sample correction area in the second space; determine, according to the spatial transformation relationship, a second transformed pixel position set corresponding to the first transformed pixel position set in the first space; and generate a second sample image according to the second transformed pixel position set.

[0112] In some embodiments of this application, the second determining module may specifically be used to: determine the position of the third sample pixel in the third sample image based on the position of the sample pixel in the first sample correction region in the second sample image and the perspective transformation parameters, wherein the third sample image is a distortion-free reference image corresponding to the first sample image; and determine position mapping relationship information based on the position of the sample pixel in the first sample image in the first sample image and the position of the third sample pixel in the third sample image.

[0113] In some embodiments of this application, the first building module 202 described above can be specifically used to: create a blank image with the same image size as the first image; fill the corresponding positions of the blank image with the pixel values ​​of the uncorrected regions in the first image according to their positions in the first image, thereby obtaining a second image.

[0114] In some embodiments of this application, the first adjustment module 204 described above can be specifically used to: fill the pixel position of the first pixel in the correction area according to the pixel value of the first pixel at the first pixel position by a preset pixel filling method to obtain a third image; wherein, the preset pixel filling method includes at least one of the following: bilinear interpolation and hardware texture interpolation.

[0115] In some embodiments of this application, the camera is mounted on a vehicle; the image processing device described above may further include a third determining module and a fourth determining module.

[0116] The third determining module is used to determine the driving scenario in which the vehicle is located based on the first image; the fourth determining module is used to determine the reference correction area associated with the driving scenario as the correction area based on the correlation between the reference driving scenario and the reference correction area.

[0117] The various modules of the image processing apparatus 200 provided in this application embodiment can realize Figures 1 to 4 It provides the functions for each step of the image processing method and can achieve the corresponding technical effects. For the sake of brevity, it will not be described in detail here.

[0118] Figure 6 The illustration shows a schematic diagram of the hardware structure of a computer device provided in some embodiments of this application.

[0119] The computer device may include a processor 301 and a memory 302 storing computer program instructions.

[0120] For example, the computer device provided in this application embodiment can be deployed in various application scenarios that require real-time image processing, including but not limited to: In-vehicle computing platforms: such as intelligent cockpit domain controllers, autonomous driving domain controllers, or in-vehicle infotainment systems installed in vehicles, which process images captured by in-vehicle cameras such as surround view, rear view, or panoramic parking cameras to provide assisted driving views.

[0121] Mobile terminal devices, such as smartphones, tablets, and laptops, can be used to run image processing applications to perform real-time correction and enhancement of wide-angle or fisheye images captured by the device's camera.

[0122] Edge computing devices: Deployed in edge nodes such as monitoring systems, robots, or drones, they are used to perform low-latency distortion correction on local camera video streams to support tasks such as target detection and scene analysis.

[0123] Cloud server: Receives image or video data uploaded from terminal devices on the server side, performs batch or high-precision correction processing, and returns the results to the terminal.

[0124] Specifically, the processor 301 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0125] Memory 302 may include mass storage for data or instructions. For example, and not limitingly, memory 302 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 302 may include removable or non-removable (or fixed) media. Where appropriate, memory 302 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 302 is non-volatile solid-state memory.

[0126] In certain embodiments, memory 302 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Therefore, typically, memory 302 includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the image processing method in the above embodiments according to this application.

[0127] The processor 301 implements any of the image processing methods described in the above embodiments by reading and executing computer program instructions stored in the memory 302.

[0128] In one example, the aforementioned computer device may further include a communication interface 303 and a bus 310. Wherein, as... Figure 6 As shown, the processor 301, memory 302, and communication interface 303 are connected through bus 310 and complete communication with each other.

[0129] The communication interface 303 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0130] Bus 310 includes hardware, software, or both, that couples components of a computer device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 310 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0131] The computer device can execute the image processing method described in the embodiments of this application, thereby achieving the combination Figures 1 to 5 The image processing method and apparatus described herein.

[0132] Furthermore, in conjunction with the image processing methods in the above embodiments, this application embodiment can provide a computer-readable storage medium for implementation. This computer-readable storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the image processing methods in the above embodiments. Examples of computer-readable storage media include non-transitory computer-readable storage media, such as portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, etc.

[0133] Furthermore, in conjunction with the image processing methods in the above embodiments, this application embodiment can provide a computer program product for implementation. This program product is stored in a storage medium and may specifically include a computer program or instructions. When executed by a processor, the computer program or instructions implement any of the image processing methods in the above embodiments. This program product is executed by at least one processor to implement the various processes of the above image processing method embodiments and achieves the same technical effects; to avoid repetition, further details are omitted here.

[0134] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0135] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0136] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0137] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0138] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. An image processing method, characterized by, The method comprises: obtaining a first image captured by a camera; constructing a second image according to a non-corrected region of the first image, the second image comprising the non-corrected region and a corrected region, the second image having the same image size as the first image, and the non-corrected region in the second image having the same position as the non-corrected region in the first image; determining a first pixel position in the first image of a first pixel in the corrected region according to position mapping relationship information, the position mapping relationship information comprising a mapping relationship between a pixel position in a camera-captured image and a pixel position in an image constructed based on the camera-captured image; adjusting a pixel value of the first pixel position in the corrected region according to a pixel value of the first pixel position, to obtain a third image.

2. The method of claim 1, wherein, The method further comprises: obtaining a first sample image captured by a sample camera, the first sample image comprising a first sample corrected region, the first sample image having the same image size as the first image, and the first sample corrected region having the same position in the first sample image as the corrected region in the first image; obtaining a second sample image according to the first sample corrected region and a perspective conversion parameter corresponding to the first sample corrected region, the first sample corrected region corresponding to a non-corrected region in the second sample image; determining the position mapping relationship information according to a first sample pixel position of a sample pixel in the first sample corrected region in the first sample image and a second sample pixel position of the sample pixel in the second sample image.

3. The method of claim 2, wherein, Before the step of determining the position mapping relationship information according to a first sample pixel position of a sample pixel in the first sample corrected region in the first sample image and a second sample pixel position of the sample pixel in the second sample image, the method further comprises: determining the second sample pixel position of the sample pixel in the second sample image according to the first sample pixel position of the sample pixel in the first sample corrected region and the perspective conversion parameter corresponding to the first sample corrected region.

4. The method of claim 2, wherein, The perspective conversion parameter comprises rotation transformation information and a space conversion relationship, the space conversion relationship being a space conversion relationship between a first space corresponding to the first sample image and a second space corresponding to the sample camera; The step of obtaining a second sample image according to the first sample corrected region and a perspective conversion parameter corresponding to the first sample corrected region comprises: determining a first conversion pixel position set in the second space corresponding to a pixel position set of the first sample corrected region according to the space conversion relationship; determining a first transformation pixel position set in the second space corresponding to the first conversion pixel position set of the first sample corrected region according to the rotation transformation information; determining a second conversion pixel position set in the first space corresponding to the first transformation pixel position set according to the space conversion relationship; According to the second conversion pixel point position set, the second sample image is generated.

5. The method of claim 3, wherein, The position mapping relationship information is determined according to the first sample pixel point position of the sample pixel point in the first sample correction region in the first sample image and the second sample pixel point position of the sample pixel point in the second sample image. According to the second sample pixel point position of the sample pixel point in the first sample correction region in the second sample image and the perspective transformation parameter, the third sample pixel point position of the sample pixel point in the first sample correction region in a third sample image is determined, the third sample image being a reference reference image without distortion corresponding to the first sample image. The position mapping relationship information is determined according to the first sample pixel point position of the sample pixel point in the first sample correction region in the first sample image and the third sample pixel point position of the sample pixel point in the third sample image.

6. The method according to any one of claims 1 to 5, characterized in that, The second image is constructed according to the non-correction region of the first image, including: A blank image with the same image size as the first image is created; According to the position of the non-correction region in the first image, the pixel value of the non-correction region in the first image is filled into the corresponding position of the blank image to obtain a second image.

7. The method according to any one of claims 1 to 5, characterized in that, The pixel value of the pixel point position of the first pixel point in the correction region is adjusted according to the pixel value of the first pixel point position of the first pixel point to obtain a third image, including: The pixel point position of the first pixel point in the correction region is filled according to the pixel value of the first pixel point at the first pixel point position by a preset pixel filling method to obtain a third image; The preset pixel filling method includes at least one of the following: bilinear interpolation, hardware texture interpolation.

8. The method according to any one of claims 1 to 5, characterized in that, The camera is mounted on a vehicle; the method further comprises: According to the first image, the driving scene in which the vehicle is located is determined; According to the association relationship between the reference driving scene and the reference correction region, the reference correction region associated with the driving scene is determined as the correction region.

9. An image processing apparatus characterized by comprising: The device comprises: A first acquisition module is configured to acquire a first image captured by a camera. A first construction module is configured to construct a second image according to a non-correction region of the first image, the second image comprising the non-correction region and a correction region, the image size of the second image being the same as the image size of the first image, and the position of the non-correction region in the second image being the same as the position of the non-correction region in the first image. A first determination module is configured to determine the first pixel point position of the first pixel point in the first image in the correction region according to position mapping relationship information, the position mapping relationship information comprising the mapping relationship between the pixel point position of the pixel point in the camera-captured image and the pixel point position in the image constructed based on the camera-captured image. A first adjustment module is configured to adjust the pixel value of the pixel point position of the first pixel point in the correction region according to the pixel value of the first pixel point position to obtain a third image.

10. A computer device, comprising: The computer device comprises a processor and a memory storing computer program instructions. The processor implements the image processing method as claimed in any one of claims 1-8 when executing the computer program instructions. Or, the image processing device as claimed in claim 9 is included.

Images