An image processing method and device, an electronic device, and a storage medium

Abstract

The application provides an image processing method and device, an electronic device and a storage medium. The image processing method comprises: acquiring a to-be-processed image collected by an image collection device; storing pixel values of each pixel point in the to-be-processed image in a corresponding target mapping address based on a preset pixel mapping table to generate a target image; wherein the preset pixel mapping table comprises a target mapping address item, and the target mapping address item comprises a storage address corresponding to each pixel point in the image of the image collection device. In the embodiment of the application, the to-be-processed image is directly stored based on the processing requirement of the to-be-processed image, and the target image processed is obtained after the storage is completed. In this way, the hardware resource consumption can be reduced, and the operation time of the to-be-processed image can be reduced.

Description

Technical Field

[0001] This application relates to the field of camera technology, and more specifically, to an image processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] Currently, applications related to visual presentation typically require image processing such as distortion correction, viewpoint and perspective transformation, and image stitching for images acquired by cameras. For example, when a fisheye camera is used in a car surround-view scenario, image distortion correction is necessary first. The common algorithm for image distortion correction involves first correcting the distortion based on the camera lens's distortion function, and then resolving the camera's mounting position and lens optical axis direction from a known-sized corrected image.

[0003] Such algorithms require computation on all pixels of each image during processing, which is computationally intensive and consumes a lot of hardware resources and processing time. Summary of the Invention

[0004] The purpose of this application is to provide an image processing method, apparatus, electronic device, and storage medium to reduce the consumption of hardware resources for image processing and to reduce computation time.

[0005] This application is implemented as follows:

[0006] In a first aspect, embodiments of this application provide an image processing method, comprising: acquiring an image to be processed acquired by an image acquisition device; and storing the pixel values ​​of each pixel in the image to be processed into a corresponding target mapping address based on a preset pixel mapping table to generate a target image; wherein the preset pixel mapping table includes a target mapping address item, and the target mapping address item includes the storage address corresponding to each pixel in the image acquired by the image acquisition device.

[0007] In this embodiment, after acquiring the image to be processed, the storage process of the image to be processed is optimized. Based on the target mapping address entry in the preset pixel mapping table, the pixel values ​​of each pixel in the image to be processed are stored in the target mapping address. Then, based on the pixel values ​​corresponding to the target mapping address, the processed target image is generated. That is, in this embodiment, the image to be processed is directly stored based on the processing requirements of the image to be processed. After storage, the processed target image is obtained. This method can reduce hardware resource consumption and reduce the processing time of the image to be processed.

[0008] In conjunction with the technical solution provided in the first aspect above, in some possible implementations, the preset pixel mapping table is obtained through the following steps: acquiring a sample image acquired by the image acquisition device and a pre-determined processed image corresponding to the sample image; determining the mapping relationship between the pixels in the sample image and the pixels in the processed image; and generating the preset pixel mapping table based on the mapping relationship.

[0009] In this embodiment, a preset pixel mapping table is constructed by acquiring sample images captured by an image acquisition device and pre-determined processed images corresponding to the sample images. This allows for the direct determination of the target mapping address using the preset pixel mapping table when acquiring subsequent images to be processed from the image acquisition device, thereby obtaining the processed image. Furthermore, since the preset pixel mapping table is also obtained from images acquired by the image acquisition device, this method can also improve the accuracy of the subsequently generated target image.

[0010] In conjunction with the technical solution provided in the first aspect above, in some possible implementations, the sample image and the processed image include the same checkerboard pattern. Determining the mapping relationship between pixels in the sample image and pixels in the processed image includes: obtaining the coordinates of grid points on a first checkerboard pattern; wherein the first checkerboard pattern is a checkerboard pattern in the sample image; determining the coordinates of grid points on a second checkerboard pattern based on the coordinates of the grid points on the first checkerboard pattern; wherein the second checkerboard pattern is a checkerboard pattern in the processed image; and determining the mapping relationship between pixels in the sample image and pixels in the processed image based on the coordinates of the grid points on the first checkerboard pattern and the coordinates of the grid points on the second checkerboard pattern.

[0011] In this embodiment of the application, since the sample image and the processed image contain the same checkerboard pattern, the grid points on the checkerboard pattern of the two images can be used to determine the mapping relationship between the pixels in the sample image and the pixels in the processed image. In this way, the accuracy of the determined mapping relationship between the pixels in the sample image and the pixels in the processed image can be improved.

[0012] In conjunction with the technical solution provided in the first aspect above, in some possible implementations, determining the mapping relationship between pixels in the sample image and pixels in the processed image based on the coordinates of grid points on the first grid and the second grid includes: determining the mapping relationship between the coordinates of pixels on the grid lines of the first grid and the coordinates of pixels on the grid lines of the second grid, based on the mapping relationship between the coordinates of grid points on the first grid and the second grid; and determining the mapping relationship between all pixels in the sample image and all pixels in the processed image based on the mapping relationship between the coordinates of pixels on the grid lines of the first grid and the second grid.

[0013] In this embodiment, the mapping relationship between the coordinates of grid points on the first grid and the coordinates of grid points on the second grid is used to determine the mapping relationship between the coordinates of pixels on the grid lines of the first grid and the coordinates of pixels on the grid lines of the second grid. Finally, the mapping relationship between the coordinates of pixels on the grid lines of the first grid and the coordinates of pixels on the grid lines of the second grid is used to determine the mapping relationship between all pixels in the sample image and all pixels in the processed image. That is, from the above-described point-to-line and line-to-surface approach, the mapping relationship between all pixels in the sample image and all pixels in the processed image can be reasonably and accurately determined.

[0014] In conjunction with the technical solution provided in the first aspect above, in some possible implementations, obtaining the coordinates of grid points on the first chessboard grid includes: obtaining the coordinates of pixel points on each grid line of the first chessboard grid; performing linear fitting on the coordinates of pixel points on each grid line of the first chessboard grid to obtain the fitting curve of each grid line of the first chessboard grid; and determining the coordinates of grid points on the first chessboard grid based on the intersection of the fitting curves of each grid line of the first chessboard grid.

[0015] In this embodiment of the application, the coordinates of the grid points on the first chessboard are reasonably determined by the intersection of the fitted curves of each grid line on the first chessboard.

[0016] In conjunction with the technical solution provided in the first aspect above, in some possible implementations, the preset pixel mapping table includes a weight item; the weight item includes the weight value corresponding to each pixel in the image acquired by the image acquisition device; the step of storing the pixel value of each pixel in the image to be processed into the corresponding target mapping address includes: calculating the weight of the pixel value of each pixel in the image to be processed based on the weight item; and storing the calculated pixel value of each pixel into the corresponding target mapping address.

[0017] In this embodiment, since the preset pixel mapping table includes weight entries, the image stitching process can be implemented using this preset pixel mapping table. That is, during the image stitching process, the pixel values ​​of the overlapping stitching area can be determined using weight values, and then the pixel values ​​calculated with weights can be stored in the corresponding target mapping address to complete the image stitching process during image storage.

[0018] In conjunction with the technical solution provided in the first aspect above, in some possible implementations, the preset pixel mapping table includes: a pixel valid bit field; the pixel valid bit field is used to characterize whether each pixel in the image of the image acquisition device is valid; correspondingly, storing the pixel value of each pixel in the image to be processed into the corresponding target mapping address includes: determining the valid pixel in the image to be processed based on the pixel valid bit field; and storing the valid pixel into the corresponding target mapping address.

[0019] In this embodiment of the application, since the preset pixel mapping table includes a valid pixel bit field, only the pixel values ​​corresponding to the required pixel points can be stored. In this way, storage space can be saved. At the same time, users can also set the viewing angle, image size, etc. according to their needs to provide more application scenarios.

[0020] Thirdly, embodiments of this application provide an electronic device, including: a processor and a memory, the processor and the memory being connected; the memory being used to store a program; the processor being used to invoke the program stored in the memory to execute the method provided by the embodiments of the first aspect described above and / or some possible implementations of the embodiments of the first aspect described above.

[0021] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, the computer program executing, when run by a processor, the methods provided as described in the first aspect embodiments and / or some possible implementations of the first aspect embodiments. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a module block diagram of an electronic device provided in an embodiment of this application.

[0024] Figure 2 This is a flowchart illustrating the steps of an image processing method provided in an embodiment of this application.

[0025] Figure 3 A flowchart illustrating the steps of a method for obtaining a preset pixel mapping table, as provided in an embodiment of this application.

[0026] Figure 4 This is a schematic diagram of a sample image provided in an embodiment of this application.

[0027] Figure 5 This is a schematic diagram of a processed image provided in an embodiment of this application.

[0028] Figure 6 This is a flowchart of step S202 in a method for obtaining a preset pixel mapping table provided in an embodiment of this application.

[0029] Figure 7 This is a stitched image obtained by the image processing method provided in the embodiments of this application.

[0030] Figure 8 This is a block diagram of an image processing apparatus provided in an embodiment of this application.

[0031] Icons: 100 - Electronic device; 110 - Processor; 120 - Memory; 200 - Image processing device; 210 - Acquisition module; 220 - Processing module. Detailed Implementation

[0032] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0033] Please see Figure 1 This application provides a schematic structural block diagram of an electronic device 100 that applies an image processing method and apparatus.

[0034] Electronic device 100 can be a terminal, which can be, but is not limited to, a personal computer (PC), smartphone, tablet computer, personal digital assistant (PDA), mobile internet device (MID), etc. Electronic device 100 can also be an image processing system, such as an in-vehicle control system. Electronic device 100 can also be any device used for image processing tasks; for example, electronic device 100 can be a central monitor in a monitoring room (used to synthesize images captured by multiple cameras). This application does not limit its scope in this regard.

[0035] Structurally, the electronic device 100 may include a processor 110 and a memory 120.

[0036] The processor 110 is electrically connected directly or indirectly to the memory 120 to enable data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines. The image processing apparatus includes at least one software module that can be stored in the memory 120 or embedded in the operating system (OS) of the electronic device 100 in the form of software or firmware. The processor 110 is used to execute executable modules stored in the memory 120, such as software function modules and computer programs included in the image processing apparatus, to implement image processing methods. The processor 110 can execute the computer program after receiving an execution instruction.

[0037] The processor 110 can be an integrated circuit chip with signal processing capabilities. The processor 110 can also be a general-purpose processor, such as a Central Processing Unit (CPU), a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a discrete gate or transistor logic device, or a discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. Furthermore, the general-purpose processor can be a microprocessor or any conventional processor.

[0038] The memory 120 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), and electrically erasable programmable read-only memory (EEPROM). The memory 120 is used to store programs, which the processor 110 executes upon receiving an execution instruction.

[0039] It should be noted that, Figure 1 The structure shown is for illustrative purposes only. The electronic device 100 provided in this embodiment may also have a structure that is more advanced than the one shown in the diagram. Figure 1 Fewer or more components, or having the same Figure 1 The different configurations shown. Furthermore... Figure 1 The components shown can be implemented through software, hardware, or a combination thereof.

[0040] Please see Figure 2 , Figure 2 This is a flowchart illustrating the steps of an image processing method provided in an embodiment of this application. The method is applied to... Figure 1 The electronic device 100 shown. It should be noted that the image processing method provided in this embodiment does not... Figure 2 The following sequence is a limitation, and the method includes: step S101-step S102.

[0041] Step S101: Acquire the image to be processed acquired by the image acquisition device.

[0042] The image acquisition device is a device that includes a camera, which can be, but is not limited to, a camcorder, camera, mobile phone, tablet computer, etc. The aforementioned camera can be, but is not limited to, a fisheye camera or a panoramic camera. It should be noted that the devices and cameras listed above are only for understanding the embodiments of this application and should not be construed as limiting the scope of this embodiment.

[0043] Step S102: Based on the preset pixel mapping table, store the pixel values ​​of each pixel in the image to be processed into the corresponding target mapping address to generate the target image.

[0044] The preset pixel mapping table includes a target mapping address field, which contains the storage address corresponding to each pixel in the image acquired by the image acquisition device. Subsequent electronic devices output the target image directly from the target mapping address for display.

[0045] The processing methods for the images to be processed mentioned above can include, but are not limited to, distortion correction, viewpoint and perspective transformation, image stitching, and other image processing.

[0046] For example, when the image acquisition device is a fisheye camera, assuming that the fisheye camera needs to use normal, undistorted images in certain application scenarios, the image acquired by the fisheye camera is first subjected to distortion correction. In this embodiment, the distortion correction process occurs during image storage. Specifically, the preset pixel mapping may include the target mapping address of each pixel in the distorted image. This process is equivalent to changing the storage address of each pixel in the distorted image, thereby enabling the image stored using the target mapping address to be restored to a normal, undistorted image.

[0047] As can be seen, in this embodiment, after obtaining the image to be processed, the storage process of the image to be processed is optimized. Based on the target mapping address entry in the preset pixel mapping table, the pixel values ​​of each pixel in the image to be processed are stored in the target mapping address, and then the processed target image is generated based on the pixel values ​​corresponding to the target mapping address. That is, in this embodiment, the image to be processed is directly stored based on the processing requirements of the image to be processed. After storage, the processed target image is obtained. In this way, hardware resource consumption and processing time of the image to be processed can be reduced.

[0048] To facilitate understanding of the image processing method provided in the embodiments of this application, the construction process of the aforementioned preset pixel mapping table will first be explained. Please refer to... Figure 3 In the embodiments of this application, the process of constructing the preset pixel mapping table includes: steps S201-S203.

[0049] Step S201: Acquire the sample image acquired by the image acquisition device and the pre-determined processed image corresponding to the sample image.

[0050] The sample image can be acquired in real time by an image acquisition device, while the processed image can be obtained by processing the sample image based on an existing algorithm. For example, if the processing is distortion correction, after the sample image is acquired by the image acquisition device, it is processed based on an existing algorithm to obtain the distortion-corrected image (i.e., the processed image mentioned above).

[0051] Of course, the processed image can also be acquired by another image acquisition device. For example, in step S201, the image acquisition device is a fisheye camera, and the other image acquisition device is a standard camera. Then, the fisheye camera and the standard camera are aimed at the same target and photographed to obtain the above sample image and the processed image. Among them, the sample image is acquired by the fisheye camera, and the processed image is acquired by the standard camera.

[0052] The target mentioned above can be any object. However, in order to construct an accurate preset pixel mapping table, in this embodiment of the application, the target can be a checkerboard pattern. Please refer to... Figure 4 and Figure 5 , Figure 4 The image is a sample image captured by a fisheye camera, which includes a distorted checkerboard pattern. Figure 5 The processed image corresponds to the sample image, and includes the normal checkerboard pattern.

[0053] Step S202: Determine the mapping relationship between the pixels in the sample image and the pixels in the processed image.

[0054] After acquiring the sample image and the processed image, the mapping relationship between the pixels in the two images is determined. It should be noted that the mapping relationship here represents the position of a pixel in the sample image in the processed image after processing.

[0055] The mapping relationship can be a coordinate relationship, that is, the mapping relationship represents the coordinate position of a pixel in the sample image after processing in the processed image. For example, A(x1,y1) = B(X1,Y1), where A(x1,y1) represents the coordinates of a pixel in the sample image, and B(X1,Y1) represents the coordinates of a pixel in the processed image. Pixel A(x1,y1) in the sample image corresponds to pixel B(X1,Y1) in the processed image. In other words, pixel A(x1,y1) in the sample image corresponds to pixel B(X1,Y1) in the processed image.

[0056] Please see Figure 6 When the sample image and the processed image contain the same checkerboard pattern, step S202 may specifically include: steps S301-S303.

[0057] Step S301: Obtain the coordinates of the grid points on the first chessboard grid; where the first chessboard grid is the chessboard grid in the sample image.

[0058] It should be noted that the chessboard is composed of multiple rectangular grids, and the grid points are the vertices of the grids.

[0059] Optionally, obtaining the coordinates of grid points on the first checkerboard may specifically include: obtaining the coordinates of pixels on each grid line of the first checkerboard; performing linear fitting on the coordinates of pixels on each grid line of the first checkerboard to obtain the fitting curve of each grid line of the first checkerboard; and determining the coordinates of grid points on the first checkerboard based on the intersection of the fitting curves of each grid line of the first checkerboard.

[0060] The fitting curves for each of the grid lines mentioned above include the fitting curves for the horizontal grid lines and the fitting curves for the vertical grid lines.

[0061] The equation of the fitted curve for the horizontal grid lines can be:

[0062] y j =FH j (x) (1)

[0063] In formula (1), j represents the j-th grid point in the vertical direction, and j ranges from 0 to m; m represents the number of grid points in the vertical direction, then y j FH represents the fitted curve corresponding to the j-th grid point in the vertical direction. j (x) represents the expression for x in the fitted curve corresponding to the j-th grid point in the vertical direction. It should be noted that since the fitted curves of the horizontal grid lines are different in different scenarios, the above formula (1) is only a general expression.

[0064] The equation of the fitted curve for the vertical grid lines can be:

[0065] x j =FV i (y) (2)

[0066] In formula (2), i represents the i-th grid point in the horizontal direction, i takes values ​​from 0 to n; n represents the number of grid points in the horizontal direction, then x i FV represents the fitted curve corresponding to the i-th grid point in the horizontal direction. i (y) represents the expression for y in the fitted curve corresponding to the i-th grid point in the horizontal direction. It should be noted that since the fitted curves of the vertical grid lines are different in different scenarios, the above formula (2) is only a general expression.

[0067] After obtaining the fitted curves of each horizontal grid line and each vertical grid line on the first chessboard, the intersection point (x) of the fitted curves of each grid line is determined. ij ,y ij Among them, the intersection point (x) ij ,y ij The coordinates are the grid points on the first determined chessboard square.

[0068] For example, (x 12 ,y 12 The intersection of the fitted curves of the first vertical grid line and the fitted curves of the second horizontal grid line is indicated by the number ).

[0069] As can be seen, in this embodiment of the application, the coordinates of the grid points on the first chessboard are reasonably determined by the intersection of the fitted curves of each grid line on the first chessboard.

[0070] Step S302: Determine the coordinates of the grid points on the second chessboard based on the coordinates of the grid points on the first chessboard; wherein, the second chessboard is the chessboard in the processed image.

[0071] Assuming the chessboard grid consists of square grids, the number of pixels between any two adjacent grid points in the second chessboard grid is p.

[0072] The coordinates of the grid points on the second chessboard, determined based on the coordinates of the grid points on the first chessboard, are (u ij ,v ij ).

[0073] That is, the grid points on two chessboard squares satisfy the following correspondence:

[0074] (u ij ,v ij )=(x ij ,y ij (3)

[0075] In formula (3), u ij =i*p;v ij =j*p.

[0076] It should be noted that (x ij ,y ij (u) represents a pixel in the first square of the chessboard. ij ,v ij ) represents the second chessboard square that is connected to (x) ij ,y ij The corresponding pixel, pixel (x) ij ,y ij ) and the pixels on the second chessboard square (u ij ,v ij Corresponding to ) That is, the pixel (x) in the first chessboard square. ij ,y ij ) corresponds to the pixel on the second chessboard square (u ij ,v ij The two are a mapping pair.

[0077] Step S303: Based on the coordinates of the grid points on the first grid and the coordinates of the grid points on the second grid, determine the mapping relationship between the pixels in the sample image and the pixels in the processed image.

[0078] After determining the coordinates of the grid points on the second chessboard through the aforementioned steps, the mapping relationship between the grid points is determined. Then, based on the mapping relationship between the grid points, the mapping relationship between all pixels in the image is determined.

[0079] Specifically, step S303 above can include: step one and step two.

[0080] Step 1: Based on the mapping relationship between the coordinates of grid points on the first grid and the coordinates of grid points on the second grid, determine the mapping relationship between the coordinates of pixels on the grid lines of the first grid and the coordinates of pixels on the grid lines of the second grid.

[0081] Step one is equivalent to determining the mapping of the line to which the point lies from the mapping of the point.

[0082] Specifically, the analytical process for the horizontal grid lines is as follows:

[0083] Curve fitting is performed on all grid points along the horizontal grid lines, that is, the x-coordinate u of the grid points is... ij x ij (Including the x-coordinate of the point on the first grid and the x-coordinate of the point on the second grid) Perform curve fitting to obtain the x-coordinate u on the horizontal grid line. ij and x ij The mapping relationship between them. The expression corresponding to this mapping relationship can be specifically defined as:

[0084] x j =f j (u) (4)

[0085] In formula (4), u = 0, 1, ..., n*p, and formula (4) is a general expression. Formula (4) represents the mapping relationship between all the horizontal coordinates on the horizontal grid line, that is, the horizontal coordinate u on the second grid on the horizontal grid line. ij The x-coordinate of this horizontal grid line on the first grid square ij The mapping relationship between them.

[0086] Then, x in formula (4) j Substituting these values ​​into formula (1), we can obtain the ordinate v of the horizontal grid line. ij and y ij The mapping relationship between them, that is, the vertical coordinate v on the second grid on the horizontal grid line. ijThe y-coordinate of the horizontal grid line on the first checkerboard grid. ij The mapping relationship between them.

[0087] By using the above method, the mapping relationship between the coordinates of pixels on the horizontal grid lines of the first chessboard and the coordinates of pixels on the horizontal grid lines of the second chessboard can be determined.

[0088] The analytical process for vertical grid lines is as follows:

[0089] Perform curve fitting on all grid points along the vertical grid lines, that is, for the ordinate v of each grid point... ij y ij (Including the ordinate of the point on the first grid and the ordinate of the point on the second grid) Perform curve fitting to obtain the ordinate v on the vertical grid line. ij and y ij The mapping relationship between them. The expression corresponding to this mapping relationship can be specifically defined as:

[0090] y j =f j (v) (5)

[0091] In formula (5), v = 0, 1, ..., m*p, and formula (5) is a general expression. Formula (5) represents the mapping relationship between all the ordinates on the vertical grid line, that is, the ordinate v on the second grid on the vertical grid line. ij The vertical grid line and its y-coordinate on the first grid square. ij The mapping relationship between them.

[0092] Then, y in formula (5) j Substituting these values ​​into formula (2), we can obtain the ordinate v of the vertical grid line. ij and y ij The mapping relationship between them, that is, the ordinate v on the second grid cell of the vertical grid line. ij The vertical grid line and its y-coordinate on the first grid square. ij The mapping relationship between them.

[0093] By using the above method, the mapping relationship between the coordinates of pixels on the vertical grid lines of the first chessboard and the coordinates of pixels on the vertical grid lines of the second chessboard can be determined.

[0094] In summary, the mapping relationship between the coordinates of pixels on the grid lines of the first chessboard and the coordinates of pixels on the grid lines of the second chessboard can be determined.

[0095] Step 2: Based on the mapping relationship between the coordinates of pixels on the grid lines of the first chessboard and the coordinates of pixels on the grid lines of the second chessboard, determine the mapping relationship between all pixels in the sample image and all pixels in the processed image.

[0096] Step two is equivalent to determining the mapping of the plane containing the line from the mapping of the line.

[0097] Specifically, based on the mapping coordinate relationship of the grid lines, the mapping coordinates of all pixels can be further analyzed column by column or row by row.

[0098] The column-by-column analysis is performed based on the mapped coordinates of the horizontal grid lines. For example, the analysis process for the k-th column is as follows:

[0099] a. Extract the pixels in column k along each horizontal grid line (u kj ,v kj ), (x kj ,y kj Then, a linear fit is performed to obtain a polynomial curve. The expression for the polynomial curve is:

[0100] y k =FV k (x) (6)

[0101] b. The ordinate v of the k-th column kj y kj Perform curve fitting to obtain the x-coordinate u of the k-th fitted line. kj and x kj The mapping relationship between them. The expression corresponding to this mapping relationship can be specifically defined as:

[0102] x k =f k (v) (7)

[0103] c. Replace x in formula (7) k Substituting these values ​​into formula (6) sequentially, we can obtain the ordinate x of the k-th column fitted line. kj and y kj The mapping relationship between them.

[0104] d. By repeating steps a, b, and c above, the mapping relationship between all pixels in the sample image and all pixels in the processed image can be obtained.

[0105] It should be noted that row-by-row analysis is based on the mapped coordinates of the vertical grid lines. Since its principle and process are basically the same as column-by-column analysis, it will not be elaborated here.

[0106] As can be seen, in this embodiment, since the sample image and the processed image include the same checkerboard pattern, the grid points on the checkerboard of the two images can be used to determine the mapping relationship between pixels in the sample image and pixels in the processed image. This method improves the accuracy of the determined mapping relationship between pixels in the sample image and pixels in the processed image. Furthermore, in the above implementation process, the mapping relationship between the coordinates of grid points on the first checkerboard and the coordinates of grid points on the second checkerboard is used to determine the mapping relationship between the coordinates of pixels on the grid lines of the first checkerboard and the coordinates of pixels on the grid lines of the second checkerboard. Finally, the mapping relationship between the coordinates of pixels on the grid lines of the first checkerboard and the coordinates of pixels on the grid lines of the second checkerboard is used again to determine the mapping relationship between all pixels in the sample image and all pixels in the processed image. That is, from the above point-to-line and then line-to-surface approach, the mapping relationship between all pixels in the sample image and all pixels in the processed image can be reasonably and accurately determined.

[0107] Step S203: Generate a preset pixel mapping table based on the mapping relationship.

[0108] Finally, based on the mapping relationship between the pixels mentioned above, the preset pixel mapping table can be determined. For example, by pre-determining the storage address of each pixel in the processed image, and then combining it with the previously determined mapping relationship, the mapping address of the pixel in each sample image can be determined (this mapping address is the storage address of the corresponding pixel in the processed image).

[0109] In summary, in this embodiment, a preset pixel mapping table is constructed by acquiring sample images captured by an image acquisition device and pre-determined processed images corresponding to the sample images. This allows the target mapping address to be directly determined using the preset pixel mapping table when the image to be processed is acquired by the image acquisition device, thereby obtaining the processed image. Furthermore, since the preset pixel mapping table is also obtained from images acquired by the image acquisition device, this method can also improve the accuracy of the subsequently generated target image.

[0110] The application of the preset pixel mapping table is explained below.

[0111] In one embodiment, the preset pixel mapping table may only include target mapping address entries. Each target mapping address corresponds to the original storage address of the image. For example, after acquiring the image to be processed from the image acquisition device, the electronic device determines the correspondence between the original storage address and the target mapping address of each pixel in the image to be processed based on the preset pixel mapping table, and then stores the pixel value of each pixel in the image to be processed into the corresponding target mapping address.

[0112] Of course, the preset pixel mapping table can also include the original storage address entry.

[0113] In one embodiment, the preset pixel mapping table may further include weight entries. The weight entries include the weight values ​​corresponding to each pixel in the image acquired by the image acquisition device.

[0114] Accordingly, storing the pixel values ​​of each pixel in the image to be processed into the corresponding target mapping address includes: calculating the weights of the pixel values ​​of each pixel in the image to be processed based on the weight term; and storing the calculated pixel values ​​of each pixel into the corresponding target mapping address.

[0115] It should be noted that the weighting term is mainly used in image stitching. For example, in a car surround view scenario, the images captured by the four cameras (front, rear, left, and right) of the car need to be stitched into a panoramic image. In this case, the images to be processed captured by the four cameras need to be stitched together.

[0116] During the stitching process, the images to be processed captured by the four cameras can be stored in their respective target mapping addresses based on the target mapping addresses of the preset pixel mapping table. The weight term is mainly used to achieve reasonable stitching of the overlapping areas (intersection areas) of the four images to be processed.

[0117] For example, in overlapping areas, the weight value of pixels can be set to be less than 1; for instance, the weight value of pixels in the overlapping areas of two images can be set to 0.5. In non-overlapping areas, the weight value of pixels is 1.

[0118] After determining the weight value of each pixel in the image to be processed through the weight term, the pixel value of each pixel in the image to be processed is weighted and then the calculated pixel value of each pixel is stored in the corresponding target mapping address.

[0119] The stitched image obtained using the above method can be used as a reference. Figure 7 .

[0120] As can be seen, in this embodiment of the application, since the preset pixel mapping table includes a weight item, the image stitching process can be implemented using this preset pixel mapping table. That is, during the image stitching process, the pixel values ​​of the overlapping stitching area can be determined using the weight values, and then the pixel values ​​calculated with weights can be stored in the corresponding target mapping address to complete the image stitching during the image storage process.

[0121] In one embodiment, the preset pixel mapping table may further include a pixel validity bit field. The pixel validity bit field is used to characterize whether each pixel in the image of the image acquisition device is valid.

[0122] Accordingly, storing the pixel values ​​of each pixel in the image to be processed into the corresponding target mapping address includes: determining the valid pixels in the image to be processed based on the valid bit entries of the pixels; and storing the valid pixels into the corresponding target mapping address.

[0123] It should be noted that the effective pixel bit field can be applied to scenarios such as image correction, viewpoint changes, and image stitching. Users can determine the required pixels by adding effective pixel bits. For example, if a user only needs an image of a specific area in a panoramic image captured by a panoramic camera, they can pre-mark the pixels in that area as effective pixels, while the pixels in the other area are considered ineffective pixels.

[0124] As can be seen, in this embodiment of the application, since the preset pixel mapping table includes a valid pixel bit field, only the pixel values ​​corresponding to the required pixel points can be stored. In this way, storage space can be saved. At the same time, users can also set the viewing angle, image size, etc. according to their needs to provide more application scenarios.

[0125] It should be noted that in other embodiments, the preset pixel table may also include a target mapping address field, a weight field, and a pixel valid bit field, which is not limited in this application. Of course, the preset pixel table described above can also be applied to more scenarios, and is not limited to image correction, viewpoint change, and image stitching.

[0126] Please see Figure 8 Based on the same inventive concept, embodiments of this application also provide an image processing apparatus 200, which includes:

[0127] The acquisition module 210 acquires the image to be processed acquired by the image acquisition device.

[0128] The processing module 220 is used to store the pixel values ​​of each pixel in the image to be processed into the corresponding target mapping address based on a preset pixel mapping table to generate a target image; wherein, the preset pixel mapping table includes a target mapping address item, and the target mapping address item includes the storage address corresponding to each pixel in the image of the image acquisition device.

[0129] Optionally, the device further includes a construction module. The construction module is used to acquire sample images acquired by the image acquisition device and pre-determined processed images corresponding to the sample images; determine the mapping relationship between pixels in the sample images and pixels in the processed images; and generate the preset pixel mapping table based on the mapping relationship.

[0130] Optionally, the sample image and the processed image include the same checkerboard pattern. The construction module is specifically used to obtain the coordinates of the grid points on the first checkerboard pattern; wherein, the first checkerboard pattern is the checkerboard pattern in the sample image; determine the coordinates of the grid points on the second checkerboard pattern based on the coordinates of the grid points on the first checkerboard pattern; wherein, the second checkerboard pattern is the checkerboard pattern in the processed image; and determine the mapping relationship between the pixels in the sample image and the pixels in the processed image based on the coordinates of the grid points on the first checkerboard pattern and the coordinates of the grid points on the second checkerboard pattern.

[0131] Optionally, the construction module is further specifically used to determine the mapping relationship between the coordinates of pixels on the grid lines of the first chessboard and the coordinates of pixels on the grid lines of the second chessboard, based on the mapping relationship between the coordinates of grid points on the first chessboard and the coordinates of grid points on the second chessboard; and to determine the mapping relationship between all pixels in the sample image and all pixels in the processed image, based on the mapping relationship between the coordinates of pixels on the grid lines of the first chessboard and the coordinates of pixels on the grid lines of the second chessboard.

[0132] Optionally, the construction module is further specifically used to obtain the coordinates of the pixels on each grid line of the first chessboard; to perform linear fitting on the coordinates of the pixels on each grid line of the first chessboard to obtain the fitting curve of each grid line of the first chessboard; and to determine the coordinates of the grid points on the first chessboard based on the intersection of the fitting curves of each grid line of the first chessboard.

[0133] Optionally, the preset pixel mapping table includes a weight item; the weight item includes the weight value corresponding to each pixel in the image acquired by the image acquisition device; accordingly, the processing module is specifically used to perform weight calculation on the pixel value of each pixel in the image to be processed based on the weight item; and store the calculated pixel value of each pixel in the corresponding target mapping address.

[0134] Optionally, the preset pixel mapping table includes: a pixel valid bit field; the pixel valid bit field is used to characterize whether each pixel in the image of the image acquisition device is valid; correspondingly, the processing module is specifically used to determine the valid pixels in the image to be processed based on the pixel valid bit field; and store the valid pixels in the corresponding target mapping address.

[0135] It should be noted that, as those skilled in the art will clearly understand, for the sake of convenience and brevity, the specific working processes of the systems, devices and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0136] Based on the same inventive concept, embodiments of this application also provide a computer-readable storage medium storing a computer program thereon, which executes the methods provided in the above embodiments when the computer program is run.

[0137] The storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

[0138] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0139] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0140] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0141] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0142] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. An image processing method, characterized by, include: Acquire the image to be processed from the image acquisition device; Based on a preset pixel mapping table, the pixel values ​​of each pixel in the image to be processed are directly stored into the corresponding target mapping address to generate the processed target image; wherein, the preset pixel mapping table includes a target mapping address item, and the target mapping address item includes the storage address corresponding to each pixel in the image of the image acquisition device. The preset pixel mapping table is obtained through the following steps: The image acquisition device acquires a sample image and a pre-determined processed image corresponding to the sample image; wherein the sample image and the processed image include the same checkerboard pattern. Obtain the coordinates of grid points on the first chessboard grid; wherein, the first chessboard grid is the chessboard grid in the sample image; The coordinates of the grid points on the second chessboard are determined based on the coordinates of the grid points on the first chessboard grid; wherein, the second chessboard grid is the chessboard grid in the processed image; Based on the coordinates of the grid points on the first chessboard and the coordinates of the grid points on the second chessboard, the mapping relationship between the pixels in the sample image and the pixels in the processed image is determined. Based on the mapping relationship, the preset pixel mapping table is generated.

2. The method of claim 1, wherein, Determining the mapping relationship between pixels in the sample image and pixels in the processed image based on the coordinates of grid points on the first and second checkerboard grids includes: Based on the mapping relationship between the coordinates of the grid points on the first grid and the coordinates of the grid points on the second grid, the mapping relationship between the coordinates of the pixels on the grid lines of the first grid and the coordinates of the pixels on the grid lines of the second grid is determined. Based on the mapping relationship between the coordinates of pixels on the grid lines of the first chessboard and the coordinates of pixels on the grid lines of the second chessboard, the mapping relationship between all pixels in the sample image and all pixels in the processed image is determined.

3. The method according to claim 1, characterized in that, The step of obtaining the coordinates of the grid points on the first chessboard grid includes: Obtain the coordinates of the pixels on each grid line of the first chessboard. Linear fitting is performed on the coordinates of the pixels on each grid line of the first chessboard to obtain the fitting curve of each grid line on the first chessboard. The coordinates of the grid points on the first chessboard are determined based on the intersection points of the fitted curves of each grid line on the first chessboard.

4. The method according to any one of claims 1-3, characterized in that, The preset pixel mapping table includes a weight item; the weight item includes the weight value corresponding to each pixel in the image acquired by the image acquisition device. The step of storing the pixel values ​​of each pixel in the image to be processed into the corresponding target mapping address includes: Based on the weight term, the pixel values ​​of each pixel in the image to be processed are weighted and calculated. The calculated pixel values ​​of each pixel are stored in the corresponding target mapping address.

5. The method according to any one of claims 1-3, characterized in that, The preset pixel mapping table includes: a pixel valid bit field; the pixel valid bit field is used to characterize whether each pixel in the image of the image acquisition device is valid; Accordingly, storing the pixel values ​​of each pixel in the image to be processed into the corresponding target mapping address includes: Based on the pixel valid bit entries, the valid pixels in the image to be processed are determined; The valid pixels are stored in the corresponding target mapping address.

6. An image processing apparatus, characterized in that, include: The acquisition module acquires the image to be processed acquired by the image acquisition device; The processing module is used to directly store the pixel values ​​of each pixel in the image to be processed into the corresponding target mapping address based on a preset pixel mapping table, so as to generate a processed target image; wherein, the preset pixel mapping table includes a target mapping address item, and the target mapping address item includes the storage address corresponding to each pixel in the image of the image acquisition device. The preset pixel mapping table is obtained through the following steps: The image acquisition device acquires a sample image and a pre-determined processed image corresponding to the sample image; wherein the sample image and the processed image include the same checkerboard pattern. Obtain the coordinates of grid points on the first chessboard grid; wherein, the first chessboard grid is the chessboard grid in the sample image; The coordinates of the grid points on the second chessboard are determined based on the coordinates of the grid points on the first chessboard grid; wherein, the second chessboard grid is the chessboard grid in the processed image; Based on the coordinates of the grid points on the first chessboard and the coordinates of the grid points on the second chessboard, the mapping relationship between the pixels in the sample image and the pixels in the processed image is determined. Based on the mapping relationship, the preset pixel mapping table is generated.

7. An electronic device, characterized in that, include: A processor and a memory, wherein the processor and the memory are connected; The memory is used to store programs; The processor is used to run a program stored in the memory and perform the method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a computer, performs the method as described in any one of claims 1-5.

Images