An image processing method, apparatus and electronic device

Abstract

This disclosure relates to an image processing method, apparatus, and electronic device, belonging to the field of computer information processing technology. The image processing method includes: acquiring a two-dimensional image to be processed; converting the pixel coordinates of the two-dimensional image into world coordinates in real-world space through camera calibration; segmenting the two-dimensional image based on key points calibrated in the two-dimensional image to obtain a target two-dimensional image; marking a second target block corresponding to a first target block in the target two-dimensional image in a virtual three-dimensional model space, the virtual three-dimensional model space being simulated based on real-world space; and mapping at least one target point in the first target block to the virtual three-dimensional model space containing the second target block based on the pixel coordinates and world coordinates of at least one vertex of the first target block. This disclosure achieves reasonable coordinate mapping and high-fidelity reproduction of the real-world two-dimensional image in a virtual three-dimensional model space.

Description

Technical Field

[0001] This disclosure belongs to the field of computer information processing technology, and specifically relates to an image processing method, apparatus and electronic device. Background Technology

[0002] In existing image processing, due to the low accuracy of camera calibration and the errors in the restoration of artistic scenes, it is impossible to convert 2D (2-dimensional) planar images into 3D (3-dimensional) model space with high fidelity, let alone restore the 3D real world and 3D model space with high fidelity.

[0003] The main reasons for the low accuracy of camera calibration are: In traditional camera calibration, the calibration accuracy is affected by factors such as the accuracy of world coordinates (or the printing and measurement accuracy of the calibration plate), the number of feature points, the accuracy of feature point extraction, the number of calibration images, and the angle at which the images are taken. These factors may be unavoidable errors, making it difficult to obtain high-precision calibration.

[0004] The main reason for the error in the reproduction of art scenes is that when building and reproducing 3D real-world space in UE (Unreal Engine), there are limitations such as geographical, cost and human factors, which make it impossible to completely and accurately reproduce 3D real-world space 1:1. This results in the coordinates of the 3D real-world space obtained by latitude and longitude coordinate conversion being deviated in the position of the constructed simulation 3D model space. Summary of the Invention

[0005] The purpose of this disclosure is to provide an image processing method, apparatus, and electronic device that achieves reasonable coordinate mapping and high-fidelity restoration between virtual 3D space and real space. The solution disclosed herein has good robustness and accuracy, and can achieve a relatively ideal mapping effect even when there are errors in the modeling of the virtual 3D model space.

[0006] In a first aspect, embodiments of this disclosure provide an image processing method, the method comprising:

[0007] Obtain the two-dimensional image to be processed;

[0008] The pixel coordinates of a two-dimensional image are converted into world coordinates in real-world space through camera calibration.

[0009] Based on the key points calibrated in the two-dimensional image, the two-dimensional image is segmented to obtain a target two-dimensional image containing at least one first target block;

[0010] In a virtual three-dimensional model space, at least one second target block corresponding to at least one first target block in the target two-dimensional image is marked. The virtual three-dimensional model space is simulated based on the real-world space.

[0011] Based on the pixel coordinates and world coordinates of at least one vertex of the first target block, at least one target point in the first target block is mapped to the virtual 3D model space where the second target block is located.

[0012] Optionally, the pixel coordinates of the two-dimensional image are converted into world coordinates in real-world space through camera calibration, including:

[0013] Obtain template images by taking pictures of two-dimensional images from at least two angles using a camera;

[0014] Identify at least one feature point in the template image;

[0015] Obtain the camera's intrinsic parameters, extrinsic parameters, and radial distortion coefficients;

[0016] Based on intrinsic parameters, extrinsic parameters, and radial distortion coefficients, the pixel coordinates of at least one feature point in a two-dimensional image are transformed to obtain world coordinates in real-world space.

[0017] Optionally, based on intrinsic parameters, extrinsic parameters, and radial distortion coefficients, the pixel coordinates of at least one feature point in the two-dimensional image are transformed to obtain world coordinates in real-world space, including:

[0018] Based on intrinsic parameters, extrinsic parameters, and radial distortion coefficients, the pixel coordinates of at least one feature point in the two-dimensional image are converted into world coordinates to obtain world coordinates in real-world space.

[0019] Optionally, based on the key points calibrated in the two-dimensional image, the two-dimensional image is segmented to obtain a target two-dimensional image containing at least one first target block, including:

[0020] Based on the key points calibrated in the two-dimensional image, the two-dimensional image is divided into at least one polygonal first target block, and each vertex of the first target block is a key point;

[0021] Mark the pixel coordinates of the vertices of the first target block and the identifier of the first target block on each first target block to obtain a two-dimensional image of the target.

[0022] Optionally, marking at least one second target block corresponding to at least one first target block in the target two-dimensional image in the virtual three-dimensional model space includes:

[0023] Based on the world coordinates of the vertices of the first target block and the identifier of the first target block, mark the second target block corresponding to the position of the first target block in the virtual 3D model space, and obtain the second target block corresponding to the first target block.

[0024] Optionally, based on the pixel coordinates and world coordinates of at least one vertex of the first target block, at least one target point in the first target block is mapped to the virtual 3D model space where the second target block resides, including:

[0025] Obtain any target point in the target 2D image;

[0026] The first target block containing the target point is segmented to obtain two triangular blocks corresponding to the first target block;

[0027] Determine the centroid coordinates of the target point in the centroid coordinate system of the triangle containing the target point;

[0028] Based on the centroid coordinates of the target point and the coordinates of the vertices of the triangular block in the virtual 3D model space, the coordinates of the target point in the virtual 3D model space are obtained.

[0029] Optionally, the centroid coordinates of the target point are determined in the centroid coordinate system of the triangular block containing the target point, including:

[0030] Determine the vector values ​​between the target point and the vertices of the triangle block;

[0031] Based on the vector values, the centroid coordinates of the target point are obtained.

[0032] In a second aspect, embodiments of this disclosure provide an image processing apparatus, comprising:

[0033] The acquisition module is used to acquire the two-dimensional image to be processed.

[0034] The processing module is used to convert the pixel coordinates of a 2D image into world coordinates in real-world space through camera calibration; to segment the 2D image based on the key points calibrated in the 2D image to obtain a target 2D image containing at least one first target block; to mark at least one second target block corresponding to at least one first target block in the target 2D image in a virtual 3D model space, the virtual 3D model space being simulated based on real-world space; and to map at least one target point in the first target block to the virtual 3D model space where the second target block is located based on the pixel coordinates and world coordinates of the vertices of the at least one first target block.

[0035] Optionally, the pixel coordinates of the two-dimensional image are converted into world coordinates in real-world space through camera calibration, including:

[0036] Obtain template images by taking pictures of two-dimensional images from at least two angles using a camera;

[0037] Identify at least one feature point in the template image;

[0038] Obtain the camera's intrinsic parameters, extrinsic parameters, and radial distortion coefficients;

[0039] Based on intrinsic parameters, extrinsic parameters, and radial distortion coefficients, the pixel coordinates of at least one feature point in a two-dimensional image are transformed to obtain world coordinates in real-world space.

[0040] Optionally, based on intrinsic parameters, extrinsic parameters, and radial distortion coefficients, the pixel coordinates of at least one feature point in the two-dimensional image are transformed to obtain world coordinates in real-world space, including:

[0041] Based on intrinsic parameters, extrinsic parameters, and radial distortion coefficients, the pixel coordinates of at least one feature point in the two-dimensional image are converted into world coordinates to obtain world coordinates in real-world space.

[0042] Optionally, based on the key points calibrated in the two-dimensional image, the two-dimensional image is segmented to obtain a target two-dimensional image containing at least one first target block, including:

[0043] Based on the key points calibrated in the two-dimensional image, the two-dimensional image is divided into a first target block of at least one polygon, and each vertex of the first target block is a key point.

[0044] Mark the pixel coordinates of the vertices of the first target block and the identifier of the first target block on each first target block to obtain a two-dimensional image of the target.

[0045] Optionally, marking at least one second target block corresponding to at least one first target block in the target two-dimensional image in the virtual three-dimensional model space includes:

[0046] Based on the world coordinates of the vertices of the first target block and the identifier of the first target block, mark the second target block corresponding to the position of the first target block in the virtual 3D model space, and obtain the second target block corresponding to the first target block.

[0047] Optionally, based on the pixel coordinates and world coordinates of at least one vertex of the first target block, at least one target point in the first target block is mapped to the virtual 3D model space where the second target block resides, including:

[0048] Obtain any target point in the target 2D image;

[0049] The first target block containing the target point is segmented to obtain two triangular blocks corresponding to the first target block;

[0050] Determine the centroid coordinates of the target point in the centroid coordinate system of the triangle containing the target point;

[0051] The coordinates of the target point in the virtual 3D model space are calculated based on the centroid coordinates of the target point and the coordinates of the vertices of the triangular block in the virtual 3D model space.

[0052] Optionally, the centroid coordinates of the target point are determined in the centroid coordinate system of the triangular block containing the target point, including:

[0053] Determine the vector values ​​between the target point and the vertices of the triangle block;

[0054] Based on the vector values, the centroid coordinates of the target point are obtained.

[0055] Thirdly, embodiments of this disclosure provide an electronic device, including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the image processing method described above.

[0056] Fourthly, embodiments of this disclosure provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the image processing method described above.

[0057] In this embodiment, a two-dimensional image to be processed is acquired; the pixel coordinates of the two-dimensional image are converted into world coordinates in real-world space through camera calibration; the two-dimensional image is segmented according to the key points calibrated in the two-dimensional image to obtain a target two-dimensional image containing at least one first target block; at least one second target block corresponding to at least one first target block in the target two-dimensional image is marked in a virtual three-dimensional model space, which is simulated based on real-world space; at least one target point in the first target block is mapped to the virtual three-dimensional model space where the second target block is located based on the pixel coordinates and world coordinates of the vertices of at least one first target block, thus achieving reasonable coordinate mapping and high fidelity between the virtual three-dimensional space and the real space. The solution disclosed herein has good robustness and accuracy, and can obtain a relatively ideal mapping effect even when there are errors in the modeling of the virtual three-dimensional model space, making it widely applicable. Attached Figure Description

[0058] Figure 1 A schematic diagram of the world coordinate system provided by an embodiment of this disclosure is shown;

[0059] Figure 2 A schematic diagram illustrating the correspondence between the camera coordinate system and the image coordinate system provided in an embodiment of this disclosure is shown.

[0060] Figure 3A schematic diagram illustrating the relationship between the image coordinate system and the pixel coordinate system provided in the embodiments of this disclosure is shown;

[0061] Figure 4 A flowchart of an image processing method provided by an embodiment of this disclosure is shown;

[0062] Figure 5 A flowchart illustrating a specific embodiment of the camera calibration method provided in this disclosure is shown.

[0063] Figure 6 This illustration shows a segmentation effect diagram of a two-dimensional image captured by a camera in a specific embodiment provided in this disclosure;

[0064] Figure 7 This illustration shows the effect of determining a corresponding second target block in a virtual three-dimensional model space based on a first target block in a specific embodiment provided in this disclosure;

[0065] Figure 8 A schematic diagram of centroid coordinates provided in an embodiment of this disclosure is shown;

[0066] Figure 9 The illustration shows a schematic diagram of the effect of determining a second target block in a virtual three-dimensional model space in a specific embodiment provided in this disclosure;

[0067] Figure 10 A flowchart illustrating the image processing method in a specific embodiment provided in this disclosure is shown;

[0068] Figure 11 A schematic diagram of the structure of the image processing apparatus provided in an embodiment of this disclosure is shown;

[0069] Figure 12 A schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure is shown;

[0070] Figure 13 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure is shown. Detailed Implementation

[0071] The technical solutions of the embodiments of this disclosure will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure are within the scope of protection of this disclosure.

[0072] The terms "first," "second," etc., used in this disclosure and in the claims are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this disclosure can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0073] The embodiments disclosed herein mainly relate to the mapping of images from real-world space to virtual three-dimensional model space. The mapping from real-world space to virtual three-dimensional model space is mainly achieved by acquiring images of real-world space using a camera device (such as a camera) to obtain a two-dimensional image. The pixel coordinates of the two-dimensional image are converted into world coordinates in real-world space through camera calibration. Furthermore, the target point is mapped to the virtual three-dimensional model space based on the centroid coordinates of the target point in the two-dimensional image.

[0074] The virtual three-dimensional model space involved in this disclosure can be obtained by simulating the real environment in reality. Of course, in one possible implementation, the scene in the virtual three-dimensional model space can also correspond to the real scene.

[0075] It should be noted that the scene within the virtual 3D model space may include various virtual objects. Each virtual object can be a virtual representation of a user or other entity within the virtual scene, and this disclosure does not limit its form. Each virtual object has its own shape, structure, and volume within the virtual scene, occupying a portion of the space within the virtual scene.

[0076] In this embodiment, the two-dimensional image to be processed is obtained by capturing real-world space using a real camera device. During the capturing process, the camera device transforms the three-dimensional real-world space in the world coordinate system into the camera coordinate system through a rigid transformation. A rigid transformation refers to a transformation in which the distance between two points in the image remains unchanged before and after the transformation, such as rotation, translation, or flipping. Then, the camera coordinate system is transformed into the image coordinate system through perspective projection. Furthermore, the image coordinate system is transformed into the pixel coordinate system through an affine transformation. The two-dimensional image is a two-dimensional image in the pixel coordinate system.

[0077] The world coordinate system, camera coordinate system, image coordinate system, and pixel coordinate system are explained here:

[0078] (1) World coordinate system: The world coordinate system refers to the absolute coordinate system in the objective three-dimensional space, which can also be called the objective coordinate system. For a camera in three-dimensional space, the position of the camera in three-dimensional space is determined by establishing a world coordinate system. This determines both the position of the camera and the position of the target object in three-dimensional space. Figure 1 A schematic diagram of the world coordinate system provided by an embodiment of this disclosure is shown, such as... Figure 1 As shown, once the world coordinate system (Ow-XwYwZw) is determined, the world coordinates of any point P within the world coordinate system are (Xw,Yw,Zw).

[0079] (2) Camera coordinate system: Figure 2 A schematic diagram illustrating the correspondence between the camera coordinate system and the image coordinate system provided in embodiments of this disclosure is shown, such as... Figure 2 As shown, the camera coordinate system is a coordinate system established with the optical center of the camera as the origin and the optical axis of the camera as the Zc axis of the camera coordinate system. The Xc and Yc axes of the camera coordinate system are parallel to the X and Y axes of the image coordinate system, respectively. The camera coordinates of any spatial point P in the camera coordinate system (Oc-XcYcZc) are (Xc,Yc,Zc).

[0080] (3) Image coordinate system: such as Figure 2 As shown, the image coordinate system is a coordinate system established with the center of the camera imaging plane as the origin O, and the X-axis and Y-axis parallel to the two vertical edges of the image plane respectively; the image coordinates of any point P in the image coordinate system (O-xy) are (x,y).

[0081] It should be noted that in the image coordinate system, the position of a pixel in the image is represented by physical units; under an ideal image imaging model, the coordinates of any three-dimensional point P in the image coordinate system are p(x...). u ,y u Considering the presence of distortion, the coordinates of this 3D spatial point P in the image coordinate system are P(X). d ,Y d In an ideal situation, the two coordinates of the three-dimensional point P (p(x)) are... u ,y u ) and P(X d ,Y d )) are overlapping.

[0082] (4) Pixel coordinate system: The pixel coordinate system takes the position of the upper left corner of the camera image plane as the origin, and the two coordinate axes parallel to the image coordinate system are the u axis and the v axis. The pixel coordinates of any point P in the pixel coordinate system (O-uv) are (u,v).

[0083] It should be noted that the images captured by the camera are stored as an M×N array. The value of each element in the M rows and N columns of the image represents the gray level of the image point. Each such element is called a pixel, and the pixel coordinate system is the image coordinate system with pixels as the unit.

[0084] Figure 3 A schematic diagram illustrating the relationship between the image coordinate system and the pixel coordinate system provided in embodiments of this disclosure is shown, such as... Figure 3 As shown, the origin of image 30 in the image coordinate system is the center point O2 of the image, and the x-axis and y-axis in the image coordinate system are parallel to the two vertical edges of the image plane, respectively.

[0085] The origin of image 30 in the pixel coordinate system is point O1, which is located at the upper left corner of the image. The u-axis and v-axis in the pixel coordinate system are parallel to the two coordinate axes of the image coordinate system, respectively. In the pixel coordinate system (O1-uv), the coordinates of the origin of the image coordinate system are (u0, v0).

[0086] Here, the transformation relationships between the world coordinate system, camera coordinate system, image coordinate system, and pixel coordinate system are explained:

[0087] The world coordinate system is (Ow-XwYwZw), and the world coordinates of any point within the world coordinate system are (Xw,Yw,Zw); the camera coordinate system is (Oc-XcYcZc), and the camera coordinates of any point within the camera coordinate system are (Xc,Yc,Zc); the image coordinate system is (O-xy), and the image coordinates of any point within the image coordinate system are (x,y); the pixel coordinate system is (O-uv), and the pixel coordinates of any point within the pixel coordinate system are (u,v).

[0088] The first transformation relationship between the pixel coordinate system (O-uv) and the image coordinate system (O-xy) is as follows:

[0089]

[0090]

[0091] The first transformation relationship can be represented by a matrix of homogeneous coordinates as follows:

[0092]

[0093] Where (u0, v0) are the coordinates of the origin of the image coordinate system in the pixel coordinate system, and d x and d y These represent the physical dimensions of each pixel along the x-axis and y-axis of the image plane, respectively.

[0094] Image coordinate system (O-xy) and camera coordinate system (Oc-X) c Yc Z c The second transformation relationship between ) is:

[0095]

[0096]

[0097] This second transformation relationship can be represented by a matrix of homogeneous coordinates as follows:

[0098]

[0099] Where f is the focal length of the camera, x and y are the image coordinates of the point in the image coordinate system, and Xc, Yc, and Zc are the camera coordinates of the point in the camera coordinate system.

[0100] Camera coordinate system (Oc-X) c Y c Z c ) and the world coordinate system is (Ow-X w Y w Z w The third transformation relationship between them is:

[0101]

[0102] Where R is a 3×3 orthogonal identity matrix, R is t is a three-dimensional translation vector, t is

[0103] Combining the first and second transformation relationships in the matrix form of homogeneous coordinates described above, we can obtain:

[0104]

[0105] After simplification, it becomes:

[0106]

[0107] Among them, c x It is the scale factor at the image level, c y It is the vertical scale factor of the image. The H matrix is ​​the intrinsic parameter matrix, which includes parameters determined by the camera's internal structure (camera intrinsic parameters), such as focal length and the coordinates of the principal point. x c y u0 and v0 are both camera intrinsic parameters; the M matrix is ​​the camera extrinsic parameter matrix, which contains the rotation matrix R and the translation vector t. The rotation matrix R and the translation vector t are determined by the position of the camera coordinate system relative to the world coordinate system. The camera intrinsic and extrinsic parameters can be determined through camera calibration.

[0108] It should be noted that the types of coordinate systems and the transformation relationships between them described above are not limited to this disclosure. In one possible implementation, other forms of coordinate systems and their corresponding transformation relationships may also be included. Based on coordinate system transformation and camera calibration, this disclosure displays two-dimensional images more accurately in the virtual three-dimensional model space.

[0109] The image processing method, apparatus, and electronic device provided in this disclosure will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0110] Figure 4 This is a flowchart of the image processing method provided in the embodiments of this disclosure, see below. Figure 4 The method may include the following steps:

[0111] Step 401: Obtain the two-dimensional image to be processed; preferably, the two-dimensional image is captured by a real camera device; after obtaining the two-dimensional image to be processed, the process may further include: performing image preprocessing on the two-dimensional image to obtain a preprocessed two-dimensional image. The preferred image preprocessing method is to remove noise from the two-dimensional image using a filtering algorithm, which improves the image quality and facilitates the selection, annotation, and calibration of key points in the image.

[0112] Step 402: Convert the pixel coordinates of the two-dimensional image into world coordinates in real-world space using camera calibration.

[0113] Step 403: Based on the key points calibrated in the two-dimensional image, the two-dimensional image is segmented to obtain a target two-dimensional image containing at least one first target block;

[0114] Step 404: Mark at least one second target block in the virtual three-dimensional model space that corresponds to at least one first target block in the target two-dimensional image. The virtual three-dimensional model space is obtained by simulating the real world space.

[0115] Step 405: Based on the pixel coordinates and world coordinates of at least one vertex of the first target block, map at least one target point in the first target block to the virtual 3D model space where the second target block is located.

[0116] The method in this embodiment converts the pixel coordinates of a two-dimensional image into world coordinates in real-world space through camera calibration; it segments the two-dimensional image based on the key points calibrated in the two-dimensional image to obtain a target two-dimensional image containing at least one first target block; it marks at least one second target block corresponding to at least one first target block in the target two-dimensional image in a virtual three-dimensional model space, which is simulated based on real-world space; and it maps at least one target point in the first target block to the virtual three-dimensional model space where the second target block is located based on the pixel coordinates and world coordinates of the vertices of the at least one first target block. This achieves reasonable coordinate mapping and high-fidelity restoration from a two-dimensional planar image in the real world to a three-dimensional image in the virtual three-dimensional model space, improving the fault tolerance, robustness, and applicability of coordinate positioning in the virtual three-dimensional model space.

[0117] In an optional embodiment of this disclosure, step 402 includes:

[0118] Step 4021: Obtain template images by taking pictures of the two-dimensional image from at least two angles using a camera; here, template images refer to at least two template images obtained by taking pictures from at least two angles.

[0119] Step 4022: Determine at least one feature point in the template image. It should be noted that since the two-dimensional image is an image in pixel coordinate system obtained by a camera, and the template image is also obtained by a camera taking pictures from different angles, the template image is also an image in pixel coordinate system. Template images at different angles are used to calibrate the intrinsic parameters, extrinsic parameters, and radial distortion coefficients of the camera. The feature point here is a pixel coordinate point that can reflect the image content of the template image. It should be noted that the template image is preferably a checkerboard image. When the template image is a checkerboard image, the feature point can be determined more accurately as the vertex of the checkerboard grid in the checkerboard image.

[0120] Step 4023: Obtain the intrinsic parameters, extrinsic parameters, and radial distortion coefficients of the camera;

[0121] Here, the process of obtaining the camera's extrinsic parameters is explained:

[0122] If the world coordinates of a point in real-world space are M = [X, Y, Z, 1] T ;

[0123] The pixel coordinates of a point within a two-dimensional image are m = [u, v, 1]. T Then the correspondence between the template image used for calibration and the two-dimensional image is: sm=A[R,t]M;

[0124] Where s is the scale factor from the world coordinate system to the image coordinate system, and the transformation relationship between the image coordinate system and the pixel coordinate system is as follows: Figure 3 As shown;

[0125] A is the camera's intrinsic parameter matrix. (u0,v0) are the pixel coordinates of a point in the two-dimensional image, α and β are the fusion values ​​of focal length and pixel aspect ratio, γ is the radial distortion coefficient, R is the rotation matrix, and t is the translation vector.

[0126] At this point, let Z = 0 when the chessboard image corresponds to the world coordinate system, and let the i-th column of the rotation matrix R be r. i Then we have:

[0127]

[0128] Let H = [h1 h2 h3] = λA[r1 r2 t], then the mapping from real-world space to a two-dimensional image or template image is: sm = HM;

[0129] Where H is the homography matrix, and r1 = λA -1 h1, r2 = λA -1 h2, r3 = r1 × r2, t = λA -1 h1,

[0130] Here, the process of obtaining the camera's intrinsic parameters is explained based on the camera's extrinsic parameters:

[0131] Based on the fact that r1 and r2 are orthogonal and their moduli are equal, the first constraint condition is determined as follows: The second constraint is:

[0132] definition:

[0133]

[0134]

[0135] The unknowns in B can be represented as a six-dimensional vector b, where b = [B 11 B 12 B 22 B 13 B 23 B 33 Let the i-th column in H be h. i h i =[h i1 h i2 h i3 ] T It can be determined Among them, v ij =[h i1 h j1 h i1 h j2 +h i2 h j1 h i2 h j2 h i3 h j1 +h i1 h j3 h i3 h j2 +h i2 h j3 h i3 h j3 ] T ,

[0136] According to the least squares method, the solution to Vb = 0 is V T The eigenvector corresponding to the smallest eigenvalue of V can be used to directly estimate b, which is then used to solve for the intrinsic parameters.

[0137] Specifically, the intrinsic parameters are determined through Cholesky decomposition, yielding the following intrinsic parameters:

[0138]

[0139]

[0140] γ=-B 12 α 2 β / λ;

[0141]

[0142] The process of obtaining the radial distortion coefficients of the camera, based on the camera's extrinsic and intrinsic parameters, is explained below:

[0143]

[0144]

[0145] Where (u,v) are the pixel coordinates under ideal conditions (i.e., without distortion), (u0,v0) represent the principal point, (x,y) are the pixel coordinates within the ideal 2D image, and k1 and k2 are the distortion parameters for the first two orders.

[0146] Convert it to matrix form as follows:

[0147]

[0148] If Dk = d, then the solution for the radial distortion coefficient is k = [k1 k2] T =(D T D) -1 D T d.

[0149] The above process for calculating the camera's intrinsic parameters, extrinsic parameters, and radial distortion coefficient yields highly accurate parameters that are easy to operate.

[0150] Figure 5 A flowchart illustrating a specific embodiment of the camera calibration method provided in this disclosure is shown, such as... Figure 5 As shown, in a specific embodiment, camera calibration is performed on any camera. A template with a checkerboard pattern is printed and pasted on a plane. Several template images are taken from different angles using the camera. Feature points of the checkerboard pattern in the template images are detected. Here, the feature points of the checkerboard pattern are the vertices of the checkerboard. The intrinsic and extrinsic parameters of the camera under ideal conditions (i.e., without distortion) are solved. The accuracy of the intrinsic and extrinsic parameters can be improved by maximum likelihood estimation.

[0151] Furthermore, the radial distortion coefficients of the camera are solved using least squares. Based on the solved intrinsic parameters, extrinsic parameters, and radial distortion coefficients, the accuracy is further improved by maximum likelihood estimation. At this point, the obtained intrinsic parameters, extrinsic parameters, and radial distortion coefficients of the camera have high accuracy and are better used for image processing.

[0152] Step 4024: Based on the intrinsic parameters, extrinsic parameters, and radial distortion coefficients, perform coordinate transformation on the pixel coordinates of at least one feature point in the two-dimensional image to obtain world coordinates in real-world space;

[0153] In the specific implementation of this step, the image coordinates of at least one feature point in the 2D image are (u,v), and (x,y,z) are the world coordinates; the transformation relationship between pixel coordinates and image coordinates is as follows: Figure 3 As shown;

[0154] The image coordinates are converted to world coordinates using the following formula:

[0155]

[0156] Among them, matrix This is the camera intrinsic parameter matrix;

[0157] matrix This is the camera extrinsic parameter matrix;

[0158] In this formula, the image coordinates are known, and the camera intrinsic parameter matrix has been obtained through calibration;

[0159]

[0160] Where M is the camera intrinsic parameter matrix, R is the rotation matrix, t is the translation matrix, and Z is the distance between the origin of the actual coordinate system and the origin of the camera coordinate system on the Z-axis.

[0161] After the above transformation, we get:

[0162]

[0163] When R, M, t, and Z are known, s is the only variable and can be calculated. Thus, the world coordinates of at least one feature point in the two-dimensional image can be obtained through the above transformation formula.

[0164] In an optional embodiment of this disclosure, step 403 may specifically include:

[0165] Step 4031: Based on the key points calibrated in the two-dimensional image, divide the two-dimensional image into at least one polygonal first target block, where each vertex of the first target block is a key point.

[0166] The first target block is preferably a quadrilateral block, but it can also be a target block of other shapes. This disclosure does not limit it to these shapes.

[0167] Here, when the first target block is a quadrilateral block, the two-dimensional image is divided into at least one quadrilateral first target block based on at least four key points in the two-dimensional image. This first target block is preferably obtained by connecting line segments of adjacent key pixels. The key points are preferably determined based on the image content of the two-dimensional image. For example, if the two-dimensional image is a crossroads, then the key pixel can be determined as each turning point of the crossroads; the selection of key pixels depends on the actual requirements.

[0168] Step 4032: Mark the pixel coordinates of the vertices of the first target block and the identifier of the first target block on each first target block to obtain the target two-dimensional image.

[0169] In this embodiment, the pixel coordinates of the vertices of each first target block are determined based on the two-dimensional image in the pixel coordinate system, and the first target blocks are identified, preferably by a number for each first target block.

[0170] Figure 6 This illustration shows a schematic diagram of the segmentation effect of a two-dimensional image captured by a camera in a specific embodiment provided in this disclosure, such as... Figure 6As shown, in a specific embodiment, the two-dimensional image 60 is a two-dimensional image captured by a surveillance camera at a certain intersection at a certain moment. Since the image content of the two-dimensional image 60 is a road intersection, the key points are determined as the points of the road traffic signs. The key points are connected to obtain several quadrilateral first target blocks. Further, based on the two-dimensional image 60, the pixel coordinates of the vertices of each first target block are determined, and each first target block is numbered in a certain order to obtain the identifier of each first target block (i.e., Figure 6 (The numerical number of the first target block of several quadrilaterals).

[0171] In an optional embodiment of this disclosure, step 404 may include:

[0172] Based on the world coordinates of the vertices of the first target block and the identifier of the first target block, mark the second target block corresponding to the position of the first target block in the virtual 3D model space, and obtain the second target block corresponding to the first target block;

[0173] In this embodiment, the first target block is obtained by segmenting a two-dimensional image. A top view corresponding to the two-dimensional image is found in the virtual three-dimensional model space. The top view is segmented according to the first target block to obtain the second target block corresponding to the first target block.

[0174] Specifically, based on the markers corresponding to the vertices of the first target block in the two-dimensional image, the points corresponding to the markers in the virtual three-dimensional model space are found as the vertices of the second target block. It should be noted that the points corresponding to the markers can be determined manually or through a machine learning model, and this disclosure is not limited to this.

[0175] Simultaneously, based on the first identifier of the first target block, each second target block is numbered in the same order as the two-dimensional image numbering process to obtain the second identifier of each second target block. At least one second target block can be determined by the position of the vertex of each second target block and the second identifier.

[0176] Figure 7 This illustration shows a specific embodiment provided in this disclosure, illustrating the effect of determining a corresponding second target block in a virtual three-dimensional model space based on a first target block. Figure 7 As shown, in another specific embodiment, a top view 70 corresponding to the two-dimensional image 60 is found, and the top view 70 is segmented in the virtual three-dimensional model space to obtain multiple second target blocks. Here, the second target blocks correspond to the first target blocks in the two-dimensional image 60. Furthermore, the position of the vertices and the second identifier of the second target blocks can also be determined.

[0177] In an optional embodiment of this disclosure, step 405 may specifically include:

[0178] Step 4051: Obtain any target point in the target two-dimensional image; the target point can be randomly selected or manually selected, and this disclosure does not impose any restrictions.

[0179] Step 4052: The first target block containing the target point is segmented to obtain two triangular blocks corresponding to the first target block. It should be noted that this disclosure does not limit the way the first target block is divided into triangular blocks, but it is preferred to divide it based on any diagonal line on the first target block.

[0180] Step 4053: Determine the centroid coordinates of the target point in the centroid coordinate system of the triangular block where the target point is located; here, the centroid coordinate system refers to the coordinate system in the geometric structure in which the position of a point in the figure is relative to each vertex; taking a triangle as an example, the points in the triangle can be represented by a matrix, which is related to each vertex of the triangle.

[0181] Step 4054: Calculate the coordinates of the target point in the virtual 3D model space based on the centroid coordinates of the target point and the coordinates of the vertices of the triangle block in the virtual 3D model space; the coordinates of the vertices of the triangle block in the virtual 3D model space are obtained directly by the software that constructs the space.

[0182] In an optional embodiment of this disclosure, step 4053 may include:

[0183] Step 40531: Determine the vector values ​​between the target point and the vertices of the triangle block;

[0184] Step 40532: Obtain the centroid coordinates of the target point based on the vector values.

[0185] Figure 9 A schematic diagram of centroid coordinates provided in an embodiment of this disclosure is shown, as follows: Figure 9 As shown, the formula for calculating the coordinates of the centroid of point P inside triangle ΔABC is:

[0186] P=A+m(BA)+n(CA), that is, P=(1-mn)A+mB+n(CA);

[0187] Given the vertices A, B, and C of the triangle, we can obtain the coordinates of P. We then need to determine the parameters m and n.

[0188] Let PA be denoted as vector v2, BA as vector v0, and CA as vector v1. Then the formula for calculating the centroid coordinates can be written as:

[0189] Then multiply by v0 and v1 respectively to get

[0190] v2·v0=m(v0·v0)+n(v1v0)

[0191] v2·v1=m(v0·v1)+n(v1v1)

[0192] Where, d 20 =v2·v0,d 21 =v2·v1,d 00 =v0·v0,d 01 =v0·v1,d 11 =v1·v1;

[0193] Right now

[0194] According to Leibniz's formula, the coordinates of the centroid of any point inside triangle ΔABC are: in,

[0195] The centroid coordinates of the above embodiment are relative to the triangle block and are independent of the coordinate system of the image where the triangle block is located. Therefore, based on the centroid coordinates of the target point, the corresponding coordinates of the target point on the triangle block in the virtual three-dimensional model space can be determined based on the vertices of the triangle block in the virtual three-dimensional model space. It should be noted that the point should be within the plane formed by the vertices of the triangle block in the virtual three-dimensional model space.

[0196] Figure 9 This illustration shows a schematic diagram of the effect of determining a second target block in a virtual three-dimensional model space in a specific embodiment provided in this disclosure, such as... Figure 9 As shown, the two-dimensional image pixel space (i.e., the pixel coordinate system of the two-dimensional image 60) obtained by perspective projection of the two-dimensional image 60 is obtained. Key pixels are determined according to the image content in the two-dimensional image 60. Based on the key pixels, the two-dimensional image is divided into multiple quadrilateral first target blocks, and vertex pixel coordinates and first identifiers are obtained for each first target block.

[0197] Figure 9 The first dashed box 91 in the figure shows the vertex coordinates and first identifier of a first target block. The first target block is a quadrilateral block with the first identifier being 5. The coordinates of each vertex are (651,739), (1009,755), (483,1073), and (1013,1077) respectively.

[0198] The 2D image 60 is calibrated using a camera calibration method, converting the pixel coordinates of the 2D image 60 into world coordinates in real-world space. At least one second target block corresponding to the first target block is then marked in the virtual 3D model space. Figure 9The image shows a second identifier of a second target block within the second dashed frame 92. The second identifier of the second target block is 5, meaning that the second target block within the second dashed frame 92 corresponds to the first target block within the first dashed frame 91.

[0199] Based on the first target block and the second target block, we obtain the following: Figure 9 The orthogonal projection top view in the virtual 3D model space shown in the figure is that, based on the pixel coordinates and world coordinates of the vertices of at least one first target block and the vertices of the second target block, at least one target point in the first target block is mapped to the virtual 3D model space where the second target block is located. The first target block after the mapping is identified as 5, and its coordinates are (-5648, -68081.0), (-5270.0, -68081.0), (-5644.0, -67655.0), and (-5268.0, -67660.0) respectively.

[0200] This eliminates the effect of perspective on objects appearing larger when closer and smaller when farther away, thus achieving a more accurate correspondence between the coordinates of a two-dimensional image and the spatial coordinates of a virtual three-dimensional model.

[0201] Figure 10 A flowchart illustrating the image processing method in a specific embodiment provided in this disclosure is shown, such as... Figure 10 As shown, a two-dimensional image of a crossroads in real three-dimensional space is captured by a camera. The specific process of image processing is as follows:

[0202] Step 1001: Perform image preprocessing on the two-dimensional image to obtain a processed two-dimensional image, which has higher image quality.

[0203] Step 1002: Convert the pixel coordinates of the two-dimensional image into world coordinates in real-world space using camera calibration.

[0204] Step 1003: Based on at least four key points calibrated in the two-dimensional image, the two-dimensional image is segmented to obtain a target two-dimensional image containing at least one first target block;

[0205] Step 1004: Segment the first target block where the target point is located to obtain two triangular blocks corresponding to the first target block, and determine the coordinates of the vertices of the triangular blocks in the virtual three-dimensional model space;

[0206] Step 1005: Determine the coordinates of the target point in the virtual 3D model space based on the coordinates of the vertices of the triangle block in the virtual 3D model space and the centroid coordinates of the triangle block in the target 2D image.

[0207] Step 1006: Complete the processing of the two-dimensional image so that the two-dimensional image corresponds to the virtual three-dimensional model space; such a correspondence between the two-dimensional image and the virtual three-dimensional model space after two processing can eliminate the deviation of near objects appearing larger and far objects appearing smaller due to perspective.

[0208] The image processing method in this embodiment converts the pixel coordinates of a two-dimensional image into world coordinates in real-world space through camera calibration; it segments the two-dimensional image based on at least four key points calibrated to obtain a target two-dimensional image; it marks a second target block corresponding to a first target block in the target two-dimensional image in a virtual three-dimensional model space, which is simulated based on real-world space; and it maps at least one target point in the first target block to the virtual three-dimensional model space where the second target block is located based on the pixel coordinates and world coordinates of at least one vertex of the first target block. This achieves reasonable coordinate mapping and height restoration from a two-dimensional planar image in the real world to a three-dimensional image in the virtual three-dimensional model space, and can eliminate the deviation problem of near objects appearing larger than distant objects caused by perspective, thereby improving the fault tolerance, robustness, and applicability of coordinate positioning in the virtual three-dimensional model space.

[0209] All of the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of this disclosure, and will not be described in detail here.

[0210] Figure 11 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this disclosure. See also... Figure 11 The device 1100 includes:

[0211] The acquisition module 1101 is used to acquire the two-dimensional image to be processed;

[0212] The processing module 1102 is used to convert the pixel coordinates of a two-dimensional image into world coordinates in real-world space through camera calibration; to segment the two-dimensional image based on the key points calibrated in the two-dimensional image to obtain a target two-dimensional image containing at least one first target block; to mark at least one second target block corresponding to at least one first target block in the target two-dimensional image in a virtual three-dimensional model space, the virtual three-dimensional model space being simulated based on real-world space; and to map at least one target point in the first target block to the virtual three-dimensional model space where the second target block is located based on the pixel coordinates and world coordinates of the vertices of the at least one first target block.

[0213] Optionally, the pixel coordinates of the two-dimensional image are converted into world coordinates in real-world space through camera calibration, including:

[0214] Obtain template images by taking pictures of two-dimensional images from at least two angles using a camera;

[0215] Identify at least one feature point in the template image;

[0216] Obtain the camera's intrinsic parameters, extrinsic parameters, and radial distortion coefficients;

[0217] Based on intrinsic parameters, extrinsic parameters, and radial distortion coefficients, the pixel coordinates of at least one feature point in a two-dimensional image are transformed to obtain world coordinates in real-world space.

[0218] Optionally, based on intrinsic parameters, extrinsic parameters, and radial distortion coefficients, the pixel coordinates of at least one feature point in the two-dimensional image are transformed to obtain world coordinates in real-world space, including:

[0219] Based on intrinsic parameters, extrinsic parameters, and radial distortion coefficients, the pixel coordinates of at least one feature point in the two-dimensional image are converted into world coordinates to obtain world coordinates in real-world space.

[0220] Optionally, based on the key points calibrated in the two-dimensional image, the two-dimensional image is segmented to obtain a target two-dimensional image containing at least one first target block, including:

[0221] Based on the key points calibrated in the two-dimensional image, the two-dimensional image is divided into a first target block of at least one polygon, and each vertex of the first target block is a key point.

[0222] Mark the pixel coordinates of the vertices of the first target block and the identifier of the first target block on each first target block to obtain a two-dimensional image of the target.

[0223] Optionally, marking at least one second target block corresponding to at least one first target block in the target two-dimensional image in the virtual three-dimensional model space includes:

[0224] Based on the world coordinates of the vertices of the first target block and the identifier of the first target block, mark the second target block corresponding to the position of the first target block in the virtual 3D model space, and obtain the second target block corresponding to the first target block.

[0225] Optionally, based on the pixel coordinates and world coordinates of at least one vertex of the first target block, at least one target point in the first target block is mapped to the virtual 3D model space where the second target block resides, including:

[0226] Obtain any target point in the target 2D image;

[0227] The first target block containing the target point is segmented to obtain two triangular blocks corresponding to the first target block;

[0228] Determine the centroid coordinates of the target point in the centroid coordinate system of the triangle containing the target point;

[0229] Based on the centroid coordinates of the target point and the coordinates of the vertices of the triangular block in the virtual 3D model space, the coordinates of the target point in the virtual 3D model space are obtained.

[0230] Optionally, the centroid coordinates of the target point are determined in the centroid coordinate system of the triangular block containing the target point, including:

[0231] Determine the vector values ​​between the target point and the vertices of the triangle block;

[0232] Based on the vector values, the centroid coordinates of the target point are obtained.

[0233] The apparatus provided in this embodiment performs camera calibration processing on a two-dimensional image by means of camera calibration, dividing the two-dimensional image into multiple first target blocks, and marking multiple second target blocks corresponding to the first target blocks in a virtual three-dimensional model space. Further, based on the first target blocks and the second target blocks, the target points in the first target blocks are mapped to the virtual three-dimensional model space where the second target blocks are located, so that the two-dimensional image and the virtual three-dimensional model space correspond, realizing the simulation of the real three-dimensional space in the virtual three-dimensional model space. The resulting coordinate positioning in the virtual three-dimensional model space has high fault tolerance, strong robustness and wide applicability.

[0234] It should be noted that the image processing apparatus provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the apparatus can be divided into different functional modules to complete all or part of the functions described above. In addition, the image processing apparatus and the image processing method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0235] The image processing device in this disclosure can be a virtual device, or a component, integrated circuit, or chip in a server or terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network-attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This disclosure does not impose specific limitations.

[0236] The image processing apparatus in this embodiment can be a device with an operating system. The operating system can be Android, iOS, or other possible operating systems; this embodiment does not specifically limit the specific operating system.

[0237] The image processing apparatus provided in this disclosure can achieve... Figures 1 to 10 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0238] Optionally, such as Figure 12 As shown, this disclosure also provides an electronic device 1200, including a processor 1201, a memory 1202, and a program or instructions stored in the memory 1202 and executable on the processor 1201. When the program or instructions are executed by the processor 1201, they implement the various processes described in the above-described processing method embodiments and achieve the same technical effects. To avoid repetition, further details are omitted here. It should be noted that the electronic devices in this disclosure include the aforementioned mobile electronic devices and non-mobile electronic devices.

[0239] Figure 13 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this disclosure.

[0240] The electronic device 1300 includes, but is not limited to, components such as: radio frequency unit 1301, network module 1302, audio output unit 1303, input unit 1304, sensor 1305, display unit 1306, user input unit 1307, interface unit 1308, memory 1309, and processor 1310.

[0241] Those skilled in the art will understand that the electronic device 1300 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 1310 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 13 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0242] It should be understood that, in this embodiment of the disclosure, the input unit 1304 may include a graphics processing unit (GPU) 13041 and a microphone 13042. The GPU 13041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 1306 may include a display panel 13061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 1307 includes a touch panel 13071 and other input devices 13072. The touch panel 13071 is also called a touch screen. The touch panel 13071 may include a touch detection device and a touch controller. Other input devices 13072 may include, but are not limited to, a physical keyboard, function keys (such as volume control buttons, power buttons, etc.), a trackball, a mouse, and a joystick, which will not be described in detail here. The memory 1309 can be used to store software programs and various data, including but not limited to applications and operating systems. Processor 1310 may integrate an application processor and a modem processor. The application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into processor 1310.

[0243] This disclosure also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described image processing method embodiments and achieve the same technical effects. To avoid repetition, further details are omitted here.

[0244] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0245] This disclosure also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above-described image processing method embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0246] It should be understood that the chip mentioned in the embodiments of this disclosure may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0247] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this disclosure is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0248] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solutions of this disclosure, in essence or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this disclosure.

[0249] The embodiments of this disclosure have been described above with reference to the accompanying drawings. However, this disclosure is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this disclosure without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this disclosure.

Claims

1. An image processing method, characterized by, The method comprises: acquiring a two-dimensional image to be processed; converting pixel coordinates of the two-dimensional image into world coordinates in a real world space through camera calibration; segmenting the two-dimensional image according to key points of calibration of the two-dimensional image to obtain a target two-dimensional image containing at least one first target block; labeling at least one second target block corresponding to the at least one first target block in the target two-dimensional image in a virtual three-dimensional model space, the virtual three-dimensional model space being simulated according to the real world space; mapping at least one target point in the first target block to the virtual three-dimensional model space where the second target block is located according to pixel coordinates of vertices of the at least one first target block.

2. The image processing method of claim 1, wherein, Converting pixel coordinates of the two-dimensional image into world coordinates in a real world space through camera calibration comprises: obtaining template images photographed at least two angles by using a camera; determining at least one feature point in the template images; obtaining intrinsic parameters, extrinsic parameters and radial distortion coefficients of the camera; converting pixel coordinates of the at least one feature point in the two-dimensional image into world coordinates in a real world space according to the intrinsic parameters, the extrinsic parameters and the radial distortion coefficients.

3. The image processing method of claim 2, wherein, Converting pixel coordinates of the at least one feature point in the two-dimensional image into world coordinates in a real world space according to the intrinsic parameters, the extrinsic parameters and the radial distortion coefficients comprises: converting pixel coordinates of a pixel coordinate system of the at least one feature point in the two-dimensional image into coordinates of a world coordinate system to obtain world coordinates in a real world space according to the intrinsic parameters, the extrinsic parameters and the radial distortion coefficients.

4. The image processing method of claim 1, wherein, Segmenting the two-dimensional image according to key points of calibration of the two-dimensional image to obtain a target two-dimensional image containing at least one first target block comprises: dividing the two-dimensional image into at least one polygonal first target block according to key points of calibration of the two-dimensional image, each vertex of the first target block being a key point; labeling pixel coordinates of the vertices of the first target block and an identification of the first target block on each first target block to obtain a target two-dimensional image.

5. The image processing method of claim 4, wherein, Labeling at least one second target block corresponding to the at least one first target block in the target two-dimensional image in a virtual three-dimensional model space comprises: labeling a second target block corresponding to a position of the first target block in the virtual three-dimensional model space according to world coordinates of the vertices of the first target block and the identification of the first target block to obtain the second target block corresponding to the first target block.

6. The image processing method of claim 1, wherein, Mapping at least one target point in the first target block to the virtual three-dimensional model space where the second target block is located according to pixel coordinates of vertices of the at least one first target block comprises: acquiring any target point in the target two-dimensional image; segmenting the first target block where the target point is located to obtain two triangular blocks corresponding to the first target block; determining a centroid coordinate of the target point in a centroid coordinate system of the triangular block where the target point is located; According to the centroid coordinates of the target point and the coordinates of the vertices of the triangular block in the virtual three-dimensional model space, the coordinates of the target point in the virtual three-dimensional model space are obtained.

7. The image processing method of claim 6, wherein, In the centroid coordinate system of the triangular block where the target point is located, the centroid coordinates of the target point are determined, including: Determining the vector values between the target point and the vertices of the triangular block; According to the vector values, the centroid coordinates of the target point are obtained.

8. An image processing apparatus characterized by comprising: Comprising: An acquisition module is configured to acquire a two-dimensional image to be processed; A processing module is configured to convert pixel coordinates of the two-dimensional image into world coordinates in a real world space by a camera calibration method, and segment the two-dimensional image according to key points of the two-dimensional image to obtain a target two-dimensional image containing at least one first target block; At least one second target block corresponding to the at least one first target block in the target two-dimensional image is marked in a virtual three-dimensional model space, which is simulated according to the real world space; According to pixel coordinates of the vertices of the at least one first target block, at least one target point in the first target block is mapped to the virtual three-dimensional model space where the second target block is located.

9. An electronic device, comprising: A processor, a memory, and a program or instructions stored on the memory and executable on the processor are included, and the program or instructions are executed by the processor to implement the steps of the image processing method according to any one of claims 1-7.

10. A readable storage medium, characterized by, A program or instructions are stored on the readable storage medium, and the program or instructions are executed by the processor to implement the steps of the image processing method according to any one of claims 1-7.

Images