An image processing method, apparatus, electronic device, and storage medium

Abstract

The application provides an image processing method and device, electronic equipment and storage medium, the method comprises the following steps: acquiring a first dot array corresponding to a target projection channel; rotating a virtual plane from an initial position to a target position corresponding to current eye point information, and determining the coordinates of a plurality of target vertexes on the virtual plane after rotation; determining the coordinates of each second pixel point based on the coordinates of the plurality of target vertexes after rotation; for each original pixel point in the original dot array, determining a third pixel point corresponding to the original pixel point based on the positional relationship between the plurality of target first pixel points and the original pixel point, and the plurality of third pixel points constitute a third dot array; and performing morphing processing on an input source image using the third dot array to obtain a projection image corresponding to the first dot array. Through the above image processing method, device, electronic equipment and storage medium, the problem of low correction and debugging efficiency of the projection image and the inability to adapt to the application scenario of real-time change of the eye point position is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to an image processing method and device, electronic equipment and a storage medium. BACKGROUND

[0002] Virtual simulation is a computer system that can create and experience a virtual world. Such a virtual world is generated by a computer and can be a reproduction of the real world or an imagined world. Users can interact with the virtual world through various sensory channels such as vision, hearing, and touch. Virtual simulation fusion technology is often used in virtual simulation processes. Virtual simulation fusion refers to creating a three-dimensional interface through projection fusion. An observer stands at the visual center of the projection, forming an immersive interactive scene. Virtual simulation fusion is a typical application of fusion splicing and has a large application market in education, scientific research, military, and aerospace. Virtual simulation fusion creates a three-dimensional interface through projection fusion.

[0003] However, in existing virtual simulation fusion scenarios, such as flight simulator scenarios, the eye point position of the driver is fixed on the driver's seat. Once the eye point position changes, the frustum calibration map will also change, and the projection image needs to be re-geometrically corrected and debugged, resulting in low efficiency of projection image correction and debugging, which cannot adapt to application scenarios with real-time dynamic changes in eye point position. SUMMARY

[0004] Therefore, the purpose of the present application is to provide an image processing method and device, electronic equipment and a storage medium to solve the problem of low efficiency of projection image correction and debugging, which cannot adapt to application scenarios with real-time dynamic changes in eye point position.

[0005] In a first aspect, an image processing method is provided, comprising:

[0006] Obtaining a first point array corresponding to a target projection channel, the first point array being known and used to describe the projection effect of the input source image projected by the target projection channel;

[0007] In a virtual simulation scenario, rotating a virtual plane corresponding to the target projection channel from an initial position to a target position corresponding to current eye point information, and determining the coordinates of a plurality of target vertices on the virtual plane after rotation;

[0008] Determining the coordinates of each second pixel point in a second point array based on the coordinates of the plurality of target vertices after rotation, the second point array being a matrix of a plurality of intersection points formed by a plurality of lines intersecting the virtual plane, and the plurality of lines being lines between each pixel point on the virtual projection screen and the current eye point position.

[0009] For each original pixel point in the original dot array, based on the positional relationship between the plurality of target first pixel points corresponding to the original pixel point and the original pixel point, a third pixel point corresponding to the original pixel point in the rotated virtual plane is determined, a third dot array is formed by the plurality of third pixel points, and the plurality of target first pixel points are pixel points in the first dot array;

[0010] The third dot array is used for morphing processing on the input source image to obtain a projection image corresponding to the first dot array projected on the actual projection screen.

[0011] Optionally, the current eye point information includes a current eye point position, a current observation direction, and a physical visual angle parameter, the physical visual angle parameter is a projection angle of the input source image, the virtual plane corresponding to the target projection channel is rotated from an initial position to a target position corresponding to the current eye point information, and the rotated coordinates of the plurality of target vertices on the virtual plane are determined, including: based on the physical visual angle parameter and the current eye point position, initial coordinates of the plurality of target vertices when the virtual plane is located at the initial position are determined, the initial position is a position corresponding to a projection direction of the target projection channel; according to an offset angle between the current observation direction and the projection direction of the target projection channel, a rotation angle of the virtual plane is determined; the virtual plane is rotated according to the rotation angle to obtain a rotated virtual plane; and based on the initial coordinates of the plurality of target vertices and the rotation angle, the rotated coordinates of the plurality of target vertices on the virtual plane are determined.

[0012] Optionally, the plurality of target vertices include a first target vertex, a second target vertex, and a third target vertex, the physical visual angle parameters include a first horizontal included angle, a second horizontal included angle, a first vertical included angle, and a second vertical included angle; based on the physical visual angle parameters and the current eye point position, determining initial coordinates of the plurality of target vertices when the virtual plane is at the initial position includes: taking the sum of the X coordinate of the current eye point position and a first horizontal distance, the sum of the Y coordinate of the current eye point position and a first vertical distance, and the sum of the Z coordinate of the current eye point position and a set distance as the X axis initial coordinate, the Y axis initial coordinate, and the Z axis initial coordinate of the first target vertex respectively, the first horizontal distance being the product of the tangent value of the first horizontal included angle and the set distance, and the first vertical distance being the product of the tangent value of the first vertical included angle and the set distance; taking the sum of the X coordinate of the current eye point position and a first horizontal distance, the sum of the Y coordinate of the current eye point position and a second vertical distance, and the sum of the Z coordinate of the current eye point position and a set distance as the X axis initial coordinate, the Y axis initial coordinate, and the Z axis initial coordinate of the second target vertex respectively, the second vertical distance being the product of the tangent value of the second vertical included angle and the set distance; and taking the sum of the X coordinate of the current eye point position and a second horizontal distance, the sum of the Y coordinate of the current eye point position and a first vertical distance, and the sum of the Y coordinate of the current eye point position and a set distance as the X axis initial coordinate, the Y axis initial coordinate, and the Z axis initial coordinate of the third target vertex respectively, the second horizontal distance being the product of the tangent value of the second horizontal included angle and the set distance.

[0013] Optionally, based on the initial coordinates of the plurality of target vertices and the rotation angle, determining the rotated coordinates of the plurality of target vertices on the virtual plane includes: in accordance with a set rotation sequence, sequentially determining an X axis rotation matrix, a Y axis rotation matrix, and a Z axis rotation matrix; taking the product of the three as a target rotation matrix; and taking the product of the target rotation matrix and the initial coordinates of the first target vertex, the product of the target rotation matrix and the initial coordinates of the second target vertex, and the product of the target rotation matrix and the initial coordinates of the third target vertex as the rotated coordinates of the first target vertex, the rotated coordinates of the second target vertex, and the rotated coordinates of the third target vertex respectively.

[0014] Optionally, based on the rotated coordinates of the plurality of target vertices, determining the coordinates of each second pixel point in the second dot matrix includes: obtaining an effective field of view angle range, the effective field of view angle range being an observation range corresponding to an actual projection screen; obtaining the coordinates of each fourth pixel point in a fourth dot matrix on a virtual simulation screen according to the number of pixel points in the horizontal direction and the vertical direction of the actual projection screen, the pixel interval in the horizontal direction and the vertical direction, and the effective field of view angle range; for each fourth pixel point in the fourth dot matrix, connecting the fourth pixel point with the current eye point position to obtain a corresponding connecting line of the fourth pixel point; and determining the coordinates of the intersection of each connecting line with the virtual plane to obtain the coordinates of each second pixel point in the second dot matrix.

[0015] Optionally, the third pixel point corresponding to the original pixel point in the rotated virtual plane is determined based on a positional relationship between the plurality of target first pixel points corresponding to the original pixel point and the original pixel point, including: selecting four first pixel points adjacent to the original pixel point in the first dot array as the plurality of target first pixel points; determining a horizontal proportional relationship in a horizontal direction and a vertical proportional relationship in a vertical direction between the original pixel point and the plurality of target first pixel points; selecting a plurality of target second pixel points corresponding to the plurality of target first pixel points in the second dot array; and applying the horizontal proportional relationship and the vertical proportional relationship on the plurality of target second pixel points to determine the third pixel point corresponding to the original pixel point in the rotated virtual plane.

[0016] Optionally, the method further includes: selecting a plurality of actual projection pixel points on the actual projection screen projected by the target projection channel; selecting a plurality of actual projection pixel points on the actual projection screen; determining actual coordinates of the plurality of actual projection pixel points in the world coordinate system and virtual coordinates of the plurality of actual projection pixel points in the virtual simulation scene; establishing an equation set based on the actual coordinates and the virtual coordinates of the plurality of actual projection pixel points to obtain a mapping matrix; and determining current eye point information corresponding to the target projection channel in the virtual simulation scene by using the mapping matrix, a current eye point position and a current observation direction of an observer in the actual observation scene.

[0017] In a second aspect, an image processing device is also provided, and the device includes:

[0018] A first dot array acquisition module is configured to acquire a first dot array corresponding to the target projection channel, the first dot array being known and used to describe a dot array of a projection effect expected to be achieved by an input source image projected by the target projection channel;

[0019] A vertex coordinate determination module is configured to rotate a virtual plane corresponding to the target projection channel from an initial position to a target position corresponding to the current eye point information in the virtual simulation scene, and determine rotated coordinates of a plurality of target vertices on the virtual plane.

[0020] A second dot array determination module is configured to determine coordinates of each second pixel point in a second dot array based on the rotated coordinates of the plurality of target vertices, the second dot array being a matrix of a plurality of intersection points of a plurality of lines with the virtual plane, the plurality of lines being lines between each pixel point on a virtual projection screen on which the input source image is projected and the current eye point position.

[0021] The third lattice determination module is configured to determine, for each original pixel point in the original lattice, a third pixel point corresponding to the original pixel point in the rotated virtual plane based on a positional relationship between the original pixel point and a plurality of target first pixel points corresponding to the original pixel point, and the plurality of third pixel points form a third lattice, and the plurality of target first pixel points are pixel points in the first lattice.

[0022] The deformation processing module is configured to perform deformation processing on the input source image by using the third lattice to obtain a projection image corresponding to the first lattice on the actual projection screen.

[0023] In a third aspect, an embodiment of the present application further provides an electronic device, including a processor, a memory and a bus, the memory stores machine readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, and the machine readable instructions are executed by the processor to perform the steps of the image processing method as described above.

[0024] In a fourth aspect, an embodiment of the present application further provides a computer readable storage medium, and the computer readable storage medium stores a computer program, when the computer program is run by a processor, the steps of the image processing method as described above are executed.

[0025] The embodiments of the present application have the following beneficial effects:

[0026] The image processing method, device, electronic device and storage medium provided by the embodiments of the present application first determine the desired projection effect of the deformed input source image, when the current eye point information of the observer changes dynamically, the virtual plane can be adjusted to the target position corresponding to the current eye point information, and the deformed parameters corresponding to the target projection channel after the current eye point information changes, that is, the third lattice, are further determined, and the input source image is deformed by using the third lattice and then directly projected on the actual projection screen to obtain the projection image corresponding to the shape of the first lattice, without the need for multiple geometric corrections on the input source image. Compared with the image processing method in the prior art, the method solves the problem of low correction and debugging efficiency of the projection image and the problem that the method cannot adapt to the application scenario of real-time dynamic changes of the eye point position.

[0027] In order to make the above objectives, characteristics and advantages of the present application more apparent and easy to understand, the following preferred embodiments are described in detail below, and the accompanying drawings are described as follows. BRIEF DESCRIPTION OF DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some of the embodiments of the present application, and therefore should not be regarded as limiting the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor.

[0029] Figure 1 A flow chart of the image processing method provided by the embodiments of the present application is shown;

[0030] Figure 2 A schematic diagram of the virtual plane located at the initial position provided by the embodiments of the present application is shown;

[0031] Figure 3 A schematic diagram of the virtual plane located at the target position provided by the embodiments of the present application is shown;

[0032] Figure 4 A schematic diagram of the horizontal proportional relationship and the vertical proportional relationship provided by the embodiments of the present application is shown;

[0033] Figure 5 A position schematic diagram of the third pixel point provided by the embodiments of the present application is shown;

[0034] Figure 6 A structure schematic diagram of the image processing device provided by the embodiments of the present application is shown;

[0035] Figure 7 A structure schematic diagram of the electronic device provided by the embodiments of the present application is shown. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solutions and advantages of the embodiments of the present application more clear, the following will combine the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, every other embodiment obtained by those skilled in the art without creative labor belongs to the scope of the present application.

[0037] It is worth noting that before the present application is proposed, virtual simulation is a computer system that can create and experience a virtual world. Such a virtual world is generated by a computer and can be a reproduction of the real world or an imagined world. Users can interact with the virtual world through various sensory channels such as vision, hearing, and touch. Virtual simulation fusion technology is often used in virtual simulation. Virtual simulation fusion refers to creating a three-dimensional interface through projection fusion. The observer stands at the visual center of the projection, forming an immersive interactive scene. Virtual simulation fusion is a typical application of fusion splicing and has a large application market in education, scientific research, military, aerospace, and other fields. Virtual simulation fusion creates a three-dimensional interface through projection fusion. However, in existing virtual simulation fusion scenarios, such as flight simulator scenarios, the eye point position of the driver is fixed on the driver's seat. Once the eye point position changes, the frustum calibration diagram will also change, and the projection image needs to be re-geometrically corrected and debugged, resulting in low efficiency of projection image correction and debugging, which cannot adapt to application scenarios with real-time dynamic changes in eye point position.

[0038] Based on this, the embodiment of the present application provides an image processing method to improve the correction and debugging efficiency of the projection image.

[0039] Please refer to Figure 1 , Figure 1 The flowchart of the image processing method provided by the embodiment of the present application. As Figure 1 shown, the image processing method provided by the embodiment of the present application comprises:

[0040] Step S101, acquiring a first point array corresponding to a target projection channel.

[0041] In this step, the target projection channel can refer to a projection channel selected from a plurality of actual projection channels, and the target projection channel is used to project an input source image corresponding to the projection channel onto an actual projection screen.

[0042] The first point array is known and is used to describe the projection effect of the input source image projected by the target projection channel.

[0043] The data in the first point array is the position coordinates of the original point array corresponding to each pixel point on the output image (i.e. the projection image) after geometric correction in the input source image. The position coordinates can be decimal numbers.

[0044] The original point array is a point array corresponding to a default picture of the projector. The default picture is a rectangular picture, and the original point array is also a rectangular point array. The original point array is also known.

[0045] The original point array will be described in detail below.

[0046] Taking the projection resolution of the projector as 1024x768 as an example, the size of the original dot array is represented by MxN, then M=ceil(w / horin)+1, N=ceil(h / verin)+1, wherein w represents the number of pixel points of the default picture of the projector in the horizontal direction, h represents the number of pixel points of the default picture of the projector in the vertical direction, horin represents the pixel interval of the default picture of the projector in the horizontal direction, and verin represents the pixel interval of the default picture of the projector in the vertical direction. Here, taking 32 as horin and verin, then M=ceil(1024 / 32)+1=33, N=ceil(768 / 32)+1=25, wherein ceil represents rounding up. The form of the original dot array is as follows:

[0047] (0, 0), (32, 0), (64, 0), …, (1024, 0);

[0048] (0, 32), (32, 32), (64, 32), …, (1024, 32);

[0049] …

[0050] (0, 768), (32, 768), (64, 768), …, (1024, 768).

[0051] Here, the dot array size of the first dot array obtained after the geometric correction is the same as that of the original dot array, which is a dot array of 33x25.

[0052] In the embodiment of the present application, after the multiple actual projection channels project the respective input source images on the actual projection screen, due to the influence of the projection angle, the projection position, and the type of the actual projection screen, the problems of blurred projection images or disordered projection positions may occur, and it is necessary to use the prior art to perform geometric correction on each actual projection channel to obtain the first dot array corresponding to each actual projection channel. After one-time geometric correction, if the eye point position of the observer changes dynamically, the third dot array corresponding to the current eye point position can be calculated in real time according to the image processing method in the present application, and the input source image is deformed using the third dot array, so as to solve the influence of the deformation of the projection image caused by the change of the eye point position, without re-performing geometric correction on the input source image as in the prior art.

[0053] Here, the existing technology can be used to correct the geometry of each actual projection channel to obtain the respective deformation parameters A corresponding to each actual projection channel. The deformation parameters A are a point array composed of multiple coordinate points, referred to as a first point array. Alternatively, the first point array can be obtained by direct setting. The first point array is a shape parameter expected to be achieved after the input source image is projected onto the actual projection screen, and is known. The pixel points in the first point array are first pixel points.

[0054] In an optional embodiment, the method further comprises: selecting a plurality of actual projection pixel points on the actual projection screen projected by the target projection channel; selecting a plurality of actual projection pixel points on the actual projection screen; determining actual coordinates of the plurality of actual projection pixel points in the world coordinate system and virtual coordinates of the plurality of actual projection pixel points in the virtual simulation scene; establishing an equation set based on the actual coordinates and the virtual coordinates of the plurality of actual projection pixel points to obtain a mapping matrix; and determining the current eye point information corresponding to the target projection channel in the virtual simulation scene by using the mapping matrix, the current eye point position and the current observation direction of the observer in the actual observation scene.

[0055] Here, first, a site including the actual projection screen and a movable area where the observer is located is established with an XYZ three-dimensional coordinate system, i.e., a world coordinate system. The three-dimensional coordinates of the eye point of the observer in the world coordinate system are determined in real time using motion capture technology. The eye point can refer to a point near the eyes of the observer (user), and the motion capture technology includes but is not limited to: optical motion capture technology, inertial motion capture technology, and image recognition motion capture technology.

[0056] The optical motion capture technology is to paste infrared reflective marker balls or marker points on the head or body of the user, use a motion capture camera to obtain the picture of the user, calculate the two-dimensional coordinates of all the marker points in the pixel coordinate system, use software to collect the two-dimensional coordinates of the marker points calculated by all the motion capture cameras at the same time, and then calculate the three-dimensional coordinates of the marker points in the world coordinate system at that moment. According to the three-dimensional coordinates of the marker points and the relative position between the marker points and the eye point, the three-dimensional coordinates of the eye point can be calculated. The optical motion capture technology belongs to the prior art, and will not be described here.

[0057] The inertial motion capture technology is to bind inertial sensors to the body of the user, use software to obtain the data of the inertial sensors and calculate the three-dimensional coordinates of the inertial sensors, and according to the three-dimensional coordinates, the eye point position information in the world coordinate system can be calculated. The inertial motion capture technology belongs to the prior art, and will not be described here.

[0058] Image recognition motion capture technology is to use monocular or binocular camera to obtain the picture of the user, to recognize the joint and bone information of the user through algorithm, especially the information of feature points such as the two eyes and two ears of the head, and to calculate the eye point position in the world coordinate system. The image recognition motion capture technology also belongs to the prior art, which will not be described here.

[0059] Specifically, more than four actual projection pixel points are selected on the actual projection screen, the actual coordinates of the more than four actual projection pixel points in the world coordinate system and the virtual coordinates in the virtual simulation system are obtained, and an equation group is established by using the actual coordinates and the virtual coordinates of the actual projection pixel points, so that the specific values of the rotation matrix and the translation matrix can be obtained. The mapping matrix is composed of the rotation matrix and the translation matrix.

[0060] The mapping matrix can be represented by the following formula:

[0061]

[0062] In the above formula, R represents the rotation matrix, t represents the translation matrix, represents a three-dimensional matrix with all values being 0, x represents the X-axis coordinate in the virtual simulation scene, y represents the Y-axis coordinate in the virtual simulation scene, z represents the Z-axis coordinate in the virtual simulation scene, x w represents the X-axis coordinate in the world coordinate system, y w represents the Y-axis coordinate in the world coordinate system, and z w represents the Z-axis coordinate in the world coordinate system.

[0063] The obtained mapping matrix can be used to map the current eye point position and the current observation direction of the observer in the actual observation scene to the current eye point information in the virtual simulation scene.

[0064] In step S102, in the virtual simulation scene, the virtual plane corresponding to the target projection channel is rotated from the initial position to the target position corresponding to the current eye point information, and the coordinates of the plurality of target vertices on the virtual plane after rotation are determined.

[0065] In this step, the virtual simulation scene can refer to the simulation scene corresponding to the actual observation scene, and the virtual simulation scene is used to simulate the observer position, the actual projection screen information and the actual projection channel in the actual observation scene.

[0066] The actual projection screen information includes the actual projection screen size, the actual projection screen type and the actual projection screen position.

[0067] Exemplarily, the actual projection screen type includes but is not limited to a spherical screen, a cylindrical screen and a planar screen.

[0068] The actual projection channels include a plurality of actual projection channels, and the plurality of actual projection channels jointly project onto an actual projection screen to obtain a fused projection image.

[0069] Each actual projection channel includes an input source, a projector, and a projection path, the input source sends an input source image to the projector, and the projector projects the input source image onto an actual projection screen to obtain a projection image.

[0070] The projection angles and the projector positions of the plurality of actual projection channels can be different, causing the projection paths to be different.

[0071] The virtual plane can refer to a rectangular plane formed at a set distance in a virtual simulation scene after being projected in a direction indicated by a preset physical viewing angle parameter from a current eye point position.

[0072] The initial position can refer to a position of the virtual plane when an eye point deflection angle of an observer at the current eye point position is 0.

[0073] The initial position of the virtual plane corresponds to the current eye point position, and when the virtual plane is at the initial position, the virtual plane is perpendicular to the Z axis.

[0074] In the embodiments of the present application, if the actual projection screen is a spherical screen, a virtual spherical screen is established in the virtual simulation scene, the spherical radius of the virtual spherical screen is r, and the spherical center O is located at the origin of the spatial direct coordinate system with coordinates (0, 0, 0). If the actual projection screen is a cylindrical screen, a virtual cylindrical screen is established in the virtual simulation scene, the equation of the straight generatrix of the virtual cylindrical screen is x = 0, z = r, and the equation of the directrix of the cylindrical screen is x 2 +z 2 =r 2 . If the actual projection screen is a planar screen, a virtual planar screen is established in the virtual simulation scene, and the plane equation of the virtual planar screen is z = r.

[0075] In an optional embodiment, the current eye point information includes a current eye point position, a current observation direction, and a physical viewing angle parameter, the physical viewing angle parameter is a projection angle of the input source image; a virtual plane corresponding to a target projection channel is rotated from an initial position to a target position corresponding to the current eye point information, the rotated coordinates of a plurality of target vertices on the virtual plane are determined, including: based on the physical viewing angle parameter and the current eye point position, the initial coordinates of the plurality of target vertices when the virtual plane is located at the initial position are determined, the initial position is a position corresponding to a projection direction of the target projection channel; according to an offset angle between the current observation direction and the projection direction of the target projection channel, a rotation angle of the virtual plane is determined; the virtual plane is rotated according to the rotation angle to obtain a rotated virtual plane; based on the initial coordinates of the plurality of target vertices and the rotation angle, the rotated coordinates of the plurality of target vertices on the virtual plane are determined.

[0076] Here, the physical visual angle parameter can refer to a range parameter used to indicate the input source image, and the physical visual angle parameter can represent the picture size of the input source image.

[0077] The initial coordinates of the plurality of target vertices can refer to the coordinates of the plurality of target vertices when the virtual plane is located at the initial position corresponding to the current eye point position.

[0078] Specifically, since the initial position of the virtual plane corresponds to the current eye point position, when the current eye point position changes, the initial position of the virtual plane also changes. Since the initial position of the virtual plane is the position when the eye point deflection angle is 0, the Z-axis coordinate of the virtual plane is equal to the sum of the Z-axis coordinate of the current eye point position and the set distance, and the size of the virtual plane is determined by the physical visual angle parameter.

[0079] In order to determine the coordinates of each pixel point on the virtual plane, the initial coordinates of the plurality of target vertices on the virtual plane are first determined, and then the coordinates of each pixel point in the rotated virtual plane are calculated according to the initial coordinates of the plurality of target vertices and the rotation angle.

[0080] In an optional embodiment, the plurality of target vertices include a first target vertex, a second target vertex, and a third target vertex, and the physical visual angle parameter includes a first horizontal included angle, a second horizontal included angle, a first vertical included angle, and a second vertical included angle. Based on the physical visual angle parameter and the current eye point position, the initial coordinates of the plurality of target vertices when the virtual plane is located at the initial position are determined, including: taking the sum of the X-coordinate of the current eye point position and the first horizontal distance, the sum of the Y-coordinate of the current eye point position and the first vertical distance, and the sum of the Z-coordinate of the current eye point position and the set distance as the X-axis initial coordinate, the Y-axis initial coordinate, and the Z-axis initial coordinate of the first target vertex, respectively, the first horizontal distance being the product of the tangent value of the first horizontal included angle and the set distance, and the first vertical distance being the product of the tangent value of the first vertical included angle and the set distance; taking the sum of the X-coordinate of the current eye point position and the first horizontal distance, the sum of the Y-coordinate of the current eye point position and the second vertical distance, and the sum of the Z-coordinate of the current eye point position and the set distance as the X-axis initial coordinate, the Y-axis initial coordinate, and the Z-axis initial coordinate of the second target vertex, respectively, the second vertical distance being the product of the tangent value of the second vertical included angle and the set distance; and taking the sum of the X-coordinate of the current eye point position and the second horizontal distance, the sum of the Y-coordinate of the current eye point position and the first vertical distance, and the sum of the Y-coordinate of the current eye point position and the set distance as the X-axis initial coordinate, the Y-axis initial coordinate, and the Z-axis initial coordinate of the third target vertex, respectively, the second horizontal distance being the product of the tangent value of the second horizontal included angle and the set distance.

[0081] The virtual plane will be introduced below with reference to Figure 2 .

[0082] Figure 2A schematic diagram of a virtual plane in an initial position is shown.

[0083] As shown in Figure 2 , the virtual projection screen is a virtual spherical screen, the coordinates of the current eye point position corresponding to the target projection channel are P0(x0, y0, z0), a perpendicular is drawn from the current eye point position to the virtual plane, the foot point is S, there are three target vertices on the virtual plane, which are a first target vertex a, a second target vertex b and a third target vertex c, points A and B are two points on a straight line parallel to the X axis and passing through point S, points C and D are two points on a straight line parallel to the Y axis and passing through point S, the physical visual angle parameters include a first horizontal included angle ∠AP0S, denoted as α1, a second horizontal included angle ∠BP0S, denoted as α2, a first vertical included angle ∠CP0S, denoted as β1, and a second vertical included angle ∠DP0S, denoted as β2. The distance is the distance P0S between the current eye point position and the virtual plane, and for the convenience of calculation, P0S is set to 1.

[0084] By using the right triangle angle calculation formula, the following results can be obtained:

[0085] The coordinates of point a are denoted as (x a ,y a ,z a ), x a =x0+tanα1, y a =y0+tanβ1, and z a =z0+1.

[0086] The coordinates of point b are denoted as (x b ,y b ,z b ), x b =x0+tanα2, y b =y0+tanβ1, and z b =z0+1.

[0087] The coordinates of point c are denoted as (x c ,y c ,z c ), x c =x0+tanα1, y c =y0+tanβ2, and z c =z0+1.

[0088] It should be noted that for different target projection channels, the current eye point position in the virtual simulation scene is the same, but because the projection directions of different target projection channels are different, the angles between different target projection channels and the current observation direction are also different.

[0089] In an optional embodiment, based on the initial coordinates of the plurality of target vertices and the rotation angle, the post-rotation coordinates of the plurality of target vertices on the virtual plane are determined, comprising: sequentially determining an X-axis rotation matrix, a Y-axis rotation matrix and a Z-axis rotation matrix according to a set rotation order; taking the product of the three as a target rotation matrix; taking the product of the target rotation matrix and the initial coordinates of the first target vertex, the product of the target rotation matrix and the initial coordinates of the second target vertex, and the product of the target rotation matrix and the initial coordinates of the third target vertex as the post-rotation coordinates of the first target vertex, the post-rotation coordinates of the second target vertex and the post-rotation coordinates of the third target vertex, respectively.

[0090] Specifically, the virtual plane is the same as the pixel points of the actual projection screen, the number of pixel points in the horizontal direction of the virtual plane is w, and the number of pixel points in the vertical direction of the virtual plane is h. The virtual plane can be rotated around the x-axis, y-axis and z-axis in the virtual simulation scene, and the angles of rotation are denoted as α, β and γ in sequence. The virtual plane must be rotated in the order of first around the x-axis, then around the y-axis, and finally around the z-axis. According to the mathematical calculation formula, the rotation matrices of the three axes can be constructed, which are R X (α), R Y (β) and R z (γ), respectively.

[0091] Among them, the rotation matrices of the three axes are:

[0092]

[0093]

[0094]

[0095] After the virtual plane is rotated, the post-rotation coordinates of the first target vertex a are denoted as (x1, y1, z1), the post-rotation coordinates of the second target vertex b are denoted as (x2, y2, z2), and the post-rotation coordinates of the third target vertex c are denoted as (x3, y3, z3). The specific values of the post-rotation coordinates of the first target vertex a, the post-rotation coordinates of the second target vertex b and the post-rotation coordinates of the third target vertex c can be calculated by the formula, and the specific calculation formula is as follows:

[0096]

[0097]

[0098]

[0099] In the above formula, R Z (γ)×R Y (β)×RX (a) is the target rotation matrix.

[0100] In step S103, coordinates of each second pixel point in the second point array are determined based on the coordinates of the plurality of target vertices after rotation.

[0101] In this step, the second pixel point can refer to the intersection of the plurality of lines and the virtual plane, and the second pixel point includes a plurality of intersections, each of which is an intersection of a line and a virtual plane.

[0102] The plurality of lines are lines between the pixel points of the input source image projected on the virtual projection screen and the current eye point position.

[0103] The second point array is a matrix composed of a plurality of second pixel points, that is, a matrix composed of a plurality of intersections of a plurality of lines and a virtual plane.

[0104] The second point array is a point array on the virtual plane after rotation, and the second point array is mapped from the fourth point array.

[0105] In an optional embodiment, based on the coordinates of the plurality of target vertices after rotation, the coordinates of each second pixel point in the second point array are determined, including: obtaining an effective field of view angle range, the effective field of view angle range being an observation range corresponding to the actual projection screen; obtaining the coordinates of each fourth pixel point in the fourth point array on the virtual simulation screen according to the number of pixel points in the horizontal direction and the vertical direction of the actual projection screen, the pixel interval in the horizontal direction and the vertical direction, and the effective field of view angle range; for each fourth pixel point in the fourth point array, connecting the fourth pixel point with the current eye point position to obtain the corresponding line of the fourth pixel point; determining the coordinates of the intersection of each line and the virtual plane to obtain the coordinates of each second pixel point in the second point array.

[0106] Here, the effective field of view angle range can refer to the field of view range that can be finally observed on the actual projection screen, and the effective field of view angle range is related to the range of the actual projection screen. Assuming that the upper and lower angle range of the spherical screen is ±30°, the effective field of view angle in the vertical direction is 60°. It should be noted that the spherical screen in the actual observation scene is not a 360° spherical screen, but a part of the 360° spherical screen.

[0107] The effective field of view angle range includes an effective field of view angle horizontal range and an effective field of view angle vertical range, and the effective field of view angle horizontal range is represented by [sta_h, end_h], and the effective field of view angle vertical range is represented by [sta_v, end_v].

[0108] The fourth point array can refer to the point array corresponding to the input source image, that is, the point array on the original video image, and also the point array on the curved video.

[0109] The second point array will be introduced below with reference to Figure 3 .

[0110] Figure 3 A schematic diagram of a virtual plane located at a target position is shown.

[0111] As shown in Figure 3 , the first target vertex a, the second target vertex b and the third target vertex c on the rotated virtual plane are respectively located at new positions, at which the coordinates of the first target vertex a after rotation are (x1, y1, z1), the coordinates of the second target vertex b after rotation are (x2, y2, z2), and the coordinates of the third target vertex c after rotation are (x3, y3, z3). The point U1 on the virtual simulation screen is a fourth pixel point in the fourth point array. The point U1 and the current eye point position P0 are connected to obtain a corresponding connecting line P0U1, the connecting line P0U1 passes through the virtual plane and intersects at a point T1, which is a second pixel point in the second point array corresponding to the point U1. In this way, the second pixel point corresponding to each fourth pixel point in the fourth point array can be determined, and the second point array is formed by all the second pixel points.

[0112] Specifically, an M×N point array with equal intervals can be calculated on the virtual simulation screen according to the effective field of view angle range, which is denoted as a fourth point array. In the spherical screen, the fourth pixel point U in the jth row and the ith column of the fourth point array is represented by longitude θ and latitude In the cylindrical screen, the fourth pixel point U in the jth row and the ith column of the fourth point array is represented by longitude θ and vertical direction coordinate In the planar screen, the fourth pixel point U in the jth row and the ith column of the fourth point array is represented by horizontal direction coordinate θ and vertical direction coordinate In different types of virtual simulation screens, the coordinate meanings of the fourth pixel point are different, but the calculation formula is the same.

[0113] The coordinate calculation formula of the fourth pixel point is as follows:

[0114]

[0115]

[0116] In the above formula, w represents the number of pixel points in the horizontal direction of the actual projection screen, h represents the number of pixel points in the vertical direction of the actual projection screen, horin represents the pixel interval in the horizontal direction of the actual projection screen, and verin represents the pixel interval in the vertical direction of the actual projection screen.

[0117] After the coordinates of each fourth pixel point in the fourth dot array are determined, for each fourth pixel point in the fourth dot array, the fourth pixel point and the current eye point position P0 are connected, and the connecting line intersects with the rotated virtual plane. The intersection point is the second pixel point corresponding to the fourth pixel point, and the coordinate of the second pixel point at this time is a three-dimensional coordinate. The three-dimensional coordinate of the second pixel point can be calculated by the following formula:

[0118]

[0119] x(j, i) = x A × z(j, i) + x B ;

[0120] y(j, i) = y A × z(j, i) + y B .

[0121] In the above formula, u = (y2-y1)×(z3-z1)-(y3-y1)×(z2-z1);

[0122] v = (x2-x1)×(z3-z1)-(x3-x1)×(z2-z1);

[0123] w = (x2-x1)×(y3-y1)-(x3-x1)×(y2-y1);

[0124] t = -x1×u+y1×v-z1×w;

[0125]

[0126]

[0127]

[0128]

[0129] After the three-dimensional coordinates of the second pixel points are determined, the second pixel points are mapped to a two-dimensional plane, that is, the second dot array is mapped to the two-dimensional plane corresponding to the virtual plane, to obtain the two-dimensional coordinates of the second pixel points. The two-dimensional coordinates of the second pixel point in the jth row and the ith column of the second dot array are denoted as (xpoint(j, i), ypoint(j, i)). The two-dimensional coordinates of the second pixel points can be calculated by the following formula:

[0130]

[0131]

[0132] In the above formula:

[0133]

[0134]

[0135]

[0136]

[0137] In the above formula, x, y, z respectively represent the X-axis coordinate, the Y-axis coordinate and the Z-axis coordinate of the second pixel point in the second point array.

[0138] In step S104, for each original pixel point in the original point array, based on the positional relationship between the plurality of target first pixel points corresponding to the original pixel point and the original pixel point, a third pixel point corresponding to the original pixel point in the rotated virtual plane is determined, and a plurality of third pixel points form a third point array.

[0139] In this step, the positional relationship between the plurality of target first pixel points corresponding to the original pixel point and the original pixel point can refer to a horizontal proportional relationship in the horizontal direction and a vertical proportional relationship in the vertical direction.

[0140] The plurality of target first pixel points are pixel points in the first point array.

[0141] The target first pixel point is a pixel point selected from the plurality of first pixel points.

[0142] In an optional embodiment, based on the positional relationship between the plurality of target first pixel points corresponding to the original pixel point and the original pixel point, the third pixel point corresponding to the original pixel point in the rotated virtual plane is determined, which comprises: selecting four first pixel points adjacent to the original pixel point in the first point array as the plurality of target first pixel points; determining the horizontal proportional relationship in the horizontal direction and the vertical proportional relationship in the vertical direction between the original pixel point and the plurality of target first pixel points; selecting a plurality of target second pixel points corresponding to the plurality of target first pixel points in the second point array; applying the horizontal proportional relationship and the vertical proportional relationship on the plurality of target second pixel points to determine the third pixel point corresponding to the original pixel point in the rotated virtual plane.

[0143] Here, the MxN point array uniformly interpolated on the actual projection screen is the original point array, and the pixel point coordinate of the jth row and the ith column of the original point array is denoted as EP(ex(j, i), ey(j, i)), and the following can be obtained: ex(j, i) = (i-1) x horin, ey(j, i) = (j-1) x verin.

[0144] Specifically, for each original pixel point in the original dot array, four first pixel points adjacent to the original pixel point are found in the first dot array as target first pixel points, so that the original pixel point is located in the smallest grid surrounded by the four target first pixel points. The coordinates of the four target first pixel points are recorded as op1(ox1, oy1), op2(ox2, oy2), op3(ox3, oy3), and op4(ox4, oy4), respectively. The four target first pixel points can be two first pixel points adjacent to the column number of the original pixel point on the left and right, respectively, and two first pixel points adjacent to the row number of the original pixel point on the top and bottom, respectively.

[0145] For each target first pixel point, the row number and column number of the target first pixel point are determined, and target second pixel points corresponding to the row number and column number are found in the second dot array, so that four target second pixel points are determined. The coordinates of the four target second pixel points are recorded as ip1(ix1, iy1), ip2(ix2, iy2), ip3(ix3, iy3), and ip4(ix4, iy4), respectively.

[0146] The horizontal proportion relationship U and the vertical proportion relationship V of the point EP(ex(j, i), ey(j, i)) in the four adjacent target first pixel points op1(ox1, oy1), op2(ox2, oy2), op3(ox3, oy3), and op4(ox4, oy4) are calculated.

[0147] The calculation method of the horizontal proportion relationship and the vertical proportion relationship will be described below with reference to Figure 4 .

[0148] Figure 4 A schematic diagram of the horizontal proportion relationship and the vertical proportion relationship provided by the embodiments of the present application is shown.

[0149] As shown in Figure 4 , the coordinates of the point op5 are recorded as op5(ox5, oy5), the coordinates of the point op6 are recorded as op6(ox6, oy6), the length between the point op5 and the point op6 is set to 1, and the length between the point op3 and the point op4 is set to 1. The horizontal proportion relationship and the vertical proportion relationship can be calculated by the following formulas, and the calculation formulas are as follows:

[0150]

[0151]

[0152] In the above formulas,

[0153] a=xp*yt-yq*xt, b=yp*xt-yq*xs-xp*yt+xq*ys, c=yp*xs-xp*ys.

[0154] oy5=(oy2-oy1)*U+oy1, oy6=(oy4-oy3)*U+oy3.

[0155] yP=ey(j,i)-oy1, yq=oy2-oy1, ys=oy3-oy1, yt=oy4-oy3-oy2+oy1.

[0156] xp=ex(j,i)-ox1, xq=ox2-ox1, xs=ox3-ox1, xt=ox4-ox3-ox2+ox1.

[0157] Then, the horizontal proportional relationship and the vertical proportional relationship are applied to the selected four target second pixel points, and a third pixel point corresponding to the original pixel point is obtained, and the third pixel point is denoted as CP(cx(j,i), cy(j,i)).

[0158] The calculation process of the third pixel point will be described below with reference to Figure 5 .

[0159] Figure 5 The position of the third pixel point provided by the embodiment of the present application is shown in the schematic diagram.

[0160] As shown in Figure 5 , the coordinates of the point ip5 are denoted as ip5(ix5, iy5), the coordinates of the point ip6 are denoted as ip6(ix6, iy6), and the coordinates of the third pixel point CP(cx(j,i), cy(j,i)) can be calculated by the following formula, and the calculation formula is as follows:

[0161] cx(j,i)=ix5+(ix6-ix5)*V;

[0162] cy(j,i)=iy5+(iy6-iy5)*V.

[0163] In the above formula, ix5=ix t +(ix2-ix1)*U, iy5=iy1+(iy2-iy1)*U, ix6=ix3+(ix4-ix3)*U, and iy6=iy3+(iy4-iy3)*U.

[0164] In step S105, the input source image is deformed by using the third dot matrix, and a projection image corresponding to the first dot matrix and projected on the actual projection screen is obtained.

[0165] In this step, the third point array is taken as a deformation parameter, the input source image is deformed by using the deformation parameter, and the projection image obtained after the deformation is projected on the actual projection screen.

[0166] It should be noted that the projection channels in the actual observation scene are multiple, and deformation processing needs to be performed for each projection channel. The multiple projection images obtained after the deformation processing are fused together to obtain the final projection picture.

[0167] In the specific implementation, a new input source image is rendered by the software end based on the current eye point information and the orientation data of each projection channel in real time. At the same time, the third point array is calculated by the virtual simulation fusion system, the input source image is deformed by using the third point array to obtain the projection image finally projected on the actual projection screen, so as to realize the real-time deformation of the new input source image. Since the third point array is calculated based on the first point array, the third point array can reflect the shape of the first point array, and therefore, the projection image finally projected on the actual projection screen corresponds to the shape of the first point array. Since the deformation processing of the input source image by using the third point array belongs to the prior art and is consistent with the ordinary fusion rule, please refer to the patent with the application number CN202111643036, which will not be described here. It can be understood that the image processing method of the present application is applied in the virtual simulation fusion system.

[0168] Compared with the image processing method in the prior art, the present application first determines the projection effect of the deformed input source image to be achieved. When the current eye point information of the observer changes dynamically, the virtual plane can be adjusted to the target position corresponding to the current eye point information, and the deformation parameter corresponding to the target projection channel after the change of the current eye point information, i.e., the third point array, is further determined. After the deformation processing of the input source image by using the third point array, the projection image corresponding to the shape of the first point array can be directly projected on the actual projection screen, without the need for multiple geometric corrections of the input source image. The problem of low correction and debugging efficiency of the projection image and the application scene that cannot adapt to the real-time dynamic change of the eye point position is solved.

[0169] Based on the same inventive concept, the present application also provides an image processing device corresponding to the image processing method. Since the principle of solving the problem by the device in the present application is similar to the above-mentioned image processing method of the present application, the implementation of the device can be referred to the implementation of the method, and the repeated parts will not be described here.

[0170] Please refer to Figure 6 , Figure 6 The structure of an image processing device provided in the present application is shown in the figure. Figure 6 As shown in the figure, the image processing device 200 includes:

[0171] The first dot array acquisition module 201 is configured to acquire a first dot array corresponding to the target projection channel, the first dot array being known and used to describe a dot array of a projection effect expected to be achieved by the input source image projected by the target projection channel;

[0172] The vertex coordinate determination module 202 is configured to rotate a virtual plane corresponding to the target projection channel from an initial position to a target position corresponding to current eye point information in a virtual simulation scene, and determine rotated coordinates of a plurality of target vertices on the virtual plane.

[0173] The second dot array determination module 203 is configured to determine coordinates of each second pixel point in a second dot array based on the rotated coordinates of the plurality of target vertices, the second dot array being a matrix of a plurality of intersection points of a plurality of lines and the virtual plane, the plurality of lines being lines between each pixel point on the virtual projection screen and the current eye point position.

[0174] The third dot array determination module 204 is configured to, for each original pixel point in the original dot array, determine a third pixel point corresponding to the original pixel point in the rotated virtual plane based on a positional relationship between a plurality of target first pixel points corresponding to the original pixel point and the original pixel point, the third dot array being composed of a plurality of third pixel points, the plurality of target first pixel points being pixel points in the first dot array.

[0175] The deformation processing module 205 is configured to perform deformation processing on the input source image by using the third dot array to obtain a projection image corresponding to the first dot array projected on the actual projection screen.

[0176] Please refer to Figure 7 , Figure 7 A structural schematic diagram of an electronic device provided by an embodiment of the present application is shown in FIG. 3. Figure 7 As shown in FIG. 3, the electronic device 300 includes a processor 310, a memory 320 and a bus 330.

[0177] The memory 320 stores machine readable instructions executable by the processor 310, and when the electronic device 300 is running, the processor 310 and the memory 320 communicate through the bus 330. When the machine readable instructions are executed by the processor 310, the steps of the image processing method in the method embodiment shown in the above Figure 1 The specific implementation can be referred to the method embodiment, and will not be described here.

[0178] The embodiment of the present application further provides a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is run by a processor to execute the steps of the image processing method in the method embodiment shown in the above Figure 1 The specific implementation can be referred to the method embodiment, and will not be described here.

[0179] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working processes of the system, device and unit described above can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0180] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some communication interfaces, devices or units, and can be electrical, mechanical or other forms.

[0181] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e. can be located in one place or distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0182] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically, or two or more units can be integrated into one unit.

[0183] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a non-volatile computer readable storage medium executable by a processor. Based on this understanding, the technical solutions of the present application or the essential part or part of the technical solutions that make contributions to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program code storage media.

[0184] Finally, it should be noted that the above-described embodiments are merely specific embodiments of the present application, which are used to illustrate the technical solutions of the present application, but not to limit the same. The protection scope of the present application is not limited thereto. Although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those skilled in the art that any skilled person in the art can still modify or easily think of changes to the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some of the technical features, within the technical scope disclosed by the present application. The modifications, changes or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An image processing method, characterized by, The method comprises the following steps: obtaining a first point array corresponding to a target projection channel, the first point array being known and used to describe a projection effect of an input source image projected by the target projection channel; in a virtual simulation scene, rotating a virtual plane corresponding to the target projection channel from an initial position to a target position corresponding to current eye point information, and determining coordinates of a plurality of target vertices on the virtual plane after rotation; based on the coordinates of the plurality of target vertices after rotation, determining coordinates of each second pixel point in a second point array, the second point array being a matrix of a plurality of intersection points formed by a plurality of lines intersecting the virtual plane, the plurality of lines being lines between each pixel point on the virtual projection screen and the current eye point position; for each original pixel point in the original point array, based on a positional relationship between a plurality of target first pixel points corresponding to the original pixel point and the original pixel point, determining a third pixel point corresponding to the original pixel point in the virtual plane after rotation, the plurality of third pixel points forming a third point array, the plurality of target first pixel points being pixel points in the first point array; using the third point array to perform morphing processing on the input source image to obtain a projection image corresponding to the first point array projected on an actual projection screen.

2. The method of claim 1, wherein, The current eye point information comprises a current eye point position, a current observation direction, and a physical visual angle parameter, the physical visual angle parameter being a projection angle of the input source image; The method of rotating the virtual plane corresponding to the target projection channel from the initial position to the target position corresponding to the current eye point information, and determining the coordinates of the plurality of target vertices on the virtual plane after rotation comprises: based on the physical visual angle parameter and the current eye point position, determining initial coordinates of the plurality of target vertices when the virtual plane is located at the initial position, the initial position being a position corresponding to a projection direction of the target projection channel; determining a rotation angle of the virtual plane according to an offset angle between the current observation direction and the projection direction of the target projection channel; rotating the virtual plane according to the rotation angle to obtain a virtual plane after rotation; based on the initial coordinates of the plurality of target vertices and the rotation angle, determining the coordinates of the plurality of target vertices on the virtual plane after rotation.

3. The method of claim 2, wherein, The plurality of target vertices comprises a first target vertex, a second target vertex, and a third target vertex, and the physical visual angle parameter comprises a first horizontal included angle, a second horizontal included angle, a first vertical included angle, and a second vertical included angle; The method of determining the initial coordinates of the plurality of target vertices when the virtual plane is located at the initial position based on the physical visual angle parameter and the current eye point position comprises: taking a sum of an X coordinate of the current eye point position and a first horizontal distance, a sum of a Y coordinate of the current eye point position and a first vertical distance, and a sum of a Z coordinate of the current eye point position and a set distance as an X axis initial coordinate, a Y axis initial coordinate, and a Z axis initial coordinate of the first target vertex respectively, the first horizontal distance being a product of a tangent value of the first horizontal included angle and the set distance, and the first vertical distance being a product of a tangent value of the first vertical included angle and the set distance. Sum of the X coordinate of the current eye point position and the first horizontal distance, sum of the Y coordinate of the current eye point position and the second vertical distance, and sum of the Z coordinate of the current eye point position and the set distance are taken as the X-axis initial coordinate, the Y-axis initial coordinate and the Z-axis initial coordinate of the second target vertex respectively, the second vertical distance being the product of the tangent value of the second vertical angle and the set distance; Sum of the X coordinate of the current eye point position and the second horizontal distance, sum of the Y coordinate of the current eye point position and the first vertical distance, and sum of the Y coordinate of the current eye point position and the set distance are taken as the X-axis initial coordinate, the Y-axis initial coordinate and the Z-axis initial coordinate of the third target vertex respectively, the second horizontal distance being the product of the tangent value of the second horizontal angle and the set distance.

4. The method of claim 2, wherein, The method further comprises: According to the initial coordinates of the plurality of target vertices and the rotation angles, the rotated coordinates of the plurality of target vertices on the virtual plane are determined, comprising: In sequence, the X-axis rotation matrix, the Y-axis rotation matrix and the Z-axis rotation matrix are determined according to the set rotation sequence; The product of the X-axis rotation matrix, the Y-axis rotation matrix and the Z-axis rotation matrix is taken as the target rotation matrix; 5. The method of claim 1, wherein, The product of the target rotation matrix and the initial coordinates of the first target vertex, the product of the target rotation matrix and the initial coordinates of the second target vertex, and the product of the target rotation matrix and the initial coordinates of the third target vertex are taken as the rotated coordinates of the first target vertex, the rotated coordinates of the second target vertex and the rotated coordinates of the third target vertex respectively. The method further comprises: An effective field of view angle range is obtained, the effective field of view angle range being the observation range corresponding to the actual projection screen; According to the number of pixel points in the horizontal direction and the vertical direction of the actual projection screen, the pixel interval in the horizontal direction and the vertical direction, and the effective field of view angle range, the coordinates of each fourth pixel point in the fourth dot array on the virtual simulation screen are obtained; For each fourth pixel point in the fourth dot array, a connecting line corresponding to the fourth pixel point is obtained by connecting the fourth pixel point with the current eye point position; 6. The method of claim 1, wherein, The coordinates of the intersection of each connecting line and the virtual plane are determined to obtain the coordinates of each second pixel point in the second dot array. The method further comprises: In the first dot array, four first pixel points adjacent to the original pixel point are selected as the plurality of target first pixel points; The horizontal proportional relationship of the original pixel point and the plurality of target first pixel points in the horizontal direction and the vertical proportional relationship in the vertical direction are determined; In the second dot array, a plurality of target second pixel points corresponding to the plurality of target first pixel points are selected; 7. The method of claim 1, wherein, The horizontal proportional relationship and the vertical proportional relationship are applied to the plurality of target second pixel points to determine the third pixel point corresponding to the original pixel point in the rotated virtual plane. The method further comprises: A plurality of actual projection pixel points on the actual projection screen are selected. Determine actual coordinates of the plurality of actual projection pixels in a world coordinate system and virtual coordinates in a virtual simulation scene respectively; Establish an equation group based on the actual coordinates and the virtual coordinates of the plurality of actual projection pixels, and obtain a mapping matrix; Determine current eye point information corresponding to the target projection channel in the virtual simulation scene by using the mapping matrix, a current eye point position and a current observation direction of an observer in the actual observation scene.

8. An image processing apparatus characterized by comprising: Comprise: A first dot array acquisition module is configured to acquire a first dot array corresponding to a target projection channel, the first dot array being known and used to describe a dot array of a projection effect expected to be achieved by an input source image projected by the target projection channel; A vertex coordinate determination module is configured to rotate a virtual plane corresponding to the target projection channel from an initial position to a target position corresponding to current eye point information in a virtual simulation scene, and determine rotated coordinates of a plurality of target vertices on the virtual plane; A second dot array determination module is configured to determine coordinates of each second pixel point in a second dot array based on the rotated coordinates of the plurality of target vertices, the second dot array being a matrix of a plurality of intersection points formed by a plurality of lines intersecting with the virtual plane, the plurality of lines being lines between each pixel point of the input source image projected on a virtual projection screen and the current eye point position; A third dot array determination module is configured to, for each original pixel point in the original dot array, determine a third pixel point corresponding to the original pixel point in the rotated virtual plane based on a positional relationship between the original pixel point and a plurality of target first pixel points corresponding to the original pixel point, the plurality of target first pixel points being pixel points in the first dot array, and a third dot array being formed by a plurality of third pixel points; A deformation processing module is configured to perform deformation processing on the input source image by using the third dot array, and obtain a projection image corresponding to the first dot array projected on an actual projection screen.

9. An electronic device, comprising: Comprise: A processor, a storage medium and a bus, the storage medium storing machine readable instructions executable by the processor, when the electronic device is running, the processor and the storage medium communicate through the bus, the processor executes the machine readable instructions to perform the steps of the image processing method in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, when the computer program is run by the processor, the steps of the image processing method in any one of claims 1 to 7 are executed.

Images