Projection image generation method and device and storage medium
By calculating the transformation coefficient and generating the special-shaped surface projection image, the problem of slow projection image generation speed is solved, and efficient image correction and processing speed improvement is achieved.
Patent Information
- Application Number
- CN202410484403.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-22
- Publication Date
- 2024-11-08
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing projection image generation method relies on GPU and openCV library, resulting in high computing power requirements and slow processing speed, and it is impossible to effectively correct the deformation distortion of the projection light source on the object surface.
By obtaining the image vertex coordinates of the image to be projected and the surface vertex coordinates of the special plane, the transformation coefficient is calculated using the central processor, and the image processor generates the special plane projection image, avoiding all processes in the image processor and reducing the complexity of the algorithm , improve processing speed.
This improves the speed of projected image generation, reduces the requirements for image processors, and completes projected image generation without relying on the openCV library, significantly improving processing speed.
Smart Images

Figure CN118921446A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to a projection image generation method, device and storage medium. Background Art
[0002] In 3D Mapping projection, since the distance from the projection light source to each point on the surface of the object is not equal, this will cause the image projected by the projection light source onto the object to be deformed. In order to ensure that the image projected onto the surface of the object is not distorted, it is usually necessary to perform a perspective transformation on the projected image to correct the projection result and ensure the projection effect.
[0003] The current method of generating projection images is usually that the graphics processing unit (GPU) of the projection device converts the input image pixel by pixel based on the open source computer vision library (openCV) to obtain the projection image. However, since the projection image generation requires the GPU of the projection device to rely on the library, this places high requirements on the computing power of the device and may result in slower processing speed. Summary of the invention
[0004] In order to solve the above technical problems, the present application provides a projection image generation method, device and storage medium, which can improve the speed of generating projection images.
[0005] In a first aspect, the present application provides a projection device, comprising: a communicator, configured to: obtain image parameters of an image to be projected and face vertex coordinates of a shaped surface; the image parameters include image vertex coordinates; the shaped surface is a projection surface on which an object collected at a target position receives projection; a central processing unit, configured to: calculate transformation coefficients based on image vertex coordinates and face vertex coordinates; an image processor, configured to: generate a shaped surface projection image based on the image parameters and the transformation coefficients; the shaped surface projection image is a projection image of the image to be projected projected onto the shaped surface at the target position.
[0006] In some embodiments, the communicator is specifically configured to: obtain image information of the projected object collected at the target position; construct a three-dimensional model of the object based on the image information; and determine the vertex coordinates of the irregular surface based on the three-dimensional model.
[0007] In some embodiments, the central processing unit is specifically configured to: perform coordinate transformation on the image vertex coordinates to obtain homogeneous coordinates corresponding to the image vertex coordinates; use the homogeneous coordinates corresponding to the image vertex coordinates as the coordinates before transformation and the face vertex coordinates as the coordinates after transformation, substitute them into the perspective transformation formula, and calculate the transformation coefficient.
[0008] In some embodiments, the image parameters also include pixel coordinates and pixel values of each pixel point in the image to be projected; the image processor is specifically configured to: determine the projection pixel coordinates based on the pixel coordinates and the transformation coefficient; the projection pixel coordinates are the coordinates of the pixel point in the irregular surface projection image, and one pixel coordinate corresponds to one projection pixel coordinate; the pixel value of the pixel point corresponding to the pixel coordinate is used as the pixel value of the projection pixel coordinate corresponding to the pixel coordinate to obtain the irregular surface projection image.
[0009] In some embodiments, the central processing unit is further configured to calculate the inverse transformation coefficients based on the image vertex coordinates and the face vertex coordinates; the image processor is specifically configured to obtain the pixel value of each pixel point from the image to be projected pixel by pixel based on the projection pixel coordinates and the inverse transformation coefficients, and fill them into the pixel points corresponding to the projection pixel coordinates to obtain the irregular surface projection image.
[0010] In some embodiments, the central processing unit is further configured to: create a canvas before generating the shaped surface projection image; the image processor is specifically configured to: draw the shaped surface projection image on the canvas according to the image parameters and the transformation coefficients.
[0011] In some embodiments, the image processor is further configured to: set the transparency of pixel points outside the irregular surface projection image area to fully transparent.
[0012] In a second aspect, the present application provides a projection image generation method, comprising: obtaining image parameters of an image to be projected and face vertex coordinates of an irregular surface; the image parameters include image vertex coordinates; the irregular surface is a projection surface of an object collected at a target position that receives projection; a central processing unit calculates transformation coefficients based on the image vertex coordinates and face vertex coordinates; an image processor generates an irregular surface projection image based on the image parameters and the transformation coefficients; the irregular surface projection image is a projection image of the image to be projected projected onto the irregular surface at the target position.
[0013] In some embodiments, obtaining the vertex coordinates of the irregular surface includes: obtaining image information of the projected object collected at the target position; constructing a three-dimensional model of the object according to the image information; and determining the vertex coordinates of the irregular surface according to the three-dimensional model.
[0014] In some embodiments, a central processing unit calculates transformation coefficients based on image vertex coordinates and face vertex coordinates, including: performing coordinate conversion on the image vertex coordinates to obtain homogeneous coordinates corresponding to the image vertex coordinates; substituting the homogeneous coordinates corresponding to the image vertex coordinates as the coordinates before transformation and the face vertex coordinates as the coordinates after transformation into the perspective transformation formula to calculate the transformation coefficients.
[0015] In some embodiments, the image parameters also include pixel coordinates and pixel values of each pixel point in the image to be projected; generating a shaped surface projection image by an image processor according to the image parameters and transformation coefficients, including: determining the projection pixel coordinates according to the pixel coordinates and the transformation coefficients; the projection pixel coordinates are the coordinates of the pixel point in the shaped surface projection image, and one pixel coordinate corresponds to one projection pixel coordinate; the pixel value of the pixel point corresponding to the pixel coordinate is used as the pixel value of the projection pixel coordinate corresponding to the pixel coordinate to obtain the shaped surface projection image.
[0016] In some embodiments, before generating the irregular surface projection image, the method also includes: calculating the inverse transformation coefficients based on the image vertex coordinates and the surface vertex coordinates; generating the irregular surface projection image based on the image parameters and the transformation coefficients through an image processor, including: obtaining the pixel value of each pixel point from the image to be projected pixel by pixel based on the projection pixel coordinates and the inverse transformation coefficients, and filling them into the pixel points corresponding to the projection pixel coordinates to obtain the irregular surface projection image.
[0017] In some embodiments, before generating the shaped surface projection image, the method also includes: creating a canvas through a central processing unit; generating the shaped surface projection image according to image parameters and transformation coefficients through an image processor, including: drawing the shaped surface projection image on the canvas according to the image parameters and transformation coefficients.
[0018] In some embodiments, after generating the shaped surface projection image, the method further includes: setting the transparency of pixel points outside the shaped surface projection image area to be fully transparent.
[0019] In a third aspect, the present application provides a computer-readable storage medium, comprising: a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a computing device, the computing device executes a projection image generation method provided in any embodiment of the second aspect.
[0020] In a fourth aspect, the present application provides a computer program product. When the computer program product is run on a computer, the computer executes the projection image generation method provided in any embodiment of the second aspect.
[0021] Compared with the prior art, the technical solution provided by this application has the following advantages:
[0022] In the above scheme, the image vertex coordinates of the image to be projected and the face vertex coordinates of the irregular surface are first obtained. After that, the transformation coefficients are calculated by the central processing unit according to the image vertex coordinates and the face vertex coordinates. Finally, the irregular surface projection image is generated by the image processor according to the image parameters and the transformation coefficients. Among them, the irregular surface projection image is a projection image of the image to be projected onto the irregular surface at the target position. In this way, on the one hand, the transformation coefficients can be calculated in the central processing unit in advance, and only the irregular surface projection image needs to be generated in the image processor, which avoids the problem that all processes are performed in the image processor, which has high requirements on the image processor and improves the processing speed. On the other hand, the entire projection image generation process does not rely on the openCV library, which reduces the complexity of the algorithm and further improves the processing speed. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0024] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0025] Figure 1 A schematic diagram of an application scenario architecture of a projection image generation method provided in an embodiment of the present application;
[0026] Figure 2 A schematic diagram of a hardware configuration block of a projection device provided in an embodiment of the present application;
[0027] Figure 3 A schematic diagram of software configuration of a projection device provided in an embodiment of the present application;
[0028] Figure 4 One of the flowcharts of the projection image generation method provided in the embodiment of the present application;
[0029] Figure 5 A schematic diagram of an image to be projected provided in an embodiment of the present application;
[0030] Fig. 6A One of the structural schematic diagrams of the object provided in the embodiment of the present application;
[0031] Figure 6B The second structural schematic diagram of the object provided in the embodiment of the present application;
[0032] Figure 6CThe third structural schematic diagram of the object provided in the embodiment of the present application;
[0033] Fig.6D A schematic diagram of an output image including a profiled surface provided in an embodiment of the present application;
[0034] Figure 7 A second flowchart of a projection image generation method provided in an embodiment of the present application;
[0035] Figure 8 The third flowchart of the projection image generation method provided in the embodiment of the present application;
[0036] Fig. 9 A fourth flowchart of the projection image generation method provided in an embodiment of the present application;
[0037] Fig.10 A fifth flowchart of the projection image generation method provided in an embodiment of the present application;
[0038] Fig.11 A sixth flowchart of a projection image generation method provided in an embodiment of the present application;
[0039] Fig.12 A seventh flowchart of a projection image generation method provided in an embodiment of the present application;
[0040] Fig.13 A schematic diagram of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION
[0041] In order to make the purpose and implementation method of the present application clearer, the exemplary implementation method of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, rather than all the embodiments.
[0042] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.
[0043] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.
[0044] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.
[0045] 3D Mapping projection refers to projecting images onto the surface of objects in the real world. It can break the physical space and turn the surface of any object into an animation. In this way, by configuring multiple projectors according to the size of the projection area and combining 3DMapping projection technology with special effects phantom, a three-dimensional scene with a high sense of reality and stereoscopic sense can be produced. With the addition of music and sound effects, a more intuitive shocking effect can be achieved, which is of great significance in the fields of commercial promotion and marketing. However, in 3D Mapping projection, since the distance from the projection light source to each point on the surface of the object is not equal, the image projected by the projection light source onto the object will be deformed and distorted. In order to ensure that the image projected onto the surface of the object is not distorted, it is usually necessary to perform perspective transformation on the projected image to correct the projection result and ensure the projection effect.
[0046] The current method of generating projection images is usually that the GPU of the projection device converts the input image pixel by pixel based on the openCV library to obtain the projection image. However, since the generation of projection images requires the GPU of the projection device to rely on the library, this places high requirements on the computing power of the device and may result in slower processing speed.
[0047] In view of the above problems, the present application provides a projection image generation method, which is applied to a projection device and can improve the speed of projection image generation.
[0048] Figure 1 This is a schematic diagram of the application scenario architecture of the projection image generation method provided in the embodiment of the present application. Figure 1 As shown, it includes an object 100 receiving a projection, a projection device 200 and an electronic device 300.
[0049] In some embodiments, the projection device 200 may have a variety of implementation forms, for example, it may be a projector (or projector), a portable projection device, a laser projection device, a transparent projection device, an interactive projection device, a head-mounted projection device, etc.
[0050] In some embodiments, the object 100 receiving the projection can be any object. Of course, in order to achieve a better projection effect, the object 100 can be selected to have a flat and smooth surface, a bright color, and a shape that is as consistent as possible with the projected image.
[0051] In practical applications, such as Figure 1As shown, the user can project the image to be projected onto the side surface 101 of the object 100 through the projection device 200 for the audience to watch. Of course, the selection of the projection surface on the object 100 can be made according to actual conditions, which is only an example here.
[0052] In some embodiments, the projection device 200 can also communicate data with other electronic devices 300. The projection device 200 can be allowed to communicate with the electronic device 300 through Bluetooth, Universal Serial Bus (USB), the Internet, etc. For example, the Internet can be a local area network (LAN) and a wireless local area network (WLAN). The electronic device 300 can be a server, a mobile phone, a computer, a tablet computer, a TV, a smart TV, etc. The electronic device 300 can provide various contents to the projection device 200. For example, the electronic device 300 can provide the projection device 200 with an image to be projected. Among them, when the electronic device 300 is a server, the server can be a cluster or multiple clusters, and can include one or more types of servers.
[0053] In some embodiments, the projection image generation method provided in the embodiment of the present application can be executed by the above-mentioned projection device 200. For example, when the projection image generation method provided in the embodiment of the present application is executed by the projection device 200, the process is as follows: the projection device 200 obtains the image parameters of the image to be projected and the face vertex coordinates of the irregular surface. Among them, the image parameters include the image vertex coordinates, and the irregular surface is the projection surface of the object collected at the target position to receive the projection. Afterwards, the central processing unit calculates the transformation coefficients according to the image vertex coordinates and the face vertex coordinates. Finally, the image processor generates the irregular surface projection image according to the image parameters and the transformation coefficients. Among them, the irregular surface projection image is the projection image of the image to be projected projected onto the irregular surface at the target position. The projection image generation process can be completed without relying on the openCV library, which reduces the complexity of the algorithm and improves the processing speed.
[0054] Figure 2 A schematic diagram of a hardware configuration block of a projection device provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the projection device 200 may include at least one of a tuner 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.
[0055] The display 260 includes a display screen component for presenting images, and a driving component for driving image display, which is used to receive image signals output from the controller, and display video content, image content, and menu control interface components and user control UI interface.
[0056] The display 260 may be a liquid crystal display or an OLED display.
[0057] The communicator 220 is a component for communicating with an external device or server according to various communication protocol types. For example, the communicator may include at least one of a Wifi module, a Bluetooth module, a wired Ethernet module, or other network communication protocol chips or near field communication protocol chips, and an infrared receiver. The projection device 200 may establish transmission and reception of control signals and data signals with the electronic device 300 through the communicator 220.
[0058] The user interface can be used to receive control signals input by the user through a control device (such as an infrared remote controller, etc.) or through touch or gestures.
[0059] The detector 230 is used to collect signals from the external environment or the external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.
[0060] The external device interface 240 may include, but is not limited to, any one or more of the following interfaces: a high-definition multimedia interface (HDMI), an analog or digital high-definition component input interface (component), a composite video input interface (CVBS), a USB input interface (USB), an RGB port, etc. It may also be a composite input / output interface formed by the above multiple interfaces.
[0061] The tuner-demodulator 210 receives broadcast television signals via wired or wireless reception, and demodulates audio and video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.
[0062] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.
[0063] The controller 250 controls the operation of the projection device 200 and responds to the user's operation through various software control programs stored in the memory. The controller 250 controls the overall operation of the projection device 200. For example, in response to receiving a user command for selecting a UI object to be displayed on the display 260, the controller 250 can perform an operation related to the object selected by the user command.
[0064] In some embodiments, the controller 250 includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), a RAM Random Access Memory (RAM), a ROM (Read-Only Memory, ROM), a first interface to an nth interface for input / output, a communication bus (Bus), etc.
[0065] The user may input a user command through a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the graphical user interface (GUI). Alternatively, the user may input a user command through a specific sound or gesture, and the user input interface recognizes the sound or gesture through a sensor to receive the user input command.
[0066] "User interface" is a medium interface for interaction and information exchange between an application or operating system and a user, and it realizes the conversion between the internal form of information and the form acceptable to the user. A common form of user interface is a graphical user interface (GUI), which refers to a user interface related to computer operation displayed in a graphical manner. It can be an interface element such as an icon, window, or control displayed on the display screen of the projection device 200, where a control can include visual interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets.
[0067] See also Figure 3 In some embodiments, the operating system of the projection device 200 is divided into four layers, namely, from top to bottom, the application layer (Applications) (referred to as "application layer"), the application framework layer (Application Framework) (referred to as "framework layer"), the Android runtime (Android runtime) and system library layer (referred to as "system runtime library layer"), and the kernel layer.
[0068] In some embodiments, at least one application is running in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. provided by the operating system, or applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.
[0069] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take. Through the API interface, applications can access system resources and obtain system services during execution.
[0070] like Figure 3 As shown, in the embodiment of the present application, the application framework layer includes managers, content providers, etc., wherein the manager includes at least one of the following modules: an activity manager (ActivityManager) is used to interact with all activities running in the system; a location manager (Location Manager) is used to provide system services or applications with access to system location services; a package manager (Package Manager) is used to retrieve various information related to applications currently installed on the device; a notification manager (Notification Manager) is used to control the display and clearing of notification messages; a window manager (Window Manager) is used to manage icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.
[0071] In some embodiments, the activity manager is used to manage the life cycle of each application and the common navigation back function, such as controlling the exit, opening, and back of the application. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, capturing the screen, and controlling the display window changes (for example, reducing the display window, shaking the display, distorting the display, etc.).
[0072] In some embodiments, the system runtime layer provides support for the upper layer, namely the framework layer. When the framework layer is used, the Android operating system will run the C / C++ library contained in the system runtime layer to implement the functions to be implemented by the framework layer.
[0073] In some embodiments, the kernel layer is a layer between hardware and software. Figure 3As shown, the kernel layer includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.
[0074] In some embodiments, when the projection image generation method provided in the embodiment of the present application is executed by the projection device 200, first, the communicator 220 of the projection device 200 obtains the image parameters of the image to be projected and the face vertex coordinates of the irregular surface. Among them, the image parameters include the image vertex coordinates, and the irregular surface is the projection surface of the object collected at the target position to receive the projection. Afterwards, the central processor in the controller 250 of the projection device 200 calculates the transformation coefficients according to the image vertex coordinates and the face vertex coordinates. Finally, the image processor in the controller 250 of the projection device 200 generates the irregular surface projection image according to the image parameters and the transformation coefficients. Among them, the irregular surface projection image is the projection image of the image to be projected projected onto the irregular surface at the target position. The projection image generation process can be completed without relying on the openCV library, which reduces the algorithm complexity and improves the processing speed.
[0075] Figure 4 A flowchart diagram of a projection image generation method provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, the projection image generation method includes the following steps:
[0076] S11, obtaining image parameters of the image to be projected and the vertex coordinates of the irregular surface.
[0077] The irregular surface is the projection surface of the object collected at the target position to receive the projection, and the image parameters include the coordinates of the image vertices, the pixel coordinates and pixel value of each pixel point, and the size of the image to be projected.
[0078] First, image parameters of the image to be projected are obtained.
[0079] In some embodiments, the method for obtaining the image parameters of the image to be projected can be that the projection device establishes a communication connection with other electronic devices through a communicator, and directly obtains the image to be projected and the image parameters of the image to be projected from the other electronic devices; or the image to be projected is first obtained from other electronic devices, and then the image to be projected is processed to obtain the image parameters of the image to be projected.
[0080] In some embodiments, when the image parameters include image vertex coordinates, the image to be projected is processed to obtain the image parameters of the image to be projected by, after the projection device obtains the image to be projected, establishing a rectangular coordinate system with a certain point of the image to be projected as the origin of the coordinates, and determining the position coordinates of the four vertices of the image to be projected in the rectangular coordinate system as the image vertex coordinates. For example, Figure 5 An image to be projected is provided in an embodiment of the present application, and the image to be projected includes four vertices, namely, a first vertex 501, a second vertex 502, a third vertex 503, and a fourth vertex 504; taking the lower left corner of the image to be projected as the coordinate origin, the direction from the fourth vertex 504 to the third vertex 503 as the x-axis, and the direction from the fourth vertex 504 to the first vertex 501 as the y-axis, a rectangular coordinate system is established to determine the vertex coordinates of the four vertices of the image to be projected, that is, the image vertex coordinates.
[0081] In some embodiments, when the image parameters also include the pixel coordinates and pixel values of each pixel point, the method for obtaining the pixel coordinates of each pixel point is the same as described above, and the method for obtaining the pixel value corresponding to each pixel point can be to traverse each pixel point through the function provided in the image processing library, and read the pixel value corresponding to each pixel point.
[0082] Secondly, obtain the vertex coordinates of the special-shaped surface.
[0083] Due to different viewing angles and distances, three-dimensional objects may appear in different shapes. Take the most common cube as an example. Each face of the cube is a square in real space, but when photographed by the human eye or a lens, it is imaged as a quadrilateral. The four angles vary depending on the viewing angle, and when viewed from different angles, two faces may be seen. For example, Fig. 6A When the cube shown is observed at the angle indicated by A, the following Figure 6B , only the first irregular surface 601 and the second irregular surface 602 are within the observation range. Therefore, when obtaining the vertex coordinates of the irregular surface, it is necessary to obtain the vertex coordinates of the irregular surface corresponding to the target position, and the target position should be consistent with the projection position of the projection device.
[0084] In some embodiments, Figure 7 As shown, the method of obtaining the coordinates of the vertices of the irregular surface may include the following steps:
[0085] S111, acquiring image information of a projected object collected at a target position.
[0086] Specifically, the image information of the projected object may be collected and received at the target position by an image acquisition device, such as a sensor, a camera, and the like.
[0087] S112. Construct a three-dimensional model of the object according to the image information.
[0088] S113, determining the vertex coordinates of the irregular surface according to the three-dimensional model.
[0089] Specifically, the method of determining the coordinates of the vertices of the irregular surface based on the three-dimensional model can be to establish a three-dimensional coordinate system in the three-dimensional model with a certain point in the irregular surface as the coordinate origin, and determine the position coordinates of the four vertices of the irregular surface in the three-dimensional coordinate system as the coordinates of the vertex. Among them, the position of the origin of the three-dimensional coordinate system in the irregular surface should be consistent with the position of the origin of the rectangular coordinate system in the image to be projected. For example, if the origin of the rectangular coordinate system is the lower left corner of the image to be projected, then the origin of the three-dimensional coordinate system should also be the lower left corner of the irregular surface. Figure 6C As shown, when the profiled surface is Fig. 6A and Figure 6B When the first shaped surface 601 is identified, a three-dimensional coordinate system is established with the lower left corner of the first shaped surface 601 as the coordinate origin, and the shaped vertex coordinates of the first shaped vertex 611, the second shaped vertex 612, the third shaped vertex 613, and the fourth shaped vertex 614 of the first shaped surface 601 in the three-dimensional coordinate system are determined.
[0090] In the above scheme, it is possible to collect and receive image information of the projected object from the target position, build a three-dimensional model of the object based on the image information, and then determine the vertex coordinates of the irregular surface based on the three-dimensional model, so as to subsequently determine the transformation coefficients based on the vertex coordinates to complete the projection image generation method.
[0091] S12. Calculate the transformation coefficients according to the image vertex coordinates and the surface vertex coordinates through the central processing unit.
[0092] In some embodiments, Figure 8 As shown, the method of calculating the transformation coefficients by the central processing unit according to the image vertex coordinates and the face vertex coordinates may include the following steps:
[0093] S121 . Perform coordinate transformation on the image vertex coordinates to obtain homogeneous coordinates corresponding to the image vertex coordinates.
[0094] Since the image vertex coordinates obtained in step S11 are two-dimensional coordinates and the face vertex coordinates are three-dimensional coordinates, it is also necessary to convert the two-dimensional coordinates into three-dimensional coordinates for the convenience of calculation, that is, to perform coordinate conversion on the image vertex coordinates to obtain the homogeneous coordinates corresponding to the image vertex coordinates. Among them, the way to convert the image vertex coordinates into homogeneous coordinates can be to add a component to the image vertex coordinates to convert them into homogeneous coordinates. For example, when the image vertex coordinates are (x, y), the corresponding homogeneous coordinates can be (x, y, z), where the value of z is related to the distance between the target position and the object.
[0095] S122. Substitute the homogeneous coordinates corresponding to the image vertex coordinates as the coordinates before transformation and the surface vertex coordinates as the coordinates after transformation into the perspective transformation formula to calculate the transformation coefficients.
[0096] Among them, the perspective transformation formula can be expressed as: Where (x, y, z) is the homogeneous coordinates of the image vertex coordinates, (X, Y, Z) is the face vertex coordinates, and a 11 to a 33 is the transformation coefficient of the perspective transformation matrix.
[0097] In the above scheme, the coordinates of the image vertices can be transformed by the central processing unit to obtain the homogeneous coordinates corresponding to the image vertex coordinates, and then the homogeneous coordinates corresponding to the image vertex coordinates are used as the coordinates before transformation, and the face vertex coordinates are used as the coordinates after transformation, and are substituted into the perspective transformation formula to calculate the transformation coefficient. In this way, the transformation coefficient is calculated in the central processing unit, which avoids occupying the resources of the image processor and improves the processing speed of the image processor.
[0098] S13, generating a special-shaped surface projection image according to the image parameters and the transformation coefficients through an image processor.
[0099] The irregular surface projection image is a projection image obtained by projecting the image to be projected onto the irregular surface at the target position.
[0100] Specifically, the transformation coefficients are first converted into a perspective matrix, and then the perspective matrix is right-multiplied by the pixel coordinates of each pixel point in the image to be processed to generate a projection image of the irregular surface.
[0101] In the above scheme, the image vertex coordinates of the image to be projected and the face vertex coordinates of the irregular surface are first obtained. After that, the transformation coefficients are calculated by the central processing unit according to the image vertex coordinates and the face vertex coordinates. Finally, the irregular surface projection image is generated by the image processor according to the image parameters and the transformation coefficients. Among them, the irregular surface projection image is a projection image of the image to be projected onto the irregular surface at the target position. In this way, on the one hand, the transformation coefficients can be calculated in the central processing unit in advance, and only the irregular surface projection image needs to be determined in the image processor, which avoids the problem that all processes are performed in the image processor, which has high requirements on the image processor and improves the processing speed. On the other hand, the entire projection image generation process does not rely on the openCV library, which reduces the complexity of the algorithm and further improves the processing speed.
[0102] In some embodiments, the image parameters also include the pixel coordinates and pixel values of each pixel point in the image to be projected. Fig. 9 As shown, the image processor generates a projection image of a special-shaped surface according to image parameters and transformation coefficients, and the method includes the following steps:
[0103] S131. Determine projection pixel coordinates according to pixel coordinates and transformation coefficients.
[0104] The projection pixel coordinates are the coordinates of the pixel points in the projection image of the irregular surface, and one pixel coordinate corresponds to one projection pixel coordinate.
[0105] Specifically, the transformation coefficients are first converted into a perspective matrix, and then the perspective matrix is right-multiplied by the pixel coordinates of each pixel in the image to be processed to obtain the coordinates of each pixel in the projection image of the irregular surface, that is, the projection pixel coordinates.
[0106] S132, taking the pixel value of the pixel point corresponding to the pixel coordinate as the pixel value of the projection pixel coordinate corresponding to the pixel coordinate, and obtaining the irregular surface projection image.
[0107] In the above scheme, the projection pixel coordinates can be determined by the image processor according to the pixel coordinates and the transformation coefficient, and then the pixel value of the pixel point corresponding to the pixel coordinate is used as the pixel value of the projection pixel coordinate corresponding to the pixel coordinate to obtain the irregular surface projection image. In this way, the irregular surface projection image can be generated directly according to the pixel coordinates, the pixel value of the pixel point corresponding to the pixel coordinates, and the transformation coefficient. The entire projection image generation process does not rely on the openCV library, which reduces the algorithm complexity and further improves the processing speed.
[0108] In some embodiments, before step S13, Fig.10 As shown, the projection image generation method also includes:
[0109] S1. Calculate the inverse transformation coefficients according to the image vertex coordinates and the face vertex coordinates through the central processing unit.
[0110] The method of calculating the inverse transformation coefficients is the same as the method of calculating the transformation coefficients in step S122, which will not be repeated here.
[0111] At this time, the implementation method of step S132 may be:
[0112] S2. Obtain the pixel value of each pixel from the image to be projected pixel by pixel according to the projection pixel coordinates and the inverse transformation coefficient through the image processor, and fill it into the pixel corresponding to the projection pixel coordinates to obtain the special-shaped surface projection image.
[0113] In the above scheme, the central processing unit can calculate the inverse transformation coefficient according to the image vertex coordinates and the face vertex coordinates, and then the image processor can obtain the pixel value of each pixel point from the image to be projected pixel by pixel according to the projection pixel coordinates and the inverse transformation coefficient, and fill it into the pixel point corresponding to the projection pixel coordinate to obtain the irregular surface projection image. In this way, the irregular surface projection image can be generated according to the inverse transformation matrix, the projection pixel coordinates, and the pixel value of each pixel point in the image to be projected. The entire projection image generation process does not rely on the openCV library, which reduces the algorithm complexity and further improves the processing speed.
[0114] In some embodiments, before step S13, Fig.11 As shown, the projection image generation method also includes:
[0115] S3. Create a canvas through the central processing unit.
[0116] Specifically, when the central processing unit determines that a special-shaped surface projection image of the image to be projected needs to be generated, a canvas is created for the GPU to draw the special-shaped surface projection image.
[0117] At this time, the implementation method of step S13 may be:
[0118] S4. Draw the projection image of the special-shaped surface on the canvas according to the image parameters and the transformation coefficients through the image processor.
[0119] Before drawing the projection image of the irregular surface, it is also necessary to align the coordinate system corresponding to the projection pixel coordinates with the coordinate system of the central processing unit operating environment, and at the same time align the coordinate system of the image processor operating environment with the coordinate system corresponding to the projection pixel coordinates to ensure the accuracy of the drawn projection image of the irregular surface.
[0120] In the above scheme, the central processing unit can create a canvas, and then the image processor can draw the projection image of the special-shaped surface on the canvas according to the image parameters and transformation coefficients, avoiding the problem that all processes are performed in the image processor and placing high requirements on the image processor, thereby improving the processing speed.
[0121] In some embodiments, since the acquired image to be projected, image parameters, face vertex coordinates, and transformation coefficients calculated by the central processor are all in the central processor, before step S13, the central processor is required to send the image to be projected, image parameters, face vertex coordinates, and transformation coefficients to the image processor through the communicator, so that the image processor can generate the special-shaped surface projection image according to the image to be projected, image parameters, face vertex coordinates, and transformation coefficients. Of course, if the special-shaped surface projection image needs to be drawn on the canvas, the canvas size also needs to be sent to the image processor.
[0122] In some embodiments, after step S13, Fig.12 As shown, the projection image generation method also includes:
[0123] S14, setting the transparency of the pixels outside the projection image area of the irregular surface to be fully transparent.
[0124] Specifically, the output image is usually displayed in the form of a regular quadrilateral, while the shaped surface projection image is usually an irregular quadrilateral. In order to make the shaped surface projection image fully displayed, the output image must include the non-shaped surface projection image part. Fig.6D As shown, the white area is the part displayed by the shaped surface projection image, and the black area is the part of the non-shaped surface projection image, that is, the part outside the shaped surface projection image area. At this time, the transparency of the pixels outside the shaped surface projection image area is set to fully transparent, which can avoid the influence of the projection image generation processing on the pixels outside the projection image area, and show the original color outside the projection image area to the user as much as possible, ensuring the user experience.
[0125] In some embodiments, the projection image generation method in this application can be implemented by the following code:
[0126] The CPU processes part of the code:
[0127] mMappingShader.set("m00",(float)data[0]);
[0128] mMappingShader.set("m01",(float)data[1]);
[0129] mMappingShader.set("m02",(float)data[2]);
[0130] mMappingShader.set("m10",(float)data[3]);
[0131] mMappingShader.set("m11",(float)data[4]);
[0132] mMappingShader.set("m12",(float)data[5]);
[0133] mMappingShader.set("m20",(float)data[6]);
[0134] mMappingShader.set("m21",(float)data[7]);
[0135] mMappingShader.set("m22",(float)data[8]);
[0136] mMappingShader.set("size",(float)tex.width,(float)tex.height);
[0137] mMappingShader.set("size1",(float)canvas.width,(float)canvas.height);
[0138] mMappingShader.set("tex",tex);
[0139] Where mMappingShader is the shader object, m00 to m22 are nine matrix coefficients, size is the size of the image to be projected, size1 is the size of the canvas, and tex is the image to be projected.
[0140] Image processor processing code:
[0141] vec2 uv = vec2(gl_FragCoord.x, size1.y - gl_FragCoord.y); / / Align the coordinate system corresponding to the projection pixel coordinates with the coordinate system of the CPU operating environment.
[0142] uv=uv-size1 / 2.0; / / Align the coordinate system of the image processor operating environment with the coordinate system corresponding to the projection pixel coordinates.
[0143] mat3 transform = mat3(m00,m10,m20,m01,m11,m21,m02,m12,m22); / / Transformation coefficients are converted into perspective matrix.
[0144] vec3 transformCoords = transform*vec3(uv,1.0); / / Calculate the position of the projected pixel coordinates in the image to be projected.
[0145] vec2 ts = vec2(transformCoords.x, transformCoords.y) / transformCoords.z; / / Normalization.
[0146] ts = ts / size + 0.5;
[0147] ts.y=1.0-ts.y;
[0148] if(ts.x<0.0||ts.x>1.0||ts.y<0.0||ts.y>1.0){ / / Area that needs to be fully transparent.
[0149] gl_FragColor=vec4(1.0,0.0,0.0,0.0);
[0150] return;
[0151] }
[0152] vec4 color = texture(tex, ts); / / Get the color from the corresponding position of the projection pixel coordinate in the image to be projected.
[0153] In the above scheme, on the one hand, the transformation coefficients can be calculated in advance in the central processing unit, and only the projection image of the irregular surface needs to be determined in the image processor, which avoids the problem that all processes are performed in the image processor and the image processor has high requirements, thereby improving the processing speed. On the other hand, the entire projection image generation process does not rely on the openCV library, which reduces the algorithm complexity and further improves the processing speed.
[0154] The embodiment of the present application can divide the anti-flashback device into functional modules according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing unit. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical functional division. There may be other division methods in actual implementation.
[0155] The embodiment of the present application also provides a projection image generation device, which is applied to a projection device and includes an acquisition module, a team formation module, a replacement module, and a processing module.
[0156] The acquisition module is used to acquire the image parameters of the image to be projected and the face vertex coordinates of the irregular surface; the image parameters include the image vertex coordinates; the irregular surface is the projection surface of the object collected at the target position to receive the projection; the processing module is used to calculate the transformation coefficients according to the image vertex coordinates and the face vertex coordinates through the central processing unit; the processing module is also used to generate the irregular surface projection image according to the image parameters and the transformation coefficients through the image processor; the irregular surface projection image is the projection image of the image to be projected projected onto the irregular surface at the target position.
[0157] In some embodiments, the acquisition module is specifically used to: acquire image information of the projected object collected at the target position; construct a three-dimensional model of the object according to the image information; and determine the vertex coordinates of the irregular surface according to the three-dimensional model.
[0158] In some embodiments, the processing module is specifically used to: perform coordinate transformation on the image vertex coordinates to obtain homogeneous coordinates corresponding to the image vertex coordinates; use the homogeneous coordinates corresponding to the image vertex coordinates as the coordinates before transformation and the face vertex coordinates as the coordinates after transformation, substitute them into the perspective transformation formula, and calculate the transformation coefficient.
[0159] In some embodiments, the image parameters also include pixel coordinates and pixel values of each pixel point in the image to be projected; the processing module is specifically used to: determine the projection pixel coordinates through an image processor based on the pixel coordinates and the transformation coefficient; the projection pixel coordinates are the coordinates of the pixel point in the projection image of the irregular surface, and one pixel coordinate corresponds to one projection pixel coordinate; the image processor uses the pixel value of the pixel point corresponding to the pixel coordinate as the pixel value of the projection pixel coordinate corresponding to the pixel coordinate to obtain the projection image of the irregular surface.
[0160] In some embodiments, the processing module is also used to: before generating the irregular surface projection image, calculate the inverse transformation coefficient according to the image vertex coordinates and the surface vertex coordinates through the central processing unit; the processing module is specifically used to: obtain the pixel value of each pixel point from the image to be projected pixel by pixel according to the projection pixel coordinates and the inverse transformation coefficient through the image processor, and fill it into the pixel point corresponding to the projection pixel coordinate to obtain the irregular surface projection image.
[0161] In some embodiments, the processing module is further used to create a canvas through a central processing unit before generating the irregular surface projection image; the processing module is specifically used to: draw the irregular surface projection image on the canvas according to image parameters and transformation coefficients through an image processor.
[0162] In some embodiments, the processing module is further used to set the transparency of pixel points outside the area of the special-shaped surface projection image to fully transparent through the image processor after generating the special-shaped surface projection image.
[0163] Among them, all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, and its role will not be repeated here.
[0164] like Fig.13As shown, the embodiment of the present application also provides a chip system, which can be applied to the projection device 200 in the aforementioned embodiment. The chip system includes at least one processor 1101 and at least one interface circuit 1102. The processor 1101 can be the processor in the above-mentioned projection device 200. The processor 1101 and the interface circuit 1102 can be interconnected through a line. The processor 1101 can receive and execute computer instructions from the memory of the above-mentioned projection device 200 through the interface circuit 1102. When the computer instructions are executed by the processor 1101, the projection device 200 can execute the various steps executed by the projection device 200 in the above-mentioned embodiment. Of course, the chip system can also include other discrete devices, which are not specifically limited in the embodiment of the present application.
[0165] The embodiment of the present application further provides a computer-readable storage medium for storing computer instructions executed by the projection device 200.
[0166] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer executes the projection image generating method provided by the above method embodiment.
[0167] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A projection device, characterized in that: include: The communicator is configured to: obtain image parameters of the image to be projected and the coordinates of the face vertices of the irregular surface; the image parameters include the coordinates of the image vertices; the irregular surface is a projection surface of the object collected at the target position to receive the projection; A central processing unit is configured to: calculate transformation coefficients according to the image vertex coordinates and the face vertex coordinates; The image processor is configured to generate a shaped surface projection image according to the image parameters and the transformation coefficients; the shaped surface projection image is a projection image obtained by projecting the image to be projected onto the shaped surface at a target position.
2. The projection device according to claim 1, characterized in that: The communicator is specifically configured as follows: Acquire image information of the projected object collected at the target location; constructing a three-dimensional model of the object according to the image information; The vertex coordinates of the irregular surface are determined according to the three-dimensional model.
3. The projection device according to claim 1, characterized in that: The central processing unit is specifically configured as follows: Performing coordinate transformation on the image vertex coordinates to obtain homogeneous coordinates corresponding to the image vertex coordinates; The homogeneous coordinates corresponding to the image vertex coordinates are used as the coordinates before transformation, and the surface vertex coordinates are used as the coordinates after transformation, and are substituted into the perspective transformation formula to calculate the transformation coefficient.
4. The projection device according to claim 1, characterized in that: The image parameters also include pixel coordinates and pixel values of each pixel point in the image to be projected; The image processor is specifically configured as follows: Determine the projection pixel coordinates according to the pixel coordinates and the transformation coefficients; the projection pixel coordinates are the coordinates of the pixel points in the projection image of the profiled surface, and one pixel coordinate corresponds to one projection pixel coordinate; The pixel value of the pixel point corresponding to the pixel coordinate is used as the pixel value of the projection pixel coordinate corresponding to the pixel coordinate to obtain the irregular surface projection image.
5. The projection device according to claim 4, characterized in that: The central processing unit is further configured to: Calculating inverse transformation coefficients according to the image vertex coordinates and the surface vertex coordinates; The image processor is specifically configured as follows: According to the projection pixel coordinates and the inverse transformation coefficients, the pixel value of each pixel point is obtained from the image to be projected pixel by pixel, and filled into the pixel point corresponding to the projection pixel coordinate to obtain the special-shaped surface projection image.
6. The projection device according to any one of claims 1 to 5, characterized in that: The central processor is further configured to: create a canvas before generating the shaped surface projection image; The image processor is specifically configured to draw the special-shaped surface projection image on the canvas according to the image parameters and the transformation coefficients.
7. The projection device according to any one of claims 1 to 5, characterized in that: The image processor is further configured to set the transparency of the pixels outside the projection image area of the special-shaped surface to be fully transparent.
8. A method for generating a projection image, characterized in that: include: Acquire image parameters of the image to be projected and vertex coordinates of the irregular surface; the image parameters include the vertex coordinates of the image; the irregular surface is a projection surface of the object collected at the target position to receive the projection; Calculating transformation coefficients according to the image vertex coordinates and the surface vertex coordinates by a central processing unit; An image processor generates a profiled surface projection image according to the image parameters and the transformation coefficients; the profiled surface projection image is a projection image obtained by projecting the image to be projected onto the profiled surface at a target position.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a computing device, the computing device implements the projection image generating method according to claim 8.
10. A computer program product, characterized in that When the computer program product is executed on a computer, the computer is caused to implement the projection image generating method described in claim 8 .
Citation Information
Patent Citations
Projection correction method and device, storage medium and projection equipment
CN114286066A
Projector, image correction method, and program
JP2014192688A
Image projection system, image processing apparatus, image projection method and program
JP2015103891A
Projection control device, imaging device, control method, program, and, storage medium
JP2020042099A
Perspective correct vector graphics rendering techniques
US20220270312A1
