Image processing method, device and electronic equipment

Abstract

The embodiment of the present invention discloses an image processing method, device and electronic device, which creates a first canvas and a second canvas according to a target image, wherein the first canvas is a canvas created in a memory, and the second canvas is a canvas created in a video memory and adapted to the resolution of a single projector; the first canvas is divided to obtain at least one sub-canvas, and the number of the sub-canvases is consistent with the number of the projectors; the second canvas is drawn according to the sub-canvas to obtain intermediate image data; and the intermediate image data is output to the projector through an output interface to display the target image. Thus, by using a small amount of video memory, the image fusion quality is improved and the equipment cost is reduced.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an image processing method, device and electronic equipment. Background Art

[0002] At present, there are more and more scenes that need to display ultra-wide images. In the process of projecting ultra-wide images, the existing technology usually creates a canvas of the same size as the ultra-wide image in the video memory, and then provides the image to the projector through multiple video output interfaces after geometric correction and edge feathering to achieve ultra-wide image fusion. As the size of ultra-wide images becomes larger and larger, the size of the video memory canvas is limited to a maximum width and height, resulting in the need to use a large number of video memory devices, which is costly. Summary of the invention

[0003] In view of this, an embodiment of the present invention provides an image processing method, apparatus and electronic device, which improve the image fusion quality and reduce the equipment cost by utilizing a small amount of video memory.

[0004] In a first aspect, an embodiment of the present invention provides an image processing method, the method comprising:

[0005] Creating a first canvas and a second canvas according to the target image, wherein the first canvas is a canvas created in the memory, and the second canvas is a canvas created in the video memory and adapted to the resolution of a single projector;

[0006] Divide the first canvas to obtain at least one sub-canvas, where the number of the sub-canvases is consistent with the number of the projectors;

[0007] Drawing the second canvas according to the sub-canvas to obtain intermediate image data;

[0008] The intermediate image data is output to a projector through an output interface to display the target image.

[0009] In some embodiments, creating the first canvas and the second canvas according to the target image includes:

[0010] Determine the target image;

[0011] Creating the first canvas in the memory according to the target image, and determining a first coordinate corresponding to a first component in the first canvas;

[0012] Get the resolution of a single projector;

[0013] A second canvas is created in the video memory according to the resolution of the single projector.

[0014] In some embodiments, dividing the first canvas to obtain at least one sub-canvas includes:

[0015] Divide the first canvas according to the number of projectors, the arrangement mode and the width of the blending zone to obtain at least one sub-canvas, and determine the second coordinate corresponding to the sub-canvas;

[0016] The arrangement order of the sub-canvases is consistent with the arrangement order of the projectors.

[0017] In some embodiments, the method further comprises:

[0018] Get the third coordinate of the second component in the sub-canvas;

[0019] The display coordinates and texture coordinates of the second component on the second canvas are acquired according to the third coordinates and the second coordinates.

[0020] In some embodiments, drawing the second canvas according to the sub-canvas to obtain intermediate image data includes:

[0021] Dividing the display area of ​​the output interface into a plurality of sub-areas according to the resolution of the projector;

[0022] Drawing a sub-image on the second canvas according to the display coordinates and the texture coordinates;

[0023] The sub-image is drawn onto the corresponding sub-region to obtain intermediate image data.

[0024] In some embodiments, drawing the sub-image on the second canvas according to the display coordinates and the texture coordinates comprises:

[0025] Determining a corresponding relationship according to the display coordinates and the texture coordinates;

[0026] Drawing on the second canvas according to the corresponding relationship to obtain at least one sub-image;

[0027] Performing geometric correction on the sub-image according to the geometric deformation parameters of the projection picture of the single projector;

[0028] The fusion zone of each sub-image is subjected to feathering processing, and the fusion zone is a region where each sub-image contacts each other.

[0029] In some embodiments, drawing the sub-image onto the corresponding sub-region to obtain intermediate image data comprises:

[0030] The sub-images are spliced ​​according to a preset direction to determine at least one intermediate image data.

[0031] In some embodiments, outputting the intermediate image data to a projector through an output interface to display the target image includes:

[0032] Transmitting the intermediate image data to an image segmentation processing unit via the output interface, one output interface corresponding to one image segmentation display processing unit;

[0033] Converting the resolution of the intermediate image into the resolution of a single projector through the image segmentation processing unit;

[0034] transmitting intermediate image data of the resolution of the single projector to the projector;

[0035] The intermediate image data is projected onto a display screen via a projector to display a target image.

[0036] In a second aspect, an embodiment of the present invention provides an image processing device, the device comprising:

[0037] A canvas creation unit, used to create a first canvas and a second canvas according to a target image, wherein the first canvas is a canvas created in a memory, and the second canvas is a canvas created in a video memory and adapted to a resolution of a single projector;

[0038] a segmentation unit, configured to divide the first canvas to obtain at least one sub-canvas, wherein the number of the sub-canvases is consistent with the number of the projectors;

[0039] an intermediate image acquisition unit, configured to draw the second canvas according to the sub-canvas to obtain intermediate image data;

[0040] The display unit is used to output the intermediate image data to a projector through an output interface to display the target image.

[0041] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a memory and a processor, wherein the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method described in the first aspect.

[0042] The technical solution of the embodiment of the present invention creates a first canvas and a second canvas according to a target image, wherein the first canvas is a canvas created in a memory, and the second canvas is a canvas created in a video memory and adapted to the resolution of a single projector; the first canvas is divided to obtain at least one sub-canvas, and the number of the sub-canvases is consistent with the number of the projectors; the second canvas is drawn according to the sub-canvas to obtain intermediate image data; and the intermediate image data is output to the projector through an output interface to display the target image. Thus, by using a small amount of video memory, the image fusion quality is improved and the equipment cost is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0044] Figure 1 is a schematic diagram of an image processing system according to an embodiment of the present invention;

[0045] Figure 2 is a flowchart of an image processing method according to an embodiment of the present invention;

[0046] Figure 3 is a flowchart of creating a first canvas and a second canvas according to an embodiment of the present invention;

[0047] Figure 4 is a flowchart of dividing a first canvas according to an embodiment of the present invention;

[0048] Figure 5 is a flow chart of obtaining intermediate image data according to an embodiment of the present invention;

[0049] Figure 6 is a schematic diagram of the interface display area of ​​this embodiment;

[0050] Figure 7 is a flowchart of drawing a sub-image according to an embodiment of the present invention;

[0051] Figure 8 is a flowchart of dividing a first canvas according to an embodiment of the present invention;

[0052] Fig. 9 is a schematic diagram of a target image data processing process according to an embodiment of the present invention;

[0053] Fig.10 is a schematic diagram of an image processing device according to an embodiment of the present invention;

[0054] Fig.11 is a schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0055] The present application is described below based on embodiments, but the present application is not limited to these embodiments. In the detailed description of the present application below, some specific details are described in detail. It is possible for those skilled in the art to fully understand the present application without the description of these details. In order to avoid confusing the essence of the present application, known methods, processes, flows, components and circuits are not described in detail.

[0056] In addition, persons of ordinary skill in the art will appreciate that the drawings provided herein are for illustration purposes and are not necessarily drawn to scale.

[0057] Unless the context clearly requires otherwise, the words "include", "comprising" and similar words throughout the application should be interpreted as including rather than exclusive or exhaustive; that is, the meaning is "including but not limited to".

[0058] In the description of this application, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of this application, unless otherwise specified, the meaning of "plurality" is two or more.

[0059] Figure 1 Schematic diagram of an image processing system according to an embodiment of the present invention. Figure 1 In the illustrated embodiment, the image processing system includes a data processing device 11, a projector 12, and a display screen 13. The data processing device 11 includes a system-level chip 14, and the system-level chip 14 also includes a memory 15 and a video memory 16, wherein each output interface of the system-level chip 14 is connected to an image segmentation display processing unit 17, and one output interface corresponds to one image segmentation display processing unit. The data processing device 11 is used to obtain a target image and create a first canvas in the memory 15 according to the target image through the system-level chip 14, and create a second canvas in the video memory 16. The first canvas is divided by the system-level chip 14 to obtain at least one sub-canvas, the second canvas is drawn according to the sub-canvas to obtain intermediate image data, and the intermediate image data is output to the projector 12 through the output interface. The image segmentation display processing unit 17 receives the intermediate image data at the output interface, and converts the resolution of the intermediate image into the resolution of the projector 12, and the projector 12 receives the intermediate image data with the converted resolution, and projects it to the display screen 13 to display the target image.

[0060] The memory 15 may be part of the system-on-chip 14, while the actual memory chip may be outside the chip. The memory controller is responsible for transferring data between the processor and the memory, but the actual memory storage is on an external chip. The video memory 17 is on the graphics card and may be a component inside the chip or an independent chip. In mobile devices or embedded systems, the graphics card may be integrated inside the system-on-chip. In the case of high-performance graphics processing, the graphics card may be an independent chip and connected to the chip through some interface.

[0061] Figure 2 is a flowchart of an image processing method according to an embodiment of the present invention. Figure 2 The image processing method shown specifically includes the following steps:

[0062] Step S100: creating a first canvas and a second canvas according to a target image, wherein the first canvas is a canvas created in a memory, and the second canvas is a canvas created in a video memory and adapted to a resolution of a single projector.

[0063] In this embodiment, the data processing device obtains the target image to be subjected to ultra-wide screen fusion, and creates a first canvas in the memory according to the received target image, and creates a second canvas in the video memory. The first canvas corresponds to the target image, and the second canvas corresponds to a single projector. The target image is the ultra-wide screen image finally presented on the display screen by projecting the projector.

[0064] Figure 3 The flowchart of creating the first canvas and the second canvas according to an embodiment of the present invention. Figure 3 The creation method shown specifically includes the following steps:

[0065] Step S110: determine the target image.

[0066] In this embodiment, the data processing device obtains the target image to be subjected to ultra-wide screen fusion, and determines the data of each element in the target image, such as images, texts, and lines, through the system-level chip.

[0067] Step S120: Create the first canvas in the memory according to the target image, and determine the first coordinates corresponding to the display component in the first canvas.

[0068] In this embodiment, the system-level chip creates a first canvas in the memory according to the target image to store the first coordinates of the display components corresponding to each element in the target image. The first canvas is a logical canvas and does not actually occupy the corresponding video memory, but is only used to record the first coordinates corresponding to each display component. The display component refers to a visual element or object defined and managed on the logical canvas. The display component does not directly correspond to the target image data, but as a logical structure, records the position, size and other possible attributes of each element in the target image on the canvas, such as color, shape, visibility, etc. Among them, the first coordinate includes the coordinate value of the upper left corner of each display component, the height value of the display component and the width value of the display component.

[0069] Step S130: Obtain the resolution of a single projector.

[0070] In this embodiment, the data processing device obtains the resolution of each single projector, wherein the resolution of each single projector is the same.

[0071] Step S140: Create a second canvas in the video memory according to the resolution of the single projector.

[0072] In this embodiment, the system-level chip creates an actual canvas of the same size as the resolution of a single projector in the video memory through a graphics API (Application Program Interface) according to the resolution of the single projector. Generally, this resolution is much smaller than the resolution of the ultra-wide screen and will occupy less video memory. At the same time, a frame buffer is created, which is an area in the video memory for storing rendering results. The frame buffer can be set as a rendering target, which means that all rendering commands will be sent to this buffer and image data will be generated in it.

[0073] Step S200: Divide the first canvas to obtain at least one sub-canvas, where the number of the sub-canvases is consistent with the number of the projectors.

[0074] In this embodiment, the system-level chip divides the first canvas in the memory to obtain at least one sub-canvas, and the number of the sub-canvases is consistent with the number of the projectors.

[0075] Figure 4 is a flowchart of dividing the first canvas according to an embodiment of the present invention. Figure 4 The method for dividing the first canvas shown specifically includes the following steps:

[0076] Step S210: Divide the first canvas according to the number and arrangement of projectors and the width of the blending zone to obtain at least one sub-canvas, and determine a second coordinate corresponding to the sub-canvas.

[0077] In this embodiment, the data processing device obtains the number, arrangement and width of the fusion band of the projectors, and the system-level chip divides the first canvas in the memory according to the number, arrangement and width of the fusion band of the projectors to obtain at least one sub-canvas, wherein the arrangement order of the sub-canvases is consistent with the arrangement order of the projectors.

[0078] For example, suppose there are four projectors, four projectors are arranged horizontally, and the blending zone width between every two projectors is 100 pixels. Suppose the width of the first canvas is 4000 pixels and the height is 2000 pixels. There are 4 projectors, and there will be 3 blending zones between them, so the non-blending zone width of each projector is the first canvas width minus the three blending zone widths divided by the number of projectors. In this example, the non-blending zone width of each projector is 925 pixels. From this, the blending zone width can be determined. The blending zone between the first and second projectors starts at the 925th pixel and ends at the 1024th pixel. The blending zone between the second and third projectors starts at the 1950th pixel and ends at the 2049th pixel. The blending zone between the third and fourth projectors starts at the 2975th pixel and ends at the 3074th pixel. Therefore, sub-canvas 1 starts from pixel 0 of the first canvas and ends at pixel 924, for the first projector; sub-canvas 2 starts from pixel 925 of the first canvas and ends at pixel 2049, for the second projector; sub-canvas 3 starts from pixel 2050 of the first canvas and ends at pixel 3074, for the third projector; sub-canvas 4 starts from pixel 3075 of the first canvas and ends at pixel 4000, for the fourth projector.

[0079] The system-level chip determines the second coordinates corresponding to each sub-canvas according to each acquired sub-canvas, where the second coordinates are the coordinate value of the upper left corner of the sub-canvas in the first canvas, and the height and width of the sub-canvas.

[0080] Step S220: Obtain the third coordinate of the second component in the sub-canvas.

[0081] In this embodiment, the system-level chip obtains the corresponding second component on each sub-canvas according to the obtained sub-canvas, and the second component is a display component dispersed on each sub-canvas after the first component on the first canvas is divided. The third coordinate includes the coordinate value of the upper left corner of each second component on the sub-canvas, the height value and the width value of the second component.

[0082] Step S230: Obtain display coordinates and texture coordinates of the second component on the second canvas according to the third coordinates and the second coordinates.

[0083] In this embodiment, the system-level chip adds the coordinate value of the upper left corner of the sub-canvas in the first canvas to the coordinate value of the upper left corner of each second component on the sub-canvas to obtain the corresponding display coordinates of the second component in the sub-canvas in the second canvas.

[0084] At the same time, the system-level chip also needs to obtain the texture coordinates of each second component on the sub-canvas. The texture coordinates are a coordinate system used to map the texture to a two-dimensional plane or a three-dimensional object surface. In two-dimensional image processing, the texture coordinates include the lower left corner coordinates and the upper right corner coordinates of the second component in the texture coordinate system. Specifically, the system-level chip divides the coordinate value of the upper left corner of each second component on the sub-canvas by the height value and width value of the sub-canvas to obtain the lower left corner coordinate of each second component on the sub-canvas in the texture coordinate system. At the same time, the coordinate value of the upper left corner of each second component on the sub-canvas is first added to the height value and width value of each second component on the sub-canvas, and then divided by the height value and width value of the sub-canvas to obtain the upper right corner coordinate of each second component on the sub-canvas in the texture coordinate system.

[0085] Take a second component on a sub-canvas as an example, assuming that the coordinates of the upper left corner of the sub-canvas in the first canvas are (x, y), the width of the sub-canvas is X, and the height is Y. The coordinates of the upper left corner of a second component on the sub-canvas in the sub-canvas are (a, b), the width of the second component is A, and the height is B.

[0086] The display coordinates (c, d) can be obtained by the following formula:

[0087] c=x+a

[0088] d=y+b

[0089] Among them, the display coordinates (c, d) are the coordinates of the second component on the second canvas, x is the horizontal coordinate of the sub-canvas in the first canvas, y is the vertical coordinate of the sub-canvas in the first canvas, a is the horizontal coordinate of the upper left corner of the second component in the sub-canvas, and b is the vertical coordinate of the upper left corner of the second component in the sub-canvas.

[0090] The lower left corner coordinates (u1, v1) of the second component in the texture coordinate system can be obtained by the following formula:

[0091]

[0092]

[0093] Among them, a is the horizontal coordinate of the upper left corner of the second component in the sub-canvas, X is the width of the sub-canvas, b is the vertical coordinate of the upper left corner of the second component in the sub-canvas, and Y is the height of the sub-canvas.

[0094] The upper right corner coordinates (u2, v2) of the second component in the texture coordinate system can be obtained by the following formula:

[0095]

[0096] Among them, a is the horizontal coordinate of the upper left corner of the second component in the sub-canvas, X is the width of the sub-canvas, b is the vertical coordinate of the upper left corner of the second component in the sub-canvas, Y is the height of the sub-canvas, A is the width of the second component, and B is the height of the second component.

[0097] Step S300: Draw the second canvas according to the sub-canvas to obtain intermediate image data.

[0098] In this embodiment, the system-level chip divides the display area of ​​the output interface into multiple sub-areas based on the resolution of the projector, draws sub-images on the second canvas according to the display coordinates and texture coordinates, and then maps these sub-images to the corresponding sub-areas to generate intermediate image data.

[0099] Figure 5 It is a flowchart of obtaining intermediate image data according to an embodiment of the present invention. Figure 5 The method for obtaining intermediate image data shown specifically includes the following steps:

[0100] Step S310: Divide the display area of ​​the output interface into a plurality of sub-areas according to the resolution of the projector.

[0101] In this embodiment, the system-level chip has multiple physical video output interfaces, and generally the resolution is higher than the resolution of a single projector. Each physical video output interface can be divided into multiple display sub-areas according to the resolution of the projector. Figure 6 Schematic diagram of the interface display area of ​​this embodiment. For example, the output resolution of the projector is 1920*1200, and the picture block area is 8192*4320, so a 4*4 total of 16 Figure 6 In the display area shown, each small display area is a sub-area.

[0102] Step S320: Draw a sub-image on the second canvas according to the display coordinates and the texture coordinates.

[0103] In this embodiment, the image content corresponding to the second component on the sub-canvas can be mapped to the second canvas according to the display coordinates and the texture coordinates, and the sub-image can be displayed on the second canvas.

[0104] Figure 7 is a flow chart of drawing a sub-image according to an embodiment of the present invention. Figure 7 The method for drawing a sub-image shown specifically includes the following steps:

[0105] Step S321, determining a corresponding relationship according to the display coordinates and the texture coordinates.

[0106] In this embodiment, the system-level chip maps the display coordinates of the second component to the lower left corner coordinates of the texture, mapping the display coordinate value to the texture coordinate (0,0). At the same time, the lower right corner coordinates of the second component correspond to the upper right corner coordinates of the texture, mapping the display coordinate value to the texture coordinate (1,1), thereby obtaining the corresponding relationship.

[0107] Step S322: Drawing is performed on the second canvas according to the corresponding relationship to obtain at least one sub-image.

[0108] In this embodiment, the system-level chip maps the target image corresponding to each second component on each sub-canvas to the second canvas according to the corresponding relationship, and one sub-canvas corresponds to one sub-image.

[0109] Step S323: geometrically correct the sub-image according to the geometric deformation parameters of the projection screen of the single projector.

[0110] In this embodiment, a calibration pattern is first used for projection, and the geometric shape of the projected image is observed to see if it is regular. According to the calibration function of the projector, the horizontal and vertical parameters are adjusted until the geometric shape of the projected image returns to normal. During the calibration of the projector, the geometric deformation parameters of the projected image are obtained. According to the geometric deformation parameters of the projector and the characteristics of the sub-image, a suitable geometric correction model is selected. A pair of ground control points at the same position on the distorted image and the reference image is selected, and the correction model parameters are calculated. Using the calculated correction model parameters, the distorted sub-image is coordinate transformed to eliminate or reduce the geometric deformation.

[0111] Step S324: Feathering the fusion band of each sub-image, where the fusion band is the area where each sub-image contacts each other.

[0112] In this embodiment, firstly, the areas where the sub-images touch each other are identified, which are called blending bands. Then, a feathering tool or algorithm is selected in the image editing software to process the blending bands.

[0113] Step S330: Draw the sub-image onto the corresponding sub-region to obtain intermediate image data.

[0114] In this embodiment, the system-level chip draws the sub-image drawn in the second canvas onto the sub-area of ​​the output interface, and the image data displayed in the display area of ​​the entire output interface is the intermediate image data.

[0115] Step S331 : stitching the sub-images according to a preset direction to determine at least one intermediate image data.

[0116] In this embodiment, the system-level chip splices each drawn sub-image in order from left to right and from top to bottom, and outputs it to the display area of ​​the corresponding output interface to determine at least one intermediate image data.

[0117] When the display area of ​​one output interface is drawn, steps S310 - S330 are executed again until the display areas of all output interfaces are drawn.

[0118] Step S400: outputting the intermediate image data to a projector through an output interface to display the target image.

[0119] In this embodiment, the acquired intermediate image data is output to corresponding projectors through multiple output interfaces of the system-level chip, and projected onto a display screen through multiple projectors to display a target image of an ultra-wide screen.

[0120] Figure 8 is a flowchart of dividing the first canvas according to an embodiment of the present invention. Figure 8 The method for dividing the first canvas shown specifically includes the following steps:

[0121] Step S410: Transmit the intermediate image data to the image segmentation processing unit through the output interface, where one output interface corresponds to one image segmentation display processing unit.

[0122] In this embodiment, each output interface is pre-configured with a corresponding relationship with a specific image segmentation processing unit, one output interface corresponds to one image segmentation display processing unit, and the system-level chip transmits the intermediate image data to the image segmentation processing unit. Among them, the image segmentation processing unit can be an FPGA (Field-Programmable Gate Array), which is a highly flexible programmable logic device that can realize different functions according to specific needs. In the field of image processing, FPGA has the advantages of high performance, high speed and high flexibility, and is particularly suitable for scenarios that require processing of large amounts of image data.

[0123] FPGA resolution conversion usually involves coordinate transformation and pixel resampling in image processing. Determine the input and output resolution: First, clarify the resolution of the intermediate image data and the resolution of a single projector. This can be done by traversing all the pixels of the intermediate image and calculating their corresponding positions in the target image, or traversing all the pixels of the target image and calculating their corresponding positions in the intermediate image. Since the resolution of the intermediate image data is different from the resolution of a single projector, the pixels need to be interpolated or filtered to obtain the pixel values ​​at the target resolution. Common interpolation methods include nearest neighbor interpolation, bilinear interpolation, bicubic interpolation, etc. Use hardware description language (write FPGA code to implement coordinate transformation and pixel resampling algorithms to convert the resolution of the intermediate image data into the resolution of a single projector.

[0124] Step S420: converting the resolution of the intermediate image into the resolution of a single projector through the image segmentation processing unit.

[0125] In this embodiment, the image segmentation processing unit reads the intermediate image data from the input interface, calculates the scaling ratio according to the resolution of the projector and the resolution of the intermediate image, and scales the intermediate image to convert the resolution of the intermediate image into the resolution of a single projector.

[0126] Step S430: Transmit the intermediate image data of the resolution of the single projector to the projector.

[0127] In this embodiment, the image segmentation processing unit transmits the scaled intermediate image data to the corresponding projector.

[0128] Step S440: projecting the intermediate image data onto a display screen via a projector to display a target image.

[0129] In this embodiment, after receiving the intermediate image data from the image segmentation processing unit, the projector starts decoding and preparing for projection. During the decoding process, the projector converts the received digital signal back into image information and projects the corresponding intermediate image onto the display screen. By projecting the corresponding intermediate image onto the display screen through multiple projectors, a target image of an ultra-wide screen can be obtained.

[0130] The embodiment of the present invention creates a first canvas and a second canvas according to a target image, wherein the first canvas is a canvas created in a memory, and the second canvas is a canvas created in a video memory and adapted to the resolution of a single projector; the first canvas is divided to obtain at least one sub-canvas, and the number of the sub-canvases is consistent with the number of the projectors; the second canvas is drawn according to the sub-canvas to obtain intermediate image data; and the intermediate image data is output to the projector through an output interface to display the target image. Thus, by using a small amount of video memory, the image fusion quality is improved and the equipment cost is reduced.

[0131] Fig. 9 FIG. 1 is a schematic diagram of a target image data processing process according to an embodiment of the present invention. Fig. 9As shown, the embodiment of the present invention creates a first canvas 71 in the memory according to the target image, and creates a second canvas 72 in the video memory according to the resolution of a single projector, wherein the first canvas includes the rectangular coordinates of the first component of all elements of the target image. The first canvas is divided into a plurality of sub-canvases according to the number of projectors, the arrangement order, and the width of the fusion band. For example, the number of projectors is 16, and from left to right and from top to bottom, they are projectors D1-D4, D5-D8, D9-D12, and D13-D16. The first canvas is also divided into sub-canvases A1-A16 according to the number of projectors, the arrangement order, and the width of the fusion band, i.e., one sub-canvas corresponds to one projector. At the same time, the rectangular coordinates of the second component in the sub-canvas are converted into display coordinates and texture coordinates in the second canvas 72. Since each second component contains its position information on the sub-canvas and properties related to rendering and interaction, when rendering this canvas to the second canvas, the rendering engine will traverse these second components and draw the sub-image on the second canvas according to their properties and position information. The system-level chip has multiple output interfaces, one interface corresponds to a projector. Usually, the resolution of the output interface is higher than the resolution of a single projector. Each output interface can be divided into multiple display sub-areas according to the resolution of a single projector. According to the converted display coordinates and texture coordinates, the image content corresponding to the display components on the sub-canvases A1-A4 is drawn onto the second canvas to obtain sub-images B1-B4, the image content corresponding to the display components on the sub-canvases A5-A8 is drawn onto the second canvas to obtain sub-images B5-B8, the image content corresponding to the display components on the sub-canvases A9-A12 is drawn onto the second canvas to obtain sub-images B9-B12, and the image content corresponding to the display components on the sub-canvases A13-A16 is drawn onto the second canvas to obtain sub-images B13-B16, wherein the drawn sub-images are adapted according to the geometric deformation parameters of the projected screen of the projector, and the edge feathering is performed on the fusion band of each sub-image. According to the position of the corresponding sub-area in the display area of ​​an output interface, each sub-image is spliced ​​to obtain intermediate image data. When the target image data corresponding to the sub-canvases A1-A4 are spliced ​​in the display area of ​​the output interface, the target image data corresponding to the sub-canvases A5-A8 are drawn on the second canvas and divided into four sub-images, and each sub-image is spliced ​​in the position of the display area of ​​an output interface to obtain intermediate image data until the target image data corresponding to all sub-canvases are drawn. All intermediate image data are transmitted to the image segmentation display processing unit through the output interface. In the embodiment of the present invention, the image segmentation display processing unit can be an FPGA processor. The image segmentation display processing unit C1 transmits the intermediate image data 73 composed of the sub-image B1, sub-image B2, sub-image B3 and sub-image B4 to the projectors D1-D4.Based on the above specific implementation steps, the image segmentation display processing unit C2 transmits the intermediate image data 74 composed of sub-images B5, B6, B7 and B8 to projectors D5-D8; the image segmentation display processing unit C3 transmits the intermediate image data 75 composed of sub-images B9, B10, B11 and B12 to projectors D9-D12; the image segmentation display processing unit C4 transmits the intermediate image data 76 composed of sub-images B13, B14, B15 and B16 to projectors D13-D16. Since the target image is an ultra-wide image, the corresponding intermediate image data are outputted respectively according to the arrangement order of the projectors D1-D4, D5-D8, D9-D12 and D13-D16, that is, from left to right and from top to bottom, and the intermediate image data are merged so that the target image data can be fully displayed in the end.

[0132] The embodiment of the present invention creates a first canvas and a second canvas according to a target image, wherein the first canvas is a canvas created in a memory, and the second canvas is a canvas created in a video memory and adapted to the resolution of a single projector; the first canvas is divided to obtain at least one sub-canvas, and the number of the sub-canvases is consistent with the number of the projectors; the second canvas is drawn according to the sub-canvas to obtain intermediate image data; and the intermediate image data is output to the projector through an output interface to display the target image. Thus, by using a small amount of video memory, the image fusion quality is improved and the equipment cost is reduced.

[0133] Fig.10 Schematic diagram of an image processing device according to an embodiment of the present invention. Fig.10 As shown, the image annotation device of the embodiment of the present invention includes a canvas creation unit 81, which is used to create a first canvas and a second canvas according to a target image, wherein the first canvas is a canvas created in a memory, and the second canvas is a canvas created in a video memory and adapted to the resolution of a single projector; a segmentation unit 82, which is used to divide the first canvas to obtain at least one sub-canvas, and the number of the sub-canvases is consistent with the number of the projectors; an intermediate image acquisition unit 83, which is used to draw the second canvas according to the sub-canvas to obtain intermediate image data; and a display unit 84, which is used to output the intermediate image data to the projector through an output interface to display the target image.

[0134] In this embodiment, the canvas creation unit 81 includes:

[0135] A data determination module, used for determining a target image;

[0136] A first creation module, configured to create the first canvas in a memory according to the target image, and determine a first coordinate corresponding to a first component in the first canvas;

[0137] The projector resolution acquisition module is used to obtain the resolution of a single projector;

[0138] The second creation module is used to create a second canvas in the video memory according to the resolution of the single projector.

[0139] In this embodiment, the segmentation unit 82 includes:

[0140] A first division module, used for dividing the first canvas according to the number of projectors, the arrangement mode and the width of the blending zone to obtain at least one sub-canvas, and determining a second coordinate corresponding to the sub-canvas;

[0141] The arrangement order of the sub-canvases is consistent with the arrangement order of the projectors.

[0142] In this embodiment, the segmentation unit 82 further includes:

[0143] A third coordinate acquisition module, used to acquire the third coordinate of the second component in the sub-canvas;

[0144] A coordinate conversion module is used to obtain the display coordinates and texture coordinates of the second component on the second canvas according to the third coordinates and the second coordinates.

[0145] In this embodiment, the intermediate image acquisition unit 83 includes:

[0146] A second division module, used for dividing the display area of ​​the output interface into a plurality of sub-areas according to the resolution of the projector;

[0147] A sub-image acquisition module, used for drawing a sub-image on the second canvas according to the display coordinates and the texture coordinates;

[0148] The drawing module is used to draw the sub-image onto the corresponding sub-area to obtain intermediate image data.

[0149] In this embodiment, the sub-image acquisition module includes:

[0150] A corresponding relationship acquisition module, used to determine the corresponding relationship according to the display coordinates and the texture coordinates;

[0151] A drawing submodule, used for drawing on the second canvas according to the corresponding relationship to obtain at least one sub-image;

[0152] A correction module, used for performing geometric correction on the sub-image according to the geometric deformation parameters of the projection picture of a single projector;

[0153] The feathering module is used to perform feathering processing on the fusion band of each sub-image, and the fusion band is the area where each sub-image contacts each other.

[0154] In this embodiment, the drawing module includes:

[0155] The stitching module is used to stitch the sub-images according to a preset direction to determine at least one intermediate image data.

[0156] In this embodiment, the display unit 84 includes:

[0157] A first transmission module, used to transmit the intermediate image data to the image segmentation processing unit through the output interface, one output interface corresponds to one image segmentation display processing unit;

[0158] A resolution conversion module, used for converting the resolution of the intermediate image into the resolution of a single projector through the image segmentation processing unit;

[0159] A second transmission module, used to transmit the intermediate image data of the resolution of the single projector to the projector;

[0160] The projection module is used to project the intermediate image data onto a display screen through a projector to display a target image.

[0161] The embodiment of the present invention creates a first canvas and a second canvas according to a target image, wherein the first canvas is a canvas created in a memory, and the second canvas is a canvas created in a video memory and adapted to the resolution of a single projector; the first canvas is divided to obtain at least one sub-canvas, and the number of the sub-canvases is consistent with the number of the projectors; the second canvas is drawn according to the sub-canvas to obtain intermediate image data; and the intermediate image data is output to the projector through an output interface to display the target image. Thus, by using a small amount of video memory, the image fusion quality is improved and the equipment cost is reduced.

[0162] Fig.11 Schematic diagram of an electronic device according to an embodiment of the present invention. Fig.11 As shown, Fig.11The electronic device shown is a general address query device, which includes a general computer hardware structure, which at least includes a processor 91 and a memory 92. The processor 91 and the memory 92 are connected via a bus 93. The memory 92 is suitable for storing instructions or programs executable by the processor 91. The processor 91 can be an independent microprocessor or a collection of one or more microprocessors. Thus, the processor 91 executes the instructions stored in the memory 92, thereby executing the method flow of the embodiment of the present invention as described above to realize the processing of data and the control of other devices. The bus 93 connects the above-mentioned multiple components together, and at the same time connects the above-mentioned components to the display controller 94 and the display device and the input / output (I / O) device 95. The input / output (I / O) device 95 can be a mouse, a keyboard, a modem, a network interface, a touch input device, a somatosensory input device, a printer, and other devices known in the art. Typically, the input / output device 95 is connected to the system via an input / output (I / O) controller 96.

[0163] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices (equipment) or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application may adopt a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0164] The present application is described with reference to flowcharts of methods, apparatuses (devices) and computer program products according to embodiments of the present application. It should be understood that each process in the flowchart can be implemented by computer program instructions.

[0165] These computer program instructions may be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device that implements the process Figure 1 A function specified in a process or multiple processes.

[0166] These computer program instructions may also be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce the instructions for implementing the process Figure 1 A device that specifies functions in a process or multiple processes.

[0167] Another embodiment of the present invention relates to a non-volatile storage medium for storing a computer-readable program, wherein the computer-readable program is used for a computer to execute part or all of the above method embodiments.

[0168] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by specifying relevant hardware through a program, and the program is stored in a storage medium, including several instructions for a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0169] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Creating a first canvas and a second canvas according to the target image, wherein the first canvas includes a first coordinate stored in a memory, and the second canvas is a canvas created in a video memory and adapted to a resolution of a single projector; Divide the first canvas to obtain at least one sub-canvas, where the number of the sub-canvases is consistent with the number of the projectors; Drawing the second canvas according to the sub-canvas to obtain intermediate image data; Outputting the intermediate image data to a projector via an output interface to display the target image; The dividing the first canvas to obtain at least one sub-canvas comprises: Divide the first canvas according to the number of projectors, the arrangement mode and the width of the blending zone to obtain at least one sub-canvas, and determine a second coordinate corresponding to the sub-canvas, wherein the arrangement order of the sub-canvases is consistent with the arrangement order of the projectors, and the second coordinate is the coordinate value of the upper left corner of the sub-canvas in the first canvas, the height value and the width value of the sub-canvas; Get the third coordinates of the second components in the sub-canvas, where the third coordinates include the coordinate value of the upper left corner of each second component on the sub-canvas, the height value and the width value of the second component; The display coordinates and texture coordinates of the second component on the second canvas are acquired according to the third coordinates and the second coordinates.

2. The method according to claim 1, characterized in that The step of creating the first canvas and the second canvas according to the target image comprises: Determine the target image; Creating the first canvas in memory according to the target image; Get the resolution of a single projector; A second canvas is created in the video memory according to the resolution of the single projector.

3. The method according to claim 1, characterized in that Drawing the second canvas according to the sub-canvas to obtain intermediate image data includes: Dividing the display area of ​​the output interface into a plurality of sub-areas according to the resolution of the projector; Drawing a sub-image on the second canvas according to the display coordinates and the texture coordinates; The sub-image is drawn onto the corresponding sub-region to obtain intermediate image data.

4. The method according to claim 3, characterized in that Drawing the sub-image on the second canvas according to the display coordinates and the texture coordinates comprises: Determining a corresponding relationship according to the display coordinates and the texture coordinates; Drawing on the second canvas according to the corresponding relationship to obtain at least one sub-image; Performing geometric correction on the sub-image according to the geometric deformation parameters of the projection picture of the single projector; The fusion zone of each sub-image is subjected to feathering processing, and the fusion zone is a region where each sub-image contacts each other.

5. The method according to claim 3, characterized in that: Drawing the sub-image onto the corresponding sub-region to obtain intermediate image data comprises: The sub-images are spliced ​​according to a preset direction to determine at least one intermediate image data.

6. The method according to claim 1, characterized in that Outputting the intermediate image data to the projector through the output interface to display the target image includes: Transmitting the intermediate image data to an image segmentation processing unit via the output interface, one output interface corresponding to one image segmentation display processing unit; Converting the resolution of the intermediate image into the resolution of a single projector through the image segmentation processing unit; transmitting intermediate image data of the resolution of the single projector to the projector; The intermediate image data is projected onto a display screen via a projector to display a target image.

7. An image processing device, characterized in that: The device comprises: A canvas creation unit, used to create a first canvas and a second canvas according to a target image, wherein the first canvas is a canvas created in a memory, and the second canvas is a canvas created in a video memory and adapted to a resolution of a single projector; a segmentation unit, configured to divide the first canvas to obtain at least one sub-canvas, wherein the number of the sub-canvases is consistent with the number of the projectors; an intermediate image acquisition unit, configured to draw the second canvas according to the sub-canvas to obtain intermediate image data; A display unit, used for outputting the intermediate image data to a projector through an output interface to display the target image; The segmentation unit also includes: a first division module, configured to divide the first canvas into at least one sub-canvas according to the number of projectors, the arrangement mode and the width of the blending zone, and determine a second coordinate corresponding to the sub-canvas, wherein the arrangement order of the sub-canvases is consistent with the arrangement order of the projectors, and the second coordinate is a coordinate value of an upper left corner of the sub-canvas in the first canvas, a height value and a width value of the sub-canvas; A third coordinate acquisition module, used to acquire the third coordinates of the second components in the sub-canvas, wherein the third coordinates include the coordinate value of the upper left corner of each second component on the sub-canvas, and the height and width of the second component; A coordinate conversion module is used to obtain the display coordinates and texture coordinates of the second component on the second canvas according to the third coordinates and the second coordinates.

8. An electronic device, comprising a memory and a processor, characterized in that: The memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method according to any one of claims 1 to 6.

Images