Image processing method and device, electronic equipment and storage medium

Abstract

The invention provides an image processing method and device, electronic equipment and a storage medium, and relates to the technical field of image processing. The method comprises the steps of determining an original image and a target image; the original image is a to-be-corrected image received by the projection equipment, and the target image is used for indicating the shape of an image to be output by the projection equipment; performing downsampling processing on the original image according to the target image to obtain a processed image; storing the processed image to a storage area of the projection equipment; reading the processed image from the storage area, and performing zooming processing on the processed image to obtain an output image; the output image is an image output to a screen by the projection equipment. According to the scheme, the image output effect of the projection equipment is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to an image processing method and device, electronic equipment and a storage medium. BACKGROUND

[0002] In an image projection scenario, trapezoidal distortion often occurs when an image is projected due to the placement angle of a projection device and other reasons.

[0003] In related technologies, a projection device can determine a target image and correct an original image according to the target image, so that the image output by the projection device is a normal image. Specifically, the projection device can store the original image in a storage area. Then, the projection device determines the mapping coordinates of each pixel point in the target image in the original image row by row and pixel by pixel in the display stage, reads the adjacent pixel values of the pixel point in the original image according to the mapping coordinates, and performs interpolation to obtain an output image.

[0004] However, since the projection device needs to read the adjacent pixel values of the pixel point in the original image, there is a problem of a sharp increase in instantaneous peak bandwidth in the image processing process, thereby causing the output image of the projection device to have poor effect. SUMMARY

[0005] The present application provides an image processing method and device, electronic equipment and a storage medium to improve the effect of the output image of the projection device.

[0006] In a first aspect, the present application provides an image processing method applied to a projection device, which includes:

[0007] determining an original image and a target image; the original image is a to-be-corrected image received by the projection device, and the target image is used to indicate the shape of the image to be output by the projection device;

[0008] performing down-sampling processing on the original image according to the target image to obtain a processed image;

[0009] storing the processed image in a storage area of the projection device;

[0010] reading the processed image from the storage area and performing scaling processing on the processed image to obtain an output image; the output image is an image output by the projection device onto a screen.

[0011] In a possible implementation, the down-sampling processing on the original image according to the target image to obtain the processed image includes:

[0012] determining at least one vertical sampling region and at least one horizontal sampling region in the target image; each vertical sampling region corresponds to the vertical dimension of the target image, and each horizontal sampling region corresponds to the horizontal dimension of the target image.

[0013] down-sampling in at least one vertical sampling region to determine at least one first pixel value;

[0014] down-sampling in at least one horizontal sampling region to determine at least one second pixel value;

[0015] determining a processed image according to the at least one first pixel value and the at least one second pixel value.

[0016] In a possible implementation, the down-sampling in the at least one vertical sampling region to determine the at least one first pixel value comprises:

[0017] determining at least one first sampling point in the at least one vertical sampling region;

[0018] for each first sampling point in the at least one first sampling point, performing the following operations:

[0019] mapping a first target position of the first sampling point in the target image to the original image to determine a first original position of the first sampling point in the original image;

[0020] determining a pixel value at the first original position in the original image as a pixel value corresponding to the first sampling point;

[0021] wherein the at least one first pixel value comprises the pixel value corresponding to each first sampling point.

[0022] In a possible implementation, the down-sampling in the at least one horizontal sampling region to determine the at least one second pixel value comprises:

[0023] determining at least one second sampling point in the at least one horizontal sampling region; the at least one second sampling point is respectively a starting point of each of the at least one horizontal sampling region;

[0024] for each second sampling point in the at least one second sampling point, performing the following operations:

[0025] determining a second target position corresponding to the second sampling point according to a position of the second sampling point in the target image;

[0026] mapping the second target position to the original image to determine at least one second original position of the second sampling point in the original image;

[0027] performing interpolation processing on pixel values at the at least one second original position to determine a pixel value corresponding to the second sampling point;

[0028] wherein the at least one second pixel value comprises the pixel value corresponding to the second sampling point in each horizontal sampling region.

[0029] In one possible implementation, scaling the processed image to obtain an output image includes:

[0030] Divide the target image into at least one region;

[0031] For each region in at least one region, perform the following operations:

[0032] Based on the region and the original image, determine the output state of the region; the output state can be one of the following: zoomed-in state, zoomed-out state, or original state.

[0033] When the output state is either reduced or original, the region corresponding to the region in the processed image is determined as the target region.

[0034] When the output state is magnified, the target region is determined based on the region corresponding to the region in the processed image;

[0035] The output image includes at least one target region.

[0036] In one possible implementation, determining the output state of the region based on the region and the original image includes:

[0037] Map the region to the original image to determine the original region corresponding to the region in the original image;

[0038] Determine the ratio between the height of the determined region and the height of the original region;

[0039] If the ratio is less than 1, the output state is determined to be amplified.

[0040] When the ratio is equal to 1, the output state is determined to be the original state;

[0041] If the ratio is greater than 1, the output state is determined to be the reduced state.

[0042] In one possible implementation, determining the target region based on the region corresponding to the region in the processed image includes:

[0043] Based on the region corresponding to the region in the processed image, at least one pixel of the region is determined;

[0044] Interpolate at least one pixel to determine at least one supplementary pixel;

[0045] The target region is determined based on at least one pixel and at least one supplementary pixel.

[0046] Secondly, this application provides an image processing apparatus, comprising:

[0047] The determination module is used to determine the original image and the target image; the original image is the image to be corrected received by the projection device, and the target image is used to indicate the shape of the image to be output by the projection device;

[0048] The first processing module is used to downsample the original image based on the target image to obtain the processed image;

[0049] The storage module is used to store the processed image in the storage area of ​​the projection device;

[0050] The second processing module is used to read the processed image from the storage area and scale the processed image to obtain the output image; the output image is the image output to the screen by the projection device.

[0051] In one possible implementation, the first processing module is specifically used for:

[0052] In the target image, at least one vertical sampling region and at least one horizontal sampling region are determined; each vertical sampling region corresponds to the vertical dimension of the target image, and each horizontal sampling region corresponds to the horizontal dimension of the target image.

[0053] Downsampling is performed in at least one vertical sampling region to determine at least one first pixel value;

[0054] Downsampling is performed in at least one horizontal sampling region to determine at least one second pixel value;

[0055] The processed image is determined based on at least one first pixel value and at least one second pixel value.

[0056] In one possible implementation, the first processing module is specifically used for:

[0057] Determine at least one first sampling point in at least one vertical sampling region;

[0058] For each of the first sampling points in at least one first sampling point, perform the following operation:

[0059] Map the first target position of the first sampling point in the target image to the original image to determine the first original position of the first sampling point in the original image;

[0060] The pixel value at the first original position in the original image is determined as the pixel value corresponding to the first sampling point;

[0061] Wherein, at least one first pixel value includes the pixel value corresponding to at least one first sampling point.

[0062] In one possible implementation, the first processing module is specifically used for:

[0063] At least one second sampling point is determined in at least one horizontal sampling region; the at least one second sampling point is the starting point of each of the at least one horizontal sampling regions.

[0064] For each sampling point in at least one second sampling point, perform the following operation:

[0065] Based on the position of the second sampling point in the target image, determine the position of the second target corresponding to the second sampling point;

[0066] Map the second target location onto the original image to determine at least one second original location of the second sampling point in the original image;

[0067] Interpolate the pixel values ​​at at least one second original location to determine the pixel value corresponding to the second sampling point;

[0068] Among them, at least one second pixel value includes the pixel value corresponding to the second sampling point in each horizontal sampling region.

[0069] In one possible implementation, the second processing module is specifically used for:

[0070] Divide the target image into at least one region;

[0071] For each region in at least one region, perform the following operations:

[0072] Based on the region and the original image, determine the output state of the region; the output state can be one of the following: zoomed-in state, zoomed-out state, or original state.

[0073] When the output state is either reduced or original, the region corresponding to the region in the processed image is determined as the target region.

[0074] When the output state is magnified, the target region is determined based on the region corresponding to the region in the processed image;

[0075] The output image includes at least one target region.

[0076] In one possible implementation, the second processing module is specifically used for:

[0077] Map the region to the original image to determine the original region corresponding to the region in the original image;

[0078] Determine the ratio between the height of the determined region and the height of the original region;

[0079] If the ratio is less than 1, the output state is determined to be amplified.

[0080] When the ratio is equal to 1, the output state is determined to be the original state;

[0081] If the ratio is greater than 1, the output state is determined to be the reduced state.

[0082] In one possible implementation, the second processing module is specifically used for:

[0083] Based on the region corresponding to the region in the processed image, at least one pixel of the region is determined;

[0084] Interpolate at least one pixel to determine at least one supplementary pixel;

[0085] The target region is determined based on at least one pixel and at least one supplementary pixel.

[0086] Thirdly, this application provides an electronic device, comprising:

[0087] At least one processor; and

[0088] A memory that is communicatively connected to at least one processor; wherein,

[0089] The memory stores instructions that can be executed by at least one processor to cause the at least one processor to perform the methods involved in the first aspect and any possible implementation.

[0090] Fourthly, this application provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to perform the methods involved in the first aspect and any possible implementation.

[0091] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the methods involved in the first aspect and any possible implementation.

[0092] In a sixth aspect, this application provides a chip including at least one processor for executing program instructions to perform the methods involved in the first aspect and any possible implementation.

[0093] The image processing method, apparatus, electronic device, and storage medium provided in this application involve a projection device performing downsampling processing on the distorted original image based on the target image before storing the original image. The resulting processed image removes redundant pixels and compresses the data volume. This reduces the write bandwidth requirement when storing the processed image in the storage area. Furthermore, during the image output stage, the projection device only performs regional scaling on the processed image, eliminating the need to perform interpolation calculations on a large number of adjacent pixels of the original image, thus avoiding sudden spikes in peak bandwidth. Therefore, the image processing method provided in this application reduces the probability of sudden spikes in peak bandwidth, thereby improving the image output quality of the projection device. Attached Figure Description

[0094] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0095] Figure 1 A schematic diagram illustrating the reading of pixel values ​​from an image block, provided for related technologies;

[0096] Figure 2 A schematic diagram of an image processing method provided for related technologies;

[0097] Figure 3 A schematic diagram of another image processing method provided for related technologies;

[0098] Figure 4 This is a schematic diagram of the system architecture provided for an embodiment of this application;

[0099] Figure 5 A schematic flowchart of an image processing method provided in an embodiment of this application;

[0100] Figure 6 A flowchart illustrating the process of determining a processed image, as provided in an embodiment of this application;

[0101] Figure 7 A flowchart illustrating the process of determining an output image, as provided in an embodiment of this application;

[0102] Figure 8 A schematic diagram illustrating the determination of an output image provided in an embodiment of this application;

[0103] Figure 9 This is a schematic diagram of an image processing embodiment provided in this application;

[0104] Figure 10 A schematic diagram illustrating the range of scaling ratio values ​​provided in this application embodiment;

[0105] Figure 11This application provides a schematic diagram of the structure of an image processing apparatus according to an embodiment of the present application;

[0106] Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0107] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0108] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0109] The collection, storage, use, processing, transmission, provision, and disclosure of financial data or user data involved in the technical solution of this application all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0110] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0111] When projecting images, projectors often exhibit trapezoidal distortion due to the tilted angle of the projector (such as upward or side projection). For example, the image projected onto the screen may appear as a trapezoid, narrower at the top and wider at the bottom, or vice versa. Therefore, it is necessary to correct the output image of the projector to ensure that the final image displayed on the screen is a standard rectangle.

[0112] In related technologies, a projector can segment the original image into multiple independent image blocks and set deformation parameters for each image block. These deformation parameters may include, for example, scaling ratios and offsets. Then, the projector determines multiple adjusted image blocks by adjusting the geometry of each image block, and stitches these adjusted image blocks together to obtain the output image.

[0113] While offline processing of the original image using the above method does not require caching the entire original image, only reserving storage space for each image block, thus saving storage resources, online processing of the original image using the same method (processing directly through the explicit path of the display device) requires reserving a large image buffer area within the display device's chip to temporarily store the row data of multiple adjusted image blocks, since the display needs to output the image line by line and each image block has different deformation parameters.

[0114] When stitching together multiple adjusted image blocks, because each image block undergoes geometric deformation, when reading the row data of an image block, it is not enough to only read the pixel values ​​of that image block itself; the pixel values ​​of a larger area surrounding the image block must also be read. This can be combined with... Figure 1 To understand, Figure 1 This is a schematic diagram illustrating how to read pixel values ​​from an image block, as provided for related technologies. For example... Figure 1 As shown, multiple image blocks include image block A, image block B, image block C, image block D, image block E, and image block F. Taking image block B as an example, when reading the pixel values ​​in image block B, it is also necessary to read the pixel values ​​of a portion of the area surrounding image block B. Among these, Figure 1 The middle grid area represents the pixel values ​​in image block B, while the black area represents the pixel values ​​of the area surrounding image block B.

[0115] In this way, when adjacent image blocks read redundant data, there will be a large number of overlapping areas, causing the same pixel value to be read repeatedly, resulting in a surge in instantaneous peak bandwidth when reading data.

[0116] In some embodiments, the projection device includes a reserved storage area for the original image. The projection device can store the original image in this storage area. When displaying the original image, a target image for correction can be determined first. Since the image needs to be projected onto the screen line by line, the projection device can determine the position coordinates of multiple pixels in the current line within the original image. For any pixel among the multiple pixels, if its position coordinates in the original image are not integers, the projection device needs to read the pixel values ​​surrounding those position coordinates and determine the pixel value at those position coordinates based on the surrounding pixel values.

[0117] Specifically, it can be combined with Figure 2 To understand, Figure 2 A schematic diagram illustrating an image processing method provided for related technologies. For example... Figure 2 As shown, the projection device can first determine the position coordinates of the four vertices in the original image, and then combine formula (1) to determine the eight coefficients in formula (1):

[0118] (1)

[0119] Where x is the horizontal coordinate of the pixel in the original image, and y is the vertical coordinate of the pixel in the original image. The horizontal coordinates of the pixel in the target image. This represents the vertical coordinates of the pixel in the target image. - For the eight coefficients in formula (1), the coordinates of four vertices in the original image and the coordinates of four vertices in the target image can be input into formula (1) to calculate the results. - The value of .

[0120] For each pixel in the target image, the position coordinates of the pixel in the original image can be determined by combining the position coordinates of the pixel in the target image with formula (1). However, the calculated position coordinates may not be represented by integers. In this case, the projection device needs to read the pixel values ​​around the position coordinates and perform interpolation based on the pixels around the position coordinates to obtain the pixel value at the position coordinates.

[0121] However, the projection device experiences a sudden spike in peak bandwidth when reading pixel values ​​around the location coordinates.

[0122] In some embodiments, even if vertical gradient correction is performed on the original image in the manner described above—that is, only the trapezoidal shape in the vertical direction of the original image is corrected—a sudden surge in instantaneous peak bandwidth still occurs. Specifically, this can be combined with... Figure 3 To understand, Figure 3 A schematic diagram of another image processing method provided for related technologies.

[0123] like Figure 3 As shown, the projection device does not need to adjust the horizontal position of each pixel in the original image, but only the vertical position of the pixels. Therefore, the projection device can determine the vertical coordinates of each pixel in the target image in the original image by combining formula (2).

[0124] (2)

[0125] Where y is the vertical coordinate of the pixel in the original image. This represents the vertical coordinates of the pixel in the target image. and For the coefficients in formula (2), the coordinates of the four vertices in the original image and the four vertices in the target image can be input into formula (1) to calculate the results. and The value of .

[0126] For each pixel in the target image, the vertical coordinates of the pixel in the original image can be determined by combining the vertical coordinates of the pixel in the target image with formula (2). However, the calculated vertical coordinates may not be represented by integers. In this case, the projection device needs to read the pixel values ​​around the vertical coordinates and perform interpolation based on the pixels around the vertical coordinates to obtain the pixel value at that position coordinate.

[0127] Therefore, even if the original image is vertically gradient corrected using the methods described above, the problem of a sudden surge in peak bandwidth still occurs.

[0128] In summary, during the image processing process, a sudden surge in peak bandwidth can occur. If this peak bandwidth exceeds the storage area's rated read / write bandwidth, it can lead to data transmission delays, stuttering, and even data loss, resulting in distortions such as screen tearing, glitches, and ghosting on the projected image. Therefore, image processing methods described above result in poor image quality from the projector.

[0129] Based on this, this application provides an image processing method in which the projection device performs downsampling processing on the distorted original image based on the target image before storing the original image. The resulting processed image removes redundant pixels and compresses the data volume. This reduces the write bandwidth requirement when storing the processed image in the storage area. Furthermore, during the image output stage, the projection device only performs regional scaling on the processed image, eliminating the need for interpolation calculations on a large number of adjacent pixels of the original image, thus avoiding sudden spikes in bandwidth. Therefore, the image processing method provided in this application reduces the probability of sudden spikes in bandwidth, thereby improving the image output quality of the projection device.

[0130] To facilitate understanding, the following will be combined with... Figure 4 The system architecture applicable to the embodiments of this application will be described.

[0131] Figure 4 This is a schematic diagram of the system architecture provided for an embodiment of this application. Figure 4 As shown, it includes a projection device 41 and a screen 42.

[0132] In practical applications, the original image can be input into the projection device 41. After receiving the original image, the projection device 41 can correct the original image to obtain the output image. Then, the projection device 41 projects the output image onto the screen 42.

[0133] It should be noted that, Figure 4This is merely an example to illustrate a system architecture diagram, and is not a limitation on system architecture diagrams.

[0134] It should be noted that the execution subject in each embodiment of this application can be a chip, chip module, processor, microprocessor, etc., or it can be a device integrating the above-mentioned chips, chip modules, processors, or microprocessors, such as a server. The specific execution subject in each embodiment of this application is not limited, and it can be selected and set according to actual needs. In the following embodiments, a server integrating the above-mentioned chips, chip modules, processors, or microprocessors is used as an example for description, which does not constitute a limitation on the actual execution subject.

[0135] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0136] Figure 5 This is a schematic flowchart illustrating an image processing method provided in an embodiment of this application. Figure 5 As shown, the method may include the following steps:

[0137] S501. Determine the original image and the target image; the original image is the image to be corrected received by the projection device, and the target image is used to indicate the shape of the image to be output by the projection device.

[0138] The original image is the image input to the projection device, and it is typically a rectangular image. A projection device is an electronic device that converts image or video signals into optical images and projects them onto a medium such as a screen or wall. Its core function is to magnify and display images to meet the needs of multiple viewers and large-scale visual presentations. An example of a projection device is a projector.

[0139] In some embodiments, the original image can be sent to the projection device via a device such as a mobile phone or computer.

[0140] The target image is the output reference object of the projection device; it is a preset, ideal image shape template used to guide the correction process. The target image itself is not actual image data, but rather a set of geometric parameters (such as the length, width, and vertex coordinates of a rectangle) indicating the shape of the final output image. Its core function is to define the shape the projection device needs to correct the original image to. The shape of the target image is the opposite of the shape of the image on the screen from which the projection device projects the original image. For example, assuming the shape of the image projected onto the screen is a trapezoid that is narrower at the top and wider at the bottom, the target image can also be a trapezoid that is wider at the top and narrower at the bottom.

[0141] Depending on the placement and angle of the projection device, the image projected onto the screen may exhibit trapezoidal distortion. Therefore, the projection device can preprocess the original image to ensure that the shape of the output image is the same as the target image, thus ensuring that the image projected onto the screen is identical in shape to the original image.

[0142] In some embodiments, the projection device stores multiple candidate target images. The projection device can determine the target image from the multiple candidate target images based on the current position and angle.

[0143] S502. Downsample the original image based on the target image to obtain the processed image.

[0144] The processed image refers to the intermediate image data generated after downsampling with varying scaling, resulting in data compression. The processed image removes redundant pixels from the original image, retains effective pixel areas that match the shape of the target image, and the vertical scaling ratio is limited to 1 (without pre-enlargement).

[0145] In some embodiments, the projection device can adjust the sampling interval (e.g., a larger sampling interval for narrow-side trapezoidal regions and a smaller sampling interval for wide-side regions) based on the matching degree between different regions of the original image and the target image, and determine at least one sampling point in the target image according to the sampling interval. Then, the projection device determines the original position of each of the at least one sampling point in the original image, and determines the pixel value of each of the at least one sampling point based on its original position. Finally, the processed image is obtained based on the pixel values ​​of each of the at least one sampling point.

[0146] S503. Store the processed image in the storage area of ​​the projection device.

[0147] The projection device includes a storage area for storing the processed image. This storage area can be, for example, Double Data Rate Synchronous Dynamic Random-Access Memory (DDR SDRAM).

[0148] S504. Read the processed image from the storage area and scale the processed image to obtain the output image; the output image is the image output to the screen by the projection device.

[0149] The output image specifically refers to the image generated after downsampling preprocessing, storage and retrieval, and region scaling. It should be noted that the shape of the output image is the same as the shape of the target image. Therefore, when the projection device projects the output image onto the screen, the image displayed on the screen is an image with the same shape as the original image.

[0150] In some embodiments, when the projection device needs to project the processed image, it can read the pixel values ​​of the processed image line by line and determine whether to enlarge the image region of that line in the processed image based on the target image and the original image.

[0151] If it is not necessary to enlarge the image region of that row in the processed image, the pixel values ​​of that row in the processed image are projected onto the screen. If it is necessary to process the image region of that row in the processed image, the image region of that row is enlarged based on the pixel values ​​of that row in the processed image, the pixel values ​​of the enlarged region are obtained, and the pixel values ​​of the enlarged region are projected onto the screen.

[0152] exist Figure 5 In the illustrated embodiment, before storing the original image, the projection device first performs downsampling processing on the distorted original image based on the target image. The resulting processed image removes redundant pixels and compresses the data volume. This reduces the write bandwidth requirement when storing the processed image in the storage area. Furthermore, during the image output stage, the projection device only performs regional scaling on the processed image, eliminating the need to perform interpolation calculations on a large number of adjacent pixels of the original image, thus avoiding sudden spikes in peak bandwidth. Therefore, the image processing method provided in this application reduces the probability of sudden spikes in peak bandwidth, thereby improving the image output quality of the projection device.

[0153] exist Figure 5 Based on the illustrated embodiment, the following is combined with Figure 6 The method of downsampling the original image based on the target image to obtain the processed image in the embodiments of this application will be further explained.

[0154] Figure 6 This is a schematic flowchart illustrating the process of determining a processed image, as provided in an embodiment of this application. Figure 6 As shown, the process may include the following steps:

[0155] S601. Determine at least one vertical sampling region and at least one horizontal sampling region in the target image; each vertical sampling region corresponds to the vertical dimension of the target image, and each horizontal sampling region corresponds to the horizontal dimension of the target image.

[0156] The vertical sampling region is a sampling area divided by the projection device based on the target image in the vertical dimension. The vertical sampling region has the same vertical coordinate range as the target image, and the sampling density of the vertical sampling region can be determined by the vertical scaling ratio between the target image and the original image. In some embodiments, the smaller the vertical scaling ratio, the smaller the sampling density of the vertical sampling region; the larger the vertical scaling ratio, the larger the sampling density of the vertical sampling region.

[0157] The horizontal sampling region is a sampling area defined by the projection device in the horizontal dimension based on the target image. The horizontal sampling region corresponds to the vertical coordinate range of the target image.

[0158] S602. Perform downsampling processing in at least one vertical sampling region to determine at least one first pixel value.

[0159] In some embodiments, the projection device determines at least one first pixel value in the following manner: determining at least one first sampling point in at least one vertical sampling region; for each of the at least one first sampling point, performing the following operations: mapping a first target position of the first sampling point in the target image to the original image, determining a first original position of the first sampling point in the original image; determining the pixel value at the first original position in the original image as the pixel value corresponding to the first sampling point; wherein, the at least one first pixel value includes the pixel value corresponding to each of the at least one first sampling point.

[0160] In some embodiments, the projection device can determine the vertical sampling density of the target image based on the mapping relationship between the vertical scaling ratio and the sampling density, and the vertical scaling ratio between the original image and the target image. Then, sampling is performed in at least one vertical sampling region according to the vertical sampling density to obtain at least one first sampling point.

[0161] The first target location is the position of the first sampling point in the target image. In some embodiments, the first target location can be represented in coordinate form. For example, the first target location can be represented as (x, y).

[0162] The first original location is the position of the first sampling point in the original image. In some embodiments, the first target location can be represented by coordinates.

[0163] In some embodiments, for each of the first sampling points in at least one first sampling point, the projection device can input the first target position of the first sampling point into formula (1) to obtain the first original position of the first sampling point.

[0164] Then, the projection device determines the pixel value at the first original position in the original image as the pixel value corresponding to the first sampling point.

[0165] It should be noted that in some embodiments, since the horizontal coordinates of the first original position and the first target position are the same, the projection device can set the horizontal coordinates of both the first original position and the first target position to 0. In this case, the projection device can input the vertical coordinates of the first target position into formula (2) to obtain the vertical coordinates of the first sampling point in the first original position. Then, the first original position is determined based on the vertical coordinates of the first original position and the horizontal coordinates of the first target position. This improves the computational efficiency of the projection device and reduces its computational cost.

[0166] S603. Perform downsampling processing in at least one horizontal sampling region to determine at least one second pixel value.

[0167] In some embodiments, the projection device determines at least one second pixel value in the following manner: determining at least one second sampling point in at least one horizontal sampling region; the at least one second sampling point being the starting point of each of the at least one horizontal sampling region; for each of the at least one second sampling point, performing the following operations: determining a second target position corresponding to the second sampling point based on the position of the second sampling point in the target image; mapping the second target position to the original image to determine a second original position of the second sampling point in the original image; determining the pixel value at the second original position in the original image as the pixel value corresponding to the second sampling point; wherein, the at least one second pixel value includes the pixel value corresponding to the second sampling point in each horizontal sampling region.

[0168] The second target location is the position of the second sampling point in the target image. In some embodiments, the second target location can be represented by coordinates. The second original location is the position of the second sampling point in the original image. In some embodiments, the second original location can be represented by coordinates.

[0169] In some embodiments, for each of the at least one horizontal sampling regions, the projection device may determine the starting point of that horizontal sampling region as a second sampling point. Therefore, at least one second sampling point is the leftmost sampling point in the target image.

[0170] In some embodiments, the second sampling point is located at the leftmost edge of the target image. Taking a trapezoidal target image as an example, the second sampling point is located on the leftmost edge of the trapezoid.

[0171] The projection device can calculate the position of at least one second sampling point in the target image based on the coordinates of its upper-left and lower-left corners. It should be noted that the position of each second sampling point in the target image can be represented by coordinates. Furthermore, since the position of the second sampling point in the target image is calculated, its coordinates may not be integers.

[0172] In some embodiments, when the coordinates of the second sampling point in the target image are represented by integers, the projection device determines the coordinates of the second sampling point in the target image as the second target position.

[0173] At this point, the projection device can input the second target position of the second sampling point into formula (1) to obtain the second original position of the second sampling point. Then, the projection device determines the pixel value at the second original position in the original image as the pixel value corresponding to the second sampling point.

[0174] If the coordinates of the second sampling point in the target image are not integers, the projection device can perform rounding up and rounding down operations on these coordinates to obtain two coordinates corresponding to the second sampling point. Then, the projection device determines these two coordinates as the second target position of the second sampling point.

[0175] At this point, the projection device can input the two second target positions of the second sampling point into formula (1) to obtain the two mapped positions of the second target positions in the original image. The projection device determines the pixel values ​​of these two mapped positions in the original image and performs interpolation processing on the pixel values ​​of these two mapped positions in the original image to obtain the pixel value of the second sampling point.

[0176] S604. Determine the processed image based on at least one first pixel value and at least one second pixel value.

[0177] In some embodiments, for each of the at least one first pixel values, the projection device can establish a two-dimensional pixel grid based on the vertical and horizontal dimensions of the target image, and then determine the position of the first pixel value in the pixel grid according to the first original position corresponding to the first pixel value. Similarly, the positions of each second pixel value in the pixel grid are determined based on the above method. Then, the projection device fills in the missing pixel values ​​between adjacent pixel values ​​in the pixel grid using a spatial interpolation algorithm, and removes redundant pixel regions in the pixel grid that exceed the shape range of the target image, thereby obtaining the processed image.

[0178] exist Figure 6In the illustrated embodiment, the projection device first divides the sampling area based on the vertical and horizontal dimensions of the target image to ensure that the sampling range accurately matches the shape of the final output image and avoids sampling of invalid areas. Furthermore, image reconstruction is performed by combining the pixel values ​​obtained from the vertical and horizontal sampling. This ensures that the geometry of the processed image matches the target image while minimizing redundant pixels, laying a low-bandwidth, high-fidelity data foundation for subsequent scaling processing at the output end. This reduces the probability of instantaneous peak bandwidth spikes, thereby improving the quality of the image output by the projection device.

[0179] Based on the above embodiments, the following is combined with Figure 7 The method of scaling the processed image to obtain the output image in the embodiments of this application will be further explained.

[0180] Figure 7 This is a schematic diagram illustrating a process for determining an output image, as provided in an embodiment of this application. Figure 7 As shown, the process may include the following steps:

[0181] S701. Divide the target image into at least one region.

[0182] In some embodiments, the projection device can decompose the geometric space of the target image into one or more regions. Where there are multiple regions, each region is an independent unit.

[0183] S702. For each region in at least one region, determine the output state of the region based on the region and the original image; the output state is one of zoomed-in state, zoomed-out state, and original state.

[0184] The output state of a region indicates how that region is processed during the output stage. The "Magnified" state means that the projection device can perform interpolation magnification on that region during output to match the shape of the target image; this operation is only performed during the output stage. The "Shrinked" state means that the projection device can perform downsampling on that region during output to remove redundant pixels and further compress the data. The "Original" state means that the projection device can directly output the pixel data of that region without additional scaling processing.

[0185] In some embodiments, for each region in at least one region, the output state of that region can be determined as follows: mapping the region to the original image to determine the original region corresponding to the region in the original image; determining the ratio between the height of the region and the height of the original region; if the ratio is less than 1, determining the output state as a magnified state; if the ratio is equal to 1, determining the output state as an original state; if the ratio is greater than 1, determining the output state as a reduced state.

[0186] The original region is the mapped region of the region in the original image. In some embodiments, the projection device can determine the coordinates of the four vertices of the region, then determine the mapped coordinates of these four vertices in the original image, and determine the original region based on the mapped coordinates of the four vertices.

[0187] In some embodiments, the projection device may determine the height of the region and the height of the original region, and then calculate the ratio between the height of the region and the height of the original region.

[0188] If the ratio is less than 1, it means that the height of the region is less than the height of the original region mapped in the original image. In other words, the projection device needs to enlarge the region when outputting it, so the output state is determined to be the enlarged state.

[0189] When the ratio is equal to 1, it means that the height of the region is equal to the height of the original region mapped in the original image. That is, when the projection device outputs the region, it does not need to enlarge or reduce the region. Therefore, the output state is determined to be the original state.

[0190] If the ratio is greater than 1, it means that the height of the region is greater than the height of the original region mapped in the original image. In other words, when the projection device outputs the region, it needs to reduce the size of the region. Therefore, the output state is determined to be the reduced state.

[0191] S703. For each region in at least one region, when the output state is either reduced or original, the region corresponding to that region in the processed image is determined as the target region.

[0192] In some embodiments, when the output state is either reduced or original, it indicates that the size of the region in the processed image already matches the shape requirements of the target image. Furthermore, if the output state of the region is reduced, since downsampling has already been performed on the region, there is no need to further reduce its size.

[0193] Therefore, when the output state is either reduced or original, the region corresponding to that region in the processed image can be directly identified as the target region.

[0194] S704. When the output state is in the magnified state, determine the target region based on the region corresponding to the region in the processed image; wherein the output image includes at least one target region.

[0195] In some embodiments, when the output state is a reduced state or the original state, it indicates that the size of the region corresponding to the processed image is smaller than the shape requirement of the target image. In this case, it is necessary to supplement pixel data based on the region and in combination with the size requirement of the target image, and expand the region size by means of interpolation operations, so as to finally determine the target region that meets the shape requirement of the target image.

[0196] Specifically, the projection device can determine the target area in the following way: based on the area corresponding to the area in the processed image, determine at least one pixel of the area; perform interpolation on the at least one pixel to determine at least one supplementary pixel; and determine the target area based on the at least one pixel and the at least one supplementary pixel.

[0197] In some embodiments, when the output state of a region is in a magnified state, the projection device can extract at least one pixel within the region based on the region corresponding to that region in the processed image.

[0198] Since the size of the region corresponding to this area in the processed image is smaller than the shape requirement of the target image, the size of this region needs to be expanded. Therefore, the projection device can calculate the pixel data of the missing position based on at least one pixel using an interpolation algorithm (such as linear interpolation or bilinear interpolation) to obtain at least one supplementary pixel, thereby filling the pixel gap in the enlarged area.

[0199] Finally, the projection device integrates at least one pixel with at least one supplementary pixel, arranging them according to the geometry of the area to determine the target area that matches the size of the area and provides a smooth image.

[0200] Can be combined Figure 8 To understand, Figure 8 This is a schematic diagram illustrating the determination of an output image according to an embodiment of this application. Figure 8 As shown, the projection device can divide the target image into region 1, region 2, and region 3. Region 1 is output in a magnified state, region 2 in a reduced state, and region 3 in its original state. The projection device magnifies region 1 based on its output state to obtain target region 1. The projection device designates region 2 as target region 2 and region 3 as target region 3. Then, the projection device stitches target regions 1, 2, and 3 together to obtain the output image.

[0201] existFigure 7 In the illustrated embodiment, the projection device first divides the target image into multiple sub-regions, and then determines the output state of each region based on the region height ratio. For regions in the reduced or original state, the corresponding region of the processed image is directly reused without additional interpolation operations, reducing the computational load of the projection device. For regions in the enlarged state, pixels are extracted based on the corresponding region of the processed image, and interpolation is performed to generate supplementary pixels. Only the regions that need to be enlarged are locally refined, reducing the bandwidth pressure caused by global interpolation, while ensuring the smoothness of the image in the enlarged region.

[0202] Based on the above embodiments, the following, in conjunction with Figure 9 The image processing method provided in the embodiments of this application will be further described.

[0203] Figure 9 This is a schematic diagram illustrating an image processing method provided in an embodiment of this application. For example... Figure 9 As shown, the projection device can downsample the original image based on the vertical and horizontal scaling ratios to determine the processed image, and then store the processed image in the storage area. When it is necessary to project the processed image, the processed image can be read line by line, and each region in the processed image can be scaled according to the output scaling ratio to determine the output image, which is then projected onto the screen. The output scaling ratio is the scaling ratio for each region in the processed image.

[0204] For example, the range of values ​​for the vertical scaling ratio, horizontal scaling ratio, and output scaling ratio can be as follows: Figure 10 As shown, Figure 10 This diagram illustrates a range of scaling ratio values ​​provided in an embodiment of this application. For example... Figure 10 As shown, during the downsampling process, the horizontal scaling ratio can range from 0.5 to 1, the vertical scaling ratio can range from 0.4 to 1, and the output scaling ratio can range from 1 to 1.6.

[0205] Figure 11 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this application. For example... Figure 11 As shown, the image processing device 110 includes: a determining module 111, a first processing module 112, a storage module 113, and a second processing module 114, wherein,

[0206] The determining module 111 is used to determine the original image and the target image; the original image is the image to be corrected received by the projection device, and the target image is used to indicate the shape of the image to be output by the projection device;

[0207] The first processing module 112 is used to perform downsampling processing on the original image based on the target image to obtain the processed image;

[0208] Storage module 113 is used to store the processed image to the storage area of ​​the projection device;

[0209] The second processing module 114 is used to read the processed image from the storage area and scale the processed image to obtain an output image; the output image is the image output to the screen by the projection device.

[0210] In one possible implementation, the first processing module 112 is specifically used for:

[0211] In the target image, at least one vertical sampling region and at least one horizontal sampling region are determined; each vertical sampling region corresponds to the vertical dimension of the target image, and each horizontal sampling region corresponds to the horizontal dimension of the target image.

[0212] Downsampling is performed in at least one vertical sampling region to determine at least one first pixel value;

[0213] Downsampling is performed in at least one horizontal sampling region to determine at least one second pixel value;

[0214] The processed image is determined based on at least one first pixel value and at least one second pixel value.

[0215] In one possible implementation, the first processing module 112 is specifically used for:

[0216] Determine at least one first sampling point in at least one vertical sampling region;

[0217] For each of the first sampling points in at least one first sampling point, perform the following operation:

[0218] Map the first target position of the first sampling point in the target image to the original image to determine the first original position of the first sampling point in the original image;

[0219] The pixel value at the first original position in the original image is determined as the pixel value corresponding to the first sampling point;

[0220] Wherein, at least one first pixel value includes the pixel value corresponding to at least one first sampling point.

[0221] In one possible implementation, the first processing module 112 is specifically used for:

[0222] At least one second sampling point is determined in at least one horizontal sampling region; the at least one second sampling point is the starting point of each of the at least one horizontal sampling regions.

[0223] For each sampling point in at least one second sampling point, perform the following operation:

[0224] Based on the position of the second sampling point in the target image, determine the position of the second target corresponding to the second sampling point;

[0225] Map the second target location onto the original image to determine at least one second original location of the second sampling point in the original image;

[0226] Interpolate the pixel values ​​at at least one second original location to determine the pixel value corresponding to the second sampling point;

[0227] Among them, at least one second pixel value includes the pixel value corresponding to the second sampling point in each horizontal sampling region.

[0228] In one possible implementation, the second processing module 114 is specifically used for:

[0229] Divide the target image into at least one region;

[0230] For each region in at least one region, perform the following operations:

[0231] Based on the region and the original image, determine the output state of the region; the output state can be one of the following: zoomed-in state, zoomed-out state, or original state.

[0232] When the output state is either reduced or original, the region corresponding to the region in the processed image is determined as the target region.

[0233] When the output state is magnified, the target region is determined based on the region corresponding to the region in the processed image;

[0234] The output image includes at least one target region.

[0235] In one possible implementation, the second processing module 114 is specifically used for:

[0236] Map the region to the original image to determine the original region corresponding to the region in the original image;

[0237] Determine the ratio between the height of the determined region and the height of the original region;

[0238] If the ratio is less than 1, the output state is determined to be amplified.

[0239] When the ratio is equal to 1, the output state is determined to be the original state;

[0240] If the ratio is greater than 1, the output state is determined to be the reduced state.

[0241] In one possible implementation, the second processing module 114 is specifically used for:

[0242] Based on the region corresponding to the region in the processed image, at least one pixel of the region is determined;

[0243] Interpolate at least one pixel to determine at least one supplementary pixel;

[0244] The target region is determined based on at least one pixel and at least one supplementary pixel.

[0245] The image processing apparatus 110 provided in this application embodiment can execute the technical solution of the image processing method in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be described again here.

[0246] Figure 12 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 120 includes:

[0247] At least one processor 122; and

[0248] Memory 121 is communicatively connected to at least one processor 112; wherein,

[0249] The memory 121 stores instructions that can be executed by at least one processor 122, which, when executed by at least one processor 122, causes the at least one processor 122 to perform the image processing method involved in the above method embodiments.

[0250] Optionally, the processor mentioned above can be a central processing unit (CPU), or it can be a GPU, other general-purpose processors, digital signal processors (DSPs), or application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in the processor.

[0251] The electronic device 120 provided in this application embodiment can execute the image processing method involved in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be described again here.

[0252] This application provides a non-transitory computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, which are used to cause the computer to perform the image processing method involved in the above method embodiments.

[0253] This application provides a computer program product, including a computer program that, when executed by an electronic device, implements the image processing method involved in the above method embodiments.

[0254] This application provides a chip, which includes at least one processor. The processor is used to run program instructions to perform the image processing method involved in the above method embodiments.

[0255] This application provides a chip module on which a computer program is stored. When the computer program is executed by the chip module, it implements the image processing method involved in the above method embodiments.

[0256] All or part of the steps in the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a readable memory. When the program is executed, it performs the steps of the above method embodiments; and the aforementioned memory (storage medium) includes: read-only memory (ROM), RAM, flash memory, hard disk, solid-state drive, magnetic tape, floppy disk, optical disk, and any combination thereof.

[0257] This application describes embodiments of methods, apparatus (systems), and computer program products according to embodiments of this application with reference to flowchart illustrations and / or block diagrams. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, special-purpose computer, embedded processor, or other programmable terminal device to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0258] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable terminal device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0259] These computer program instructions can also be loaded onto a computer or other programmable terminal device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0260] Obviously, those skilled in the art can make various modifications and variations to the embodiments of this application without departing from the spirit and scope of this application. Therefore, if these modifications and variations to the embodiments of this application fall within the scope of the claims of this application and their equivalents, this application also intends to include these modifications and variations.

[0261] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0262] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. An image processing method, characterized in that, Applied to a projection device, the method includes: Determine the original image and the target image; the original image is the image to be corrected received by the projection device, and the target image is used to indicate the shape of the image to be output by the projection device; The original image is downsampled based on the target image to obtain the processed image; The processed image is stored in the storage area of ​​the projection device; The processed image is read from the storage area and scaled to obtain an output image; the output image is the image output to the screen by the projection device.

2. The method according to claim 1, characterized in that, The step of downsampling the original image based on the target image to obtain the processed image includes: In the target image, at least one vertical sampling region and at least one horizontal sampling region are determined; each vertical sampling region corresponds to the vertical dimension of the target image, and each horizontal sampling region corresponds to the horizontal dimension of the target image. Downsampling is performed in the at least one vertical sampling region to determine at least one first pixel value; Downsampling is performed in the at least one horizontal sampling region to determine at least one second pixel value; The processed image is determined based on the at least one first pixel value and the at least one second pixel value.

3. The method according to claim 2, characterized in that, The downsampling process performed in the at least one vertical sampling region to determine at least one first pixel value includes: Determine at least one first sampling point in the at least one vertical sampling region; For each of the at least one first sampling points, the following operations are performed: Map the first target position of the first sampling point in the target image to the original image to determine the first original position of the first sampling point in the original image; The pixel value at the first original position in the original image is determined as the pixel value corresponding to the first sampling point; The at least one first pixel value includes the pixel value corresponding to each of the at least one first sampling point.

4. The method according to claim 2 or 3, characterized in that, The step of downsampling in the at least one horizontal sampling region to determine at least one second pixel value includes: At least one second sampling point is determined within the at least one horizontal sampling region; the at least one second sampling point is the starting point of each of the at least one horizontal sampling regions. For each of the at least one second sampling points, the following operations are performed: Based on the position of the second sampling point in the target image, determine the second target position corresponding to the second sampling point; Map the second target location onto the original image to determine at least one second original location of the second sampling point in the original image; Interpolate the pixel values ​​at at least one second original position to determine the pixel value corresponding to the second sampling point; Wherein, the at least one second pixel value includes the pixel value corresponding to the second sampling point in each horizontal sampling region.

5. The method according to any one of claims 1-3, characterized in that, The scaling process of the processed image to obtain the output image includes: The target image is divided into at least one region; For each of the at least one region, perform the following operations: Based on the region and the original image, the output state of the region is determined; the output state is one of a zoomed-in state, a zoomed-out state, and an original state. When the output state is the reduced state or the original state, the region corresponding to the region in the processed image is determined as the target region; When the output state is the magnified state, the target region is determined based on the region corresponding to the region in the processed image. The output image includes at least one target region.

6. The method according to claim 5, characterized in that, Determining the output state of the region based on the region and the original image includes: Map the region to the original image to determine the original region corresponding to the region in the original image; Determine the ratio between the height of the region and the height of the original region; If the ratio is less than 1, the output state is determined to be the amplified state; When the ratio is equal to 1, the output state is determined to be the original state; If the ratio is greater than 1, the output state is determined to be the reduced state.

7. The method according to claim 5, characterized in that, Determining the target region based on the region corresponding to the region in the processed image includes: Based on the region corresponding to the region in the processed image, at least one pixel of the region is determined; Interpolation is performed on the at least one pixel to determine at least one supplementary pixel; The target region is determined based on the at least one pixel and the at least one supplementary pixel.

8. An image processing apparatus, characterized in that, The device includes: A determination module is used to determine the original image and the target image; the original image is the image to be corrected received by the projection device, and the target image is used to indicate the shape of the image to be output by the projection device; The first processing module is used to downsample the original image based on the target image to obtain the processed image; A storage module is used to store the processed image in the storage area of ​​the projection device; The second processing module is used to read the processed image from the storage area and scale the processed image to obtain an output image; the output image is the image output to the screen by the projection device.

9. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to cause the at least one processor to perform the method of any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, in, The computer instructions are used to cause the computer to perform the method according to any one of claims 1 to 7.

Images