CPU and GPU combined real-time image smoothing method and system, medium and program product

Abstract

The invention relates to a real-time image smoothing method and system based on combination of a CPU and a GPU, a medium and a program product, in the method, the CPU is controlled to carry out preprocessing operation on a to-be-projected image to obtain a preprocessed image, and the preprocessing operation comprises denoising, brightness adjustment and contrast adjustment; controlling the CPU to perform edge detection on the preprocessed image by using a preset edge detection algorithm, and marking an edge region in the preprocessed image; controlling the CPU to perform anti-aliasing operation on the edge area to obtain a preliminary projection image; establishing a transmission channel between the CPU and the GPU, and enabling the CPU to send the preliminary projection image to the GPU; controlling the GPU to perform keystone correction operation on the preliminary projection image to obtain a corrected projection image; adjusting and correcting the projection image on the GPU by using a preset anti-aliasing algorithm to obtain a smooth projection image; and controlling the GPU to enhance image details of the smooth projection image to obtain a target projection image. The method and the device are used for improving image smoothness, further enhancing image quality and improving and optimizing image processing efficiency.

Description

Technical Field

[0001] The present application belongs to the field of image processing, and in particular, relates to a real-time image smoothing method, system, medium and program product combining a CPU and a GPU. Background Art

[0002] With the continuous development of projection technology, people have higher and higher requirements for the quality of projected images. Early projected images often had problems such as distortion and blur, which seriously affected the viewing experience. Therefore, how to improve the clarity and smoothness of projected images has become an important issue.

[0003] In the related art, the keystone correction technology widely used in projectors mainly relies on GPU to achieve geometric transformation and correction of images through OpenGL API. However, due to the uneven performance of GPUs in different projectors, the edges of the corrected images often appear jagged, which reduces the quality of the images projected by the projector. Summary of the invention

[0004] The present application provides a real-time image smoothing method, system, medium and program product combining a CPU and a GPU, which are used to improve image smoothness, thereby enhancing image quality and improving the efficiency of optimizing image processing.

[0005] In a first aspect, the present application provides a real-time image smoothing method combining a CPU with a GPU, which controls the CPU to perform a preprocessing operation on an image to be projected to obtain a preprocessed image, wherein the preprocessing operation includes denoising, brightness adjustment, and contrast adjustment; Controlling the CPU to perform edge detection on the pre-processed image using a preset edge detection algorithm, and marking edge areas in the pre-processed image; Control the CPU to perform anti-aliasing operation on the edge area to obtain a preliminary projection image; Establish a transmission channel between the CPU and the GPU so that the CPU sends the preliminary projection image to the GPU; Control the GPU to perform a trapezoidal correction operation on the preliminary projection image to obtain a corrected projection image; Use the preset anti-aliasing algorithm on the GPU to adjust and correct the projected image to obtain a smooth projected image; The GPU is controlled to enhance the image details of the smooth projection image to obtain a target projection image.

[0006] By adopting the above technical solution, first, the CPU is responsible for image preprocessing, edge detection and preliminary anti-aliasing operations, which are suitable for the serial processing capabilities of the CPU. Preprocessing operations (denoising, brightness adjustment and contrast adjustment) improve the basic quality of the image and lay the foundation for subsequent processing. Edge detection provides a precise target area for anti-aliasing operations and improves processing efficiency. The preliminary anti-aliasing operation is performed on the CPU, which can reduce the burden on the GPU. Subsequently, the processed image is transferred to the GPU to make full use of the parallel computing power of the GPU for more complex image processing tasks. Operations such as trapezoidal correction, advanced anti-aliasing and image detail enhancement performed by the GPU can give full play to the advantages of the GPU in graphics processing and achieve higher quality image output. This way of CPU and GPU working together not only improves the overall processing efficiency, but also achieves better image quality. Especially in real-time application scenarios, such as projection display, while ensuring image quality, the processing delay is significantly reduced to provide a smoother visual experience. The effect of improving image smoothness, enhancing image quality, and improving the efficiency of optimizing image processing is achieved.

[0007] In combination with some embodiments of the first aspect, in some embodiments, controlling the CPU to perform edge detection on the preprocessed image using a preset edge detection algorithm to mark an edge area in the preprocessed image specifically includes: Controlling the CPU to grayscale the preprocessed image to obtain a grayscale image; Applying Gaussian filter to the grayscale image results in a blurred image; Perform gradient calculation on the blurred image to obtain a gradient image; Apply thresholding to the gradient image to obtain a binary edge image; Perform edge detection on the binary edge image and mark the edge area in the preprocessed image.

[0008] By adopting the above technical solution, first, the pre-processed image is grayed. This step can convert the color image into a single-channel image, greatly reducing the amount of data for subsequent processing and improving the processing speed. Secondly, a Gaussian filter is applied to obtain a blurred image. This step can effectively remove the noise in the image, making the edge detection more stable and reliable. Next, the gradient calculation is performed on the blurred image, which can highlight the edge information in the image and provide a good foundation for subsequent edge detection. Then, a binary edge image is obtained by thresholding. This step can present the edge information more clearly and facilitate subsequent precise positioning. Finally, edge detection is performed on the binary edge image to mark the edge area in the pre-processed image. This multi-step edge detection method can not only accurately identify the edges in the image, but also effectively suppress the influence of noise and improve the robustness of edge detection. The high-quality edge detection result provides a precise target area for the subsequent anti-aliasing process, thereby improving the effect and efficiency of the anti-aliasing process.

[0009] In combination with some embodiments of the first aspect, in some embodiments, controlling the CPU to perform an anti-aliasing operation on the edge area to obtain a preliminary projection image specifically includes: identifying a number of pixels to be processed in the edge region; Perform sub-pixel sampling on a number of pixels to be processed to obtain sampling results; According to the sampling result, the CPU is controlled to perform color mixing processing on a number of pixels to be processed to obtain a preliminary projection image.

[0010] By adopting the above technical solution, firstly, by identifying the pixels to be processed in the edge area, the method can accurately locate the area that needs to be anti-aliased, avoiding unnecessary processing of the entire image, thereby improving the processing efficiency. Secondly, sub-pixel sampling is performed on the pixels to be processed. This technology can obtain more refined image information based on the original pixel resolution, and provide more accurate data support for subsequent color mixing processing. Finally, color mixing is performed on the pixels to be processed according to the sampling results. This step can effectively smooth the edges, reduce the jagged phenomenon, and improve the visual quality of the image. This CPU-based anti-aliasing method has high flexibility and controllability, and can be fine-tuned according to different image features and processing requirements. At the same time, since only the pixels to be processed in the edge area are processed, the amount of calculation is greatly reduced, so that the CPU can efficiently complete the preliminary anti-aliasing operation. This not only improves the overall processing efficiency, but also reduces the burden on subsequent GPU processing, and realizes better task allocation and collaborative work between the CPU and the GPU.

[0011] In conjunction with some embodiments of the first aspect, in some embodiments, identifying a number of pixels to be processed in the edge area specifically includes: Traverse every pixel in the edge area; In the case where it is determined that there is a non-edge pixel among the neighboring pixels of the existing pixel, the pixel is determined as a pixel to be processed.

[0012] By adopting the above technical solution, firstly, by traversing every pixel in the edge area, the method ensures that no potential pixels to be processed are missed, and the integrity of the anti-aliasing process is guaranteed. Secondly, by judging whether there are non-edge pixels in the adjacent pixels of the pixel, the method can accurately locate the junction of the edge and non-edge areas. The pixels at these junctions are the main cause of the jagged phenomenon of the image and are also the targets that need to be focused on. By identifying the pixels to be processed in this way, the anti-aliasing process can be accurately applied to the most needed areas, avoiding unnecessary operations on pixels that do not need to be processed. This not only improves the efficiency of the anti-aliasing process, but also maximizes the clarity and details of the original image. At the same time, this identification method also provides precise targets for subsequent sub-pixel sampling and color mixing processing, so that these operations can be more targeted, thereby improving image quality while optimizing the utilization of computing resources. In general, this method of identifying pixels to be processed provides an efficient and accurate starting point for the entire anti-aliasing process, laying the foundation for achieving high-quality image output.

[0013] In conjunction with some embodiments of the first aspect, in some embodiments, before controlling the CPU to perform a preprocessing operation on the image to be projected to obtain the preprocessed image, the method further includes: Get the native resolution and projected image of the projection device; Adaptively adjust the resolution of the projected image to make the resolution of the projected image equal to the native resolution; The projected image is adjusted using a bicubic interpolation algorithm; The projection image is converted into a color space according to the color space of the projection device to obtain an image to be projected.

[0014] By adopting the above technical solution, firstly, by obtaining the native resolution of the projection device, the method can be optimized for a specific hardware environment to ensure that the output image matches the device performance. Adaptive adjustment of the resolution of the projected image to make it consistent with the native resolution of the projection device can avoid image distortion or blurring caused by resolution mismatch, thereby improving the image quality of the final output. Using the bicubic interpolation algorithm for image adjustment is a key step. Compared with simple linear interpolation, bicubic interpolation can maintain better image quality when enlarging or reducing the image, reducing jagged and blurring effects. This has a positive impact on subsequent edge detection and anti-aliasing processing because it provides a higher quality initial image. Finally, color space conversion is performed according to the color space of the projection device to ensure the accurate presentation of image colors and avoid color distortion caused by color space mismatch. This series of preprocessing steps not only optimizes image quality, but also reduces the computational burden in subsequent processing by making necessary adjustments in advance, thereby improving overall processing efficiency. At the same time, the implementation of these steps also provides high-quality, standardized input for subsequent CPU and GPU processing, improving the consistency and reliability of the entire image processing process.

[0015] In combination with some embodiments of the first aspect, in some embodiments, after establishing the transmission channel between the CPU and the GPU, the method further includes: Processing the preliminary projection image using a data compression algorithm based on run-length coding to obtain compressed image data; Divide the image data into a number of data blocks; Several data blocks are sent to the GPU through a transmission channel according to an asynchronous transmission mechanism.

[0016] By adopting the above technical solution, firstly, the preliminary projection image is processed using a data compression algorithm based on run-length coding. This step can significantly reduce the amount of data that needs to be transmitted. Run-length coding is particularly suitable for processing images with large areas of the same color, such as the background area in the projection image, and can achieve efficient lossless compression. The reduction in the amount of data can not only speed up the transmission speed, but also reduce the bandwidth occupancy and energy consumption during the transmission process. Secondly, the compressed image data is divided into several data blocks. This block strategy creates conditions for parallel transmission and processing. Small data blocks are easier to manage, and the transmission channel can be used more flexibly to improve the concurrency of transmission. Finally, the data blocks are sent to the GPU using an asynchronous transmission mechanism. This method allows the CPU to continue to perform other tasks while sending data without waiting for each data block to be transmitted. Asynchronous transmission makes full use of the parallel processing capabilities of modern computer systems, significantly improving the utilization of the CPU and the responsiveness of the overall system.

[0017] In combination with some embodiments of the first aspect, in some embodiments, after controlling the GPU to enhance the image details of the smoothed projection image to obtain the target projection image, the method further includes: Calculate the number of frames that need to be inserted based on the difference between the current frame rate and the target frame rate; According to the number of frames, a motion estimation algorithm is applied to calculate the motion vector between adjacent frames; Generate intermediate frames based on motion vectors; The intermediate frames are merged with the original frames in chronological order to obtain the final output sequence with increased frame rate.

[0018] By adopting the above technical solution, firstly, the number of frames to be inserted is determined by calculating the difference between the current frame rate and the target frame rate. This adaptive method can be dynamically adjusted according to the actual situation, avoiding the problem of unnecessary calculation or poor effect caused by fixed frame rate interpolation. Secondly, the motion vector between adjacent frames is calculated using the motion estimation algorithm. This step can accurately capture the motion information of objects in the image and provide key data for the subsequent generation of intermediate frames. The accuracy of motion estimation directly affects the quality of the interpolated frame, so this step is crucial to maintain the coherence and naturalness of the motion. Then, the intermediate frame is generated according to the motion vector. This motion-based frame interpolation method can better maintain the smoothness of the motion than simple linear interpolation, and reduce the jitter and unnatural phenomenon of the picture. Finally, the generated intermediate frame is merged with the original frame in chronological order to obtain the final output sequence after the frame rate is increased. This frame rate increase method not only increases the smoothness of the video, but also effectively reduces the motion blur and stuttering in dynamic scenes, greatly improving the viewing experience. Especially in the case of high-speed motion or fast scene switching, this method can significantly improve the visual effect and make the picture more coherent and natural.

[0019] In a second aspect, an embodiment of the present application provides a real-time smooth image system combining a CPU and a GPU, wherein the real-time smooth image system combining a CPU and a GPU comprises: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code comprises computer instructions, and the one or more processors call the computer instructions so that the system executes the method described in the first aspect and any possible implementation method of the first aspect.

[0020] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, comprising instructions, which, when executed on a system, causes the system to execute the method described in the first aspect and any possible implementation of the first aspect.

[0021] In a fourth aspect, an embodiment of the present application provides a computer program product, characterized in that when the computer program product runs on a system, the system executes the method described in any possible implementation manner in the first aspect.

[0022] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. The present application provides a real-time image smoothing method combining CPU and GPU. First, the CPU is responsible for image preprocessing, edge detection and preliminary anti-aliasing operations, which are suitable for the serial processing capabilities of the CPU. Preprocessing operations (denoising, brightness adjustment and contrast adjustment) improve the basic quality of the image and lay the foundation for subsequent processing. Edge detection provides a precise target area for anti-aliasing operations and improves processing efficiency. The preliminary anti-aliasing operation is performed on the CPU, which can reduce the burden on the GPU. Subsequently, the processed image is transferred to the GPU to make full use of the parallel computing power of the GPU for more complex image processing tasks. Operations such as trapezoidal correction, advanced anti-aliasing and image detail enhancement performed by the GPU can give full play to the advantages of the GPU in graphics processing and achieve higher quality image output. This way of working together between the CPU and GPU not only improves the overall processing efficiency, but also achieves better image quality. Especially in real-time application scenarios, such as projection display, while ensuring image quality, the processing delay is significantly reduced to provide a smoother visual experience. The effect of improving image smoothness, enhancing image quality and improving the efficiency of optimizing image processing is achieved.

[0023] 2. The present application provides a real-time image smoothing method combining a CPU and a GPU. By traversing every pixel in the edge area, the method ensures that no potential pixels to be processed are missed, and the integrity of the anti-aliasing process is guaranteed. Secondly, by determining whether there are non-edge pixels in the adjacent pixels of the pixel, the method can accurately locate the junction of the edge and non-edge areas. The pixels at these junctions are the main cause of the jagged phenomenon of the image and are also the targets that need to be focused on. By identifying the pixels to be processed in this way, the anti-aliasing process can be accurately applied to the most needed areas, avoiding unnecessary operations on pixels that do not need to be processed. This not only improves the efficiency of the anti-aliasing process, but also maximizes the clarity and details of the original image. At the same time, this recognition method also provides precise targets for subsequent sub-pixel sampling and color mixing processing, so that these operations can be more targeted, thereby improving image quality while optimizing the utilization of computing resources. In general, this method of identifying pixels to be processed provides an efficient and accurate starting point for the entire anti-aliasing process, laying the foundation for achieving high-quality image output.

[0024] 3. The present application provides a real-time image smoothing method combining CPU and GPU. The number of frames to be inserted is determined by calculating the difference between the current frame rate and the target frame rate. This adaptive method can be dynamically adjusted according to the actual situation, avoiding the problem of unnecessary calculation or poor effect caused by fixed frame rate interpolation. Secondly, the motion vector between adjacent frames is calculated using a motion estimation algorithm. This step can accurately capture the motion information of objects in the image and provide key data for the subsequent generation of intermediate frames. The accuracy of motion estimation directly affects the quality of the interpolated frame, so this step is crucial to maintaining the coherence and naturalness of the motion. Then, the intermediate frame is generated according to the motion vector. This motion-based frame interpolation method can better maintain the smoothness of the motion than simple linear interpolation, and reduce the jitter and unnatural phenomenon of the picture. Finally, the generated intermediate frame is merged with the original frame in chronological order to obtain the final output sequence after the frame rate is increased. This frame rate increase method not only increases the smoothness of the video, but also effectively reduces motion blur and stuttering in dynamic scenes, greatly improving the viewing experience. Especially in the case of high-speed motion or fast scene switching, this method can significantly improve the visual effect and make the picture more coherent and natural. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a flow chart of a method for real-time image smoothing using a CPU and a GPU in an embodiment of the present application.

[0026] Figure 2 This is another flowchart of a real-time image smoothing method combining a CPU and a GPU in an embodiment of the present application.

[0027] Figure 3 It is a schematic diagram of the physical device structure of a real-time smooth image system combining a CPU and a GPU provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification and appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear indication to the contrary in the context. It should also be understood that the term "and / or" used in the present application refers to any or all possible combinations comprising one or more listed items.

[0029] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0030] In modern business and education environments, projection technology has become an indispensable visual communication tool. From large conference rooms to small classrooms, from corporate presentations to academic lectures, projectors are used everywhere. However, with the continuous improvement of content quality and the growing expectations of audiences, the limitations of traditional projection technology are gradually revealed.

[0031] The same is true in the field of education. For example, in a physics laboratory at a famous university, a professor is demonstrating a sophisticated particle collision simulation to students. This simulation needs to show extremely subtle particle trajectories and energy changes. However, due to the distortion and blurring of the projected image, many key details become difficult to discern. Students have to frequently walk to the screen to observe carefully, which greatly reduces classroom efficiency and affects the learning experience.

[0032] Another example is that in an architectural design company, designers are presenting a new urban planning project to clients. This project contains a large number of detailed 3D models and complex structural designs. However, when these highly detailed images are presented through a projector, many tiny but important design elements become blurred. It is difficult for clients to accurately evaluate the pros and cons of the design, which may lead to delays or misunderstandings in project approval.

[0033] These scenes clearly show that traditional projection technology faces severe challenges in processing high-quality, high-detail images. Especially in applications that require accurate display of text, graphic edges or subtle structures, the decline in image quality not only affects the visual experience, but is also likely to cause distortion and misunderstanding of information transmission. In order to solve the above technical problems, this application provides a real-time image smoothing method combining a CPU with a GPU, which is used to improve image smoothness, thereby enhancing image quality and improving the efficiency of optimizing image processing.

[0034] Combine the following Figure 1 , a real-time image smoothing method combining a CPU and a GPU in an embodiment of the present application is described: See also Figure 1 , which is a flow chart of a real-time image smoothing method combining a CPU and a GPU in an embodiment of the present application.

[0035] S101, controlling the CPU to perform a preprocessing operation on the image to be projected to obtain a preprocessed image; Before this step, the system also needs to obtain the native resolution and projected image of the projection device; Adaptively adjust the resolution of the projected image to make the resolution of the projected image equal to the native resolution; The projected image is adjusted using a bicubic interpolation algorithm; The projection image is converted into a color space according to the color space of the projection device to obtain an image to be projected.

[0036] After obtaining the image to be projected, the system controls the CPU to perform preprocessing operations on the image to be projected to obtain a preprocessed image. The preprocessing operations include denoising, brightness adjustment, and contrast adjustment.

[0037] In the presentation environment of a modern architectural design company, the system first obtains the native resolution of the projection device, such as 1920x1080. Suppose the designer wants to show a 3D rendering of a city plan with a 4K resolution (3840x2160). The system first uses a bicubic interpolation algorithm to adjust the image resolution to 1920x1080 to match the native resolution of the projector. This step can maintain the clarity of the image while avoiding distortion caused by resolution mismatch.

[0038] Next, the system converts the image to the color space of the projection device (such as sRGB). This ensures that the projected colors accurately reflect the original design intent, which is especially important for the materials and light and shadow effects in architectural renderings.

[0039] The CPU then performs a series of preprocessing operations: Denoising: Use the non-local means denoising algorithm to remove subtle noise that may be introduced by image compression or the rendering process. This is important to maintain the smoothness of the building surface.

[0040] Brightness Adjustment: Increases the overall brightness of the image by 5% to adapt to the lighting conditions of the projection environment and ensures that the projected image does not appear too dark.

[0041] Contrast Adjustment: Slightly increase contrast (10% increase) to make building outlines and details stand out more.

[0042] These pre-processing operations can significantly improve the initial quality of the image and lay the foundation for subsequent fine processing. Through this step, the subtle defects that may have been caused by compression or rendering are corrected, and the overall image is clearer and richer in layers.

[0043] S102, controlling the CPU to perform edge detection on the preprocessed image using a preset edge detection algorithm, and marking edge areas in the preprocessed image; The system controls the CPU to perform edge detection on the pre-processed image using a preset edge detection algorithm, and marks the edge area in the pre-processed image. Specifically: the system controls the CPU to perform grayscale processing on the pre-processed image to obtain a grayscale image; Applying Gaussian filter to the grayscale image results in a blurred image; Perform gradient calculation on the blurred image to obtain a gradient image; Apply thresholding to the gradient image to obtain a binary edge image; Perform edge detection on the binary edge image and mark the edge area in the preprocessed image.

[0044] In this step, the system uses the CPU to perform edge detection on the pre-processed 3D rendering of the urban plan, which is essential for accurately presenting the details of building outlines, street layouts, and landscaping designs.

[0045] First, the system converts the image into a grayscale image. Suppose the exterior wall of a modern-style building in the original image is light gray (RGB: 200, 200, 200), which contrasts with the dark background (RGB: 50, 50, 50). After conversion to grayscale, the grayscale value of the exterior wall is about 200, and the background is about 50.

[0046] Next, the system applies a Gaussian filter to blur the grayscale image. A 5x5 Gaussian kernel with σ=1.4 is used. This step reduces the noise in the image and makes edge detection more accurate. After Gaussian filtering, the grayscale value of the building's exterior wall may become 195, and the background may become 55, with a smoother transition.

[0047] The system then calculates the gradient of the image. The Sobel operator is used to calculate the gradient in the x- and y-directions and merge them to get a gradient magnitude image. The gradient value may be as high as 140 at the building outline and close to 0 in smooth areas.

[0048] Next, the system thresholds the gradient image. Suppose the threshold is set to 50, pixels below this value are set to 0 (non-edge), and pixels above this value are set to 255 (edge). In this way, important features such as the main outline of the building and the edge of the window are marked as edges.

[0049] Finally, the system thins the binary edge image to obtain an accurate single-pixel width edge. This step uses the thinning algorithm in morphological operations to ensure that the edge accurately represents the outline and details of the building.

[0050] Through this series of operations, the system successfully marked the key edges in the urban planning map, including building outlines, road boundaries, landscape elements, etc. This provides precise target areas for subsequent anti-aliasing and image enhancement operations, ensuring that these key details are clearly presented in the final projection.

[0051] S103, controlling the CPU to perform anti-aliasing operation on the edge area to obtain a preliminary projection image; The system controls the CPU to perform anti-aliasing operations on the edge area to obtain a preliminary projection image, specifically: identifying a number of pixels to be processed in the edge area; Sub-pixel sampling is performed on a number of pixels to be processed to obtain sampling results. Specifically, each pixel in the edge area is traversed; In the case where it is determined that there is a non-edge pixel among the neighboring pixels of the existing pixel, the pixel is determined as a pixel to be processed; According to the sampling result, the CPU is controlled to perform color mixing processing on a number of pixels to be processed to obtain a preliminary projection image.

[0052] In this step, the system performs anti-aliasing on the previously detected edge areas to eliminate jagged effects and make the edges of the image smoother. This is especially important for rendering fine details of architectural designs.

[0053] First, the system identifies the pixels to be processed in the edge area. Specifically, the system traverses each pixel marked as an edge and checks its 8 neighboring pixels. If there are non-edge pixels among the neighboring pixels of an edge pixel, it is marked as a pixel to be processed. For example, on the outline of a high-rise building, the system may identify about 10,000 pixels to be processed.

[0054] Next, the system performs sub-pixel sampling on these pixels to be processed. Using a 4x4 supersampling grid, each pixel is divided into 16 sub-pixels. For each pixel to be processed, the system calculates how many of these 16 sub-pixels fall inside the edge. Assume that at a pixel position of the building outline, 10 of the 16 sub-pixels fall inside the building (value 1) and 6 fall in the background (value 0).

[0055] Then, the system performs color mixing on the processed pixel based on the sampling result. Continuing with the above example, if the building color is dark gray (RGB: 80, 80, 80) and the background is light blue (RGB: 200, 220, 240), the final pixel color will be: R = (80 * 10 + 200 * 6) / 16 ≈ 125 G = (80 * 10 + 220 * 6) / 16 ≈ 132 B = (80 * 10 + 240 * 6) / 16 ≈ 140 This blended color (RGB: 125, 132, 140) will make the building outlines look smoother and less jagged.

[0056] The system repeats this process for all pixels to be processed. For example, for a complex urban plan, there may be hundreds of thousands of edge pixels to be processed. This process is applied not only to building outlines, but also to road edges, outlines of landscape elements, etc.

[0057] Through this fine anti-aliasing process, the system obtains a preliminary projection image. In this image, the outline of the building is smoother, the curves are more natural, and the details of small buildings and landscape elements are well preserved. This greatly improves the visual quality of the image, allowing designers to more accurately display their ideas and customers to more clearly understand the design intent.

[0058] S104, establishing a transmission channel between the CPU and the GPU; The system establishes a transmission channel between the CPU and the GPU, so that the CPU sends the preliminary projection image to the GPU. After that, the preliminary projection image is processed using a data compression algorithm based on run-length coding to obtain compressed image data; Divide the image data into a number of data blocks; Several data blocks are sent to the GPU through a transmission channel according to an asynchronous transmission mechanism.

[0059] In this step, the system establishes an efficient transmission channel between the CPU and GPU to prepare for subsequent GPU processing. This step is crucial to achieve real-time processing and smooth projection.

[0060] First, the system uses a data compression algorithm based on run-length encoding (RLE) to process the preliminary projection image. For example, for a row of pixel data in a city planning map: Original data: 255 255 255 255 255 255 200 200 200 200 150 150 150 255 255255 After compression: (255, 6) (200, 4) (150, 3) (255, 3) This compression method is particularly suitable for processing large areas of the same color, such as building surfaces, sky or water. For a 1920x1080 image, compression may reduce the amount of data from about 6.2MB to about 2.5MB.

[0061] Next, the system divides the compressed image data into multiple data blocks. Assume that the system divides the data into 64 blocks, each of which is about 40KB in size. This block strategy allows the system to achieve parallel transmission and processing.

[0062] Then, the system established a shared memory high-speed transmission channel. Shared Memory: The CPU and GPU usually share physical memory (usually DDR SDRAM), which means they can exchange data through memory. The GPU writes the rendered image data to the shared memory, and the CPU can directly read the data for processing, and vice versa. The system also implements a ring buffer mechanism. When the GPU finishes processing a block of data, the CPU will immediately send the next block of data to maintain the continuity of the data flow. This mechanism ensures that the GPU always has data to process, maximizing processing efficiency.

[0063] In addition, the system also implements dynamic priority scheduling. For key areas in the image (such as major buildings or important landscape elements), the system will give priority to transmitting and processing data blocks in these areas. For example, if there is a landmark building in the center of the city planning map, the data block containing this building will be given the highest priority.

[0064] Through this efficient transmission mechanism, the system can transfer the entire image data to the GPU in a very short time (usually less than 16.7ms, i.e. one frame time at a 60Hz refresh rate). This provides a solid foundation for subsequent real-time GPU processing and ensures the smoothness and real-time performance of the projection process.

[0065] S105, controlling the GPU to perform a trapezoidal correction operation on the preliminary projection image to obtain a corrected projection image; In this step, the system uses the parallel computing power of the GPU to perform keystone correction on the preliminary projected image to compensate for the image deformation caused by the projection angle. This is crucial to ensure that the urban planning map can maintain the correct proportion and shape in various projection environments.

[0066] First, the system obtains the geometric information of the projection surface through the camera or user input. Suppose in the conference room of an architectural design company, the projector is placed at an angle, resulting in trapezoidal distortion of the image. The system detects the coordinates of the four corners of the projected image as: Top left corner: (100, 50) Top right corner: (1820, 100) Lower left corner: (50, 1000) Lower right corner: (1870, 1030) This shows that the top of the image is narrower than the bottom, and the right side is taller than the left side, typical keystoning.

[0067] Next, the system calculates the perspective transformation matrix from this deformed quadrilateral to the ideal rectangle. The system then transforms the image using vertex and fragment shaders on the GPU. The vertex shader is responsible for adjusting the geometry of the image, while the fragment shader is responsible for interpolation and color mapping.

[0068] This way, the system can efficiently perform keystone correction on the GPU, typically within 1-2 milliseconds for a 1920x1080 image.

[0069] After the correction, the buildings and roads in the urban planning map have restored their correct proportions and shapes. For example, the high-rise building that originally appeared tilted due to the projection angle has now become straight, and the circular square has returned to its perfect circle shape. This ensures that the designer's intentions can be accurately conveyed and the client can correctly understand the urban layout and building proportions.

[0070] S106, using a preset anti-aliasing algorithm on the GPU to adjust and correct the projected image to obtain a smooth projected image; In this step, the system further performs anti-aliasing on the GPU to obtain smoother visual effects. This is especially important for presenting fine details in architectural design, such as window outlines, decorative elements, etc.

[0071] The system uses multi-sampling anti-aliasing (MSAA) technology, which is an anti-aliasing method that is efficiently implemented on the GPU. Specifically, the system uses 8x MSAA, which means that each pixel is sampled 8 times.

[0072] First, the system sets up OpenGL's multi-sampling buffer, then creates a multi-sampling frame buffer object (FBO). During the rendering process, the system samples each pixel in the image 8 times. For example, for the edge pixels of a building in a city plan, the system may get the following sampling results: Sample 1: Building color (RGB: 180, 180, 180) Sample 2: Building color (RGB: 180, 180, 180) Sample 3: Background color (RGB: 220, 230, 240) Sample 4: Building color (RGB: 180, 180, 180) Sample 5: Building color (RGB: 180, 180, 180) Sample 6: Background color (RGB: 220, 230, 240) Sample 7: Building color (RGB: 180, 180, 180) Sample 8: Building color (RGB: 180, 180, 180) The system then calculates the average of these 8 samples: R = (180 * 6 + 220 * 2) / 8 = 190 G = (180 * 6 + 230 * 2) / 8 = 192.5 B = (180 * 6 + 240 * 2) / 8 = 195 Finally, the color of this edge pixel is set to (RGB: 190, 193, 195), and this transition color makes the edge of the building look smoother.

[0073] For a full 1920x1080 image, this process may require processing millions of pixels. However, due to the parallel processing capabilities of the GPU, the entire MSAA process can usually be completed in 2-3 milliseconds.

[0074] After processing is complete, the system resolves the image in the multisample buffer into a standard resolution frame buffer.

[0075] S107 , controlling the GPU to enhance the image details of the smoothed projection image to obtain a target projection image.

[0076] In this final step, the system uses the powerful computing power of the GPU to enhance the details of the smoothed projection image to obtain the final high-quality target projection image. This step is crucial for highlighting the fine features of the architectural design and enhancing the overall visual effect.

[0077] The system uses a super-resolution algorithm called ESPCN (Efficient Sub-Pixel Convolutional Neural Network) based on a convolutional neural network (CNN). This algorithm can effectively improve image resolution and enhance details while maintaining low computational complexity, making it suitable for real-time processing.

[0078] The above embodiments have the following beneficial effects: The CPU is responsible for image preprocessing, edge detection, and preliminary anti-aliasing operations, which are suitable for the serial processing capabilities of the CPU. Preprocessing operations (denoising, brightness adjustment, and contrast adjustment) improve the basic quality of the image and lay the foundation for subsequent processing. Edge detection provides a precise target area for anti-aliasing operations, improving processing efficiency. Preliminary anti-aliasing operations are performed on the CPU, which can reduce the burden on the GPU. Subsequently, the processed image is transferred to the GPU to fully utilize the parallel computing capabilities of the GPU for more complex image processing tasks. Operations such as keystone correction, advanced anti-aliasing, and image detail enhancement performed by the GPU can give full play to the advantages of the GPU in graphics processing and achieve higher quality image output. This way of CPU and GPU working together not only improves overall processing efficiency, but also achieves better image quality. Especially in real-time application scenarios, such as projection display, while ensuring image quality, it significantly reduces processing delays and provides a smoother visual experience. The effect of improving image smoothness, enhancing image quality, and improving the efficiency of optimizing image processing is achieved.

[0079] Combine the following Figure 2 , another real-time image smoothing method combining CPU and GPU in the embodiment of the present application is described: See also Figure 2 , is another flow chart of a real-time image smoothing method combining a CPU and a GPU in an embodiment of the present application.

[0080] S201, calculating the number of frames to be inserted according to the difference between the current frame rate and the target frame rate; In this step, the system first detects the actual frame rate of the current video stream, then compares it with the preset target frame rate and calculates the number of frames that need to be inserted. This process involves frame rate detection and basic arithmetic operations.

[0081] In specific implementation, the system can use a high-precision timer to measure the time interval between several consecutive frames to obtain the current actual frame rate. For example, if the system measures the total time of 10 consecutive frames to be 333 milliseconds, the current frame rate can be calculated as: 30 frames / second (10 frames / 0.333 seconds).

[0082] Next, the system compares the actual frame rate with the preset target frame rate. Assuming the target frame rate is set to 60 frames per second, the system needs to calculate the number of frames that need to be inserted between every two original frames. The calculation formula can be expressed as: Insertion frame number = (target frame rate / current frame rate) - 1.

[0083] In this example, the number of inserted frames = (60 / 30) - 1 = 1. This means that the system needs to insert a new frame between every two original frames to increase the frame rate from 30 fps to 60 fps.

[0084] When the system performs this step, it may use floating-point operations to handle possible non-integer results, and perform rounding or other rounding operations based on actual conditions. At the same time, the system also needs to take into account that the frame rate may change dynamically, so it will periodically recalculate this value to adapt to possible frame rate fluctuations.

[0085] S202, applying a motion estimation algorithm according to the number of frames to calculate a motion vector between adjacent frames; After determining the number of frames that need to be inserted, the system begins to execute the motion estimation algorithm to calculate the motion vectors between adjacent frames. This step is the most complex and computationally intensive part of the entire process, and usually requires the parallel computing power of the GPU to accelerate the processing.

[0086] The core idea of ​​the motion estimation algorithm is to divide the current frame into multiple small blocks (usually 8x8 or 16x16 pixel blocks), and then search for the best matching area in the next frame to obtain the motion direction and distance of each block, that is, the motion vector.

[0087] A commonly used algorithm is the Block Matching Algorithm (BMA). Taking a 16x16 block size as an example, the system selects a 16x16 block in the current frame and then searches for the most similar 16x16 region in the search window of the next frame (for example, the surrounding 32x32 region). Similarity is usually measured using the mean squared error (MSE) or the sum of absolute differences (SAD).

[0088] Suppose in a 16x16 block, the motion vector calculated by the system is (5, -3), which means that the block moves 5 pixels to the right and 3 pixels up in the next frame. The system will calculate such a motion vector for each block, and finally get a motion vector field that describes the motion of the entire picture.

[0089] To improve efficiency, the system may adopt a multi-resolution search strategy, first performing a rough search on a low-resolution image, and then a fine search on a high-resolution image. At the same time, the system may also use a predictive search to predict the possible motion direction of the current block based on the motion vectors of the surrounding blocks, thereby reducing the search range.

[0090] When implemented on a GPU, the system can utilize parallel computing frameworks such as CUDA or OpenCL to assign motion estimation tasks for different blocks to different GPU cores and perform them simultaneously, greatly improving the processing speed.

[0091] S203, generating an intermediate frame according to the motion vector; With the motion vector, the system can start generating intermediate frames. This process is usually called Motion Compensated Frame Interpolation (MCFI). The system will calculate the position and color value of each pixel at the intermediate moment based on the motion vector between two adjacent frames.

[0092] Specifically, suppose the system needs to insert an intermediate frame C between frame A and frame B. For each pixel in the intermediate frame C, the system calculates the color value of the pixel based on the motion vector of its corresponding position in the A and B frames.

[0093] For example, suppose the position of a pixel in frame A is (100, 100) and its corresponding motion vector is (10, 5), which means that the pixel moves to (110, 105) in frame B. Then in the intermediate frame C, the position of this pixel should be (105, 102.5), and its color value can be obtained by taking the weighted average of the pixel at position (100, 100) in frame A and the pixel at position (110, 105) in frame B.

[0094] The weights are usually calculated based on the temporal interpolation ratio. If the intermediate frame C is exactly halfway between A and B, then the weights might be 0.5 and 0.5. But if C is closer to a frame in A or B, the weights will be adjusted accordingly.

[0095] In actual implementation, the system needs to handle some special cases, such as occlusion (some pixels are blocked in the next frame) and exposure (some pixels appear in the next frame). For these cases, the system may need to use more complex interpolation strategies, such as bidirectional prediction, area filling and other techniques.

[0096] In addition, to improve image quality, the system may apply some post-processing techniques such as deblocking filtering, sharpening, etc. These operations can reduce blur or artifacts caused by interpolation.

[0097] When implemented on the GPU, this step can also be highly parallelized, and the interpolation calculation of each pixel can be assigned to different GPU threads to perform simultaneously, thus achieving real-time processing.

[0098] S204: Merge the intermediate frames with the original frames in chronological order to obtain a final output sequence with an increased frame rate.

[0099] After the intermediate frames are generated, the system needs to merge these newly generated frames with the original frames in the correct time sequence to form the final high frame rate video sequence. This step may seem simple, but it actually involves precise time control and frame buffer management.

[0100] First, the system needs to establish an accurate timestamp system. Assuming the original video is 30 frames per second, the system inserts intermediate frames to make it 60 frames per second. Then the timestamp may be as follows: Original frame A: 0ms Insert frame C1: 16.67ms Original frame B: 33.33ms Insert frame C2: 50ms Original frame C: 66.67ms ...and so on The system needs to organize the output order of frames precisely according to this time sequence. This requires the system to maintain a frame buffer, which stores a certain number of original frames and generated intermediate frames. The size of the buffer needs to balance latency and smoothness: the larger the buffer, the more complex the changes between frames that can be processed, but it also increases latency.

[0101] In the actual output process, the system needs to decide when to output each frame according to the refresh rate of the display device. For example, if the refresh rate of the display device is 144Hz, the system needs to further interpolate the 60fps video sequence to 144fps, or use frame repetition technology to fill in extra frames.

[0102] In addition, the system also needs to deal with audio synchronization issues. Since the video frame rate has changed, the audio playback speed needs to be adjusted accordingly to keep the audio and picture synchronized. This is usually achieved through audio resampling technology, which resamples the original audio stream to a sampling rate that matches the new video frame rate.

[0103] When processing real-time streaming media, the system also needs to consider network latency and jitter. It may need to implement adaptive buffering technology to dynamically adjust the buffer size according to network conditions to strike a balance between smoothness and real-time performance.

[0104] Finally, the system needs to ensure that the latency of the entire processing process remains within an acceptable range. For real-time applications, such as online games or video conferencing, end-to-end latency is usually required to be no more than 100ms. This requires the system to optimize performance as much as possible at each step, and may need to use prediction technology to start processing when some frame data has not yet arrived to reduce overall latency.

[0105] In the above embodiment, the number of frames to be inserted is determined by calculating the difference between the current frame rate and the target frame rate. This adaptive method can be dynamically adjusted according to the actual situation, avoiding the problem of unnecessary calculation or poor effect that may be caused by fixed frame rate interpolation. Secondly, the motion vector between adjacent frames is calculated using a motion estimation algorithm. This step can accurately capture the motion information of objects in the image and provide key data for the subsequent generation of intermediate frames. The accuracy of motion estimation directly affects the quality of the interpolated frame, so this step is crucial to maintaining the coherence and naturalness of the motion. Then, the intermediate frame is generated according to the motion vector. This motion-based frame interpolation method can better maintain the smoothness of the motion than simple linear interpolation, and reduce the jitter and unnatural phenomenon of the picture. Finally, the generated intermediate frame is merged with the original frame in chronological order to obtain the final output sequence after the frame rate is increased. This frame rate increase method not only increases the smoothness of the video, but also effectively reduces motion blur and stuttering in dynamic scenes, greatly improving the viewing experience. Especially in the case of high-speed motion or fast scene switching, this method can significantly improve the visual effect and make the picture more coherent and natural.

[0106] The following describes the system in the embodiment of the present invention from the perspective of hardware processing. Figure 3 , which is a schematic diagram of the physical device structure of a real-time smooth image system combining a CPU and a GPU provided in an embodiment of the present application.

[0107] It should be noted that Figure 3 The structure of the system shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0108] like Figure 3 As shown, the system includes a central processing unit (CPU) 301, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 302 or the program loaded from the storage part 308 to the random access memory (RAM) 303, such as executing the method in the above embodiment. In RAM 303, various programs and data required for system operation are also stored. CPU 301, ROM 302 and RAM 303 are connected to each other through a bus 304. Input / output (I / O) interface 305 is also connected to bus 304.

[0109] The following components are connected to the I / O interface 305: an input section 306 including a camera, an infrared sensor, etc.; an output section 307 including a liquid crystal display (LCD) and a speaker, etc.; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to the I / O interface 305 as needed. A removable medium 311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 310 as needed so that a computer program read therefrom is installed into the storage section 308 as needed.

[0110] In particular, according to an embodiment of the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 309, and / or installed from a removable medium 311. When the computer program is executed by the central processing unit (CPU) 301, various functions defined in the present invention are performed.

[0111] It should be noted that the computer-readable medium shown in the embodiment of the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable computer program. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.

[0112] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. Among them, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0113] As another aspect, the present invention further provides a computer-readable storage medium, which may be included in the system described in the above embodiment; or may exist independently without being assembled into the system. The above storage medium carries one or more computer programs, and when the above one or more computer programs are executed by a processor of a system, the system implements the method provided in the above embodiment.

[0114] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0115] As used in the above embodiments, the term "when..." may be interpreted to mean "if..." or "after..." or "in response to determining..." or "in response to detecting...", depending on the context. Similarly, the phrases "upon determining..." or "if (the stated condition or event) is detected" may be interpreted to mean "if determining..." or "in response to determining..." or "upon detecting (the stated condition or event)" or "in response to detecting (the stated condition or event)", depending on the context.

[0116] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0117] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

Claims

1. A real-time image smoothing method combining CPU and GPU, characterized in that: include: Controlling the CPU to perform a preprocessing operation on the image to be projected to obtain a preprocessed image, wherein the preprocessing operation includes denoising, brightness adjustment, and contrast adjustment; Controlling the CPU to perform edge detection on the pre-processed image using a preset edge detection algorithm, and marking an edge area in the pre-processed image; Controlling the CPU to perform an anti-aliasing operation on the edge area to obtain a preliminary projection image; Establishing a transmission channel between the CPU and the GPU so that the CPU sends the preliminary projection image to the GPU; Controlling the GPU to perform a trapezoidal correction operation on the preliminary projection image to obtain a corrected projection image; Using a preset anti-aliasing algorithm on the GPU to adjust the corrected projection image to obtain a smooth projection image; The GPU is controlled to enhance image details of the smoothed projection image to obtain a target projection image.

2. The method according to claim 1, characterized in that The controlling the CPU to perform edge detection on the pre-processed image using a preset edge detection algorithm to mark an edge area in the pre-processed image specifically includes: Controlling the CPU to perform grayscale processing on the preprocessed image to obtain a grayscale image; Applying a Gaussian filter to the grayscale image to obtain a blurred image; Performing gradient calculation on the blurred image to obtain a gradient image; Applying thresholding processing to the gradient image to obtain a binary edge image; Perform edge detection on the binary edge image and mark the edge area in the preprocessed image.

3. The method according to claim 1, characterized in that The controlling the CPU to perform an anti-aliasing operation on the edge area to obtain a preliminary projection image specifically includes: Identifying a number of pixels to be processed in the edge region; Performing sub-pixel sampling on the plurality of pixels to be processed to obtain sampling results; According to the sampling result, the CPU is controlled to perform color mixing processing on the plurality of pixels to be processed to obtain a preliminary projection image.

4. The method according to claim 3, characterized in that The identifying a number of pixels to be processed in the edge area specifically includes: Traversing each pixel in the edge area; In the case where it is determined that there is a non-edge pixel among the neighboring pixels of the existing pixel, the pixel is determined as a pixel to be processed.

5. The method according to claim 1, characterized in that Before the control CPU performs a preprocessing operation on the image to be projected to obtain the preprocessed image, the method further includes: Get the native resolution and projected image of the projection device; Adaptively adjusting the resolution of the projected image so that the resolution of the projected image is equal to the native resolution; Using a bicubic interpolation algorithm to adjust the projected image; The projection image is subjected to color space conversion according to the color space of the projection device to obtain an image to be projected.

6. The method according to claim 1, characterized in that After establishing the transmission channel between the CPU and the GPU, the method further includes: Processing the preliminary projection image using a data compression algorithm based on run-length coding to obtain compressed image data; Dividing the image data into a plurality of data blocks; The plurality of data blocks are sent to the GPU through the transmission channel according to an asynchronous transmission mechanism.

7. The method according to claim 1, characterized in that After controlling the GPU to enhance the image details of the smoothed projection image to obtain a target projection image, the method further includes: Calculate the number of frames that need to be inserted based on the difference between the current frame rate and the target frame rate; Applying a motion estimation algorithm to calculate motion vectors between adjacent frames according to the number of frames; generating an intermediate frame according to the motion vector; The intermediate frames are combined with the original frames in time order to obtain a final output sequence with an increased frame rate.

8. A real-time smooth image system combining CPU and GPU, characterized in that: The system comprises: One or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to cause the system to execute the method as described in any one of claims 1-7.

9. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on a system, the system is caused to execute the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that When the computer program product is run on a system, the system is caused to execute the method according to any one of claims 1 to 7.

Images