Screen projection display processing method and device, vehicle and electronic equipment

Abstract

The invention discloses a screen projection display processing method and device, a vehicle and electronic equipment, and relates to the technical field of vehicle screen projection, and the method comprises the steps: obtaining an original frame to be subjected to screen projection in response to a screen projection request, and determining a target region of interest from the original frame to be subjected to screen projection; based on the initial resolution and the target resolution of the target region of interest, image super-division or image enhancement is carried out on the target region of interest to obtain target projection screen display data of the target region of interest, and the target resolution is the resolution of the target region of interest when the target region of interest is displayed at a projection screen receiving end; and displaying the target region of interest on the projection screen receiving end based on the target projection screen display data. According to the method and the device, the projection screen processing efficiency can be improved, and the computing power loss of a vehicle machine is reduced.

Description

Technical Field

[0001] This application relates to the field of vehicle projection technology, and in particular to a projection display processing method, device, vehicle, and electronic device. Background Technology

[0002] With the development of smart cockpit technology, the interconnection and screen projection function between in-vehicle infotainment systems and various devices (such as mobile phones and rear-seat screens) has become a key feature for enhancing user experience. In this scenario, the vehicle system typically projects its own or mobile application screens to the passenger-side screen or rear-seat entertainment screens via wireless or wired means.

[0003] Currently, the rendering results are typically scaled and cropped before being output for display. However, this method can lead to compression of details during scaling, encoding, and transmission, resulting in compression artifacts such as block noise, mosquito noise, and color banding. This causes blurry text and a blocky appearance at the edges of the user interface. To address these issues, a uniform image quality optimization operation is usually performed on the scaled and cropped full-screen image.

[0004] However, the full-screen display includes various static element areas preset by the vehicle system, as well as invalid areas such as black borders at the top, bottom, left, and right caused by the mismatch between the aspect ratio of the source image and the projection window in mirrored screen projection. Therefore, when performing image quality optimization operations, unnecessary algorithm processing will be applied to the static element areas and invalid areas, which will destroy the original clear visual experience of the elements in the image. At the same time, indiscriminately applying enhancement algorithms to the full-screen display will also significantly increase the computing power consumption of the vehicle system.

[0005] Therefore, how to improve the display effect of the projected screen while increasing the efficiency of the projected screen processing and reducing the consumption of computing power has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0006] In view of the above problems, this application provides a screen projection display processing method, apparatus, vehicle, and electronic device to overcome or at least partially solve the above problems. The technical solution is as follows: A screen projection display processing method is applied to the screen projection output layer. The method includes: in response to a screen projection request, obtaining an original frame to be projected, and determining a target region of interest from the original frame to be projected; Based on the initial resolution and target resolution of the target region of interest, image super-resolution or image enhancement is performed on the target region of interest to obtain target projection display data of the target region of interest, wherein the target resolution is the resolution of the target region of interest when displayed on the projection receiving end; the target region of interest is displayed on the projection receiving end based on the target projection display data.

[0007] This application obtains the original frame to be projected and determines the target region of interest in response to the projection request. Then, based on its initial resolution and target resolution, it selects image super-resolution or enhancement to generate target projection display data and display it. This solves the image quality problems such as detail compression and block noise caused by traditional full-screen scaling and cropping. By precisely optimizing only the target region of interest, it avoids redundant algorithm processing of static element areas, invalid black borders and other redundant parts. It not only preserves the original clear elements of the image, but also improves the display effect of core content in a targeted manner. At the same time, compared with full-screen indiscriminate enhancement, it significantly reduces computing power consumption. While improving the display quality of the projected image, it effectively improves the projection processing efficiency and reduces the computing power consumption of the vehicle's system.

[0008] Optionally, the step of performing image super-resolution or image enhancement on the target region of interest based on the initial resolution and the target resolution to obtain target projection display data of the target region of interest includes: calculating the scaling factor from the initial resolution to the target resolution; if the scaling factor is less than a first threshold or greater than a second threshold, then using an image super-resolution network to perform super-resolution reconstruction on the target region of interest to obtain the target projection display data; the second threshold is greater than the first threshold; if the scaling factor is greater than or equal to the first threshold and less than or equal to the second threshold, then using image enhancement processing to perform same-resolution image quality enhancement on the target region of interest to obtain the target projection display data.

[0009] This optional implementation calculates the scaling factor from the initial resolution to the target resolution and, based on a quantized judgment of a first threshold and a second threshold, adaptively selects an image super-resolution network or image enhancement processing method to obtain the target projection display data. This approach can address large scaling scenarios where the scaling factor is less than the first threshold or greater than the second threshold, by accurately reconstructing lost details through the image super-resolution network, solving problems such as blurring, block noise, and jagged text caused by traditional interpolation scaling, thus achieving a leap in image quality. Conversely, for small scaling scenarios or scenarios with the same resolution where the scaling factor is between the first and second thresholds, lightweight image enhancement processing eliminates compression artifacts, slight noise, and edge blurring, improving image quality while maintaining the original resolution and effectively avoiding over-processing or under-processing. Simultaneously, the adaptive selection mechanism can allocate computing power on demand. The low computing power consumption of image enhancement combined with the efficient inference of the lightweight super-resolution network significantly reduces the hardware burden in computing-constrained scenarios such as in-vehicle systems.

[0010] Optionally, determining the target region of interest from the original frame to be projected includes: extracting projection layer data corresponding to the target application from the original frame to be projected to generate an initial region of interest; obtaining the initial aspect ratio of the initial region of interest and the size information of the display window of the projection receiving end; adjusting the aspect ratio of the initial region of interest according to the initial aspect ratio and the size information of the display window to generate a first initial region of interest matching the display window; if there is an invalid black border area in the first initial region of interest, then removing the invalid black border area in the first initial region of interest to generate the target region of interest; if there is no invalid black border area in the first initial region of interest, then using the first initial region of interest as the target region of interest.

[0011] This optional implementation extracts the target layer and adapts the scale, then removes the invalid black border areas that appear after adaptation. This achieves the filtering from the original screen-casting frame to the precise target area of ​​interest, ensuring the compatibility between the screen-casting content that the user is interested in and the display window, while avoiding the waste of computing power on invalid areas during subsequent image quality optimization.

[0012] Optionally, the step of extracting the projection layer data corresponding to the target application from the original frame to be projected to generate an initial region of interest includes: obtaining the application corresponding to the content displayed in the original frame to be projected, and obtaining the application identifier corresponding to the application; determining whether the application identifier belongs to the target application in a preset application list; if so, extracting the projection layer data corresponding to the target application from the original frame to be projected to generate the initial region of interest.

[0013] This optional implementation method obtains the application identifier of the application corresponding to the original screen-casting frame and determines whether it belongs to the target application in the preset application list. If it matches, it extracts the screen-casting layer data corresponding to the target application to generate an initial region of interest. This not only focuses on the core screen-casting content of the target application, avoiding redundant information from occupying computing power and greatly reducing the amount of data processed in subsequent processing to adapt to computing-limited scenarios such as in-vehicle systems, but also independently processes the exclusive screen-casting layer data of the target application, matches its display characteristics to customize optimization strategies, and ensures the targeting and accuracy of image quality optimization.

[0014] Optionally, the step of using an image super-resolution network to perform super-resolution reconstruction of the target region of interest to obtain the target projection display data includes: dividing the target region of interest into multiple display data blocks; setting overlapping regions between adjacent blocks in the multiple display data blocks; inputting each display data block into the image super-resolution network for processing to obtain the corresponding target display data block; and merging the target display data blocks in the overlapping regions using a weight fusion method based on a preset window function to generate the target projection display data.

[0015] This optional implementation method, in this embodiment, divides the target region of interest into multiple display data blocks with overlapping regions and inputs them independently into the super-resolution network, which significantly reduces the amount of data processed per frame, adapts to the performance constraints of devices with limited computing power such as in-vehicle systems, and supports parallel processing to improve overall efficiency. Secondly, the overlapping regions between adjacent blocks, combined with the weight fusion method of the preset window function, can smoothly transition the processing differences at the block boundaries, completely eliminating the seam artifacts that are easy to occur in traditional block super-resolution, and ensuring the integrity and visual continuity of the image. Optionally, the step of performing image enhancement processing on the target region of interest at the same resolution to obtain the target projection display data includes: dividing the target region of interest into multiple display data blocks; setting overlapping regions between adjacent blocks in the multiple display data blocks; performing image enhancement processing on each of the multiple display data blocks to obtain corresponding target display data blocks; and merging the target display data blocks in the overlapping regions using a weighted fusion method based on a preset window function to generate the target projection display data.

[0016] This optional implementation divides the target area into overlapping display data blocks and performs image enhancement independently. This reduces the amount of data processed per block, adapting to the real-time requirements of computing-constrained scenarios such as in-vehicle systems, and also supports parallel processing of blocks to improve overall efficiency. On the other hand, the overlapping areas of adjacent blocks are combined with the weighted fusion of preset window functions to smoothly eliminate the boundary image quality differences caused by independent block enhancement, ensuring the continuity and visual consistency of the enhanced image. At the same time, this method can flexibly adapt to target areas with different resolutions. By adjusting the block size and overlap width, it can achieve precise optimization of image quality at the same resolution with low computing power overhead, balancing the enhancement effect and processing cost.

[0017] Optionally, the method further includes: obtaining other projection display data corresponding to other regions in the original projection frame other than the target region of interest; the other projection display data includes at least one of system user interface display data, status bar display data, and display data corresponding to applications not in the preset application list; and displaying the other regions on the projection receiving end based on the other projection display data.

[0018] This optional implementation acquires other projection display data outside the target region of interest in the original projection frame, and displays other areas on the projection receiving end based on this data. This ensures the integrity of the projection screen and avoids the loss of functional information such as system status and operation access caused by only displaying the core area, thus meeting the user's need for a complete projection scenario. It also optimizes the efficiency of computing power allocation, performing computing power-intensive processing such as super-resolution / enhancement only on the core target area, while other areas directly reuse the original display data, reducing the consumption of ineffective computing power to adapt to computing power-constrained environments such as vehicle systems.

[0019] A screen projection display processing device, the device comprising: The acquisition module is used to acquire the original frame to be cast in response to the screen casting request, and to determine the target region of interest from the original frame to be cast; The processing module is used to perform image super-resolution or image enhancement on the target region of interest based on the initial resolution and the target resolution, to obtain the target projection display data of the target region of interest, wherein the target resolution is the resolution of the target region of interest when it is displayed on the projection receiving end; The display module is used to display the target region of interest on the projection receiving end based on the target projection display data.

[0020] A computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the screen projection processing method described in any of the preceding claims.

[0021] A vehicle comprising the aforementioned projection display processing device.

[0022] A computer program product includes a computer program that, when executed by a processor, implements the steps of any of the above-described screen projection processing methods.

[0023] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0024] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 This application illustrates a schematic flowchart of a screen projection processing method provided in an embodiment. Figure 1 ; Figure 2 This application illustrates a schematic flowchart of a screen projection processing method provided in an embodiment. Figure 2 ; Figure 3 This application illustrates a schematic flowchart of a screen projection processing method provided in an embodiment. Figure 3 ; Figure 4 This application illustrates a schematic flowchart of a screen projection processing method provided in an embodiment. Figure 4 ; Figure 5 This application illustrates a schematic flowchart of a screen projection processing method provided in an embodiment. Figure 5 ; Figure 6 This illustration shows a schematic structural diagram of a screen projection display processing device provided in an embodiment of this application; Figure 7 A schematic structural diagram of a vehicle provided in an embodiment of this application is shown. Detailed Implementation

[0025] Exemplary embodiments of the present application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the scope of the present application to those skilled in the art.

[0026] To more clearly illustrate the embodiments of this application, the technical terms used in the embodiments will be briefly introduced below: Currently, the rendering results are typically scaled and cropped before being output for display. However, this method can lead to compression of details during scaling, encoding, and transmission, resulting in compression artifacts such as block noise, mosquito noise, and color banding. This causes blurry text and a blocky appearance at the edges of the user interface. To address these issues, a uniform image quality optimization operation is usually performed on the scaled and cropped full-screen image.

[0027] However, the full-screen display includes various static element areas preset by the vehicle system, as well as invalid areas such as black borders at the top, bottom, left, and right caused by the mismatch between the aspect ratio of the source image and the projection window in mirrored screen projection. Therefore, when performing image quality optimization operations, unnecessary algorithm processing will be applied to the static element areas and invalid areas, which will destroy the original clear visual experience of the elements in the image. At the same time, indiscriminately applying enhancement algorithms to the full-screen display will also significantly increase the computing power consumption of the vehicle system.

[0028] To overcome or at least partially solve the above problems, this application provides a screen projection display processing method, such as... Figure 1 As shown, Figure 1 This application provides a schematic flowchart of a screen projection display processing method, which is applied to the screen projection output layer. The specific steps include: S101. In response to the screen projection request, obtain the original screen projection frame and determine the target region of interest from the original screen projection frame.

[0029] In some embodiments, the screen mirroring request can be initiated by the user through the vehicle's infotainment system's interactive interface (such as a touchscreen or voice command), or automatically triggered by the target application (such as a video playback application or a navigation application) (e.g., when a screen mirroring receiver is detected to be connected). The screen mirroring request includes basic information such as the screen mirroring receiver identifier (e.g., the rear entertainment screen or an additional display screen) and target display window parameters (e.g., display size, preset aspect ratio), providing a basis for subsequent processing.

[0030] Specifically, the original frame to be projected is complete frame data obtained by the projection output layer from the display link of the vehicle system. It contains layer information corresponding to all content currently displayed on the vehicle system. For example, it may include the display layer of a user-specified application to be projected, as well as the current vehicle system user interface (UI) layer, status bar layer, etc. The acquisition method adopts zero-copy technology, which directly maps the hardware buffer (such as AHardwareBuffer) corresponding to the original frame to be projected into an image object that can be recognized by the GPU in the form of a direct memory access buffer (dma-buf). The data transmission process does not go through CPU copying, reducing data flow latency and ensuring real-time projection.

[0031] In this step, the target region of interest (ROI) is the core region in the original frame to be projected in this embodiment that needs to undergo super-resolution or enhancement processing, and is used to accurately define the processing range that needs to be subsequently optimized for image quality.

[0032] This is because the original frame to be projected may contain at least one application, system UI, status bar, and other types of display content. Only the display interface of a certain application is the object of the user's projection, and therefore the image quality needs to be improved through super-resolution / enhancement. Other applications do not need to be optimized because the user does not pay attention to them. In addition, the vehicle system UI (such as buttons and menus) and status bar (such as time and battery level) are rendered and generated by the vehicle system, and their resolution and display effect are already adapted to the projection receiver in the car, so no additional enhancement is needed.

[0033] Furthermore, by first obtaining the target region of interest through this step, the display content that needs to be optimized for subsequent image quality can be accurately separated from the original screen frame to be projected. This ensures that subsequent super-resolution reconstruction and image quality enhancement are performed only on this region, avoiding the consumption of computing power by system UI, non-target applications and other content that does not need to be optimized, and preventing problems such as over-sharpening of UI icons and color distortion caused by over-processing, thus ensuring that the optimization strategy focuses on core needs.

[0034] It should be noted that, referring to Figure 2 As shown, the specific implementation process of determining the target region of interest from the original frame to be projected in S101 includes the following steps S1011-S1015: S1011. Extract the projection layer data corresponding to the target application from the original projection frame to generate an initial region of interest.

[0035] Specifically, the original frame to be projected contains independent layers corresponding to multiple types of display content (such as a layer corresponding to at least one application, a system UI layer, a status bar layer, etc.). This step requires obtaining the application identifier (such as application package name, process ID, exclusive layer mark) corresponding to the application layer of each application in the original frame to be projected, and then comparing it with the target application identifier in the preset application list; if the match is successful, the complete pixel data of the layer (i.e., the projection layer data) is extracted and used as the initial region of interest.

[0036] This further separates the target application into an independent layer, avoiding redundant areas such as the system UI and status bar involved in subsequent super-resolution / enhancement processing. This reduces the processing scope from the source, laying the foundation for reducing computing power consumption and avoiding UI optimization errors.

[0037] S1012. Obtain the initial aspect ratio corresponding to the initial region of interest, and the size information of the display window of the projection receiver.

[0038] The initial aspect ratio can be calculated from the pixel dimensions of the initial region of interest (e.g., 1920×1080) (e.g., 1920:1080=16:9); the size information of the display window can be determined by the hardware parameters of the screen mirroring receiver or the configuration parameters in the screen mirroring request, specifying the size of the target container that the screen needs to be adapted to.

[0039] S1013. Adjust the initial region of interest according to the initial aspect ratio and the size information of the display window to generate a first initial region of interest that matches the display window.

[0040] In this step, the principle is to maximize the effective content of the screen to fill the display window. First, compare the initial aspect ratio with the aspect ratio of the display window. The aspect ratio of the display window is derived from the size information, such as 1920×1080 corresponding to 16:9.

[0041] If the initial aspect ratio is greater than the aspect ratio of the display window (e.g., 4:3 adapted to 16:9), the scaling factor k is calculated based on the height of the display window as the adaptation reference, which is the display window height divided by the initial region of interest height. If the initial aspect ratio is less than the aspect ratio of the display window (e.g., 21:9 adapted to 16:9), the scaling factor k is calculated based on the width of the display window as the adaptation reference, which is the display window width divided by the initial region of interest width.

[0042] Then, the width and height of the initial region of interest are scaled proportionally by a scaling factor k (width × k, height × k) to ensure that the scaled image proportions are consistent with the initial proportions and there is no stretching distortion. If a certain dimension (width or height) of the image does not fill the display window after scaling (due to proportional constraints), the scaled image is centered and aligned. The unfilled area will produce a black border area. Finally, a first initial region of interest is generated with a size that perfectly matches the display window and a proportion that conforms to the initial inherent properties.

[0043] It should be noted that if the initial aspect ratio is equal to that of the display window, then during the scaling adjustment, a uniform scaling factor k is calculated using both the width and height dimensions of the display window as dual references. The initial region of interest is then scaled proportionally according to this scaling factor, ensuring that the width and height dimensions of the scaled image perfectly match the width and height of the display window while maintaining the original aspect ratio. At this point, the scaled image completely fills the display window without any black borders or invalid areas. The resulting first initial region of interest is the effective image area that perfectly matches the size and aspect ratio of the display window. Subsequent black border culling is unnecessary; this first initial region of interest is used as the target region of interest.

[0044] S1014. If there is an invalid region with a black border in the first initial region of interest, then the invalid region with a black border in the first initial region of interest is removed, and the target region of interest is generated.

[0045] In this embodiment of the application, the edge strips of the first initial region of interest (e.g., the width is 5%-10% of the region height / width) can be verified at the pixel level to calculate the mean and variance of the strip brightness. If the mean is ≤5 and the variance is ≤2 (pure black feature), it is determined to be an invalid black edge region. The invalid black edge region needs to be removed before generating the target region of interest.

[0046] S1015. If the black-bordered invalid region does not exist in the first initial region of interest, then the first initial region of interest is taken as the target region of interest.

[0047] If the brightness characteristics of the edge strip do not meet the black border standard, then the first initial region of interest is directly taken as the target region of interest.

[0048] This application embodiment extracts the target layer and adapts the scale size, and then removes the invalid black border areas that appear after adaptation. This achieves the filtering from the original screen-casting frame to the precise target area of ​​interest, which not only ensures the adaptability of the screen-casting content that the user is concerned about to the display window, but also avoids the waste of computing power on invalid areas during subsequent image quality optimization.

[0049] S102. Based on the initial resolution and target resolution of the target region of interest, perform image super-resolution or image enhancement on the target region of interest to obtain target projection display data of the target region of interest.

[0050] Wherein, the target resolution is the resolution of the target region of interest when displayed on the projection receiving end.

[0051] In this embodiment of the application, the initial resolution refers to the original pixel dimension (e.g., 1280×720) of the target region of interest determined in step S101. It is determined by the inherent resolution of the target application projection layer data and reflects the original detail accuracy of the target region of interest.

[0052] The target resolution refers to the preset pixel dimension (e.g., 1920×1080) when the target area of ​​interest is displayed on the screen mirroring receiver. It is determined by the hardware display capability of the screen mirroring receiver (e.g., the physical resolution of the screen), the display parameters set by the user, or the resolution parameters specified in the screen mirroring request. It is the target benchmark for image quality optimization.

[0053] Therefore, the process of performing image super-resolution or image enhancement on the target region of interest based on the initial resolution and the target resolution in this step can be as follows: calculate the scaling factor from the initial resolution to the target resolution, and further select whether to perform image super-resolution or image enhancement on the target region of interest according to the magnitude of the scaling factor, so as to obtain the target projection display data of the target region of interest.

[0054] Specifically, image super-resolution (SR) refers to a technology that uses algorithmic models to reconstruct pixel information and complete details in low-resolution images to generate high-resolution images. Its core is to overcome the pixel limitations of the original image, restoring or reconstructing image details lost due to scaling, compression, etc., thus achieving a leap in image quality from low to high resolution. It is mainly used to solve problems such as blurring and loss of detail in scenarios with significant scaling (e.g., block noise, mosquito noise, and jagged edges on text when a low-resolution source frame is enlarged to fit a high-resolution display window); while improving image resolution, it maintains the realism and consistency of the image content, avoiding distortion or artifacts.

[0055] Image enhancement refers to optimizing the visual effect of an image through a series of signal processing algorithms without changing its original resolution. Its core function is to improve key visual indicators such as image sharpness, contrast, and signal-to-noise ratio, thereby improving the subjective perception of the image. It is mainly used to address image quality defects in scenes with small scaling or at the same resolution (such as compression artifacts, slight noise, blurred edges, and dull colors); enhance effective visual information in the image (such as text and object edges); suppress invalid interference information; and improve the readability and visual appeal of the image.

[0056] Furthermore, the specific implementation of S102 (based on the initial resolution and target resolution of the target region of interest, performing image super-resolution or image enhancement on the target region of interest to obtain the target projection display data of the target region of interest) can be refined as follows: S1021-S1023: S1021. Calculate the scaling factor from the initial resolution to the target resolution.

[0057] Specifically, the initial resolution is the original pixel dimension of the target region of interest (e.g., 1280×720), and the target resolution is the preset pixel dimension when the projection receiver displays the region (e.g., 1920×1080).

[0058] Then, the scaling factor s is calculated based on the initial resolution and the target resolution. The calculation formula is: s = total number of pixels at the target resolution ÷ total number of pixels at the initial resolution of the target region of interest (or take the average of the scaling factors in the width and height directions to ensure that the result reflects the overall scaling magnitude).

[0059] For example: if the initial resolution is 1280×720 (total number of pixels 921600) and the target resolution is 1920×1080 (total number of pixels 2073600), then the scaling factor s = 2073600 ÷ 921600 = 2.0.

[0060] In the embodiments of this application, the scaling factor s directly reflects the extent to which the target region of interest needs to be "enlarged / reduced", providing a quantitative basis for subsequent selection of super-resolution or enhancement strategies.

[0061] S1022. If the scaling factor is less than the first threshold or greater than the second threshold, then an image super-resolution network is used to perform super-resolution reconstruction of the target region of interest to obtain the target projection display data.

[0062] Wherein, the second threshold is greater than the first threshold.

[0063] For example, a first threshold (e.g., 1) and a second threshold (e.g., 1.3) are preset.

[0064] When the scaling factor is less than the first threshold, the target resolution is less than the initial resolution of the target region of interest (the screen needs to be scaled down to fit the display window). In this case, the super-resolution network can preserve details during the scaling down process and avoid blurring the image.

[0065] When the scaling factor is greater than the second threshold, the target resolution is much larger than the initial resolution (the image needs to be enlarged to improve details). At this time, the super-resolution network makes up for the loss of details caused by scaling through pixel reconstruction.

[0066] It should be noted that the super-resolution network can be a lightweight deep learning super-resolution network (such as FSRCNN×2, Mobile-ESRGAN-Mobile), which is suitable for computing-constrained scenarios such as vehicle systems; the network input is the image data of the target region of interest, and the output is the super-resolution data that matches the target resolution.

[0067] S1023. If the scaling factor is greater than or equal to the first threshold and less than or equal to the second threshold, then image enhancement processing is used to enhance the image quality of the target region of interest at the same resolution to obtain the target projection display data.

[0068] When the first threshold ≤ scaling factor ≤ second threshold (e.g., 1 ≤ s ≤ 1.3), the target resolution is close to the initial resolution of the target region of interest. There is no need to improve the resolution through super-resolution; only image quality defects need to be optimized.

[0069] Then, a lightweight serial processing algorithm is adopted to sequentially perform image quality enhancement processes such as adaptive denoising, edge-aware chroma upsampling, and limited sharpening and contrast adjustment.

[0070] This application embodiment calculates the scaling factor from the initial resolution to the target resolution and, based on the quantization judgment of the first threshold and the second threshold, adaptively selects an image super-resolution network or an image enhancement processing method to obtain the target projection display data. It can accurately reconstruct lost details through the image super-resolution network for large scaling scenarios where the scaling factor is less than the first threshold or greater than the second threshold, solving problems such as image blurring, block noise, and jagged text caused by traditional interpolation scaling, and achieving a leap in image quality. It can also eliminate compression artifacts, slight noise, edge blurring, and other defects through lightweight image enhancement processing for small scaling scenarios or scenarios with the scaling factor between the first threshold and the second threshold, improving the image quality while maintaining the original resolution, and effectively avoiding the problems of over-processing or under-processing.

[0071] Meanwhile, the adaptive selection mechanism can allocate computing power on demand. The low computing power consumption of image enhancement combined with the efficient inference of lightweight super-resolution networks significantly reduces the hardware burden in computing-constrained scenarios such as vehicle systems.

[0072] S103. Based on the target projection display data, display the target region of interest on the projection receiving end.

[0073] Then, the target projection display data generated in step S102 is transmitted to the projection receiving end through the projection link (such as HDMI, wireless projection protocol). During the transmission, a synchronization signal (such as releaseFence) is generated to synchronize the display timing of the projection receiving end, ensuring that the display rhythm of the target projection display data matches the refresh rate of the receiving end (such as 30fps, 60fps) to avoid frame drops and stuttering.

[0074] This application embodiment obtains the original frame to be projected and determines the target region of interest in response to the projection request. Then, based on its initial resolution and target resolution, it selects image super-resolution or enhancement to generate target projection display data and display it. This solves the image quality problems such as detail compression and block noise caused by traditional full-screen scaling and cropping. By precisely optimizing only the target region of interest, it avoids redundant algorithm processing of static element areas, invalid black borders and other redundant parts. It not only preserves the original clear elements of the image, but also improves the display effect of core content in a targeted manner. At the same time, compared with full-screen indiscriminate enhancement, it greatly reduces the computing power consumption. While improving the display quality of the projected image, it effectively improves the projection processing efficiency and reduces the computing power consumption of the vehicle's system.

[0075] As an extension and refinement of the above embodiments, refer to Figure 4 As shown, the above S1021 can be further refined into the following steps S401-S403: S401. Obtain the application corresponding to the content displayed in the original screen-to-be-projected frame, and obtain the application identifier corresponding to the application.

[0076] Specifically, when the original screen frame to be projected is generated by overlaying display layers of multiple running applications and system modules in the vehicle system, this step iterates through all display layers contained in the original screen frame to be projected through the system's underlying interfaces (such as WindowManager and SurfaceFlinger in Android systems) to identify the subject to which each layer belongs (i.e., the application or system module to which the displayed content belongs). For layers whose subject is an application, their unique application identifier is extracted. The application identifier is feature information that can uniquely distinguish the application, including but not limited to the application package name (such as com.example.video), application process ID (PID), and system-assigned exclusive layer tag (Layer ID).

[0077] Furthermore, by identifying the layer affiliation and extracting the application identifier, the mapping relationship between the display layer and the application is obtained, providing data basis for the accurate selection of target applications and avoiding confusion in the processing scope caused by the inability to distinguish the layer affiliation.

[0078] S402. Determine whether the application identifier belongs to the target application in the preset application list.

[0079] Specifically, the preset application list can be a set of applications that need to be optimized for screen mirroring (such as video playback applications, navigation applications, document reading applications, etc.) that are pre-configured by the system or user-defined. The list stores the application identifiers corresponding to each target application, which are consistent with the identifier type extracted in S401. Then, the application identifiers obtained in S401 are compared one by one with the identifiers in the preset application list to determine whether they are consistent. If they are consistent, the application is determined to be the target application, and the following S403 is executed.

[0080] S403. Extract the projection layer data corresponding to the target application from the original frame to be projected to generate the initial region of interest.

[0081] After identifying the application as the target application, based on the mapping relationship between the display layer and the application established in S401, the complete display layer data (i.e., projection layer data) corresponding to the target application is separated and extracted from multiple overlay layers of the original frame to be projected. The projection layer data includes all pixel information, color space parameters, resolution parameters, etc. of the content displayed by the target application. The extracted projection layer data is used as an independent image data block and defined as the initial region of interest. Its boundary is consistent with the boundary of the target application's display layer and does not contain the display content of other applications or system modules.

[0082] This application embodiment obtains the application identifier of the application corresponding to the original screen-casting frame and determines whether it belongs to the target application in the preset application list. If it matches, it extracts the screen-casting layer data corresponding to the target application to generate an initial region of interest. This allows it to focus on the core screen-casting content of the target application, avoids redundant information from consuming computing power, and significantly reduces the amount of data processed in subsequent processing to adapt to computing-limited scenarios such as in-vehicle systems. It also allows for independent processing of the exclusive screen-casting layer data of the target application, matching its display characteristics to customize optimization strategies, ensuring the targeted nature and accuracy of image quality optimization.

[0083] It should be noted that the obtained initial region of interest also needs to undergo color space conversion and standardization.

[0084] This is because the initial projection layer data of the region of interest may use different color spaces (such as RGB, YUV4:2:0, YUV4:4:4) due to differences in target applications (such as video, navigation) or system rendering. To ensure the compatibility and consistency of subsequent optimization algorithms, it is uniformly converted to a preset target color space (e.g., YUV4:2:0). This color space separates luminance (Y channel) and chrominance (U, V channels), which can be used to optimize luminance channel details (such as text, edges) while reducing the computational power consumption of the chrominance channel, adapting to the computational power constraints of the vehicle's infotainment system. The conversion process uses standard color mapping algorithms (such as BT.601, BT.709) to ensure accurate color reproduction and avoid color deviation and distortion.

[0085] In addition, pixel value standardization is performed on the converted image data, that is, the pixel value (0-255) is mapped to a preset range (such as [-1,1] or [0,1]) to eliminate the differences in pixel value distribution caused by different application rendering; mean normalization and variance adjustment are performed at the same time to make the image data meet the input requirements of subsequent super-resolution network or enhancement algorithm, improve the convergence speed and processing accuracy of the algorithm, and avoid the unstable optimization effect caused by pixel value fluctuation (such as local over-sharpening and noise amplification).

[0086] By unifying the color space and standardizing pixel data, the format differences and distribution fluctuations of the original data are eliminated, providing a unified input basis for subsequent super-resolution / enhancement processing. This ensures the adaptability of the optimization algorithm and the stability and consistency of the processing effect, while also taking into account the realism of color reproduction and the reasonableness of computing power consumption.

[0087] As an extension and refinement of the above embodiments, refer to Figure 5 As shown, the screen projection processing method provided in this application further includes the following steps: S501. Obtain other projection display data corresponding to other regions in the original projection frame other than the target region of interest.

[0088] The other screen projection display data includes at least one of the following: system user interface display data, status bar display data, and display data corresponding to applications not in the preset application list.

[0089] Specifically, other areas can be divided into two categories. The first category (content that the vehicle system can render natively) includes the system user interface (such as the vehicle system settings menu, function buttons, and pop-ups) and status bar (such as time, battery level, vehicle speed, and network status). The display style and layout of these are directly controlled by the vehicle system and do not require additional adaptation. The second category includes non-target application display content that is not in the preset application list (such as background music application windows, floating tools, and unselected third-party application interfaces). The original size and proportion of these content may not match the display window of the screen mirroring receiver and require further separate processing.

[0090] For the first type of content, its original display data (pixels, layout parameters) can be directly extracted without performing proportional size adjustments or complex optimizations. Since the vehicle system can render and adapt automatically, the output can be directly displayed on the screen receiving end in the native style, ensuring consistency with the local display of the vehicle system.

[0091] For the second type of content, we can first obtain its original aspect ratio and the size information of the display window of the projection receiver, and adapt the same aspect ratio to ensure that the content is not stretched or distorted and does not exceed the boundary of the display window, and then generate the adapted display data.

[0092] S502. Display the other areas on the screen projection receiver based on the other screen projection display data.

[0093] Furthermore, based on layer attribution recognition technology, the layer data corresponding to the first type of content and the second type of content are separated from the overlay layers of the original screen projection frame to ensure accurate extraction without redundancy; and the two types of processed display data are layer-synthesized to generate complete other screen projection display data, which is synchronously output to the screen projection receiver and displayed along with the screen projection data of the target area of ​​interest.

[0094] Preferably, in some scenarios, users can choose to cast only the display screen of a specific application or the entire in-vehicle infotainment system screen when casting.

[0095] If the screen mirroring request only requires displaying the content of the target application (such as a single screen mirrored video or navigation screen), the receiving end of the screen mirroring will directly display the target area of ​​interest based on the target screen mirroring display data, without needing to obtain other screen mirroring display data.

[0096] If the screen projection request requires displaying the complete vehicle infotainment screen (including the target application, system UI, status bar, etc.), then the layer compositing step is performed, and other screen projection display data is displayed on other areas of the screen projection receiving terminal in the manner described in the above embodiment.

[0097] This application embodiment obtains other projection display data outside the target region of interest in the original projection frame, and displays other areas on the projection receiving end based on this data. This not only ensures the integrity of the projection screen and avoids the loss of functional information such as system status and operation entry points caused by only displaying the core area, thus meeting the user's needs for a complete projection scenario, but also optimizes the efficiency of computing power allocation. It only performs computing power-intensive processing such as super-resolution / enhancement on the core target area, while other areas directly reuse the original display data, reducing the consumption of ineffective computing power to adapt to computing power-constrained environments such as vehicle systems.

[0098] As an extension and refinement of the above embodiments, S1032 can be further refined into the following steps: Step 11: Divide the target region of interest into multiple display data blocks.

[0099] Among these, adjacent blocks in the plurality of display data blocks have overlapping areas.

[0100] Specifically, based on the resolution of the target region of interest (e.g., 1920×1080) and the upper limit of hardware computing power, a fixed block specification (e.g., 320×320 pixels, 512×512 pixels) can be preset, and the target region of interest can be divided into multiple display data blocks that are fully covered without omissions, using a uniform division method.

[0101] Then, a pre-defined overlap area (e.g., 16-32 pixels) is reserved between adjacent blocks. The width of the overlap area is determined according to the block specifications and algorithm characteristics (the larger the block, the wider the overlap can be), ensuring that there is data intersection in the boundary areas of adjacent blocks.

[0102] By splitting high-resolution images, the pixel size of individual blocks is greatly reduced, which can adapt to the input limitations of super-resolution networks. At the same time, it reduces the computational power consumption and latency of single-frame processing, meeting the needs of real-time projection. Meanwhile, the setting of overlapping areas avoids feature breakage caused by independent processing of block boundaries, and eliminates seam artifacts by weight fusion of overlapping parts.

[0103] Step 12: Input each of the display data blocks into the image super-resolution network for processing to obtain the corresponding target display data blocks.

[0104] Then, each display data block is input into the super-resolution network. The network performs super-resolution reconstruction independently on each block and outputs a single target display data block that matches the target resolution (e.g., a 320×320 block becomes 640×640 after super-resolution).

[0105] Step 13: In the overlapping area, the target display data is divided into blocks and merged by a weighted fusion method based on a preset window function to generate the target projection display data.

[0106] Specifically, a window function for smooth transition (such as a cosine window or a Gaussian window) can be preset, and the effective range of the window function is consistent with the width of the overlapping area (such as a 16-pixel overlapping area corresponding to a 16-pixel window function radius). Then, for the overlapping areas of each target display data block, weights are assigned according to the window function. The weight of the center area of ​​the block is 1 (completely retaining its own data), and the weight of the overlapping area decreases linearly from the center to the boundary to 0 (completely merging the data of adjacent blocks). Then, for the overlapping areas of adjacent blocks, a weighted average of pixel values ​​is performed according to the calculated weights (e.g., the weight of the overlapping area of ​​block A is 0.7, the weight of the overlapping area of ​​block B is 0.3, and the merged pixel value = A × 0.7 + B × 0.3). Non-overlapping areas directly retain the target data of the corresponding block, and finally, the data is stitched together to form the complete target projection display data.

[0107] This application embodiment significantly reduces the amount of data processed per frame by dividing the target region of interest into multiple display data blocks with overlapping regions and inputting them independently into the super-resolution network. This adapts to the performance constraints of devices with limited computing power, such as in-vehicle systems, and supports parallel processing to improve overall efficiency. Secondly, the overlapping regions between adjacent blocks, combined with the weight fusion method of the preset window function, can smoothly transition the processing differences at the block boundaries, completely eliminating the seam artifacts that are prone to occur in traditional block super-resolution, and ensuring the integrity and visual continuity of the image.

[0108] It should be noted that after the image super-resolution network processes the image, the reconstructed resolution, i.e., super-resolution (SR), is obtained. At this point, it is also necessary to determine whether the SR is related to the target resolution required by the projection receiver.

[0109] If the SR result does not satisfy the integer multiple relationship with the target resolution (e.g., SR result is 2560×1440, target resolution is 2880×1620, scaling factor is 1.125; or SR result is 1920×1080, target resolution is 1600×900, scaling factor is 0.833), then it is necessary to first convert the SR result output by the super-resolution network (usually an sRGB non-linear space image) to linear space (e.g., through inverse gamma correction operation) to avoid brightness distortion and contrast abnormalities (e.g., loss of dark details and overexposure of bright areas) caused by resampling in non-linear space, and ensure the accuracy of pixel value calculation during the resampling process.

[0110] Then, the Lanczos3 or bicubic interpolation algorithm is selected in the linearRGB space. The SR result is directly resampled to the target resolution and restored to the target color space. After the resampling is completed, the linear space image is converted back to the target color space (such as sRGB) to ensure that the output target projection display data meets the display requirements of the projection receiver and the color reproduction is true.

[0111] If the SR result is an integer multiple of the target resolution (e.g., 2x, 3x), the SR result is output directly without additional resampling.

[0112] By directly adapting to the target size through a single resampling, the algorithm eliminates the repeated loss of pixel information and error accumulation during multi-level scaling, solving the problems of overall image blurring and detail dulling caused by traditional methods. Simultaneously, it balances detail preservation and image quality consistency. Linear space conversion ensures the accuracy of brightness and color calculations during resampling. Compared to simpler algorithms like bilinear interpolation, the Lanczos3 / bicubic interpolation algorithm better preserves image edges, textures, and other detailed features, avoiding jagged edges and blockiness caused by resampling, and ensuring that the detail quality after super-resolution is not lost in secondary processing.

[0113] As an extension and refinement of the above embodiments, S1033 can be further refined into the following steps: Step 21: Divide the target region of interest into multiple display data blocks.

[0114] Among these, adjacent blocks in the plurality of display data blocks have overlapping areas.

[0115] This step is the same as step 11 above, and will not be repeated here.

[0116] Step 22: Perform image enhancement processing on each of the multiple display data blocks to obtain the corresponding target display data blocks.

[0117] Specifically, this requires performing a series enhancement algorithm (adaptive denoising, limited sharpening, chroma upsampling, etc.) on each display data block to optimize image quality while keeping the original resolution of the block unchanged, and outputting a single target display data block.

[0118] Step 23: In the overlapping area, the target display data is divided into blocks and merged by a weighted fusion method based on a preset window function to generate the target projection display data.

[0119] This step is similar to step 13 above, both involving merging the processed target display data into blocks, which will not be elaborated here.

[0120] This application embodiment divides the target area into overlapping display data blocks and performs image enhancement independently. This reduces the amount of data processed per block, adapting to the real-time requirements of computing-constrained scenarios such as in-vehicle systems, and also supports parallel processing of blocks to improve overall efficiency. On the other hand, the overlapping areas of adjacent blocks are combined with the weighted fusion of preset window functions to smoothly eliminate boundary image quality differences (such as sudden changes in local sharpness or contrast) caused by independent block enhancement, ensuring the continuity and visual consistency of the enhanced image. At the same time, this method can flexibly adapt to target areas with different resolutions. By adjusting the block specifications and overlap width, it can achieve precise optimization of image quality at the same resolution with low computing power overhead, balancing the enhancement effect and processing cost.

[0121] In some embodiments, the screen projection processing method provided in this application further includes real-time monitoring of hardware / performance indicators such as screen projection frame rate, temperature, and frame drops. When the indicators exceed the threshold, the corresponding function can be adaptively turned off, and a "reset to prevent frequent switching" mechanism can be set.

[0122] Furthermore, when the hardware load is too high, in order to prioritize the smoothness of screen projection and avoid overheating or lag of the device, at least one of the super-resolution function and the enhancement function can be turned off; and the "reset" mechanism is used to prevent the super-resolution function and the enhancement function from repeatedly switching on and off near the critical threshold, thereby improving the stability of the experience.

[0123] The screen projection processing method provided in this application embodiment further includes, if the image quality enhancement of a certain frame is not completed within a limited time, directly using the original frame and terminating the super-resolution process of that frame. This sacrifices the image quality of a single frame in exchange for overall screen projection smoothness, avoiding global stuttering caused by single-frame processing timeouts, and adapting to scenarios with high real-time requirements for in-vehicle systems.

[0124] The screen projection processing method provided in this application embodiment also includes maintaining a lightweight hash for the results after block processing; that is, when the screen is a still frame or has slight local changes, the processing result of the previous frame is reused, and the cache is controlled by the least recently used algorithm to prevent the video memory from expanding, thereby significantly reducing the GPU computing power and bandwidth usage (no need to repeat processing in still frame scenes), avoiding excessive cache usage of hardware resources, and balancing performance and storage costs.

[0125] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0126] like Figure 6 As shown, embodiments of this application also provide a screen projection processing device, which includes: The acquisition module 601 is used to acquire the original frame to be cast in response to the screen casting request, and to determine the target region of interest from the original frame to be cast. Processing module 602 is used to perform image super-resolution or image enhancement on the target region of interest based on the initial resolution and the target resolution of the target region of interest, so as to obtain the target projection display data of the target region of interest, wherein the target resolution is the resolution of the target region of interest when it is displayed on the projection receiving end; Display module 603 is used to display the target region of interest on the projection receiving end based on the target projection display data.

[0127] In one specific embodiment, the processing module 602 is specifically used to calculate the scaling factor from the initial resolution to the target resolution; if the scaling factor is less than a first threshold or greater than a second threshold, then an image super-resolution network is used to perform super-resolution reconstruction on the target region of interest to obtain the target projection display data; the second threshold is greater than the first threshold; if the scaling factor is greater than or equal to the first threshold and less than or equal to the second threshold, then image enhancement processing is used to perform same-resolution image quality enhancement on the target region of interest to obtain the target projection display data.

[0128] In one specific embodiment, the acquisition module 601 is specifically used for: extracting the projection layer data corresponding to the target application from the original projection frame to generate an initial region of interest; acquiring the initial aspect ratio of the initial region of interest and the size information of the display window of the projection receiving end; adjusting the aspect ratio of the initial region of interest according to the initial aspect ratio and the size information of the display window to generate a first initial region of interest matching the display window; if there is a black border invalid region in the first initial region of interest, then removing the black border invalid region in the first initial region of interest to generate the target region of interest; if there is no black border invalid region in the first initial region of interest, then using the first initial region of interest as the target region of interest.

[0129] In one specific embodiment, the acquisition module 601 is specifically used to acquire the application corresponding to the content displayed in the original screen-to-cast frame, and acquire the application identifier corresponding to the application; determine whether the application identifier belongs to the target application in the preset application list; if so, extract the screen-casting layer data corresponding to the target application from the original screen-to-cast frame to generate the initial region of interest.

[0130] In one specific embodiment, the processing module 602 is specifically used to divide the target region of interest into multiple display data blocks; adjacent blocks in the multiple display data blocks are provided with overlapping regions; each display data block is input into the image super-resolution network for processing to obtain the corresponding target display data block; in the overlapping region, the target display data blocks are merged and combined by a weight fusion method based on a preset window function to generate the target projection display data.

[0131] In one specific embodiment, the processing module 602 is specifically used to divide the target region of interest into multiple display data blocks; an overlapping region is set between adjacent blocks in the multiple display data blocks; image enhancement processing is performed on the multiple display data blocks respectively to obtain the corresponding target display data blocks; in the overlapping region, the target display data blocks are merged and combined by a weighted fusion method based on a preset window function to generate the target projection display data.

[0132] In one specific embodiment, the processing module 602 is further configured to obtain other projection display data corresponding to other regions in the original projection frame other than the target region of interest; the other projection display data includes at least one of system user interface display data, status bar display data, and display data corresponding to applications not in the preset application list; and display the other regions on the projection receiving end based on the other projection display data.

[0133] Figure 7 This is a schematic diagram of the structure of a vehicle provided in an embodiment of this application.

[0134] For example, such as Figure 7 As shown, the vehicle includes a memory 701 and a processor 702. The memory 701 stores executable program code 7011, and the processor 702 is used to call and execute the executable program code 7011 to perform the screen projection display processing method.

[0135] This embodiment can divide the vehicle into functional modules according to the above method example. For example, each function can be assigned to a separate module, or two or more functions can be integrated into a single processing module. The integrated module can be implemented in hardware. It should be noted that the module division in this embodiment is illustrative and represents only one logical functional division; other division methods may be used in actual implementation. When dividing the functional modules according to their respective functions, the vehicle may include: an acquisition module 601, a processing module 602, a display module 603, etc. It should be noted that all relevant content of each step involved in the above method embodiment can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0136] The vehicle provided in this embodiment is used to execute the above-described screen projection processing method, and thus can achieve the same effect as the above-described implementation method.

[0137] When using integrated units, the vehicle may include a processing module and a storage module. The processing module is used to control and manage the vehicle's actions. The storage module supports the vehicle in executing program code and data.

[0138] The processing module may be a processor or a controller, which can implement or execute various exemplary logic blocks, modules, and circuits as disclosed in this application. The processor may also be a combination of computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc., and the storage module may be a memory.

[0139] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above-described screen projection processing method embodiments.

[0140] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described screen projection processing method embodiments when it runs.

[0141] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0142] The embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described screen projection processing method embodiments.

[0143] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described screen projection processing method embodiments.

[0144] The beneficial effects of the above embodiments can be referred to the beneficial effects of the corresponding methods provided above, and will not be repeated here.

[0145] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0146] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0147] In the description of this application, it should be understood that if the terms "upper", "lower", "front", "rear", "left" and "right" are used to indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the position or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0148] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0149] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A screen projection display processing method, characterized in that, Applied to the screen projection output layer, including: In response to a screen mirroring request, the original screen mirroring frame is obtained, and the target region of interest is determined from the original screen mirroring frame; Based on the initial resolution and target resolution of the target region of interest, image super-resolution or image enhancement is performed on the target region of interest to obtain the target projection display data of the target region of interest. The target resolution is the resolution of the target region of interest when it is displayed on the projection receiving end. Based on the target projection display data, the target region of interest is displayed on the projection receiving end.

2. The method according to claim 1, characterized in that, The step of performing image super-resolution or image enhancement on the target region of interest based on the initial resolution and target resolution of the target region of interest to obtain target projection display data of the target region of interest includes: Calculate the scaling factor from the initial resolution to the target resolution; If the scaling factor is less than a first threshold or greater than a second threshold, an image super-resolution network is used to perform super-resolution reconstruction of the target region of interest to obtain the target projection display data; the second threshold is greater than the first threshold. If the scaling factor is greater than or equal to the first threshold and less than or equal to the second threshold, then image enhancement processing is used to enhance the image quality of the target region of interest at the same resolution in order to obtain the target projection display data.

3. The method according to claim 1, characterized in that, Determining the target region of interest from the original frame to be projected includes: Extract the projection layer data corresponding to the target application from the original frame to be projected to generate an initial region of interest; Obtain the initial aspect ratio corresponding to the initial region of interest, and the size information of the display window of the projection receiver; Based on the initial aspect ratio and the size information of the display window, the initial region of interest is adjusted proportionally to generate a first initial region of interest that matches the display window. If there is an invalid region with a black border in the first initial region of interest, then the invalid region with a black border in the first initial region of interest is removed, and the target region of interest is generated; If the black-bordered invalid region does not exist in the first initial region of interest, then the first initial region of interest is taken as the target region of interest.

4. The method according to claim 3, characterized in that, The step of extracting the projection layer data corresponding to the target application from the original frame to be projected to generate an initial region of interest includes: Obtain the application corresponding to the content displayed in the original screen frame to be projected, and obtain the application identifier corresponding to the application; Determine whether the application identifier belongs to the target application in the preset application list; If so, the projection layer data corresponding to the target application is extracted from the original projection frame to generate the initial region of interest.

5. The method according to claim 2, characterized in that, The step of using an image super-resolution network to perform super-resolution reconstruction of the target region of interest to obtain the target projection display data includes: The target region of interest is divided into multiple display data blocks; adjacent blocks in the multiple display data blocks have overlapping areas. Each of the display data blocks is input into the image super-resolution network for processing to obtain the corresponding target display data block; In the overlapping area, the target display data is divided into blocks and merged using a weighted fusion method based on a preset window function to generate the target projection display data.

6. The method according to claim 2, characterized in that, The step of performing image enhancement processing on the target region of interest at the same resolution to obtain the target projection display data includes: The target region of interest is divided into multiple display data blocks; adjacent blocks in the multiple display data blocks have overlapping areas. Image enhancement processing is performed on each of the multiple display data blocks to obtain the corresponding target display data blocks; In the overlapping area, the target display data is divided into blocks and merged using a weighted fusion method based on a preset window function to generate the target projection display data.

7. The method according to claim 1, characterized in that, The method further includes: Obtain other projection display data corresponding to areas other than the target region of interest in the original frame to be projected; the other projection display data includes at least one of system user interface display data, status bar display data, and display data corresponding to applications not in the preset application list. The other areas are displayed on the screen projection receiver based on the other screen projection display data.

8. A screen projection display processing device, characterized in that, include: The acquisition module is used to acquire the original frame to be cast in response to the screen casting request, and to determine the target region of interest from the original frame to be cast; The processing module is used to perform image super-resolution or image enhancement on the target region of interest based on the initial resolution and the target resolution, to obtain the target projection display data of the target region of interest, wherein the target resolution is the resolution of the target region of interest when it is displayed on the projection receiving end; The display module is used to display the target region of interest on the projection receiving end based on the target projection display data.

9. A vehicle, characterized in that, Includes the screen projection display processing device as described in claim 8.

10. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the screen projection processing method as described in any one of claims 1 to 7 when executing the computer program.

Images