Image processing method and system for LED special-shaped screen

Abstract

The invention discloses an image processing method and system for an LED special-shaped screen, and belongs to the technical field of image processing, and the method comprises the steps: carrying out the semantic segmentation of an input image through a convolutional neural network, and generating a feature thermodynamic diagram containing visual attention scores; generating a dynamic vector grid on the screen physical topological structure to obtain image resolution adjustment data; performing phase compensation calculation on the bent part of the edge of the dynamic vector grid to obtain phase compensation data; environment parameters are monitored in real time, a brightness-color temperature compensation matrix is dynamically generated, and environment compensation data are obtained; and rendering by using the image resolution adjustment data, the phase compensation data and the environment compensation data to obtain a final image. According to the method, semantic segmentation is performed by using the convolutional neural network, the image resolution is dynamically adjusted on the LED special-shaped screen according to the visual attention score, and image quality optimization and energy efficiency improvement are realized in combination with phase compensation and environment monitoring.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an image processing method and system for an LED special-shaped screen. Background Art

[0002] With the rapid development of LED display technology, irregular-shaped LED display screens, such as arc screens, wave screens, spherical screens, etc., are increasingly widely used in fields such as architectural decoration, stage performances, virtual reality, etc. Compared with traditional rectangular screens, special-shaped screens can achieve unique visual presentation effects through free splicing, but also face multiple technical bottlenecks.

[0003] Currently, although special-shaped screens are increasingly widely used, due to their irregular shape characteristics, the image processing technology of traditional rectangular screens cannot be directly applied. Specifically, when the picture is projected onto a curved screen, geometric distortions such as stretching and twisting will occur, seriously affecting the visual effect. At the same time, the pixel compensation accuracy in the splicing gap area of the special-shaped screen is insufficient, and manual intervention is often required for adjustment, which is time-consuming and highly dependent on experience, greatly reducing work efficiency. In addition, conventional LED screens render the entire picture at a fixed resolution, resulting in unreasonable resource allocation. High-precision rendering in secondary areas wastes a large amount of computing power, while blurring will occur in high-motion areas due to insufficient resolution, affecting the viewing experience. The solutions in the industry mainly improve clarity through super-resolution algorithms, but cannot achieve regional dynamic optimization and cannot meet the requirements in different scenarios. Summary of the Invention

[0004] To solve the above problems, the present invention provides an image processing method and system for an LED special-shaped screen, which uses a convolutional neural network for semantic segmentation, dynamically adjusts the image resolution on the LED special-shaped screen according to the visual attention score, and combines phase compensation and environmental monitoring to achieve image quality optimization and energy efficiency improvement.

[0005] The above object can be achieved by the following solutions: An image processing method for an LED special-shaped screen, including: performing semantic segmentation on an input image through a convolutional neural network to generate a feature heat map containing visual attention scores; generating a dynamic vector grid on the physical topology structure of the LED screen body according to the attention scores of each region in the feature heat map to obtain image resolution adjustment data; performing phase compensation calculation on the curved part of the edge of the dynamic vector grid to obtain phase compensation data; monitoring environmental parameters in real time and dynamically generating a brightness-color temperature compensation matrix to obtain environmental compensation data; using the image resolution adjustment data, the phase compensation data, and the environmental compensation data for rendering to obtain the final image.

[0006] Optionally, the semantic segmentation of the input image by the convolutional neural network to generate a feature heat map including visual attention scores includes: parallel processing of the input image by two branches of the convolutional neural network; the first branch outputs a pixel-level mask marking the key areas of the image; the second branch generates a human eye fixation probability distribution map, and the probability distribution map predicts the probability values of the visually attended areas; the mask and the probability distribution map are weighted and fused to generate a primary heat map; the primary heat map is smoothed by applying non-uniform Gaussian filtering to obtain a feature heat map.

[0007] Optionally, generating a dynamic vector grid on the physical topology of the LED screen according to the attention scores of each area in the feature heat map includes: in the area where the attention score is greater than a preset first threshold, constructing a first grid with a single pixel point as a vertex; in the area where the attention score is greater than a preset second threshold and less than or equal to the first threshold, gradually reducing the vertex density according to a tapering function, and constructing a second grid according to the vertices with reduced density; in the area where the attention score is less than or equal to the second threshold, extracting key pixel points as vertices to construct a third grid; merging the first grid, the second grid, and the third grid to obtain a dynamic vector grid; adjusting the resolution according to the number of vertices of the dynamic vector grid.

[0008] Optionally, extracting key pixel points as vertices to construct a third grid includes: obtaining the pixel point coordinates in the area where the attention score is less than or equal to the second threshold, and constructing a coordinate set; inputting the coordinate set into the SURF algorithm to extract key pixel points as vertices; calculating the distance values between adjacent vertices and sorting them in ascending order; selecting the two vertices corresponding to the first sorted distance value for merging, and constructing an initial grid using the merged vertices; stopping merging vertices when the area of the initial grid is greater than or equal to a preset multiple of the area of the first grid to obtain a third grid.

[0009] Optionally, performing phase compensation calculation on the curved part of the edge of the dynamic vector grid to obtain compensation data includes: performing brightness mixing adjustment in the edge area where the screen curvature is greater than a preset threshold; calculating the horizontal offset of the display position of each pixel point in real time according to the curvature of the screen surface and the light and darkness of the surrounding environment; continuously monitoring the change in the brightness contrast ratio in the picture and dynamically adjusting the brightness curve; integrating the brightness ratio, the horizontal offset, and the brightness curve to obtain compensation data.

[0010] Optionally, the brightness mixing adjustment in the edge area where the screen bending radian is greater than the preset threshold includes: splitting the input image data into three independent channels of R / G / B; performing linear interpolation in each channel to preliminarily adjust the brightness value to obtain the brightness data in the RGB space; converting the brightness data in the RGB space to the logarithmic space for color enhancement to obtain the brightness data in the logarithmic space; generating mixed brightness data using the brightness data in the RGB space and the brightness data in the logarithmic space; and performing brightness adjustment using the mixed brightness data.

[0011] Optionally, the generating of the mixed brightness data using the brightness data in the RGB space and the brightness data in the logarithmic space includes: performing weighted addition on the brightness data in the RGB space and the brightness data in the logarithmic space to obtain the mixed brightness data. For the mixed brightness data , there is , wherein, is the mixing weight coefficient, is the brightness data in the RGB space, is the brightness data in the logarithmic space; adjusting the mixing weight coefficient according to the ambient light change. For the mixing weight coefficient , there is , wherein, is the illuminance lux value, is the exponential function.

[0012] Optionally, the real-time monitoring of the environmental parameters, dynamically generating the brightness-color temperature compensation matrix, and obtaining the environmental compensation data includes: real-time monitoring of the light change rate through the built-in environmental sensor; when the light change rate is greater than the preset change rate, calculating the screen brightness compensation value and the color temperature compensation value to obtain the environmental compensation data; wherein, the brightness compensation value is less than or equal to the preset compensation value.

[0013] Optionally, the rendering using the phase compensation data and the environmental compensation data to obtain the final image includes: detecting the moving area and the static area of the image and calculating the proportion of the moving area; when the proportion of the moving area is less than the preset ratio and the number of moving areas of the image is greater than the preset number value, stopping generating the dynamic vector grid; and rendering each area of the image with a unified resolution to obtain the final image.

[0014] Based on the same inventive concept, the present invention also provides an image processing system for an LED special-shaped screen, the system comprising: a feature heat map generation module, configured to perform semantic segmentation on an input image through a convolutional neural network to generate a feature heat map including visual attention scores; a resolution adjustment module, configured to generate a dynamic vector grid on the physical topology of the LED screen body according to the attention scores of each region in the feature heat map to obtain image resolution adjustment data; a phase compensation module, configured to perform phase compensation calculation on the curved portions of the edges of the dynamic vector grid to obtain phase compensation data; an environment compensation module, configured to monitor environment parameters in real time and dynamically generate a brightness-color temperature compensation matrix to obtain environment compensation data; and an image rendering module, configured to perform rendering by using the image resolution adjustment data, the phase compensation data, and the environment compensation data to obtain a final image.

[0015] Compared with the prior art, the present invention has the following advantages: 1. The present invention performs semantic segmentation on an input image through a convolutional neural network to generate a feature heat map including visual attention scores, and dynamically generates a vector grid according to the feature heat map, thereby realizing fine adjustment of the image resolution; regions with high attention will adopt a higher resolution to ensure clear details; while regions with low attention can appropriately reduce the resolution to save resources; 2. The present invention performs phase compensation calculation on the curved portions of the edges of the LED special-shaped screen, which can effectively improve the display quality of the image at the curved edges, reduce distortion and blurring phenomena, and make the image smoother and more natural; 3. The present invention monitors environment parameters such as the light change rate in real time through a built-in environment sensor, and calculates the screen brightness compensation value and the color temperature compensation value to adapt to different environmental light conditions; 4. The present invention uses a convolutional neural network for image semantic segmentation, and performs parallel processing on the input image through two branches to generate a feature heat map including visual attention scores. This semantic segmentation method can accurately identify the key regions and background regions in the image, providing strong support for subsequent image processing; the efficient semantic segmentation makes the image processing more intelligent and precise, can more accurately capture the visual attention points of the audience, and thus optimizes the display effect of the image; 5. The present invention intelligently switches the processing mode through the detection of the proportion of the moving area, stops the unified resolution of the dynamic grid when the moving area is small, reduces the calculation burden, and improves the processing speed.

[0016] Other features and advantages of the present invention will be described in the subsequent specification, and, in part, will be obvious from the specification, or will be understood by implementing the present invention. The objectives and other advantages of the present invention can be realized and obtained by the structures pointed out in the specification, the claims, and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0018] Figure 1 It is a schematic flowchart of an image processing method for an LED special-shaped screen according to an embodiment of the present invention.

[0019] Figure 2 It is an execution flowchart of an image processing method for an LED special-shaped screen according to an embodiment of the present invention.

[0020] Figure 3 It is a schematic structural diagram of a dynamic vector grid according to an embodiment of the present invention.

[0021] Figure 4 It is a schematic structural diagram of an image processing system for an LED special-shaped screen according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0023] Referring to Figure 1 , an embodiment of the present invention proposes an image processing method for an LED special-shaped screen, which uses a convolutional neural network for semantic segmentation, dynamically adjusts the image resolution on the LED special-shaped screen according to the visual attention score, and combines phase compensation and environmental monitoring to achieve image quality optimization and energy efficiency improvement.

[0024] The method of this embodiment specifically includes: Performing semantic segmentation on the input image through a convolutional neural network to generate a feature heat map containing visual attention scores; Specifically, the semantic information of the image and the human eye visual attention area are extracted through a convolutional neural network (CNN) to generate a heat map reflecting the importance of different areas.

[0025] According to the attention scores of each area in the feature heat map, a dynamic vector grid is generated on the physical topology structure of the LED screen body to obtain image resolution adjustment data; Specifically, according to the attention score of the heat map, computing resources are dynamically allocated on the physical topology of the LED special-shaped screen. Different-density grids are merged into continuous vector grids, and the local rendering resolution is dynamically adjusted according to the number of vertices. In high-attention areas, single pixels are used as vertices to construct dense grids (maintaining the original resolution). In medium-attention areas, a tapering function is used to reduce the vertex density (such as a quadratic function to reduce the number of vertices). In low-attention areas, the SURF algorithm is used to extract key points as vertices, and adjacent points are merged to construct sparse grids (reducing the resolution).

[0026] Perform phase compensation calculation on the curved parts of the edges of the dynamic vector grid to obtain phase compensation data; Specifically, this step mainly solves the geometric distortion and brightness distortion of the curved edges of the LED special-shaped screen. According to the screen curvature radius and the light incident angle, calculate the pixel position offset (for example, the smaller the curvature radius, the larger the horizontal offset). RGB linear interpolation enhances the details in the dark areas, logarithmic space enhances the colors in the bright areas, and weighted mixing is adjusted. When the ambient light intensity is high (strong sunlight), reduce the logarithmic space weight to avoid overexposure.

[0027] Real-time monitor the environmental parameters, dynamically generate a brightness-color temperature compensation matrix to obtain environmental compensation data; Specifically, the screen display parameters are adjusted in real time according to the ambient light change, and built-in sensors (such as photoresistors) monitor the light change rate (such as more than 10% change per second).

[0028] Use the image resolution adjustment data, phase compensation data, and environmental compensation data for rendering to obtain the final image.

[0029] Exemplarily, such as Figure 2As shown, assume that the LED special-shaped screen is an arc-shaped display wall for playing a live basketball game, and it is necessary to optimize the picture quality and reduce the computational load. First, a feature heat map will be generated. The image is input into a convolutional neural network (CNN). The semantic branch marks the players and the basket as key areas. The visual attention branch predicts that the audience's line of sight focuses on the moving players and the basketball. In the heat map, the attention score in the player area is 0.9 (high), and the score in the audience area is 0.2 (low). Then, a dynamic vector grid will be constructed. Each LED pixel point in the player area is used as a vertex, maintaining a 1080P resolution. The vertex interval of the venue background (medium attention) gradually increases from 2 pixels to 8 pixels, and the resolution drops to 720P. For the audience seats (low attention), SURF extracts edge points (about 1 / 20 pixel), and adjacent points are merged to generate a sparse grid, and the resolution drops to 480P. At the same time, phase compensation will be carried out. For example, the advertising text on the arc-shaped edge of the screen (curvature radius 5m) is distorted due to bending. The horizontal coordinate offset is calculated (such as 3 pixels to the right for the left-edge pixels). After brightness mixing (RGB weight 0.7, logarithmic space 0.3), text stretching and blurring are avoided. At the same time, environmental compensation will be carried out. For example, when the on-site lights suddenly increase (such as when a spotlight is turned on), the sensor detects that the light change rate is 50%, the brightness compensation increases by 20% (from 800nits to 960nits), and the color temperature changes from 5000K to 5500K. Finally, the picture will be rendered according to the above parameters. If 95% of the picture is the static area when there is a pause, the rendering mode will be switched to stop generating the dynamic grid and uniformly render the full screen at 720P to save GPU computing power.

[0030] Optionally, semantic segmentation of the input image by a convolutional neural network to generate a feature heat map including visual attention scores includes: Parallel processing of the input image through two branches of the convolutional neural network; Specifically, simultaneously capture the semantic key areas (object contours) and visual attention focuses (human gaze tendencies) in the image to make up for the limitations of a single method.

[0031] The first branch outputs a pixel-level mask marking the key areas of the image; Specifically, the network structure is based on U-Net or DeepLabv3+, and a pixel-level mask (0 / 1 binary map) is output; the first branch accurately segments the main targets of fixed categories (such as faces, vehicles, text) to ensure the integrity of important object areas.

[0032] The second branch generates a human eye fixation probability distribution map, and the probability distribution map predicts the probability values of the visually attended areas; Specifically, the network structure is an encoder using VGG-like or ResNet connected to a probability regression layer, and a probability map from 0 to 1 is output; the second branch simulates human gaze behavior and predicts dynamic high-attention areas (such as moving objects, high-contrast edges).

[0033] Perform weighted fusion of the mask and the probability distribution map to generate a primary heat map; Specifically, combine rigid segmentation and flexible attention prediction to avoid missing unstructured concerns (such as flames and flashing lights) in pure semantic segmentation; perform morphological dilation (such as dilating by 3 pixels) on the semantic mask to cover the object edge transition area; among them, the attention score , where is the mask value (0 or 1), is the fixation probability, is the weight coefficient of the mask value, is the weight coefficient of the fixation probability; the weight coefficients are optimized according to historical data.

[0034] Apply non-uniform Gaussian filtering to smooth the primary heat map to obtain a feature heat map.

[0035] Specifically, this step is mainly used to eliminate artificial boundaries (such as sharp edges of the mask) and noise points in the fused heat map, while maintaining the sharpness of key regions; adjust the standard deviation according to the local gradient of the heat map , for the standard deviation there is , where is the base standard deviation, is a constant coefficient; regions with large gradients (such as object edges) use small kernels to preserve details; flat regions use large kernels for smooth transition; the filtering process is to calculate the heat map gradient, dynamically generate the filtering kernel, filter pixel by pixel, and normalize to the [0,1] interval.

[0036] Exemplarily, assume that the input image is a dynamic scene of a red sports car drifting on a bend, the image contains high-speed movement, body reflection, and background blur; first, the first branch accurately segments the outline of the sports car, but because the body reflection area is not defined as a category, the reflective parts such as the windows are ignored by the mask; the second branch detects that the human eye fixation points are concentrated on the headlights (highlighted area) and tire smoke (motion blur area), and generates a prediction probability map; dilate the semantic mask by 2 pixels to fill the gaps between the wheels, and after weighted fusion, the body is complete and the smoke area is enhanced; there are jagged noises around the headlights in the initial heat map before filtering, and after non-uniform Gaussian filtering, the transition between the headlights and the smoke is natural, and the background noise points are eliminated, so as to output the final feature heat map. The attention scores in the sports car body area in the figure are 0.8 to 0.9, the attention scores of the headlights and the smoke are 0.7 to 0.8; the attention score of the background trees is 0.3.

[0037] Exemplarily, the first branch semantic segmentation can ensure that advertising trademarks (rigid demands) are not lost, and the second branch visual attention model can capture the highlights of promotional texts (flexible demands); when the dynamic content is dominant in the application scenario, such as sports events, the weight coefficient of the fixation probability can be adjusted to increase the proportion of the probability map; if edge preservation is required for medical imaging, the constant coefficient needs to be reduced to avoid over-smoothing the lesion area.

[0038] Optionally, as Figure 3 shown, generating a dynamic vector grid on the physical topology of the LED screen body according to the attention scores of each region in the feature heat map includes: In the region where the attention score is greater than a preset first threshold, a first grid is constructed with a single pixel point as a vertex; Specifically, according to the difference in the distribution of human eye attention, computing resources are dynamically allocated, details are retained in high-score regions, and the computing load is reduced in low-score regions; the first threshold is used to divide high-attention regions, indicating that fine rendering is required; at this time, the original resolution is retained, the vertices are equal to the pixels, and the vertex density reduction ratio is 0, which is mainly applicable to advertising subjects, human faces, etc.

[0039] In the region where the attention score is greater than a preset second threshold and less than or equal to the first threshold, the vertex density is gradually reduced according to a tapering function, and a second grid is constructed according to the vertices after the density reduction; Specifically, this step is mainly used to achieve a smooth change in vertex density in the medium-attention region to avoid visual discontinuity caused by a sudden drop in resolution; the tapering function can use a quadratic function or an exponential decay function. For example, a quadratic function can be used, such as: , where is the attention score, is the first threshold, is the second threshold; the density value is converted into a vertex interval step size, such as converting a density of 0.7 to taking 1 vertex every 2 pixels.

[0040] In the region where the attention score is less than or equal to the second threshold, key pixel points are extracted as vertices to construct a third grid; Specifically, this step is mainly used to retain structural feature points in the low-attention region to prevent contour loss caused by over-simplification.

[0041] The first grid, the second grid, and the third grid are merged to obtain a dynamic vector grid; The resolution is adjusted according to the number of vertices of the dynamic vector grid.

[0042] Specifically, by aligning the edge vertices of the first grid, the second grid, and the third grid, the boundary vertices of the medium- and low-density regions are forced to match the high-density region; the empty circle property detection can be performed on the merged vertex set using Delaunay triangulation to generate a continuous grid; for example, the vertex spacing in the high-density region is 1 pixel, the spacing in the medium-density region is 3 pixels, and transitional triangles are generated at the junction.

[0043] Exemplarily, taking an LED stage screen of a certain concert as an example, the main screen displays a close-up of the singer (high attention), stage lighting effects (medium attention), and background starry sky animation (low attention); first, the area is divided. Assuming the first threshold is 0.8, the second threshold is 0.5, the attention score of the singer's face is 0.9, the attention score of the lighting beam is 0.6, and the attention score of the starry sky background is 0.3; then, for the high-attention area of the singer's face, vertices will be generated pixel by pixel with a vertex spacing of 1 pixel; for the medium-attention area of the lighting beam, the vertex spacing will be set according to the tapering function with a vertex spacing of 3 pixels; for the low-attention area of the starry sky background, the center of the star points will be extracted using SURF as key points.

[0044] Optionally, extracting key pixel points as vertices to construct the third grid includes: Obtaining the pixel point coordinates in the area where the attention score is less than or equal to the second threshold to construct a coordinate set; Specifically, extracting the original pixel coordinates of the low-attention area (attention score less than or equal to the second threshold) provides input data for subsequent algorithm processing; traversing all pixel points in the feature heat map with attention scores not exceeding the second threshold and recording their coordinates to form a set.

[0045] Inputting the coordinate set into the SURF algorithm to extract key pixel points as vertices; Specifically, the SURF algorithm is an efficient feature point detection algorithm, which is good at extracting stable key points in images (such as corner points, edge intersection points, etc.); by using SURF, pixel points (key vertices) with significant local structures are screened out from the low-attention area, redundant points are removed, and the complexity of the subsequent grid is reduced.

[0046] Calculating the distance values between adjacent vertices and sorting them in ascending order; Specifically, based on the key vertex coordinates, the Euclidean distances between all adjacent vertices are calculated and sorted in ascending order; this provides a basis for vertex merging - giving priority to merging the two vertices with the closest distance, thereby gradually simplifying the grid.

[0047] Selecting the two vertices corresponding to the distance value ranked first for merging and using the merged vertices to construct the initial grid; Specifically, merge the two vertices with the smallest distance after sorting (for example, take their geometric center as the new vertex) to reduce the number of vertices; use Delaunay triangulation or a similar algorithm to construct an initial mesh based on the merged vertex set.

[0048] Stop merging vertices when the area of the initial mesh is greater than or equal to a preset multiple of the area of the first mesh to obtain the third mesh.

[0049] Specifically, stop merging when the area of the initial mesh is greater than or equal to a preset multiple of the area of the first mesh (for example, it can be 3 times); ensure that the mesh area in the low-concern area is expanded to a specified ratio, reduce the resolution by reducing the vertex density, and at the same time retain sufficient structural information.

[0050] Exemplarily, assume that a football game is being played on the curved LED screen of a stadium, and the non-concern areas of the screen are the distant stands and the blurred background billboards (the attention score is less than or equal to 0.3, and the second threshold is set to 0.4). It is necessary to construct a third mesh in this area to reduce the resolution to save computing resources; first extract the pixel points with an attention score less than or equal to 0.4 to obtain a coordinate set. For example, the original pixels in this area are 100×100, a total of 10,000 points; input the coordinate set into the SURF algorithm to detect local feature points (such as the edges of billboards and the outlines of the audience), and output key vertices (for example, from 10,000 points to 500 key points); calculate the adjacent distances between the 500 key points, generate a distance list and sort it in ascending order. Assume that the two vertices with the smallest distance of 2 pixels are A and B; merge A and B into vertex C (the coordinates are the midpoint of A and B), replace the original A and B with C, recalculate the adjacent distances and update the sorting; repeat the above steps, each time merging the vertex pair with the smallest current distance; assume that the area of the first mesh, that is, the single-pixel mesh in the high-concern area, is the area of 1 pixel, and the preset multiple is 100 times, then the termination condition is that the area of the third mesh is greater than or equal to the area of 100 pixels; when the area of the mesh unit formed by the merged vertices reaches the area of 100 pixels (for example, the number of vertices drops from 500 to about 50), stop merging; the original area pixel density is 10,000 points, and the number of vertices of the processed third mesh is 50 points, and the area is expanded to 100 times. When displaying, each mesh unit is rendered as a 10×10 pixel block, and the resolution drops from the original 1080P to about 100P, but the key structures (such as the outlines of advertising texts) are still retained; reduce the computing load and improve the rendering efficiency by merging redundant vertices, extract feature points based on SURF, and ensure that the visually key information (such as text edges and dynamic object outlines) in the low-concern area is not lost; control the degree of mesh sparsity through the area threshold, and can be flexibly adapted to different scenario requirements (such as adjusting the preset multiple).

[0051] Optionally, perform phase compensation calculation on the curved part of the edge of the dynamic vector mesh to obtain compensation data including: Perform brightness mixing adjustment in the edge area where the screen bending curvature is greater than the preset threshold; Specifically, this step is used to identify the area with severe optical distortion caused by bending in the LED special-shaped screen. Compare the preset curvature radius threshold in the screen physical model (for example, the bending radius less than 5m is the high curvature area). When the curvature corresponding radian exceeds the threshold, trigger the compensation operation; According to the bending degree of the screen curved surface and the light brightness of the surrounding environment, calculate the horizontal offset of the display position of each pixel point in real time; Specifically, due to the light refraction of the screen curved surface, there is a deviation between the actual display position of the pixel and the plane projection position; the position offset can be , where in the formula is the angle between the normal of the screen curved surface and the line of sight, is a constant coefficient, is the radius of curvature; the larger the curvature ( increases), and the greater the change in the incident angle of the ambient light, the greater the horizontal offset .

[0052] Continuously monitor the change in the light and dark contrast in the picture, and dynamically adjust the brightness curve; Specifically, adaptively adjust the brightness response curve according to the picture content (such as the scene with sudden change in light and dark contrast) to avoid the sudden change distortion perceived by the human eye; by monitoring the local contrast (such as the brightness difference threshold between adjacent pixels is greater than 50nits), adjust the gamma curve parameters to smoothly transition the brightness difference.

[0053] Integrate the brightness ratio, horizontal offset and brightness curve to obtain compensation data.

[0054] Optionally, performing brightness mixing adjustment in the edge area where the screen bending curvature is greater than the preset threshold includes: Split the input image data into three independent channels of R / G / B; Specifically, mainly solve the color separation problem caused by the difference in light refraction at the bending edge (such as the red offset is greater than the blue); decompose the image into three independent channels of R, G, B, and can optimize the brightness compensation parameters of each color component respectively; for example, at the curved surface edge, the blue light is easily scattered due to its short wavelength and requires a higher interpolation compensation intensity.

[0055] Perform linear interpolation in each channel, initially adjust the brightness value, and obtain the brightness data in the RGB space; Specifically, restore the dark details lost due to geometric distortion in the bending area through local linear mapping; perform piecewise linear interpolation on the pixel gray values of each channel, for example, perform gray stretching on the dark part (pixel value 0 to 50).

[0056] Convert the RGB space luminance data to the logarithmic space for color enhancement to obtain the logarithmic space luminance data; Specifically, utilize the physiological characteristic that the human eye is sensitive to dark areas, expand the low luminance dynamic range, and convert the RGB luminance values to the logarithmic domain.

[0057] Generate mixed luminance data using the RGB space luminance data and the logarithmic space luminance data; Adjust the luminance using the mixed luminance data.

[0058] Specifically, according to the ambient light intensity, select the optimal linear and logarithmic space mixing ratio; recombine the mixed luminance data with the original chrominance data, output it to the LED special-shaped screen driving module, and control the actual display luminance of the LED lamp beads through PWM dimming or current regulation.

[0059] Optionally, generating mixed luminance data using the RGB space luminance data and the logarithmic space luminance data includes: Perform weighted addition on the RGB space luminance data and the logarithmic space luminance data to obtain the mixed luminance data. For the mixed luminance data , there is , wherein, is the mixing weight coefficient, is the RGB space luminance data, is the logarithmic space luminance data; Specifically, the luminance data restored by linear interpolation of the RGB space data can retain the details of the highlight area; the logarithmic space data enhances the low dynamic range (LDR) information of the dark area sensitivity through non-linear mapping; through the dynamic mixing weight coefficient, the balanced fusion of the two data is realized. The brighter the ambient light, the higher the weight of the RGB space data (retaining the highlight without overexposure); the darker the environment, the higher the weight of the logarithmic space data (enhancing the details of the dark part).

[0060] Adjust the mixing weight coefficient according to the change of ambient light. For the mixing weight coefficient , there is , wherein, is the illuminance lux value, is the exponential function.

[0061] Specifically, in a strong light environment such as outdoors during the day, RGB space data processed linearly is preferred to prevent the screen from overexposing and becoming white; in a low light environment such as indoors at night, logarithmic space data processed logarithmically is emphasized to fully amplify shadow details; if the denominator constant 200 is changed to 100, the sensitivity is equivalently increased, which is suitable for scenarios that require rapid response to environmental changes; by dynamically perceiving the ambient light and fusing cross-space data, the display quality of the curved screen under different lighting conditions is significantly improved, and the formula parameters are flexibly adjustable to adapt to diverse application scenarios.

[0062] Exemplarily, assume that a spherical LED screen (curvature radius 4m) in a certain mall is playing an aquarium advertisement, and it is necessary to compensate for the geometric distortion and brightness distortion of the fish swimming video at the edge of the lower hemisphere of the screen; the curvature radius corresponding to the curved arc of the lower hemisphere of the screen is 3.8m, which is less than the preset threshold of 5m, so it is determined as the area that needs to be compensated; linear interpolation is performed on the R / G / B channels of the input image respectively, and the dark gray value is stretched from [0, 50] to [0, 100] to restore the shadow details of the water plants; the RGB brightness is converted to the logarithmic domain, and exponential expansion is performed on the high-light area (greater than 200nits) to enhance the reflection of the scales of the blue fish; the ambient light in the mall is relatively strong (1000lux), and according to the weight formula, the calculated mixed weight coefficient is 0.27, that is, the weight of the RGB space is 0.27, and the weight of the logarithmic space is 0.73, to prevent the overall from being too bright after mixing; assume that the visual angle when the fish swim to the curved edge is 60°, the constant coefficient is 0.2, and the curvature radius is 3.8m, then the horizontal offset is 0.2 x 3.8 x 0.866 = 0.66, so the pixels at the edge of the fish body are actually shifted 0.66 pixels to the right during actual rendering to correct the visual position; it is detected that the brightness difference between the water plants and the background suddenly increases from 20nits to 120nits, and the gamma value is temporarily adjusted from 2.2 to 1.8, so that the dark part increases faster and the bright part increases slower, and the brightness jump is smoothly transitioned; the final output compensation parameters can be {brightness mixing data: [α = 0.27, RGB_log fusion matrix], horizontal offset: Δx = 0.66pixel, brightness curve: gamma = 1.8}.

[0063] Optionally, the environmental parameters are monitored in real time, and a brightness-color temperature compensation matrix is dynamically generated to obtain environmental compensation data including: The change rate of the ambient light is monitored in real time through the built-in environmental sensor; Specifically, based on the transient response of the ambient light sensor, the change rate of the light intensity per unit time is calculated; when the light changes strongly, the screen needs to be actively adjusted to ensure visual comfort.

[0064] When the change rate of the ambient light is greater than the preset change rate, the screen brightness compensation value and the color temperature compensation value are calculated to obtain the environmental compensation data; Among them, the brightness compensation value is less than or equal to the preset compensation value.

[0065] Specifically, the brightness compensation clipping rule is that the compensation step needs to satisfy that the compensation step is between 0 and the maximum compensation step; the principle of brightness compensation is to increase or decrease the overall screen brightness to match the ambient light intensity. For example, when the sun shines directly, the increment compensation is up to 800 nits; the principle of color temperature compensation is to adjust the RGB ratio (such as reducing the blue light ratio) to alleviate the color cast of the ambient light. For example, when the light is cold on a cloudy day, the color temperature is increased to 7500K; the color temperature compensation matrix is , where , , are color temperature compensation coefficients (associated with environmental parameters), , , are the three-channel color temperature adjustment weights.

[0066] Exemplarily, when the in-vehicle display suddenly encounters a strong light switch in a tunnel and the detected light change rate is greater than the preset change rate, the current screen brightness is calculated to be 400 nits and the target brightness is 50 nits at this time, and 350 nits need to be reduced; due to the limitation of the preset compensation value (assumed to be 80 nits), the maximum brightness compensation value is 80 nits, so a brightness compensation value of -80 nits is generated; after calculation, , , , so the output compensation data can be .

[0067] Optionally, using the phase compensation data and the environmental compensation data for rendering, the final image obtained includes: Detect the moving area and the static area of the image, and calculate the proportion of the moving area; Specifically, it is mainly used to distinguish the moving area (moving object) and the static area (background or static element) in the image; the optical flow method or the inter-frame difference method can be used to identify the moving pixels (such as the Farneback algorithm in OpenCV); the adjacent moving pixels are merged into independent moving areas through a clustering algorithm (such as DBSCAN).

[0068] When the proportion of the moving area is less than the preset ratio and the number of moving areas in the image is greater than the preset number value, stop generating the dynamic vector grid; Specifically, when there are a large number of scattered small-area moving areas (such as a flock of flying birds), the calculation cost of generating grids for each area is too high, and the global processing efficiency is higher; the rendering delay and data transmission overhead caused by excessive subdivision can be reduced.

[0069] The final image is obtained by rendering each area of the image at a unified resolution.

[0070] Specifically, for static regions and limited motion regions, full-frame uniform sampling is adopted to avoid jagged or blurred phenomena caused by multi-resolution stitching; the results of phase compensation (time-axis dynamic compensation) and environment compensation (brightness and color parameter adjustment) are uniformly applied to all regions.

[0071] Exemplarily, for the processing of a real-time aerial video of a drone, the input image data is an aerial video stream with a resolution of 4K. The picture contains grasslands, woods, and a group of scattered flying birds (10 birds); the pixel area of each bird accounts for approximately 0.3% of the total picture (single region), and the total motion region accounts for 3%; the number of motion regions is 10, the preset ratio is set to 5%, and the preset quantity value is 8. Therefore, the condition for stopping dynamic grid generation is met; at this time, the local vector grid interpolation for each bird is turned off (for example, originally planned to generate a 64×64 dynamic grid for each bird body); originally, 10×64² = 40,960 grid units needed to be processed, which is now reduced to 0, significantly reducing the GPU load; according to the jitter data provided by the on-board IMU (Inertial Measurement Unit), global phase translation compensation is performed on the entire image to correct the image offset caused by the drone's shaking; according to the light sensor data, the details of the dark parts of the entire image are enhanced, and the color temperature (7500K) offset due to cloudy weather is restored to the standard 6500K; all regions are rendered at the native 4K resolution, and no jagged edges appear at the edges of the bird flock due to block processing, and the texture details of the background trees are retained intact. The processing delay is reduced from 35ms to 22ms, meeting the real-time transmission requirements; through the dynamic region criterion plus global rendering strategy, the balance problem between rendering efficiency and quality in multi-target scenarios is solved, significantly improving the performance and visual effects of the real-time image processing system, and is applicable to high-dynamic vision scenarios such as drones, vehicle-mounted monitoring, AR / VR, etc.

[0072] Based on the same inventive concept, as Figure 4 shown, the present invention also provides an image processing system for an LED special-shaped screen, the system includes: A feature heat map generation module, configured to perform semantic segmentation on the input image through a convolutional neural network to generate a feature heat map containing visual attention scores; A resolution adjustment module, configured to generate dynamic vector grids on the physical topology structure of the LED screen according to the attention scores of each region in the feature heat map to obtain image resolution adjustment data; A phase compensation module, configured to perform phase compensation calculations on the curved parts of the edges of the dynamic vector grids to obtain phase compensation data; An environment compensation module, configured to monitor environmental parameters in real time, dynamically generate a brightness-color temperature compensation matrix to obtain environment compensation data; A picture rendering module, configured to perform rendering using the image resolution adjustment data, phase compensation data, and environment compensation data to obtain the final image.

[0073] Based on the same inventive concept, the present invention also provides a storage medium storing instructions executable by at least one processor. The instructions are executed by at least one processor to enable the at least one processor to execute the method as described above.

[0074] It should be noted that the electrical connections between the above-mentioned various units do not necessarily mean direct connection of the circuits. Indirect connection methods, as long as the object of the present invention is achieved, can be applied to the embodiments of the present invention. The above are only exemplary embodiments of the present invention and cannot be used to limit the scope of the present invention.

[0075] That is, any equivalent changes and modifications made in accordance with the teachings of the present invention still fall within the scope covered by the present invention. Those skilled in the art will readily think of other implementation schemes of the present invention after considering the disclosure of the specification. This application aims to cover any variations, uses or adaptive changes of the present invention, and these variations, uses or adaptive changes follow the general principles of the present invention and include common general knowledge or conventional technical means in the technical field not recorded in the present invention.

Claims

1. An image processing method for LED special-shaped screen, characterized in that: The method comprises: Perform semantic segmentation on the input image through a convolutional neural network to generate a feature heat map containing visual attention scores; According to the attention scores of each area in the characteristic heat map, a dynamic vector grid is generated on the physical topological structure of the LED screen to obtain image resolution adjustment data; Performing phase compensation calculation on the curved portion of the edge of the dynamic vector grid to obtain phase compensation data; Monitor environmental parameters in real time, dynamically generate brightness-color temperature compensation matrix, and obtain environmental compensation data; The image resolution adjustment data, the phase compensation data and the environment compensation data are used for rendering to obtain a final image.

2. The image processing method of LED special-shaped screen according to claim 1, characterized in that: The method of performing semantic segmentation on the input image through a convolutional neural network to generate a feature heat map containing visual attention scores includes: The input image is processed in parallel by two branches of the convolutional neural network; The first branch outputs a pixel-level mask marking the key areas of the image; The second branch generates a probability distribution map of human eye gaze, wherein the probability distribution map predicts the probability value of the visual attention area; Performing weighted fusion of the mask and the probability distribution map to generate a primary heat map; The primary thermal map is smoothed by applying non-uniform Gaussian filtering to obtain a characteristic thermal map.

3. The image processing method of LED special-shaped screen according to claim 2, characterized in that: Generating a dynamic vector grid on the physical topological structure of the LED screen according to the attention score of each area in the characteristic heat map includes: In the area where the attention score is greater than a preset first threshold, constructing a first grid with a single pixel point as a vertex; In the area where the attention score is greater than a preset second threshold and less than or equal to the first threshold, gradually reducing the vertex density according to a tapering function, and constructing a second mesh based on the vertices after the density is reduced; In the area where the attention score is less than or equal to the second threshold, extract key pixel points as vertices to construct a third grid; Merging the first grid, the second grid and the third grid to obtain a dynamic vector grid; The resolution is adjusted according to the number of vertices of the dynamic vector mesh.

4. The image processing method of LED special-shaped screen according to claim 3, characterized in that: Extracting key pixel points as vertices to construct the third grid includes: Obtaining the coordinates of the pixel points in the area where the attention score is less than or equal to the second threshold, and constructing a coordinate set; Input the coordinate set into the SURF algorithm to extract key pixel points as vertices; Calculate the distance values ​​between adjacent vertices and sort them in ascending order; Select the two vertices corresponding to the first-ranked distance values ​​and merge them, and use the merged vertices to construct the initial mesh; When the area of ​​the initial mesh is greater than or equal to a preset multiple of the area of ​​the first mesh, the merging of vertices is stopped to obtain a third mesh.

5. The image processing method of LED special-shaped screen according to claim 1, characterized in that: The performing phase compensation calculation on the curved portion of the edge of the dynamic vector grid to obtain compensation data comprises: Perform brightness mixing adjustment in the edge area where the curvature of the screen is greater than a preset threshold; According to the curvature of the screen surface and the brightness of the surrounding environment, the lateral offset of the display position of each pixel is calculated in real time; Continuously monitor the changes in the light and dark contrast in the picture and dynamically adjust the brightness curve; The brightness ratio, the lateral offset and the brightness curve are integrated to obtain compensation data.

6. The image processing method of LED special-shaped screen according to claim 5, characterized in that: The brightness mixing adjustment in the edge area where the curvature of the screen is greater than a preset threshold comprises: Split the input image data into three independent channels of R / G / B; Perform linear interpolation in each channel to preliminarily adjust the brightness value to obtain RGB space brightness data; Convert the RGB space brightness data to logarithmic space for color enhancement to obtain logarithmic space brightness data; Generate mixed brightness data using the RGB space brightness data and the logarithmic space brightness data; The mixed brightness data is used to perform brightness adjustment.

7. The image processing method of LED special-shaped screen according to claim 6, characterized in that: The generating mixed brightness data by using the RGB space brightness data and the logarithmic space brightness data comprises: The RGB space brightness data and the logarithmic space brightness data are weightedly added to obtain mixed brightness data. ,have ， In the formula, is the mixing weight coefficient, is the RGB space brightness data, is the logarithmic space brightness data; The mixing weight coefficient is adjusted according to the change of the ambient light. ,have ， In the formula, is the illuminance lux value, is an exponential function.

8. The image processing method of LED special-shaped screen according to claim 1, characterized in that: The real-time monitoring of environmental parameters, the dynamic generation of a brightness-color temperature compensation matrix, and the acquisition of environmental compensation data include: Real-time monitoring of light change rate through built-in environmental sensors; When the illumination change rate is greater than a preset change rate, a screen brightness compensation value and a color temperature compensation value are calculated to obtain environmental compensation data; The brightness compensation value is less than or equal to a preset compensation value.

9. The image processing method of LED special-shaped screen according to claim 1, characterized in that: The rendering using the phase compensation data and the environment compensation data to obtain a final image comprises: Detect the moving and static areas of the image and calculate the proportion of the moving area; When the motion area ratio is less than a preset ratio and the number of image motion areas is greater than a preset number, stopping the generation of the dynamic vector grid; Each area of ​​the image is rendered with a uniform resolution to obtain the final image.

10. An image processing system for LED special-shaped screens, used to implement the image processing method for LED special-shaped screens according to any one of claims 1 to 9, characterized in that: The system comprises: The feature heat map generation module is used to perform semantic segmentation on the input image through a convolutional neural network and generate a feature heat map containing visual attention scores; A resolution adjustment module, used to generate a dynamic vector grid on the physical topological structure of the LED screen according to the attention score of each area in the characteristic heat map, and obtain image resolution adjustment data; A phase compensation module, used for performing phase compensation calculation on the curved portion of the edge of the dynamic vector grid to obtain phase compensation data; Environmental compensation module, used to monitor environmental parameters in real time, dynamically generate brightness-color temperature compensation matrix, and obtain environmental compensation data; The image rendering module is used to perform rendering using the image resolution adjustment data, the phase compensation data and the environment compensation data to obtain a final image.

Images