Special-shaped screen video segmentation processing method and system
By identifying and analyzing the shape characteristics of the special-shaped screen, segmenting, splicing, defect detection and graphics separation of video images, and performing image distortion correction, it solves the problem that traditional video segmentation methods are difficult to adapt to the special-shaped screen, and achieves high-quality video display effects.
Patent Information
- Application Number
- CN202510039838.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-10
- Publication Date
- 2025-04-25
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Traditional video segmentation methods are difficult to accurately adapt to different shapes of special-shaped screens, resulting in problems such as image distortion, obvious splicing gaps, and color distortion.
By acquiring video signals from multiple video sources, analyzing video images, identifying and analyzing the screen shape characteristics of the target display device, segmentation processing, splicing processing, defect detection and graphics separation of video images, and real-time rendering and image distortion correction of the processed images.
It enables the adaptation and correct display of standard video content on the distinctive screen, ensuring that the video display effect remains high quality and smooth on complex-shaped screens.
Smart Images
Figure CN119887803A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of video segmentation processing, and in particular, relates to a method and system for segmenting and processing videos for special-shaped screens. Background Art
[0002] With the development of display technology, special-shaped screens with various shapes, such as curved screens and multi-faceted screens, have gradually been widely used in many fields such as televisions, smart phones, and virtual reality devices. The characteristics of special-shaped screens are mainly reflected in the complexity of screen display forms, the particularity of visual effects, the difficulty of manufacturing processes, and the need to adapt display content. Its appearance breaks the limitations of traditional rectangular display screens and brings a more immersive and diverse user experience. However, the screen forms of special-shaped screens are diverse, such as different curvatures and different numbers of display surfaces, which leads to the inability of traditional planar video processing methods to effectively adapt to these screen forms and it is difficult to achieve good visual effects and user experience. Especially when displaying large-scale, continuous images or video content, traditional video segmentation methods are difficult to accurately adapt to screens of different shapes, which may cause problems such as image distortion, obvious splicing gaps, and color distortion. This requires a new video segmentation processing method that can handle problems such as geometric deformation, content adaptation, and segmentation and splicing of special-shaped screens. Summary of the invention
[0003] The object of the present invention is to provide a method and system for processing video segmentation of a special-shaped screen, which can be achieved by the following technical solutions: In a first aspect, an embodiment of the present application provides a method for segmenting and processing a video of an irregular screen, comprising the following steps: Acquire video signals from multiple video sources, and parse the video signals to obtain corresponding video images; Identify and analyze the screen shape characteristics of the target display device; Performing segmentation processing, splicing processing, defect detection and image-text separation on the input video image according to the screen shape features; Rendering the processed image obtained after the video image processing in real time and displaying it on the target display device; Correcting image distortion generated when the processed image is displayed on the target display device; An interactive operation interface for interacting with the target display device is provided for the user, and the user is supported to adjust the display effect of the video image and the adaptation parameters of the screen shape features.
[0004] Preferably, parsing the video signal to obtain a corresponding video image includes: Performing signal synchronization on the video signals acquired from multiple video sources by using time stamp alignment and frame synchronization; Decoding the video signal using a video decoder to convert it into image frames for analysis and processing; Preprocessing the decoded image frame, including denoising, image enhancement and color correction; The image frames are analyzed and the analyzed images are fused to obtain the video image. Preferably, the identifying and analyzing the screen shape features of the target display device includes: Acquiring image information of the target display device through an imaging device; Using an edge detection algorithm to extract edge information from the image information; Based on the edge information, using a contour detection algorithm to identify contour information of the screen edge of the target display device; Performing shape feature analysis on the contour information to obtain the screen shape feature includes: Shape recognition: using the RANSAC algorithm to perform shape feature analysis on the contour information; Polygon fitting: using a polygon fitting algorithm to fit the contour information and find the best fitting rectangle; Corner point detection: using a corner point detection algorithm to determine the four corner points of the optimal fit rectangle and confirm the screen shape of the target display device; Shape feature extraction: Analyze the geometric properties of the screen shape, including aspect ratio, angle and area, and use the geometric properties to extract and verify the screen shape features.
[0005] Preferably, the segmentation process of the input video image according to the screen shape feature includes: The input video image is divided into a plurality of regions according to the screen shape feature, each region corresponding to a screen surface formed by the screen shape feature; wherein the video image content displayed on the display area of each screen surface matches the screen shape feature; Detect the screen surface angle of each screen surface and perform viewing angle correction for screen surfaces with different angles or curvature, and adjust the video image content using geometric transformation according to the position and angle of each screen surface; Mapping the processed video image content to each screen surface according to the segmented areas and the transformed coordinates; Image recognition technology is used to monitor the screen morphology changes of the target display device in real time, and the image segmentation and transformation process is adaptively adjusted according to the screen morphology changes. Preferably, performing splicing processing on the input video content according to the screen shape feature includes: Obtaining seams between multiple screen surfaces obtained after segmentation processing; Calculate and calibrate the joint positions between different screen surfaces, including the width, angle and connection method of the joints; According to the determined seam position, an anti-aliasing algorithm is used to smoothly transition the image at the seam; Perform color matching and brightness correction on images at seams; Partially overlap the images at the seams to obtain a stitched image, specifically: Obtaining a preset resolution threshold according to the resolution of the image and the resolution of the screen surface; Partially overlap the images at the seams and determine whether the resolution of the overlapping portion exceeds a preset resolution threshold. If so, reduce the overlapping portion; otherwise, treat the overlapping images as stitched images and use an interpolation method to process them. The spliced images obtained between multiple screen surfaces are spliced and combined in sequence to obtain a complete screen surface image.
[0006] Preferably, the step of using an anti-aliasing algorithm to smoothly transition the image at the seam includes: The edges of the seam positions are identified by edge detection algorithms; The anti-aliasing interpolation method is used to smooth the image edges at the seams after geometric transformation; Weighted averaging and blurring are applied to the joints to eliminate hard boundaries; Anti-aliasing algorithm is used to post-process and optimize the image at the seam. Preferably, performing defect detection on the input video image according to the screen shape feature comprises: Processing the video image and obtaining a grayscale gradient image to be detected; Calculate the grayscale moment value and grayscale approximate value parameters of the grayscale gradient image by using a second-order surface algorithm; Generate a flawless gradient image according to the gray value moment value and the gray value approximation parameter fitting; Subtracting the flawless gradient image from the grayscale gradient image to obtain a difference image; The difference image is threshold segmented to obtain a dark area defective image, specifically: Using a mask of a set size to move within the difference image; Compare the grayscale value of the current image pixel with the grayscale mean value of the mask, and if the grayscale value of the current image pixel is lower than the corresponding mask grayscale mean value, determine that the current image pixel is a defective pixel; Collecting and displaying all defective pixels to obtain the dark area defective image; Extracting defects from the bad dark area image and marking them for display; Perform defect handling on marked defects.
[0007] Preferably, the step of separating the image and text of the input video image according to the screen shape feature comprises the following steps: S11, obtaining a video image to be separated according to the video image, and graying it to obtain a grayscale image; S12, calculating a local activity matrix of the grayscale image, and performing local threshold adaptive binarization on the grayscale image according to the local activity matrix, removing image portions whose local activities are lower than a preset activity threshold and obtaining a high-activity image; S13, performing a closing operation on the high-activity image using a connection kernel with a scale of 3*3 to obtain a connection element set including n independent small connection elements; S14, searching all small connection elements connected to the first connection element from the connection element set according to the prior knowledge of the text area, and forming a neighbor connection element set; S15, for each small connected element in the set of nearest connected elements, calculating the directional average distance and obtaining the distribution direction of each small connected element, thereby constructing a local adaptive connection kernel, and applying the local adaptive connection kernel to perform a closing operation on each small connected element in the set of nearest connected elements; S16, repeating steps S14 and S15 until the connection element set is empty, and obtaining a new connection element set after the adaptive closing operation; S17, performing a picture-text separation operation based on the new connection element set; The elements in the new connection element set are divided into text connection elements and image connection elements. Preferably, the correcting of image distortion generated when the processed image is displayed on the target display device comprises: forming an initialization grid on a display screen according to optical parameters of the target display device, imaging the initialization grid through a lens disposed on one side of the target display device to form a correction grid on the imaging screen of the target display device, wherein the correction grid includes a plurality of grid points to be adjusted; Establishing a two-dimensional coordinate system on the correction grid; A set of concentric circles are formed on the initialization grid with the origin of the two-dimensional coordinate system as the center and the distances from all the recorded grid points to be adjusted to the origin as the radius. The grid points to be adjusted on each circle and outside the circle are moved relative to the origin in a direction away from the center of the circle. The positions of the grid points to be adjusted whose distances are less than the radius of the circle being adjusted remain unchanged until the grid lines where the grid points to be adjusted in the correction grid are located are straightened to correct image distortion.
[0008] In a second aspect, an embodiment of the present application provides a special-shaped screen video segmentation processing system, which applies the special-shaped screen video segmentation processing method as described above, including: Video signal acquisition module: used to acquire video signals from multiple video sources, and parse the video signals to obtain corresponding video images; Screen shape detection module: used to identify and analyze the screen shape characteristics of the target display device; Video image processing module: used for performing segmentation processing, splicing processing, defect detection and image-text separation on the input video image according to the screen shape features; Real-time rendering and display module: used for rendering in real time the processed image obtained after video image processing and displaying it on the target display device; An image distortion correction module is used to correct the image distortion generated when the processed image is displayed on the target display device; Interaction and control module: used to provide users with an interactive operation interface for interacting with the target display device, supporting users to adjust the display effect of the video image and the adaptation parameters of the screen shape characteristics.
[0009] The beneficial effects of the present invention are as follows: the present invention can adapt and correctly display standard video content on display devices of various shapes such as curved screens and multi-faceted screens by processing the segmentation, splicing, deformation, correction, defect detection and content detection of video signals. The entire processing flow combines the optimization of hardware and software to ensure that the display effect of the video remains high quality and smooth on screens of complex shapes. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] For better understanding and implementation, the technical solution of the present application is described in detail below with reference to the accompanying drawings.
[0011] Figure 1 A flowchart of a method for segmenting and processing a video of an irregular screen provided in an embodiment of the present application; Figure 2 A flowchart of the steps for identifying screen shape features provided in an embodiment of the present application; Figure 3 A flowchart of the steps of image segmentation processing provided in an embodiment of the present application; Figure 4 A structural flow chart of a special-shaped screen video segmentation processing system provided in an embodiment of the present application. DETAILED DESCRIPTION
[0012] In order to further explain the technical means and effects taken by the present invention to achieve the predetermined invention purpose, exemplary embodiments will be described in detail here, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are only examples of methods and systems consistent with some aspects of the present application as detailed in the attached claims.
[0013] The terms used in this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in this article refers to any or all possible combinations of one or more associated listed items.
[0014] The specific implementation methods, features and effects of the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments.
[0015] Example 1 See also Figure 1 The embodiment of the present application provides a method for segmenting and processing a video of an irregular screen, comprising the following steps: Acquire video signals from multiple video sources, and parse the video signals to obtain corresponding video images; Identify and analyze the screen shape characteristics of the target display device; Performing segmentation processing, splicing processing, defect detection and image-text separation on the input video image according to the screen shape features; Rendering the processed image obtained after the video image processing in real time and displaying it on the target display device; Correcting image distortion generated when the processed image is displayed on the target display device; An interactive operation interface for interacting with the target display device is provided for the user, and the user is supported to adjust the display effect of the video image and the adaptation parameters of the screen shape features.
[0016] Specifically, in the prior art, when displaying large-scale, continuous images or video content, traditional video segmentation methods are difficult to accurately adapt to screens of different shapes, which may cause problems such as image distortion, obvious splicing gaps, and color distortion. Therefore, in order to deal with the above-mentioned problems caused by special-shaped screens, the present application adopts video segmentation, video splicing, image distortion correction, and rendering technologies for processing, which specifically include the following contents: first, video signals are obtained from multiple video sources, and video images corresponding to the video signals are parsed according to the video signals; then, the screen shape characteristics of the target display device are identified and analyzed; then, the input video image is segmented, spliced, defect detected, and image and text segmented according to the screen shape characteristics; then, the processed image obtained after adaptive segmentation and splicing is rendered in real time and displayed on the target display device; then, the image distortion generated when the video image is displayed on the target display device is corrected; finally, an interactive operation interface for interacting with the target display device is provided to support the user to adjust the display effect of the video image and the adaptation parameters of the screen shape characteristics.
[0017] This application can adapt and correctly display standard video content on display devices of various shapes, such as curved screens and multi-faceted screens, by processing video signal segmentation, splicing, deformation, correction, defect detection and content detection. The entire processing flow combines hardware and software optimization to ensure that the video display effect remains high quality and smooth on screens of complex shapes.
[0018] In an embodiment provided in the present application, parsing the video signal to obtain a corresponding video image includes: Performing signal synchronization on the video signals acquired from multiple video sources by using time stamp alignment and frame synchronization; Decoding the video signal using a video decoder to convert it into image frames for analysis and processing; Preprocessing the decoded image frame, including denoising, image enhancement and color correction; The image frames are analyzed and the analyzed images are fused to obtain the video image.
[0019] Specifically, since the video signals come from multiple video sources (such as cameras, storage devices, streaming media, video files, etc.), there will be problems such as signal asynchrony. Therefore, this embodiment sets a timestamp for each video source and synchronizes the video stream according to the timestamp (usually adding accurate time information during acquisition, and then adjusting the video playback order and frame synchronization through the timestamp); frame synchronization is also used to ensure that the frame rates collected from multiple video sources are consistent, avoiding image confusion due to inconsistent frame numbers. The quality of video content is also improved by preprocessing the image frames.
[0020] like Figure 2 As shown, in one embodiment provided in the present application, the identifying and analyzing the screen shape features of the target display device includes: Acquiring image information of the target display device through an imaging device; Using an edge detection algorithm to extract edge information from the image information; Based on the edge information, using a contour detection algorithm to identify contour information of the screen edge of the target display device; Performing shape feature analysis on the contour information to obtain the screen shape feature includes: Shape recognition: using the RANSAC algorithm to perform shape feature analysis on the contour information; Polygon fitting: using a polygon fitting algorithm to fit the contour information and find the best fitting rectangle; Corner point detection: using a corner point detection algorithm to determine the four corner points of the optimal fit rectangle and confirm the screen shape of the target display device; Shape feature extraction: Analyze the geometric properties of the screen shape, including aspect ratio, angle and area, and use the geometric properties to extract and verify the screen shape features.
[0021] Specifically, the goal of this embodiment is to extract the geometric shape, size, boundary and other information of the display device from the screen image. Specifically, the edge information in the image is extracted by an edge detection algorithm (such as Canny edge detection). Edge detection helps to identify the boundary of the display screen, especially when the lighting conditions are not ideal or the screen surface is highly reflective; secondly, based on the edge image, the contour detection algorithm is used to identify the contour information of the screen edge. Usually, the screen of the target display device has a regular geometric shape (rectangle or approximate rectangle), so the difficulty of contour recognition can be further simplified. Then, shape recognition and analysis are performed based on the extracted contour information; since common geometric shapes include rectangles, squares, circles, etc., the screen of the target display device is usually rectangular or close to a rectangle, so shape matching or polygon approximation (such as RANSAC algorithm) can be used to determine whether it meets the rectangular feature. Then, the extracted contour is fitted by a polygon fitting algorithm (such as least squares method) to find its optimal fitting rectangle. Finally, a corner detection algorithm (such as Harris corner detection, Shi-Tomasi corner detection) is used to determine the four corner points of the rectangle, thereby further confirming the screen shape of the display device. This embodiment can effectively identify and analyze the screen shape features of the display device by comprehensively utilizing computer vision technology and geometric analysis methods.
[0022] like Figure 3As shown, in an embodiment provided by the present application, the segmentation processing of the input video image according to the screen shape feature includes: The input video image is divided into a plurality of regions according to the screen shape feature, each region corresponding to a screen surface formed by the screen shape feature; wherein the video image content displayed on the display area of each screen surface matches the screen shape feature; The screen surface angle of each screen surface is detected and the viewing angle correction is performed on the screen surfaces with different angles or curvature, and the video image content is adjusted using geometric transformation according to the position and angle of each screen surface; the geometric transformation includes affine transformation, perspective transformation and polar coordinate transformation.
[0023] Map the processed video image content to each screen surface according to the segmented areas and the transformed coordinates; that is, remap the pixels according to the resolution of the video image and the resolution of each screen surface to ensure that the picture is not distorted; Image recognition technology is used to monitor the screen morphology changes of the target display device in real time, and the image segmentation and transformation process is adaptively adjusted according to the screen morphology changes. The image segmentation and transformation are adjusted according to the real-time changes in the screen morphology to ensure that the video content can always correctly adapt to different screen surfaces.
[0024] Specifically, this embodiment uses the above-mentioned image segmentation processing to make the display area of the video image content on each screen plane match the actual physical screen size, and has a certain adaptability and can respond to changes in the screen form in real time. By processing real-time adaptation and dynamic changes, it is ensured that the video content under different screen forms always matches.
[0025] In an embodiment provided in the present application, splicing the input video content according to the screen shape feature includes: Obtaining seams between multiple screen surfaces obtained after segmentation processing; Calculate and calibrate the joint positions between different screen surfaces, including the width, angle and connection method of the joints; According to the determined seam position, an anti-aliasing algorithm is used to smoothly transition the image at the seam to reduce image faults; Perform color matching and brightness correction on images at the seams to ensure visual consistency at the seams; Partially overlap the images at the seams to obtain a stitched image, specifically: Obtaining a preset resolution threshold according to the resolution of the image and the resolution of the screen surface; Partially overlap the images at the seam, and determine whether the resolution of the overlapping portion exceeds a preset resolution threshold. If so, reduce the overlapping portion; otherwise, treat the overlapping images as stitched images and use an interpolation method to process them. The above operation can ensure seamless transition of the images. The spliced images obtained between multiple screen surfaces are spliced and combined in sequence to obtain a complete screen surface image.
[0026] Specifically, during the splicing process, due to geometric transformation and image stretching, jagged phenomena may occur. Therefore, this embodiment uses an anti-aliasing algorithm for smoothing in image splicing to smooth the image edges and ensure that the picture is smoother and more natural. In addition, an interpolation method is used to perform pixel interpolation or resampling for the spliced video content, especially in the case of inconsistent resolution, to avoid image quality degradation. On the other hand, since different screen surfaces may have different color and brightness characteristics, it is necessary to perform color matching and brightness correction on the image to ensure visual consistency at the seams, and to achieve slight overlap of the image at the seams to ensure seamless transition, especially in curved or irregular screen layouts. Through the above-mentioned image splicing steps, this embodiment can ensure that the video content is seamlessly spliced and displayed with high quality under complex multi-sided screen configurations.
[0027] In an embodiment provided in the present application, the use of an anti-aliasing algorithm to smoothly transition the image at the seam includes: The edges of the seam positions are identified by edge detection algorithms; The anti-aliasing interpolation method is used to smooth the image edges at the seams after geometric transformation, so that the edges of the interpolated image no longer have obvious aliasing. Weighted averaging and blurring are applied to the joints to eliminate hard boundaries; Anti-aliasing algorithm is used to post-process and optimize the image at the seam.
[0028] Specifically, when an image is rotated, scaled, or affine transformed, the original pixels may become irregular. At this time, anti-aliasing interpolation methods (such as bilinear interpolation, Bicubic interpolation) are used to smooth the transformed image so that the edges of the interpolated image no longer have obvious jagged edges. In the process of image stitching, the seam area (that is, where multiple images are joined) is usually where jagged edges are generated. At this time, an anti-aliasing algorithm is used to smooth the seam area: Weighted average method: By taking a weighted average of the pixels in the seam area, the pixel values in the stitching area can be gradually transitioned to avoid hard boundaries.
[0029] Blur processing: Gaussian blur or bilateral filtering is performed on the seam area to reduce edge mutations and make the stitching area blend more naturally.
[0030] After stitching is completed, the image can be post-processed to optimize the final result through anti-aliasing algorithms, especially in the seam area, to ensure that the stitched image has no obvious jagged seams visually. Common methods include: FXAA or SMAA post-processing: Apply FXAA (Fast Approximate Anti-Aliasing) or SMAA (Subpixel Morphological Anti-Aliasing) to the stitched image to further smooth the edges and eliminate the remaining aliasing.
[0031] Dynamic frame smoothing: If it is video stitching, TAA (Temporal Anti-Aliasing) can further reduce the aliasing caused by changes between frames in dynamic scenes.
[0032] This embodiment can effectively improve the quality of the spliced image by adopting an anti-aliasing algorithm, making the edges smoother and more natural, and reducing visual abruptness and distortion.
[0033] In an embodiment provided in the present application, performing defect detection on the input video image according to the screen shape feature includes: Processing the video image and obtaining a grayscale gradient image to be detected; Calculate the grayscale moment value and grayscale approximate value parameters of the grayscale gradient image by using a second-order surface algorithm; Generate a flawless gradient image according to the gray value moment value and the gray value approximation parameter fitting; Subtracting the flawless gradient image from the grayscale gradient image to obtain a difference image; The difference image is threshold segmented to obtain a dark area defective image, specifically: Using a mask of a set size to move within the difference image; Compare the grayscale value of the current image pixel with the grayscale mean value of the mask, and if the grayscale value of the current image pixel is lower than the corresponding mask grayscale mean value, determine that the current image pixel is a defective pixel; Collecting and displaying all defective pixels to obtain the dark area defective image; Extracting defects from the bad dark area image and marking them for display; Perform defect handling on marked defects.
[0034] Specifically, this embodiment uses a second-order surface fitting algorithm to generate a flawless gradient image that matches the grayscale of the grayscale gradient image, and then subtracts the flawless gradient image from the grayscale gradient image to obtain a difference image; by extracting dark area defects from the difference image, gradient defects in the dark area are screened out, and then marked and displayed on the grayscale gradient image, the gradient defect features are clear and obvious, thereby realizing automatic detection of the display screen of the extended reality device, greatly reducing the low detection rate, missed detection, and high false detection rate caused by small gradient image differences, human eye fatigue, etc., thereby improving the efficiency and accuracy of the detection equipment in screen defect detection.
[0035] In an embodiment provided in the present application, the image and text separation of the input video image is performed according to the screen shape feature, comprising the following steps: S11, obtaining a video image to be separated according to the video image, and graying it to obtain a grayscale image; S12, calculating the local activity matrix of the grayscale image, and performing local threshold adaptive binarization on the grayscale image according to the local activity matrix, removing the image portion whose local activity is lower than a preset activity threshold and obtaining a high-activity image; this can reduce the complexity of subsequent closing processing.
[0036] S13, performing a closing operation on the high-activity image using a connection kernel with a scale of 3*3 to obtain a connection element set including n independent small connection elements; S14, searching all small connection elements connected to the first connection element from the connection element set according to the prior knowledge of the text area, and forming a neighbor connection element set; S15, for each small connected element in the set of nearest connected elements, calculating the directional average distance and obtaining the distribution direction of each small connected element, thereby constructing a local adaptive connection kernel, and applying the local adaptive connection kernel to perform a closing operation on each small connected element in the set of nearest connected elements; S16, repeating steps S14 and S15 until the connection element set is empty, and obtaining a new connection element set after the adaptive closing operation; S17, performing a picture-text separation operation based on the new connection element set; The elements in the new connection element set are divided into text connection elements and image connection elements.
[0037] The text connection element has identification features, including area occupancy rate, number of colors and overall activity, which are used to identify it from the image connection elements. The area covered by all text connection elements is the text part in the screen image, thereby realizing image and text segmentation of the screen image.
[0038] Specifically, the present embodiment adopts a design of constructing a connection kernel with adaptive scale and direction for local closing operation, which can accurately segment the text area and the image area from the screen image, so that the construction of the connection kernel is based on the characteristics of each local area. Therefore, the final detection result is independent of scale and direction, that is, the present invention can detect text of any size and direction, which not only greatly improves the accuracy of segmentation, but also avoids the occurrence of over-connection and under-connection.
[0039] In an embodiment provided in the present application, the correcting of image distortion generated when the processed image is displayed on the target display device includes: forming an initialization grid on a display screen according to optical parameters of the target display device, imaging the initialization grid through a lens disposed on one side of the target display device to form a correction grid on the imaging screen of the target display device, wherein the correction grid includes a plurality of grid points to be adjusted; Establishing a two-dimensional coordinate system on the correction grid; A set of concentric circles are formed on the initialization grid with the origin of the two-dimensional coordinate system as the center and the distances from all the recorded grid points to be adjusted to the origin as the radius. The grid points to be adjusted on each circle and outside the circle are moved relative to the origin in a direction away from the center of the circle. The positions of the grid points to be adjusted whose distances are less than the radius of the circle being adjusted remain unchanged until the grid lines where the grid points to be adjusted in the correction grid are located are straightened to correct image distortion.
[0040] Specifically, this embodiment corrects image distortion through the above steps on the one hand, thereby improving user experience; on the other hand, only the remaining grid points to be adjusted on and outside the circle where the grid points to be adjusted are located are adjusted to correct image distortion, thereby improving correction efficiency. This embodiment can effectively eliminate video distortion on a curved screen or multi-faceted screen through geometric correction and transformation technology, so that video content can be accurately presented on a special-shaped screen, ensuring distortion-free display and improving viewing experience.
[0041] Example 2 See also Figure 4 The embodiment of the present application provides a special-shaped screen video segmentation processing system, which applies the special-shaped screen video segmentation processing method as described above, including: Video signal acquisition module: used to acquire video signals from multiple video sources, and parse the video signals to obtain corresponding video images; Screen shape detection module: used to identify and analyze the screen shape characteristics of the target display device; Video image processing module: used for performing segmentation processing, splicing processing, defect detection and image-text separation on the input video image according to the screen shape features; Real-time rendering and display module: used for rendering in real time the processed image obtained after video image processing and displaying it on the target display device; An image distortion correction module is used to correct the image distortion generated when the processed image is displayed on the target display device; Interaction and control module: used to provide users with an interactive operation interface for interacting with the target display device, supporting users to adjust the display effect of the video image and the adaptation parameters of the screen shape characteristics.
[0042] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0043] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.
[0044] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0045] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention still fall within the scope of the technical solution of the present invention.
Claims
1. A method for segmenting and processing a video of an irregular screen, characterized in that: The steps include: Acquire video signals from multiple video sources, and parse the video signals to obtain corresponding video images; Identify and analyze the screen shape characteristics of the target display device; Performing segmentation processing, splicing processing, defect detection and image-text separation on the input video image according to the screen shape features; Rendering the processed image obtained after the video image processing in real time and displaying it on the target display device; Correcting image distortion generated when the processed image is displayed on the target display device; An interactive operation interface for interacting with the target display device is provided for the user, and the user is supported to adjust the display effect of the video image and the adaptation parameters of the screen shape features.
2. The method for segmenting and processing a video of an irregular screen according to claim 1, characterized in that: The step of parsing the video signal to obtain a corresponding video image includes: Performing signal synchronization on the video signals acquired from multiple video sources by using time stamp alignment and frame synchronization; Decoding the video signal using a video decoder to convert it into image frames for analysis and processing; Preprocessing the decoded image frame, including denoising, image enhancement and color correction; The image frames are analyzed and the analyzed images are fused to obtain the video image.
3. The method for segmenting and processing a video of an irregular screen according to claim 1, characterized in that: The identifying and analyzing the screen shape characteristics of the target display device includes: Acquiring image information of the target display device through an imaging device; Using an edge detection algorithm to extract edge information from the image information; Based on the edge information, using a contour detection algorithm to identify contour information of the screen edge of the target display device; Performing shape feature analysis on the contour information to obtain the screen shape feature includes: Shape recognition: using the RANSAC algorithm to perform shape feature analysis on the contour information; Polygon fitting: using a polygon fitting algorithm to fit the contour information and find the best fitting rectangle; Corner point detection: using a corner point detection algorithm to determine the four corner points of the optimal fit rectangle and confirm the screen shape of the target display device; Shape feature extraction: Analyze the geometric properties of the screen shape, including aspect ratio, angle and area, and use the geometric properties to extract and verify the screen shape features.
4. The method for segmenting and processing a video of an irregular screen according to claim 1, characterized in that: The step of segmenting the input video image according to the screen shape feature comprises: The input video image is divided into a plurality of regions according to the screen shape feature, each region corresponding to a screen surface formed by the screen shape feature; wherein the video image content displayed on the display area of each screen surface matches the screen shape feature; Detect the screen surface angle of each screen surface and perform viewing angle correction for screen surfaces with different angles or curvature, and adjust the video image content using geometric transformation according to the position and angle of each screen surface; Mapping the processed video image content to each screen surface according to the segmented areas and the transformed coordinates; Image recognition technology is used to monitor the screen morphology changes of the target display device in real time, and the image segmentation and transformation process is adaptively adjusted according to the screen morphology changes.
5. The method for segmenting and processing a video of an irregular screen according to claim 1, characterized in that: The input video content is spliced according to the screen shape feature, including: Obtaining seams between multiple screen surfaces obtained after segmentation processing; Calculate and calibrate the joint positions between different screen surfaces, including the width, angle and connection method of the joints; According to the determined seam position, an anti-aliasing algorithm is used to smoothly transition the image at the seam; Perform color matching and brightness correction on images at seams; Partially overlap the images at the seams to obtain a stitched image, specifically: Obtaining a preset resolution threshold according to the resolution of the image and the resolution of the screen surface; Partially overlap the images at the seam and determine whether the resolution of the overlapping portion exceeds a preset resolution threshold. If so, reduce the overlapping portion. Otherwise, treat the overlapping image as a stitched image and use an interpolation method to process it. The spliced images obtained between multiple screen surfaces are spliced and combined in sequence to obtain a complete screen surface image.
6. The method for segmenting and processing a video of a special-shaped screen according to claim 5, characterized in that: The anti-aliasing algorithm is used to smoothly transition the image at the seam, including: The edges of the seam positions are identified by edge detection algorithms; The anti-aliasing interpolation method is used to smooth the image edges at the seams after geometric transformation; Weighted averaging and blurring are applied to the joints to eliminate hard boundaries; Anti-aliasing algorithm is used to post-process and optimize the image at the seam.
7. The method for segmenting and processing a video of a special-shaped screen according to claim 1, characterized in that: Performing defect detection on the input video image according to the screen shape feature includes: Processing the video image and obtaining a grayscale gradient image to be detected; Calculate the grayscale moment value and grayscale approximate value parameters of the grayscale gradient image by using a second-order surface algorithm; Generate a flawless gradient image according to the gray value moment value and the gray value approximation parameter fitting; Subtracting the flawless gradient image from the grayscale gradient image to obtain a difference image; The difference image is threshold segmented to obtain a dark area defective image, specifically: Using a mask of a set size to move within the difference image; Compare the grayscale value of the current image pixel with the grayscale mean value of the mask, and if the grayscale value of the current image pixel is lower than the corresponding mask grayscale mean value, determine that the current image pixel is a defective pixel; Collecting and displaying all defective pixels to obtain the dark area defective image; Extracting defects from the bad dark area image and marking them for display; Perform defect handling on marked defects.
8. The method for segmenting and processing a video of an irregular screen according to claim 1, characterized in that: The input video image is separated into images and texts according to the screen shape feature, comprising the following steps: S11, obtaining a video image to be separated according to the video image, and graying it to obtain a grayscale image; S12, calculating a local activity matrix of the grayscale image, and performing local threshold adaptive binarization on the grayscale image according to the local activity matrix, removing image portions whose local activities are lower than a preset activity threshold and obtaining a high-activity image; S13, performing a closing operation on the high-activity image using a connection kernel with a scale of 3*3 to obtain a connection element set including n independent small connection elements; S14, searching all small connection elements connected to the first connection element from the connection element set according to the prior knowledge of the text area, and forming a neighbor connection element set; S15, for each small connected element in the set of nearest connected elements, calculating the directional average distance and obtaining the distribution direction of each small connected element, thereby constructing a local adaptive connection kernel, and applying the local adaptive connection kernel to perform a closing operation on each small connected element in the set of nearest connected elements; S16, repeating steps S14 and S15 until the connection element set is empty, and obtaining a new connection element set after the adaptive closing operation; S17, performing a picture-text separation operation based on the new connection element set; The elements in the new connection element set are divided into text connection elements and image connection elements.
9. The method for segmenting and processing a video of a special-shaped screen according to claim 1, characterized in that: The correcting of image distortion generated when the processed image is displayed on the target display device comprises: forming an initialization grid on a display screen according to optical parameters of the target display device, imaging the initialization grid through a lens disposed on one side of the target display device to form a correction grid on the imaging screen of the target display device, wherein the correction grid includes a plurality of grid points to be adjusted; Establishing a two-dimensional coordinate system on the correction grid; A set of concentric circles are formed on the initialization grid with the origin of the two-dimensional coordinate system as the center and the distances from all the recorded grid points to be adjusted to the origin as the radius. The grid points to be adjusted on each circle and outside the circle are moved relative to the origin in a direction away from the center of the circle. The positions of the grid points to be adjusted whose distances are less than the radius of the circle being adjusted remain unchanged until the grid lines where the grid points to be adjusted in the correction grid are located are straightened to correct image distortion.
10. A video segmentation processing system for a special-shaped screen, using a video segmentation processing method for a special-shaped screen as claimed in any one of claims 1 to 9, characterized in that: include: Video signal acquisition module: used to acquire video signals from multiple video sources, and parse the video signals to obtain corresponding video images; Screen shape detection module: used to identify and analyze the screen shape characteristics of the target display device; Video image processing module: used for performing segmentation processing, splicing processing, defect detection and image-text separation on the input video image according to the screen shape features; Real-time rendering and display module: used for rendering in real time the processed image obtained after video image processing and displaying it on the target display device; An image distortion correction module is used to correct the image distortion generated when the processed image is displayed on the target display device; Interaction and control module: used to provide users with an interactive operation interface for interacting with the target display device, supporting users to adjust the display effect of the video image and the adaptation parameters of the screen shape characteristics.