Cave screen video processing method and system

Abstract

The application provides a Cave screen-folding video-based processing method and system, relates to the technical field of data processing, and is used for presenting immersive naked-eye 3D visual effects. The relative parameters of the active screen and the fixed screen in the screen-folding space are determined, and the joint boundary line of the active screen and the bottom screen is obtained. When the joint boundary line does not have key elements, the background reconstruction processing is performed on the associated area of the bottom screen based on the relative parameters. When the joint boundary line has key elements, the position migration processing is performed on the key elements. The to-be-rendered picture adapted to each screen surface of the screen-folding space is obtained. The to-be-rendered picture is subjected to the rendering processing, and the naked-eye 3D video adapted to the screen-folding space is obtained. The application adapts the video picture to the actual state of each screen surface of the screen-folding space, avoids the distortion and disorder of the picture at the screen-folding joint, realizes the natural connection of the picture and the complete presentation of the key elements, and is suitable for the immersive naked-eye 3D display in various screen-folding scenes.

Description

Technical Field

[0001] This invention relates to data processing technology, and more particularly to a method and system for processing Cave folding screen video. Background Technology

[0002] Folded screen spaces are visual spaces composed of multiple screens, either enclosed or semi-enclosed, that provide users with an immersive naked-eye 3D experience. They are widely used in exhibitions, virtual simulations, cultural tourism and entertainment.

[0003] Currently, most screens in existing folding screen spaces use a fixed screen angle, which cannot accommodate users of different heights or viewing angles. Even if some screens introduce adjustable structures such as robotic arms, when the screen rotates relative to the viewer, the filling of gaps or overlapping areas is often just a simple overall image stretching or cropping. This processing method will cause disproportion of objects such as people or buildings in the image. For example, people in the image will be obviously out of proportion after being stretched, while image cropping will directly discard a part of the object, resulting in incomplete display of objects and difficulty in maintaining the true shape of objects in the image.

[0004] Therefore, how to achieve seamless image transitions and ensure the complete presentation of key elements when the screen angle changes in a folded screen space has become a critical issue that urgently needs to be addressed. Summary of the Invention

[0005] This invention provides a method and system for processing Cave folded screen videos, which can achieve seamless image transitions and ensure the complete presentation of key elements when the screen angle changes in the folded screen space.

[0006] A first aspect of the present invention provides a method for processing Cave-based folded screen video, comprising: Determine the relative rotation parameters between the movable screen and the fixed screen in the folding screen space. The fixed screen includes a front screen and a bottom screen. The movable screen is rotatably connected to the left and right sides of the front screen. Obtain the boundary line between the active screen and the bottom screen; When it is determined that there are no key elements at the intersection boundary line, the background reconstruction process is performed on the associated area of ​​the bottom screen according to the relative rotation parameters to obtain the processed image to be rendered. When it is determined that there are key elements at the boundary line, the key elements are moved according to the relative rotation parameters to obtain the processed image to be rendered. The image to be rendered is processed to obtain a naked-eye 3D video adapted to the folded screen space.

[0007] Optionally, in one possible implementation of the first aspect, determining the relative rotation parameters between the movable screen and the fixed screen in the folding space includes: The user terminal inputs customized rotation parameters as relative rotation parameters between the moving screen and the fixed screen in the folded screen space. The relative rotation parameters include the relative rotation state and the rotation angle.

[0008] Optionally, in one possible implementation of the first aspect, determining the relative rotation parameters between the movable screen and the fixed screen in the folding space includes: The user's posture height is obtained, and a preset parameter lookup table is retrieved. The preset parameter lookup table has a one-to-one correspondence between preset height ranges and preset rotation parameters. The user height range is determined by the preset height range in the preset parameter lookup table. Based on the user's height range, the corresponding preset rotation parameters are retrieved as the relative rotation parameters between the moving screen and the fixed screen in the folding screen space. The relative rotation parameters include the relative rotation state and the rotation angle.

[0009] Optionally, in one possible implementation of the first aspect, when it is determined that there are no key elements at the junction boundary line, background reconstruction processing is performed on the associated region of the bottom screen according to the relative rotation parameters, including: When it is determined that there are no key elements at the intersection boundary line, the relative rotation state in the relative rotation parameters is obtained, and the relative rotation state includes outward rotation and inward rotation; When the relative rotation state is determined to be outward rotation, the supplementary area of ​​the bottom screen is filled with background. When the relative rotation state is determined to be an inward rotation state, the occlusion area of ​​the bottom screen is deleted. The associated area includes the supplementary area and the occlusion area.

[0010] Optionally, in one possible implementation of the first aspect, when determining that the relative rotation state is outward rotation, background filling processing is performed on the supplementary area of ​​the bottom screen, including: When the relative rotation state is determined to be outward rotation, the endpoint at the junction boundary line that is in contact with both the front screen and the bottom screen is taken as the fixed endpoint. The rotation angle in the relative rotation parameters is retrieved as the external rotation angle; Using the fixed endpoint as the center, the intersection boundary line is rotated based on the outward rotation angle to obtain the supplementary area through which the intersection boundary line rotates. The background pixel values ​​are retrieved to fill the supplementary area.

[0011] Optionally, in one possible implementation of the first aspect, when determining that the relative rotation state is an inward rotation state, deleting the obstructed area of ​​the bottom screen includes: When the relative rotation state is determined to be an inward rotation state, the endpoint at the junction boundary line that is in contact with both the front screen and the bottom screen is taken as the fixed endpoint, and the remaining endpoints are taken as the extended endpoints. The rotation angle in the relative rotation parameters is taken as the inward rotation angle; With the fixed endpoint as the center, the intersection boundary line is rotated based on the inward rotation angle to obtain the rotated intersection boundary line; The rotated boundary line is extended to obtain an extended ray, and the intersection point of the extended ray and the corresponding edge line of the bottom screen is obtained. Connect the intersection point with the extended endpoint of the intersection boundary line to obtain the shading area; The obscured area is then deleted.

[0012] Optionally, in one possible implementation of the first aspect, when it is determined that there is a key element at the junction boundary line, the key element is subjected to position migration processing according to the relative rotation parameters, including: When it is determined that there is a key element at the intersection boundary line, the relative rotation state in the relative rotation parameters is obtained, and the relative rotation state includes outward rotation and inward rotation; When the relative rotation state is determined to be outward rotation, the pixels of the key elements are subjected to overall position migration processing based on the distribution ratio of the key elements. When the relative rotation state is determined to be an inward rotation state, the position migration processing is performed on the part of the key element on the bottom screen based on the inline distribution of the key element.

[0013] Optionally, in one possible implementation of the first aspect, when determining that the relative rotation state is an outward rotation, based on the distribution proportion of the key element, performing an overall position migration process on the pixels of the key element, including: When the relative rotation state is determined to be outward rotation, the first distribution ratio of the key element on the active screen and the second distribution ratio on the bottom screen are calculated respectively. Based on the movable screen and the fixed screen, the display screen is obtained; Compare the first distribution ratio with the second distribution ratio, and determine the display screen with the largest ratio as the target carrying screen, and the remaining display screens as the removal screens; The pixels of the corresponding key elements are translated as a whole to the target supporting screen until the screen no longer has the corresponding key elements, and the position of the missing pixels after translation is obtained. The background pixel values ​​are retrieved to fill the positions of the missing pixels.

[0014] Optionally, in one possible implementation of the first aspect, when determining that the relative rotation state is an inward rotation state, based on the inline distribution of the key element, position migration processing is performed on the portion of the key element on the bottom screen, including: When the relative rotation state is determined to be an inward rotation state, the occlusion area and display area of ​​the bottom screen are obtained, and the migration pixels of the key elements located in the occlusion area are obtained, and the pixel row where the migration pixel is located is taken as the migration row. Identify irrelevant pixels in the migration row within the display area that do not belong to key elements; The number of migrated pixels in the migration row is counted as the deletion count for the corresponding migration row. Irrelevant pixels in the corresponding migration row are deleted based on the deletion count to obtain the position of the filling pixel. Obtain the first direction in which the migrated pixel is located at the position of the filled pixel, and take the direction opposite to the first direction as the filling direction; Move the migrated pixels according to the filling direction until the migrated pixels enter the display area.

[0015] A second aspect of the present invention provides a processing system for Cave folding screen video, comprising: An information module is used to determine the relative rotation parameters between the movable screen and the fixed screen in the folded screen space. The fixed screen includes a front screen and a bottom screen. The movable screen is rotatably connected to the left and right sides of the front screen. The determining module is used to obtain the boundary line of the intersection between the active screen and the bottom screen; The reconstruction module is used to reconstruct the background of the associated area of ​​the bottom screen according to the relative rotation parameters when it is determined that there are no key elements at the intersection boundary line, so as to obtain the processed image to be rendered. The migration module is used to determine that there are key elements at the intersection boundary line, and then perform position migration processing on the key elements according to the relative rotation parameters to obtain the processed image to be rendered. The rendering module is used to render the image to be rendered to obtain a naked-eye 3D video adapted to the folded screen space.

[0016] The beneficial effects of this invention are as follows: 1. This invention determines the relative rotation parameters of the moving screen and the fixed screen in the folded screen space, and obtains the boundary line between the moving screen and the bottom screen. When it is determined that there are no key elements at the boundary line, the background reconstruction process is performed on the associated area of ​​the bottom screen according to the relative rotation parameters; when it is determined that there are key elements at the boundary line, the position migration process is performed on the key elements according to the relative rotation parameters. This invention uses different processing methods based on whether there are key elements at the boundary line, which can achieve seamless screen transition and ensure the complete presentation of key elements when the screen angle changes in the folded screen space.

[0017] 2. When there are no key elements at the junction boundary line, this invention distinguishes between relative rotation states as outward rotation and inward rotation. When rotating outward, the supplementary area of ​​the bottom screen is filled with background, and when rotating inward, the occluded area of ​​the bottom screen is deleted. This invention uses a fixed endpoint as the center and rotates the junction boundary line based on the rotation angle to obtain the supplementary area or occluded area, thus avoiding unfilled blank areas or overlapping redundant images after the screen is rotated.

[0018] 3. When key elements are present at the boundary line, this invention, during outward rotation, translates the key elements as a whole to the target supporting screen based on their distribution ratio on the active screen and the bottom screen, and fills the missing pixel positions after translation. During inward rotation, based on the distribution of key elements in the occluded area, delete an equal number of irrelevant pixels within the migration row, and then move the occluded migration pixels according to the positions of the deleted irrelevant pixels until the migration pixels appear in the display area. This invention avoids the problem of key elements being missing or broken due to screen rotation, and can maintain the true shape of key elements after screen rotation. Attached Figure Description

[0019] Figure 1 A flowchart illustrating a method for processing Cave-based folded screen video provided by this invention; Figure 2 This is a schematic diagram of the front viewing screen, bottom screen, and movable screen in this invention; Figure 3 This is a schematic diagram of the structure of the supplementary region in this invention; Figure 4 This is a schematic diagram of the structure of the shielding area in this invention; Figure 5 This is a schematic diagram illustrating the movement of the migrated pixels in this invention; Figure 6 This is a schematic diagram of the structure of a Cave folding screen video processing system provided by the present invention; Figure 7 This is a schematic diagram of the hardware structure of an electronic device provided by the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein.

[0022] It should be understood that in the various embodiments of the present invention, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0023] It should be understood that in this invention, "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product, or device.

[0024] It should be understood that in this invention, "multiple" refers to two or more. "And / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "Contains A, B, and C", "Contains A, B, and C" means that all three A, B, and C are contained; "Contains A, B, or C" means that one of A, B, and C is contained; "Contains A, B, and / or C" means that any one, two, or three of A, B, and C are contained.

[0025] It should be understood that in this invention, "B corresponding to A", "B corresponding to A", "A and B correspond", or "B and A correspond" means that B is associated with A, and B can be determined based on A. Determining B based on A does not mean determining B solely based on A; B can also be determined based on A and / or other information. Matching A and B is defined as a similarity between A and B that is greater than or equal to a preset threshold.

[0026] Depending on the context, "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection."

[0027] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0028] This invention provides a method for processing Cave-based folded screen video, such as... Figure 1 As shown, it includes: S1, determine the relative rotation parameters between the movable screen and the fixed screen in the folded screen space, wherein the fixed screen includes a front screen and a bottom screen, and the movable screen is rotatably connected to the left and right sides of the front screen.

[0029] It is understandable that, such as Figure 2 As shown, the screens within the folding screen space are divided into fixed screens and movable screens. Fixed screens refer to screens whose positions are fixed within the folding screen space. Fixed screens include a vertically placed front screen and a horizontally placed bottom screen. Movable screens are located on the left and right sides of the front screen and are rotatably connected to it. The movable screens can rotate around their connection points with the front screen to change the opening angle. This step first determines the state and angle to which the movable screens should rotate relative to the fixed screens.

[0030] Among them, the front screen refers to the screen that is placed vertically in front of the folded screen space, the bottom screen refers to the screen that is placed horizontally at the bottom of the folded screen space, the movable screen refers to the screen that is rotatably connected to the front screen and can be rotated, and the relative rotation parameters refer to the parameters of the movable screen's rotation relative to the fixed screen, including the relative rotation state and the rotation angle.

[0031] It should be noted that the present invention allows users to actively customize the parameters of the moving screen rotation, or it can automatically adapt the parameters of the moving screen rotation based on the user's height.

[0032] In some embodiments, step S1 (determining the relative rotation parameters between the moving screen and the fixed screen in the folding space) is implemented through user customization, including A1: A1 receives customized rotation parameters input from the user terminal as relative rotation parameters between the moving screen and the fixed screen in the folded screen space. The relative rotation parameters include the relative rotation state and the rotation angle.

[0033] Understandably, users can directly customize the rotation of the moving screen according to their own preferences. Specifically, before the user enters the folded screen space, the user terminal displays a virtual model of the moving screen and its adjustable range, receives adjustment operations applied by the user to the virtual model of the moving screen, and uses the rotation direction and rotation angle corresponding to the adjustment operation as customized rotation parameters. These customized rotation parameters are then used as the relative rotation parameters between the moving screen and the fixed screen, meaning the relative rotation parameters include the relative rotation state representing the rotation direction and the rotation angle representing the rotation angle.

[0034] Among them, the user terminal refers to the terminal device that allows users to perform interactive operations to input customized rotation parameters; customized rotation parameters refer to the parameters that users input on the user terminal to customize the rotation of the active screen; relative rotation state refers to the direction of rotation of the active screen relative to the bottom screen; rotation angle refers to the angle by which the active screen rotates.

[0035] In other embodiments, step S1 (determining the relative rotation parameters between the movable screen and the fixed screen in the folding space) is achieved by automatically adapting to the user's height, including B1-B3: B1. Obtain the user's posture height and retrieve a preset parameter lookup table. The preset parameter lookup table has a one-to-one correspondence between preset height ranges and preset rotation parameters.

[0036] It should be noted that this step pre-divides the height into several intervals and configures the corresponding rotation parameters for each height interval. Thus, after obtaining the user's height, the relative rotation parameters between the moving screen and the fixed screen are determined by looking up a table.

[0037] Among them, user posture refers to the body posture of the user entering the folded screen space; preset height range refers to the pre-divided height range; preset rotation parameters refer to the parameters for the rotation of the moving screen that are pre-set for each preset height range.

[0038] Understandably, the system obtains the user's posture height, which is equivalent to the user's height. Simultaneously, it retrieves a pre-configured preset parameter lookup table. This table stores multiple preset height ranges, and each preset height range corresponds to a set of preset rotation parameters.

[0039] B2, determine the preset height range in the preset parameter reference table as the user height range.

[0040] Understandably, the posture height obtained in step B1 is compared with the preset height intervals in the preset parameter lookup table to determine the preset height interval into which the posture height falls, and the corresponding preset height interval is taken as the user height interval. For example, if the posture height is 175 cm and there is a preset height interval of 170 cm to 180 cm in the preset parameter lookup table, then the preset height interval of 170 cm to 180 cm is taken as the user height interval.

[0041] B3, based on the user height range, retrieve the corresponding preset rotation parameters as the relative rotation parameters between the active screen and the fixed screen in the folding screen space, the relative rotation parameters including the relative rotation state and the rotation angle.

[0042] Understandably, based on the user height range determined in step B2, the preset rotation parameters corresponding to the user height range are retrieved from the preset parameter lookup table, and the retrieved preset rotation parameters are used as the relative rotation parameters between the moving screen and the fixed screen.

[0043] S2, obtain the boundary line of the intersection between the active screen and the bottom screen.

[0044] Understandably, when the movable screen rotates relative to the bottom screen, the relative position of the intersection between the movable screen and the bottom screen will change, causing the originally continuous image to be missing or broken at the intersection. This step first determines the boundary line where the movable screen and the bottom screen intersect, and subsequent image processing operations are all based on the area near this boundary line.

[0045] The junction boundary line refers to the boundary line where the moving screen intersects with the bottom screen when the moving screen does not rotate according to the relative rotation parameters.

[0046] S3, when it is determined that there are no key elements at the intersection boundary line, the background reconstruction processing of the associated area of ​​the bottom screen is performed according to the relative rotation parameters to obtain the processed image to be rendered.

[0047] Among them, key elements refer to the main objects in the video frame that serve as the subject of expression, such as people, animals, buildings, and text. For example, if there is a horse on the grassland in the video frame, the key element is the horse, and the grassland is the background. The associated area refers to the area on the bottom screen that needs to be processed after the moving screen is rotated, including the supplementary area and the occluded area. The image to be rendered refers to the image obtained after processing the image according to the relative rotation parameters and used for subsequent rendering.

[0048] It should be noted that video footage typically includes a main subject and a background. When the moving screen rotates, causing changes at the boundary line, if the change occurs in the background area where there is no main subject, the change in that area only affects the background, and it can be directly filled or deleted. This step first determines whether the boundary line falls within the background. If there are no key elements at the boundary line, only the area on the bottom screen affected by the rotation is reconstructed.

[0049] In some embodiments, step S3 (when it is determined that there are no key elements at the junction boundary line, perform background reconstruction processing on the associated area of ​​the bottom screen according to the relative rotation parameters) includes S31-S33: S31, when it is determined that there is no key element at the intersection boundary line, the relative rotation state in the relative rotation parameters is obtained, and the relative rotation state includes outward rotation and inward rotation.

[0050] It should be noted that when the movable screen rotates around its connection point with the main screen, it rotates either outwards or inwards. Rotating outwards will move the movable screen away from the center of the folded screen space, creating a blank area between the movable screen and the bottom screen; rotating inwards will move the movable screen closer to the center of the folded screen space, in which case the movable screen will obscure a portion of the bottom screen.

[0051] Understandably, after determining that there are no key elements at the junction boundary line, the relative rotation state is read from the relative rotation parameters to determine whether the relative rotation state is an outward rotation or an inward rotation.

[0052] S32, when it is determined that the relative rotation state is outward rotation, background filling processing is performed on the supplementary area of ​​the bottom screen.

[0053] It should be noted that when the active screen rotates outward, the junction between the active screen and the bottom screen opens up to form a blank area that originally had no image data. This step fills the background of this newly appeared blank area when rotating outward, so that the area blends with the background image.

[0054] It is understandable that the blank area on the bottom screen that was originally without image data after the active screen was rotated outward is used as a supplementary area, and the background pixel value is retrieved to fill the background of the supplementary area.

[0055] The background pixel value refers to the pixel value used to fill the supplementary area.

[0056] In some embodiments, step S32 (when determining that the relative rotation state is outward rotation, performing background filling processing on the supplementary area of ​​the bottom screen) includes S321-S324: S321, when it is determined that the relative rotation state is outward rotation, the endpoint at the junction boundary line that is in contact with both the front screen and the bottom screen is obtained as the fixed endpoint.

[0057] It is understandable that, among the two endpoints of the boundary line, one endpoint is in contact with both the front screen and the bottom screen, and the position of this endpoint will not change with the rotation of the moving screen, so it is regarded as a fixed endpoint; the position of the other endpoint will change with the rotation of the moving screen.

[0058] S322, retrieve the rotation angle from the relative rotation parameters as the external rotation angle.

[0059] It is understandable that the rotation angle is read from the relative rotation parameters, and the rotation angle when the active screen rotates outward is taken as the outward rotation angle.

[0060] S323, with the fixed endpoint as the center, the intersection boundary line is rotated based on the external rotation angle to obtain the supplementary area through which the intersection boundary line rotates.

[0061] Understandably, see Figure 3 , Figure 3 The diagram illustrates outward rotation from a top-down perspective. Using a fixed endpoint as the center, the boundary line is rotated outward by the specified angle. The area swept by the boundary line during this rotation is the supplementary area. The larger the outward rotation angle, the larger the area swept by the boundary line, and the larger the resulting supplementary area.

[0062] It is worth mentioning that this invention limits the supplementary area to a fan shape when the movable screen rotates outward. When the movable screen rotates outward, the trajectory traced by the outer edge of its bottom edge on the bottom screen is an arc centered at a fixed endpoint. If the supplementary area were defined as a triangle, with the outer edge of the triangle being a straight line, the arc-shaped trajectory of the bottom edge would protrude beyond this straight line during the rotation of the movable screen, causing a blank bottom surface to be exposed during the rotation, resulting in image loss. This invention uses a fan shape for the supplementary area, with the arc edge of the fan coinciding with the rotation trajectory of the outer edge of the bottom edge of the movable screen. Throughout the entire rotation process, the bottom of the movable screen is covered by the filled image from the supplementary area, preventing any image-free bottom surface from being exposed.

[0063] It can be added that, for the final state when the moving screen has finished rotating, the supplementary area can also be a triangle, that is, a triangle formed by connecting the fixed endpoint, the endpoint of the intersection boundary line that is far away from the fixed endpoint before rotation, and the endpoint of the intersection boundary line that is far away from the fixed endpoint after rotation. This can satisfy the screen connection in the final state. The fan shape adopted in this invention can take into account the dynamic process of the moving screen rotation.

[0064] S324, retrieve the background pixel values ​​and fill the supplementary area.

[0065] It should be noted that existing technologies typically fill blank areas by stretching the surrounding image, which may cause image distortion. This step directly retrieves the pixel values ​​of the background portion adjacent to the area to be filled, ensuring a natural transition of the background image and preventing distortion issues.

[0066] It is understandable that the background pixel values ​​are filled into the positions of each pixel in the supplementary area, so that the supplementary area blends with the background image.

[0067] It is worth mentioning that the background pixel value can be a pre-set pixel value based on the background image, or it can be a pixel value that is connected to the surrounding background, calculated by supplementing the known pixel values ​​at the boundary of the region and gradually transferring, spreading and extending from the boundary to the supplemented region based on existing image inpainting algorithms. For example, the image inpainting algorithm based on the fast-moving method, namely the Telea algorithm, is existing technology and will not be elaborated here.

[0068] S33, when the relative rotation state is determined to be an inward rotation state, the occlusion area of ​​the bottom screen is deleted, and the associated area includes the supplementary area and the occlusion area.

[0069] It should be noted that when the movable screen rotates inward, it will block part of the bottom screen. The image data in the blocked area cannot be seen by the user. Since there are no key elements at the junction boundary, deleting this area will not affect the user's viewing and can avoid image duplication and redundancy.

[0070] It is understandable that the area on the bottom screen that is blocked by the moving screen after the moving screen is rotated inward is taken as the blocked area, and the image data in the blocked area is deleted.

[0071] In some embodiments, step S33 (when the relative rotation state is determined to be an inward rotation state, the occlusion area of ​​the bottom screen is deleted) includes S331-S336: S331, when the relative rotation state is determined to be an inward rotation state, the endpoint at the junction boundary line that is in contact with both the front screen and the bottom screen is taken as the fixed endpoint, and the remaining endpoints are taken as the extended endpoints.

[0072] It is understandable that when the relative rotation state is determined to be an inward rotation state, referring to step S321, a fixed endpoint that is in contact with the front screen and the bottom screen and whose position does not change with the rotation of the moving screen is obtained from the two endpoints of the intersection boundary line, and the other endpoint is used as the extension endpoint.

[0073] S332, retrieve the rotation angle from the relative rotation parameters as the internal rotation angle.

[0074] It is understandable that the rotation angle is read from the relative rotation parameters, and the rotation angle when the active screen rotates inward is taken as the inward rotation angle.

[0075] S333, with the fixed endpoint as the center, the intersection boundary line is rotated based on the inward rotation angle to obtain the rotated intersection boundary line.

[0076] Understandably, see Figure 4 , Figure 4 The diagram illustrates the inward rotation from a top-down perspective. Using the fixed endpoint as the center, the boundary line is rotated inward by an inward rotation angle. The boundary line at the new position is taken as the new boundary line after rotation.

[0077] S334, the intersection boundary line after rotation is extended to obtain an extended ray, and the connection point between the extended ray and the corresponding edge line of the bottom screen is obtained.

[0078] It is understandable that the endpoints on the rotated boundary line other than the fixed endpoints are taken as the updated endpoints, and the direction from the fixed endpoints to the updated endpoints is taken as the extension direction. The rotated boundary line is then extended to obtain an extended ray, and the intersection of the extended ray and the corresponding edge line of the bottom screen is taken as the connection intersection point.

[0079] Among them, the edge line refers to the outer boundary line of the bottom screen.

[0080] S335, connect the connection point with the extended endpoint of the intersection boundary line to obtain the shading area.

[0081] Understandably, the first connecting line is obtained by connecting the intersection point to the corresponding extended endpoint of the boundary line, and the second connecting line is obtained by connecting the fixed endpoint to the intersection point. The area enclosed by the first connecting line, the second connecting line, and the boundary line is the shading area.

[0082] It is worth mentioning that this invention defines the obstructed area as a triangle when the active screen rotates inward, rather than a fan-shaped area swept by the rotating boundary line. When the active screen rotates inward, it obstructs the area of ​​the bottom screen near the boundary line, an area that the user cannot see. If the obstructed area were a fan-shaped area swept by the rotating boundary line, there would still be a missing area between the outer arc of the fan-shaped area and the edge of the bottom screen, which the user cannot see. This invention extends the rotating boundary line to the edge of the bottom screen, making the obstructed area a triangle, and deletes the obstructed area between the fan-shaped area and the edge of the bottom screen, avoiding residual image data in the obstructed area.

[0083] S336, the occluded area is deleted.

[0084] It is understandable that the image data within the obstructed area will be deleted.

[0085] S4, when it is determined that there is a key element at the intersection boundary line, the key element is moved according to the relative rotation parameters to obtain the processed image to be rendered.

[0086] It should be noted that if the boundary line is on the main object in the video frame, then there are key elements that span the screen at the boundary line. In this case, rotating the moving screen will cause the key elements to break or be partially missing. Existing technologies typically handle this situation by stretching or truncating. Stretching the image will cause the main object to deviate from its original proportions; for example, a stretched person will not be in proportion to the original. Truncation will directly discard a part of the main object, resulting in an incomplete display. Therefore, this step, when there are key elements at the boundary line, relocates the key elements according to relative rotation parameters, moving them to an area where they can be fully displayed, thus preserving the true shape of the key elements.

[0087] In some embodiments, step S4 (when it is determined that there is a key element at the junction boundary line, the key element is relocated according to the relative rotation parameters) includes S41-S43: S41, when it is determined that there is a key element at the junction boundary line, the relative rotation state in the relative rotation parameters is obtained, and the relative rotation state includes outward rotation and inward rotation.

[0088] Understandably, after determining that there are key elements at the junction boundary line, the relative rotation state is read from the relative rotation parameters to determine whether the relative rotation state is an outward rotation or an inward rotation.

[0089] S42, when the relative rotation state is determined to be outward rotation, the overall position migration processing of the pixels of the key elements is performed based on the distribution ratio of the key elements. It should be noted that when there are key elements at the boundary line, these key elements span the junction of the moving screen and the bottom screen, with part of them located on the moving screen and the other part on the bottom screen. When the moving screen rotates outward, the originally continuous key elements will be separated by the opened blank area at the junction, causing a break in the key elements. In this step, during the outward rotation, based on the proportion of the key elements on the moving screen and the bottom screen respectively, the key elements are shifted as a whole to the side with the larger proportion, allowing the key elements to regroup on a single screen and maintain their complete shape.

[0090] The distribution ratio refers to the ratio of the number of pixels of a key element on a certain screen to the total number of pixels of the key element.

[0091] In some embodiments, step S42 (when the relative rotation state is determined to be outward rotation, performing overall position migration processing on the pixels of the key elements based on the distribution ratio of the key elements) includes S421-S425: S421, when the relative rotation state is determined to be outward rotation, calculate the first distribution ratio of the key element on the active screen and the second distribution ratio on the bottom screen.

[0092] It should be noted that the key elements at the boundary line each occupy a portion on the active screen and the bottom screen. Which screen to move the key elements to depends on which screen the key elements occupy a larger proportion on.

[0093] Understandably, the ratio of the number of pixels of the key element on the active screen to the total number of pixels of the key element is used as the first distribution percentage, and the ratio of the number of pixels of the key element on the bottom screen to the total number of pixels of the key element is used as the second distribution percentage. For example, if the key element has a total of 100 pixels, with 60 pixels on the active screen and 40 pixels on the bottom screen, then the first distribution percentage is 60%, and the second distribution percentage is 40%.

[0094] S422, based on the active screen and the fixed screen, a display screen is obtained.

[0095] It is understandable that both the active screen and the bottom screen, which carry the key elements, are used as display screens.

[0096] S423, compare the first distribution ratio with the second distribution ratio, and determine the display screen with the largest ratio as the target carrier screen, and the remaining display screens as the removal screens.

[0097] Understandably, by comparing the first distribution percentage with the second distribution percentage, the display screen with the larger percentage is designated as the target display screen, and the display screen with the smaller percentage is designated as the removal display screen. For example, if the first distribution percentage is 60% and the second distribution percentage is 40%, and the first distribution percentage is greater than the second distribution percentage, then the active screen corresponding to the first distribution percentage is designated as the target display screen, and the bottom screen corresponding to the second distribution percentage is designated as the removal display screen.

[0098] S424, the pixels of the corresponding key elements are shifted as a whole to the target carrier screen until the screen no longer has the corresponding key elements, and the position of the missing pixels after the shift is obtained.

[0099] It should be noted that, in order to ensure that the key element is displayed completely on the target display screen, this step translates all the pixels of the key element as a whole onto the target display screen, so that the entire key element is moved onto the target display screen. During the translation process, the shape of the key element itself and the relative positions between each pixel remain unchanged.

[0100] Understandingly, all pixels of the key element on both the removal screen and the target screen are treated as a whole and translated towards the target screen. The relative positions of the pixels within the key element remain unchanged during the translation process until the removal screen no longer contains any pixels of the key element. At this point, the entire key element is located on the target screen. After the pixels of the key element on the removal screen are translated, the positions of the pixels that were originally occupied by the key element and are now vacant are designated as empty pixel positions.

[0101] S425, retrieve the background pixel value and fill the missing pixel position.

[0102] Understandably, referring to step S324, the background pixel value is retrieved and filled into the position of the empty pixel point, so that the empty position after the key element on the screen is moved away is covered by the background image.

[0103] S43, when it is determined that the relative rotation state is an inward rotation state, based on the inline distribution of the key element, the position migration processing is performed on the part of the key element on the bottom screen.

[0104] It should be noted that when the moving screen rotates inward, it will obscure part of the bottom screen, causing some key elements to be hidden and unable to be displayed. This step, during the inward rotation, moves the obscured key elements from the bottom screen row by row to the unobscured positions on the bottom screen, ensuring that the pixels of the key elements are not obscured.

[0105] Among them, a pixel row refers to a row of pixels arranged horizontally in the image, and the image is composed of multiple pixel rows arranged sequentially in the vertical direction; the in-row distribution refers to the distribution of pixels of key elements within each pixel row of the bottom screen.

[0106] In some embodiments, step S43 (when determining that the relative rotation state is an inward rotation state, performing position migration processing on the portion of the key element on the bottom screen based on the inline distribution of the key element) includes S431-S435: S431, when it is determined that the relative rotation state is an inward rotation state, the occlusion area and display area of ​​the bottom screen are obtained, and the migration pixel points of the key elements located in the occlusion area are obtained, and the pixel row where the migration pixel points are located is taken as the migration row.

[0107] It should be noted that after the moving screen rotates inward, the bottom screen is divided into an area obscured by the moving screen and an area that is not obscured and can still be displayed. Pixels of key elements located in the obscured area cannot be displayed and need to be moved to the unobscured area. Since the number of obscured key element pixels varies in different pixel rows, this step processes the pixels row by row. First, the pixel rows containing key element pixels in the obscured area are identified, and then the migration is performed row by row.

[0108] Understandably, referring to steps S331-S335, the occlusion area on the bottom screen that is blocked by the active screen is obtained, and the remaining area on the bottom screen is used as the display area. Pixels within the occlusion area of ​​the key element are used as migration pixels, and the pixel rows containing the migration pixels are used as migration rows.

[0109] S432, Identify irrelevant pixels in the migration row within the display area that do not belong to key elements.

[0110] Understandably, within the display area, all pixels in the migration row that are not key elements are treated as irrelevant pixels.

[0111] S433, count the number of migrated pixels in the migration row as the deletion count for the corresponding migration row, and delete irrelevant pixels in the corresponding migration row based on the deletion count to obtain the position of the filling pixel.

[0112] It should be noted that for a pixel to be migrated into the display area, the same number of pixel positions within the display area must be vacated. This step counts the number of migrated pixels in each migration row and, based on this count, deletes irrelevant pixels from the corresponding migration row within the display area, vacating positions equal to the number of migrated pixels.

[0113] Understandably, the number of migrated pixels within each migration row is counted, and this count is used as the deletion count for that migration row. Based on this deletion count, irrelevant pixels in the corresponding migration row are deleted from the display area, and the resulting empty pixel positions are used as fill pixel positions. For example, if a migration row contains 2 migrated pixels, then the deletion count for that migration row is 2. Deleting these 2 irrelevant pixels from the display area results in 2 fill pixel positions.

[0114] S434, obtain the first direction in which the migrated pixel is located at the position of the filled pixel, and take the direction opposite to the first direction as the filling direction.

[0115] Understandably, within the migration row, the direction of the migrated pixel relative to the fill pixel's position is determined. The direction of the migrated pixel relative to the fill pixel's position is taken as the first direction, and the direction opposite to the first direction is taken as the fill direction. The fill direction is the direction the migrated pixel moves. For example, if the migrated pixel is to the right of the fill pixel's position, then the first direction is right, the fill direction is left, and the migrated pixel should move to the left.

[0116] S435, move the migrated pixel based on the filling direction until the migrated pixel enters the display area.

[0117] It should be noted that in this step, when moving the migrated pixels, all pixels from the migrated pixel to the position of the filling pixel are moved together as a whole along the filling direction until the migrated pixels in the migration row enter the display area, ensuring the continuity and integrity of key elements in the migration row.

[0118] Understandably, the starting pixel in the migration row, furthest from the fill pixel, can be used as the starting pixel. All pixels from the starting pixel to the fill pixel are then moved together along the fill direction as a whole until the migrated pixel enters the display area. For example, ... Figure 5 As shown, there are two migrated pixels in the occluded area within a certain migration row, namely A and B, with A being the starting pixel. There are four pixels in the key elements of the display area, namely C, D, E, and F. Two irrelevant pixels in the migration row are deleted within the display area, resulting in two filling pixel positions, namely g and h. The filling direction is from A to h. All pixels from A to h are moved together as a whole along the filling direction by a distance of 2 pixels. That is, A, B, C, D, E, and F are moved together as a whole. After the movement, E enters g, F enters h, and A and B enter the display area.

[0119] In addition, after the migration of pixels is completed, the remaining image data in the occluded area is deleted.

[0120] S5, render the image to be rendered to obtain a naked-eye 3D video adapted to the folded screen space.

[0121] Understandably, after background reconstruction or repositioning, the image to be rendered is obtained. Based on the positional relationship of each screen within the folded screen space after the moving screen rotates, the image to be rendered is processed, and the rendered video is used as a naked-eye 3D video adapted to the folded screen space and distributed to each screen in the folded screen space for playback.

[0122] See Figure 6 This is a schematic diagram of a Cave folding screen video processing system provided in an embodiment of the present invention. The system includes: An information module is used to determine the relative rotation parameters between the movable screen and the fixed screen in the folded screen space. The fixed screen includes a front screen and a bottom screen. The movable screen is rotatably connected to the left and right sides of the front screen. The determining module is used to obtain the boundary line of the intersection between the active screen and the bottom screen; The reconstruction module is used to reconstruct the background of the associated area of ​​the bottom screen according to the relative rotation parameters when it is determined that there are no key elements at the intersection boundary line, so as to obtain the processed image to be rendered. The migration module is used to determine that there are key elements at the intersection boundary line, and then perform position migration processing on the key elements according to the relative rotation parameters to obtain the processed image to be rendered. The rendering module is used to render the image to be rendered to obtain a naked-eye 3D video adapted to the folded screen space.

[0123] See Figure 7 This is a schematic diagram of the hardware structure of an electronic device 70 provided in an embodiment of the present invention. The electronic device 70 includes: a processor 71, a memory 72, and a computer program; wherein... The memory 72 is used to store the computer program, and the memory may also be flash memory. The computer program is, for example, an application program or functional module that implements the above method.

[0124] The processor 71 is configured to execute the computer program stored in the memory to implement the various steps performed by the device in the above method. For details, please refer to the relevant descriptions in the preceding method embodiments.

[0125] Alternatively, the memory 72 can be either standalone or integrated with the processor 71.

[0126] When the memory 72 is a device independent of the processor 71, the device may further include: Bus 73 is used to connect the memory 72 and the processor 71.

[0127] The present invention also provides a readable storage medium storing a computer program, which, when executed by a processor, is used to implement the methods provided in the various embodiments described above.

[0128] The readable storage medium can be a computer storage medium or a communication medium. A communication medium includes any medium that facilitates the transfer of computer programs from one location to another. A computer storage medium can be any available medium accessible to a general-purpose or special-purpose computer. For example, a readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application-Specific Integrated Circuit (ASIC). Alternatively, the ASIC can be located in a user equipment. Of course, the processor and the readable storage medium can also exist as discrete components in a communication device. The readable storage medium can be a read-only memory (ROM), random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0129] The present invention also provides a program product including executable instructions stored in a readable storage medium. At least one processor of the device can read the executable instructions from the readable storage medium, and the at least one processor executes the executable instructions to cause the device to implement the methods provided in the various embodiments described above.

[0130] In the embodiments of the above-described device, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.

[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for processing video based on Cave folding screen, characterized in that, include: Determine the relative rotation parameters between the movable screen and the fixed screen in the folding screen space. The fixed screen includes a front screen and a bottom screen. The movable screen is rotatably connected to the left and right sides of the front screen. Obtain the boundary line between the active screen and the bottom screen; When it is determined that there are no key elements at the intersection boundary line, the background reconstruction process is performed on the associated area of ​​the bottom screen according to the relative rotation parameters to obtain the processed image to be rendered. When it is determined that there are key elements at the boundary line, the key elements are moved according to the relative rotation parameters to obtain the processed image to be rendered. The image to be rendered is processed to obtain a naked-eye 3D video adapted to the folded screen space.

2. The method according to claim 1, characterized in that, Determining the relative rotation parameters between the movable screen and the fixed screen in the folding screen space includes: The user terminal inputs customized rotation parameters as relative rotation parameters between the moving screen and the fixed screen in the folded screen space. The relative rotation parameters include the relative rotation state and the rotation angle.

3. The method according to claim 1, characterized in that, Determining the relative rotation parameters between the movable screen and the fixed screen in the folding screen space includes: The user's posture height is obtained, and a preset parameter lookup table is retrieved. The preset parameter lookup table has a one-to-one correspondence between preset height ranges and preset rotation parameters. The user height range is determined by the preset height range in the preset parameter lookup table. Based on the user's height range, the corresponding preset rotation parameters are retrieved as the relative rotation parameters between the moving screen and the fixed screen in the folding screen space. The relative rotation parameters include the relative rotation state and the rotation angle.

4. The method according to claim 1, characterized in that, When it is determined that there are no key elements at the intersection boundary line, background reconstruction processing is performed on the associated area of ​​the bottom screen according to the relative rotation parameters, including: When it is determined that there are no key elements at the intersection boundary line, the relative rotation state in the relative rotation parameters is obtained, and the relative rotation state includes outward rotation and inward rotation; When the relative rotation state is determined to be outward rotation, the supplementary area of ​​the bottom screen is filled with background. When the relative rotation state is determined to be an inward rotation state, the occlusion area of ​​the bottom screen is deleted. The associated area includes the supplementary area and the occlusion area.

5. The method according to claim 4, characterized in that, When the relative rotation state is determined to be outward rotation, background filling processing is performed on the supplementary area of ​​the bottom screen, including: When the relative rotation state is determined to be outward rotation, the endpoint at the junction boundary line that is in contact with both the front screen and the bottom screen is taken as the fixed endpoint. The rotation angle in the relative rotation parameters is retrieved as the external rotation angle; Using the fixed endpoint as the center, the intersection boundary line is rotated based on the outward rotation angle to obtain the supplementary area through which the intersection boundary line rotates. The background pixel values ​​are retrieved to fill the supplementary area.

6. The method according to claim 4, characterized in that, When the relative rotation state is determined to be an inward rotation state, the occlusion area of ​​the bottom screen is deleted, including: When the relative rotation state is determined to be an inward rotation state, the endpoint at the junction boundary line that is in contact with both the front screen and the bottom screen is taken as the fixed endpoint, and the remaining endpoints are taken as the extended endpoints. The rotation angle in the relative rotation parameters is taken as the inward rotation angle; With the fixed endpoint as the center, the intersection boundary line is rotated based on the inward rotation angle to obtain the rotated intersection boundary line; The rotated boundary line is extended to obtain an extended ray, and the intersection point of the extended ray and the corresponding edge line of the bottom screen is obtained. Connect the intersection point with the extended endpoint of the intersection boundary line to obtain the shading area; The obscured area is then deleted.

7. The method according to claim 1, characterized in that, When it is determined that there is a key element at the boundary line, the key element is relocated according to the relative rotation parameters, including: When it is determined that there is a key element at the intersection boundary line, the relative rotation state in the relative rotation parameters is obtained, and the relative rotation state includes outward rotation and inward rotation; When the relative rotation state is determined to be outward rotation, the pixels of the key elements are subjected to overall position migration processing based on the distribution ratio of the key elements. When the relative rotation state is determined to be an inward rotation state, the position migration processing is performed on the part of the key element on the bottom screen based on the inline distribution of the key element.

8. The method according to claim 7, characterized in that, When the relative rotation state is determined to be outward rotation, based on the distribution ratio of the key elements, the pixels of the key elements are subjected to overall position migration processing, including: When the relative rotation state is determined to be outward rotation, the first distribution ratio of the key element on the active screen and the second distribution ratio on the bottom screen are calculated respectively. Based on the movable screen and the fixed screen, the display screen is obtained; Compare the first distribution ratio with the second distribution ratio, and determine the display screen with the largest ratio as the target carrying screen, and the remaining display screens as the removal screens; The pixels of the corresponding key elements are translated as a whole to the target supporting screen until the screen no longer has the corresponding key elements, and the position of the missing pixels after translation is obtained. The background pixel values ​​are retrieved to fill the positions of the missing pixels.

9. The method according to claim 7, characterized in that, When determining that the relative rotation state is an inward rotation state, based on the inline distribution of the key element, the portion of the key element on the bottom screen is subjected to position migration processing, including: When the relative rotation state is determined to be an inward rotation state, the occlusion area and display area of ​​the bottom screen are obtained, and the migration pixels of the key elements located in the occlusion area are obtained, and the pixel row where the migration pixel is located is taken as the migration row. Identify irrelevant pixels in the migration row within the display area that do not belong to key elements; The number of migrated pixels in the migration row is counted as the deletion count for the corresponding migration row. Irrelevant pixels in the corresponding migration row are deleted based on the deletion count to obtain the position of the filling pixel. Obtain the first direction in which the migrated pixel is located at the position of the filled pixel, and take the direction opposite to the first direction as the filling direction; Move the migrated pixels according to the filling direction until the migrated pixels enter the display area.

10. A processing system for Cave folding screen video, characterized in that, include: An information module is used to determine the relative rotation parameters between the movable screen and the fixed screen in the folded screen space. The fixed screen includes a front screen and a bottom screen. The movable screen is rotatably connected to the left and right sides of the front screen. The determining module is used to obtain the boundary line of the intersection between the active screen and the bottom screen; The reconstruction module is used to reconstruct the background of the associated area of ​​the bottom screen according to the relative rotation parameters when it is determined that there are no key elements at the intersection boundary line, so as to obtain the processed image to be rendered. The migration module is used to determine that there are key elements at the intersection boundary line, and then perform position migration processing on the key elements according to the relative rotation parameters to obtain the processed image to be rendered. The rendering module is used to render the image to be rendered to obtain a naked-eye 3D video adapted to the folded screen space.

Images