A data processing apparatus, method and computer readable storage medium

Abstract

The application provides a data processing device, method and computer readable storage medium, and belongs to the technical field of data processing. The device comprises: an information determination module configured to determine mapping information and synchronization information, wherein the mapping information is used to represent a spatial correspondence relationship between a control position in transmittance control data and a transmittance modulation object, and the synchronization information is used to represent a time correspondence relationship between the transmittance control data and display data; and an encoding module connected with the information determination module, configured to establish an association relationship between the transmittance control data and the mapping information and the synchronization information, and to encode the transmittance control data, the mapping information and the synchronization information to obtain transparent display encoding data. The application can preserve the spatial relationship and the time relationship corresponding to the transmittance control data in the encoding process.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data processing apparatus, method, and computer-readable storage medium. Background Technology

[0002] A transparent display system can output display content through transparent display units and control the transmittance of different positions or areas through a transmittance modulation structure set on the light transmission path. Transmittance control data can be provided as independent data, auxiliary bitstream, data track, or configuration data.

[0003] In existing transparent display data processing methods, the correspondence between transmittance control data and transparent display execution process is unclear. The playback side has difficulty in determining the transmittance modulation object corresponding to the control position, as well as the display data or display time corresponding to the corresponding transmittance control data. Summary of the Invention

[0004] This application provides a data processing apparatus, method, and computer-readable storage medium that can preserve the spatial and temporal relationships corresponding to transmittance control data during the encoding process.

[0005] The technical solution of this application is implemented as follows: In a first aspect, this application provides a data processing apparatus applied to the content side, comprising: an information determination module for determining mapping information and synchronization information, wherein the mapping information is used to characterize the spatial correspondence between the control position in the transmittance control data and the transmittance modulation object, and the synchronization information is used to characterize the temporal correspondence between the transmittance control data and the display data; and an encoding module connected to the information determination module for establishing the association between the transmittance control data and the mapping information and the synchronization information, and encoding the transmittance control data, the mapping information and the synchronization information to obtain transparent display encoded data.

[0006] Secondly, this application provides a data processing apparatus applied on the playback side, comprising: a decoding module for decoding transparent display encoded data to obtain interrelated transmittance control data, mapping information, and synchronization information; a spatial processing module connected to the decoding module for mapping the control position in the transmittance control data to a target transmittance modulation object according to the mapping information to obtain target transmittance control data; a time processing module connected to the decoding module for determining the output timing of the target transmittance control data according to the synchronization information; and a control output module connected to both the spatial processing module and the time processing module for outputting the target transmittance control data according to the output timing.

[0007] Thirdly, this application provides a data processing method applied to the content side, comprising: determining mapping information and synchronization information, wherein the mapping information represents the spatial correspondence between the control position in the transmittance control data and the transmittance modulation object, and the synchronization information represents the temporal correspondence between the transmittance control data and the display data; establishing the association between the transmittance control data and the mapping information and the synchronization information; and encoding the transmittance control data, the mapping information and the synchronization information to obtain transparent display encoded data.

[0008] Fourthly, this application provides a data processing method applied to the playback side, comprising: decoding transparent display encoded data to obtain interrelated transmittance control data, mapping information and synchronization information; mapping the control position in the transmittance control data to a target transmittance modulation object according to the mapping information to obtain target transmittance control data; determining the output timing of the target transmittance control data according to the synchronization information; and outputting the target transmittance control data according to the output timing.

[0009] Fifthly, this application provides an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the data processing method as described in the third or fourth aspect.

[0010] In a sixth aspect, this application provides a computer-readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the data processing method as described in the third or fourth aspect.

[0011] This application provides a data processing apparatus, method, and computer-readable storage medium. Transmittance control data can only represent transmittance control requirements and cannot directly indicate the transmittance modulation object and display time corresponding to the control value. By determining mapping information and synchronization information, a spatial correspondence between the control position and the transmittance modulation object, as well as a temporal correspondence between transmittance control data and display data, are established. Then, the association between these three is established and encoded, enabling the transparent display encoded data to retain and restore the corresponding spatial and temporal relationships, providing a data foundation for the correct application of transmittance control data on the playback side. Attached Figure Description

[0012] Figure 1 This is a schematic diagram of the structure of a transparent display data processing system provided in this application.

[0013] Figure 2 This is a schematic diagram illustrating the light propagation between the transparent display unit, the transmittance modulation layer, and the background display unit provided in this application.

[0014] Figure 3 A structural block diagram of a first data processing device provided in this application.

[0015] Figure 4 This application provides a structural block diagram of a second data processing device.

[0016] Figure 5 This is a flowchart illustrating a data processing method provided in this application.

[0017] Figure 6 This is a schematic diagram of the hardware structure of an electronic device provided in this application. Detailed Implementation

[0018] The technical solutions in the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0019] The terms "first," "second," etc., used in this application's specification are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, the first object can be one or more.

[0020] Figure 1 In this transparent display data processing system, a content side 100 and a playback side 200 may be included. The playback side 200 includes a transparent display unit 300 and a transmittance modulation layer 400. The content side 100 is used for encoding data, and the playback side 200 is used for decoding data. The content side 100 may include a first data processing device 110, and the playback side 200 may also include a second data processing device 220. The first data processing device 110 and the second data processing device 220 may be implemented by the same electronic device, or they may be implemented by a content processing device and a playback processing device, respectively.

[0021] like Figure 2The diagram illustrates the light propagation between the transparent display unit 300, the transmittance modulation layer 400, and the background display unit 500. The transparent display unit 300 is used to output the foreground content corresponding to the display data. The display data may include video data, image data, or graphic data. The transparent display unit 300 can be a light-emitting diode (LED) grid screen, a transparent LED display screen, a transparent organic light-emitting diode display screen, a transparent micro-light-emitting diode display screen, a transparent liquid crystal display screen, or a transparent projection medium.

[0022] A transmittance modulation layer 400 is disposed on the light transmission path of the transparent display unit 300 and includes at least one transmittance modulation object 410, the light transmission path being shown as a dashed arrow in the figure. The transmittance modulation object 410 is an object within the transmittance modulation layer 400 that can serve as a transmittance control target; different fill sizes of the transmittance modulation object 410 in the figure represent different transmittances. For example, the transmittance modulation object 410 can be a light valve unit, a pixel, a pixel group, a light bar, a modulation zone, or an image area. The transmittance modulation layer 400 can be located on the front side, rear side, internal interlayer of the transparent display unit 300, or in an independent optical layer adjacent to the transparent display unit 300.

[0023] The background object is located behind the transparent display unit 300 and can be observed by the observer 310 through the transparent display unit 300. The background object can be an environmental background, exhibits, or other physical objects. The background display unit 500 is an active display device capable of outputting background display content, such as a liquid crystal display screen, a light-emitting diode display screen, an organic light-emitting diode display screen, or a projection display device. The background display unit 500 is a specific form of background object. The data processing device can process transmittance control data, mapping information, and synchronization information without the background display unit 500.

[0024] The transmittance modulation object 410 can correspond to the display pixels, pixel groups, light-emitting units, light strips, light-transmitting openings, display zones, or image areas of the transparent display unit 300. In an LED grid screen scenario, the transmittance modulation object 410 can cover the light-emitting unit and its surrounding light-transmitting openings, allowing the transmittance control data to act on the location of the light-emitting unit and control the background light corresponding to the light-transmitting gaps between the light-emitting units. The transmittance control data is used to indicate the transmittance of light passing through each transmittance modulation object 410.

[0025] For example, the transparent display unit 300 displays a foreground graphic, the background display unit 500 displays a background image, and the transmittance modulation layer 400 reduces the transmittance of the corresponding area of ​​the foreground graphic and increases the transmittance of the background observation area based on existing transmittance control data. This example only illustrates the usage environment of the transmittance control data and does not limit its generation process.

[0026] like Figure 3 As shown, the first data processing device 110 is disposed on the content side 100. The first data processing device 110 includes an information determination module 112 and an encoding module 114.

[0027] The information determination module 112 is used to determine the mapping information and the synchronization information; the encoding module 114 is connected to the information determination module 112 and is used to establish the correlation between the transmittance control data, the mapping information and the synchronization information, and to encode and generate transparent display encoded data.

[0028] The information determination module 112 can read mapping information and synchronization information from configuration files, calibration data, content description data, data header fields or external control data, and can also perform calculations, selections, conversions or updates based on spatial parameters and time identifiers.

[0029] Transmittance control data can include multiple control values. A control value can represent a control location or a transmittance control requirement corresponding to a transmittance modulation object. A control location can represent a point location or a region location. The control values ​​in the transmittance control data can be logical control values. Logical control values ​​can be converted into data that the transmittance modulation hardware can execute during subsequent processing.

[0030] For example, transmittance control data includes control value T1 and control value T2. Control value T1 can indicate that a first transmittance control state is required for a first location, and control value T2 can indicate that a second transmittance control state is required for a second region.

[0031] Transmittance control data is used to describe the transmittance control requirements for transparent display content. Different transparent display contents have different spatial structures and variation characteristics, and accordingly, transmittance control data can be at least one of raster data, area parameter data, vector data, or differential data.

[0032] Raster data consists of multiple raster cells arranged in a predetermined row and column structure, with control values ​​set for each raster cell. A single raster cell represents a control location within the transmittance control data space. Raster data may include raster width, raster height, and multiple control values ​​arranged in raster order. The raster width represents the number of horizontal raster cells, and the raster height represents the number of vertical raster cells.

[0033] For example, transmittance control data uses a 100×60 grid. 100 indicates 100 grid cells horizontally, and 60 indicates 60 grid cells vertically. Each grid cell is configured with a transmittance control value. The grid cells located in column 20 and row 15 are configured with control value T1, which represents the transmittance control requirement for the corresponding control position.

[0034] Raster data is suitable for representing content where control positions are continuously distributed or where transmittance control requirements vary with spatial location. The raster resolution in raster data refers to the data representation resolution and is not required to be the same as the actual resolution of the transmittance modulation hardware.

[0035] Zone parameter data, when representing a zone by a control position, indicates the transmittance control requirements for that zone. Zone parameter data can include control zone boundaries, control zone outlines, or control zone position parameter definitions. Zone parameter data can also include multiple control zones. Different control zones can be configured with different control values.

[0036] For example, the region parameter data includes a rectangular region R1 and a control value T2. The rectangular region R1 is defined by the upper left corner (100, 80) and the lower right corner (300, 240). The control value T2 applies to the spatial range covered by the rectangular region R1.

[0037] Vector data represents transmittance control requirements through geometric objects and associated control parameters. Geometric objects can include points, line segments, curves, polygons, contours, or composite objects formed by multiple geometric elements.

[0038] Vector data can include geometric object types, spatial parameters of the geometric objects, and control parameters. The geometric object type is used to determine the interpretation rules of the geometric object, the spatial parameters are used to determine the position and shape of the geometric object, and the control parameters are used to determine the transmittance control requirements within the coverage area of ​​the geometric object.

[0039] For example, vector data includes a polygon P1 and a control value T4. Polygon P1 is defined by vertices V1, V2, V3, and V4. Another example is a closed contour C1. The closed contour C1 is used to describe the outer boundary of a displayed object. Vector data is suitable for representing transparent display content with clear outlines, well-defined geometric structures, or control areas that can be described by geometric objects.

[0040] Differential data represents the difference between the current transmittance control data and the reference transmittance control data. The reference transmittance control data is the transmittance control data used as the basis for comparison. The current transmittance control data is the transmittance control data that needs to be encoded. Differential data may include control positions or control values ​​that have changed. The encoding module 114 compares the current transmittance control data with the reference transmittance control data, determines the changed data portion, and encodes the changed data portion.

[0041] For example, the baseline transmittance control data includes 100 × 60 control values. Currently, only the control values ​​within region R3 change in the transmittance control data. Encoding module 114 can generate differential data including region R3 and the changed control value T5, without repeatedly encoding the control values ​​outside region R3 that have not changed.

[0042] For example, the control position P2 in the baseline transmittance control data has a control value T6, and the control position P2 in the current transmittance control data has a control value T7. Differential data can record the control position P2 and the control value T7.

[0043] Transmittance control data can be expressed in one format or a combination of multiple formats. For example, transmittance control data can use region parameter data to describe a large control area, while simultaneously using vector data to describe local objects with irregular contours within the control area. Encoding module 114 encodes the region parameter data and vector data separately, maintaining the relationship between the region parameter data, vector data, and the same transmittance control data. As another example, the baseline transmittance control data can use raster data, and subsequent transmittance control data can use differential data. Raster data provides the complete control state, while differential data provides the data portions that change relative to the complete control state.

[0044] By setting multiple transmission rate control data expression formats, the encoding module 114 can select the corresponding data structure according to the spatial granularity, regional structure, geometric features, or change features of the transmission rate control content.

[0045] Synchronization information is used to characterize the time correspondence between transmittance control data and display data. The time correspondence refers to the relationship between a transmittance control data and a display data, a display stage, or a display moment.

[0046] For example, display data D1 represents the first displayed content, and transmittance control data T1 represents the transmittance control requirements in conjunction with the first displayed content. Synchronization information can indicate that transmittance control data T1 corresponds to display data D1. After the playback side obtains the synchronization information, it can use transmittance control data T1 when processing display data D1.

[0047] The mapping information is used to characterize the spatial correspondence between the control position in the transmittance control data and the transmittance modulation object 410. Spatial correspondence refers to the relationship between spatial objects in the transmittance control data and objects on the transmittance modulation execution side. The spatial object can be a control position, and the execution side object can be the transmittance modulation object 410.

[0048] For example, the mapping information can indicate that control position P1 corresponds to transmittance modulation object O1, and control region R1 corresponds to transmittance modulation object O2. After the playback side obtains the mapping information, the playback side can determine that the control value corresponding to control position P1 should be used for transmittance modulation object O1, and can determine that the control value corresponding to control region R1 should be used for transmittance modulation object O2.

[0049] The information determination module 112 acquires the control position in the transmittance control data and determines the transmittance modulation object corresponding to the control position. Based on this, the information determination module 112 forms mapping information to characterize the spatial correspondence. When determining synchronization information, the information determination module 112 determines the display data or display stage corresponding to the transmittance control data. Based on this, the information determination module 112 forms or acquires synchronization information to characterize the temporal correspondence.

[0050] The information determination module 112 provides mapping information and synchronization information to the encoding module 114. The connection between the information determination module 112 and the encoding module 114 can be a physical connection or a logical connection for transmitting data.

[0051] Association refers to the data relationship between a mapping information and a synchronization information, specifying which transmittance control data it corresponds to. Association ensures that transmittance control data, mapping information, and synchronization information can still be mutually identified after encoding.

[0052] For example, the encoding module 114 acquires transmittance control data T1, mapping information M1, and synchronization information S1. Mapping information M1 indicates that the control position P1 in transmittance control data T1 corresponds to transmittance modulation object O1, and synchronization information S1 indicates that transmittance control data T1 corresponds to display data D1. The encoding module 114 establishes a set of corresponding data relationships between transmittance control data T1, mapping information M1, and synchronization information S1. After the decoding side acquires this set of data relationships, it can determine that both mapping information M1 and synchronization information S1 are applicable to transmittance control data T1.

[0053] Encoding module 114 encodes the transmittance control data, mapping information, and synchronization information to obtain transparent display encoded data. Transparent display encoded data refers to data that retains the transmittance control data, mapping information, and synchronization information, as well as the relationships between them.

[0054] In this embodiment of the application, by establishing and encoding the association between transmittance control data and mapping information and synchronization information, the resulting transparent display encoded data can characterize the transmittance modulation object to which the transmittance control data is applicable and the corresponding display data or display time, thereby providing a data basis for the playback side to restore and apply spatial and temporal correspondences.

[0055] The specific process for determining the mapping information can be as follows: the information determination module 112 can obtain the first spatial parameter and the second spatial parameter, and determine the mapping information based on the first spatial parameter, the second spatial parameter and the spatial transformation relationship between them.

[0056] The first spatial parameter describes the spatial attributes of the transmittance control data space, and the transmittance modulation space represents the spatial domain of the position and arrangement of the transmittance modulation object 410 in the transmittance modulation layer 400. The first spatial parameter may include at least one of the following: spatial size, spatial resolution, coordinate origin, coordinate axis direction, position spacing, region boundary, or position index range.

[0057] For example, transmittance control data includes 1920 control positions horizontally and 1080 control positions vertically. The transmittance control data space can be represented as a two-dimensional discrete space of 1920 × 1080. The first spatial parameters may include the number of horizontal positions (1920), the number of vertical positions (1080), the origin of the upper left coordinate system, and the direction of coordinate increment. The control position (960, 540) is located in the central region of the transmittance control data space.

[0058] The second spatial parameter is used to describe the spatial properties of the transmittance modulation space. The second spatial parameter may include at least one of the following: the number of transmittance modulation objects 410, the number of rows and columns, the arrangement, the object size, the object spacing, the origin of the coordinate system, the direction of the coordinate axes, the physical size, or the position index.

[0059] For example, the transmittance modulation layer 400 includes 120 transmittance modulation objects 410 laterally and 72 transmittance modulation objects 410 vertically. The transmittance modulation space can be represented as a two-dimensional discrete space of 120×72. The second spatial parameter may include the number of objects laterally (120), the number of objects vertically (72), the arrangement direction of each transmittance modulation object 410, and its position index. The position (60, 36) may correspond to a transmittance modulation object 410 in the central region of the transmittance modulation layer 400.

[0060] For example, the transmittance modulation layer 400 includes a plurality of strip-shaped transmittance modulation objects 410 arranged longitudinally. The second spatial parameter may include the number, width, starting position, and extension direction of the strip-shaped transmittance modulation objects 410. One strip-shaped transmittance modulation object 410 may correspond to multiple control positions in the transmittance control data space.

[0061] The first spatial parameter enables the information determination module 112 to identify the position of the control position in the transmittance control data space. The second spatial parameter enables the information determination module 112 to identify the position of the transmittance modulation object 410 in the transmittance modulation space. The information determination module 112 can determine the positional difference between the two spaces based on the first and second spatial parameters, and then convert the control position to the transmittance modulation space according to the spatial transformation relationship.

[0062] The spatial transformation relationship is the spatial conversion rule between the transmittance control data space and the transmittance modulation space. The spatial transformation relationship is used to convert the source position in the transmittance control data space into the target position or transmittance modulation object 410 in the transmittance modulation space.

[0063] For example, if the number of horizontal positions in the transmittance control data space is 1920 and the number of horizontal transmittance modulation objects 410 in the transmittance modulation space is 120, then the horizontal scaling ratio can be 16, meaning that 16 consecutive horizontal control positions in the transmittance control data space can correspond to one horizontal position in the transmittance modulation space. If the number of vertical positions in the transmittance control data space is 1080 and the number of vertical transmittance modulation objects 410 in the transmittance modulation space is 72, then the vertical scaling ratio can be 15, meaning that 15 consecutive vertical control positions in the transmittance control data space can correspond to one vertical position in the transmittance modulation space.

[0064] For example, the transmittance control data space and the transmittance modulation space have the same spatial resolution, but the transmittance modulation layer 400 has a positional offset relative to the transmittance control data space. The spatial transformation relationship can include lateral translation and longitudinal translation. The information determination module 112 can add the lateral coordinate of the control position to the lateral translation and add the longitudinal coordinate of the control position to the longitudinal translation to determine the target transmittance modulation object 410.

[0065] By using the first spatial parameter, the second spatial parameter, and the spatial transformation relationship, the information determination module 112 can determine the spatial correspondence between the control position in the transmittance control data and the transmittance modulation object 410, i.e., the mapping information.

[0066] Mapping information can be represented using a mapping table or spatial transformation parameters. A mapping table is a data set recording the correspondence between control positions and transmittance modulation objects 410. A mapping table can include multiple mapping entries, and each entry can include a source position identifier and a target object identifier. The source position identifier identifies the control position in the transmittance control data space, and the target object identifier identifies the transmittance modulation object in the transmittance modulation space.

[0067] For example, the mapping table includes the following mapping entries: control position P1 corresponds to transmittance modulation object O1; control position P2 corresponds to transmittance modulation object O2; and control region R1 corresponds to transmittance modulation object O3.

[0068] Spatial transformation parameters are parameters used to perform spatial coordinate transformations. The information determination module 112 can calculate the target position corresponding to the control position or control area based on the spatial transformation parameters. Spatial transformation parameters may include at least one of scaling parameters, translation parameters, rotation parameters, affine transformation parameters, perspective transformation parameters, mesh transformation parameters, or nonlinear correction parameters.

[0069] Scaling parameters describe the size ratio between two spaces. For example, the horizontal scaling parameter is 1 / 16, and the vertical scaling parameter is 1 / 15. Translation parameters describe the positional offset between two spaces. For example, the horizontal translation parameter is 5 position units, and the vertical translation parameter is 3 position units. Rotation parameters describe the directional difference between two spaces. For example, the transmittance modulation layer 400 is rotated 90 degrees relative to the transmittance control data space. Affine transformation parameters can simultaneously describe scaling, translation, rotation, or shearing relationships. The information determination module 112 can input the coordinates of the control position into the affine transformation relationship and use the calculation result as the target coordinates in the transmittance modulation space. Perspective transformation parameters can describe the perspective relationship between two planes formed by different viewing directions or installation postures. For example, when the transmittance modulation layer 400 is not parallel to the display plane, the shape and proportion of the same size area may be different in the two spaces. The information determination module 112 can convert a rectangular area in the transmittance control data space into a quadrilateral area in the transmittance modulation space according to the perspective transformation parameters.

[0070] Mesh transformation parameters can describe different spatial transformation relationships between multiple local regions. The information determination module 112 can divide the transmittance control data space into multiple mesh regions and configure corresponding local transformation parameters for different mesh regions. Mesh transformation parameters are suitable for situations where different locations have different offsets or deformations. Nonlinear correction parameters can describe spatial differences that cannot be represented by a uniform linear transformation. For example, when the transmittance modulation layer 400 has bending or local deformation, the information determination module 112 can correct the target position corresponding to different control positions according to the nonlinear correction parameters.

[0071] The information determination module 112 can select a mapping table or spatial transformation parameters according to the needs of expressing spatial relationships. When the correspondence between discrete locations has been determined, the mapping table can directly record the corresponding results. When spatial relationships can be represented by unified calculation rules, spatial transformation parameters can reduce the number of mapping entries recorded item by item. When local regions have different spatial relationships, the mapping table and spatial transformation parameters can also jointly constitute mapping information.

[0072] Through mapping tables or spatial transformation parameters, mapping information can represent the spatial correspondence between control positions, control regions, and transmittance modulation objects 410 in a queryable or computable manner.

[0073] Specifically, the process of determining synchronization information involves the information determination module 112 acquiring a first time identifier of the transmittance control data and a second time identifier of the display data, and determining synchronization information based on the first time identifier and the second time identifier.

[0074] Transmittance control data and display data can be formed into transmittance control data sequences and display data sequences, respectively. The transmittance control data sequence includes multiple transmittance control data items arranged according to their temporal relationship. The display data sequence includes multiple display data items arranged in display order.

[0075] The first time identifier is used to identify a transmittance control data point within the transmittance control data sequence. The second time identifier is used to identify a display data point within the display data sequence. Time identifiers are information used to locate the position of data in time sequence or on a timeline.

[0076] The first and second time identifiers may include at least one of the following: frame sequence number, timestamp, frame duration, or keyframe index.

[0077] Frame numbers are assigned according to the order in which the data is arranged. Frame numbers in the transmittance control data sequence identify the position of different transmittance control data. Frame numbers in the display data sequence identify the position of different display data.

[0078] For example, the first time identifier of transmittance control data T10 is frame number 10, and the second time identifier of display data D10 is frame number 10. The information determination module 112 can determine that transmittance control data T10 and display data D10 have a time correspondence based on the same frame number.

[0079] A timestamp is a numerical value representing the corresponding moment of data on a timeline. A timestamp can represent the start time of the data, the target display time, or other predefined time positions. For example, the first timestamp of transmittance control data T20 is 3.200 seconds, and the second timestamp of display data D20 is 3.200 seconds. The information determination module 112 can determine that transmittance control data T20 and display data D20 correspond to the same moment on the same timeline based on the same timestamp. The first and second timestamps can also have a predefined time offset. For example, the first timestamp of transmittance control data T21 is 3.180 seconds, and the second timestamp of display data D21 is 3.200 seconds, with a predefined time offset of 0.020 seconds. The information determination module 112 can determine that transmittance control data T21 corresponds to display data D21 based on the time offset relationship.

[0080] Frame duration represents the duration of a data item on the time axis. The information determination module 112 can determine whether two duration ranges overlap based on the frame duration of the transmittance control data and the frame duration of the display data. For example, the time range corresponding to transmittance control data T30 is 5.000 seconds to 5.040 seconds, and the time range corresponding to display data D30 is also 5.000 seconds to 5.040 seconds. The information determination module 112 can determine that transmittance control data T30 and display data D30 have a time correspondence based on their shared time range.

[0081] Keyframe indexes are used to identify data locations that can serve as the basis for interpreting subsequent data. When the transmittance control data sequence includes baseline transmittance control data and subsequent data formed relative to the baseline transmittance control data, the keyframe index can identify the location of the baseline transmittance control data. When the display data sequence includes key display frames, the keyframe index can identify the location of the key display frames. For example, the first time identifier of transmittance control data T40 includes keyframe index K5, and the second time identifier of display data D40 includes keyframe index K5. The information determination module 112 can determine, based on keyframe index K5, that transmittance control data T40 and display data D40 belong to the same keyframe location.

[0082] When the information determination module 112 determines the synchronization information, it acquires a first time identifier and a second time identifier, and compares the correspondence between the first time identifier and the second time identifier. The comparison result may indicate that the frame sequence number is the same, the timestamp is the same, the timestamp meets a predetermined offset relationship, the duration range corresponds, or the keyframe index is the same.

[0083] The information determination module 112 can determine the comparison result as synchronization information. The synchronization information is used to represent the time correspondence between the transmittance control data and the display data.

[0084] By using the first and second time identifiers, the information determination module 112 can establish a recognizable time correspondence between the transmittance control data sequence and the display data sequence. Synchronization information provides a basis for the encoding module 114 to establish the association between transmittance control data and time.

[0085] After encoding the transmittance control data, mapping information and synchronization information, the encoding module 114 can use at least one of the following carrying methods to form transparent display encoded data: auxiliary code stream, independent data track, extended data area or metadata.

[0086] The bearer method refers to the data storage structure or logical organization location of transmittance control data, mapping information, and synchronization information in the transparent display encoded data. The bearer method determines the structure in which the encoded results are stored; it does not indicate the network communication method or limit the physical transmission channel.

[0087] The auxiliary bitstream is a sequence of encoded data formed separately from the display content bitstream and used to carry data related to transparent display. The auxiliary bitstream can arrange transmittance control data, mapping information, and synchronization information into independent data units. When the encoding module 114 uses the auxiliary bitstream, it encodes the transmittance control data into control data units within the auxiliary bitstream and encodes the mapping and synchronization information into associated data units within the auxiliary bitstream. The data structure within the auxiliary bitstream enables the decoding process to distinguish between control data units and associated data units. The auxiliary bitstream can be formed independently of the encoding process of the ordinary display content.

[0088] An independent data track is a data track that is distinguished from other data tracks within a data container and is used to store data related to transparent display. A data container is a data structure capable of holding multiple data tracks and describing the organizational relationships between them. When the encoding module 114 uses an independent data track, it establishes a transparent display data track within the data container and writes transmittance control data, mapping information, and synchronization information into the transparent display data track. The track description information within the data container can indicate the data type and reading rules of the transparent display data track.

[0089] For example, the data container includes a display data track and a transparency display data track. The display data track stores display-encoded data, and the transparency display data track stores transparency display-encoded data. The first data item in the transparency display data track stores transmittance control data T2, the second data item stores mapping information M2, and the third data item stores synchronization information S2. An independent data track is used to distinguish display data from transparency display-related data at the container organization level. The independent data track does not require the transmittance control data to be in video frame format.

[0090] An extended data region is a data area reserved or set in the encoded data structure for additional data. The extended data region can be located at the beginning, end, or a predetermined extension position of the encoded data unit. When the encoding module 114 uses an extended data region, it writes transmittance control data, mapping information, or synchronization information into the extended data region and records the type or length of the extended data region in the encoded data structure, enabling the decoding process to locate the extended data region.

[0091] For example, an encoded data unit includes a basic data area and an extended data area. The basic data area stores the basic encoded content, while the extended data area stores transmittance control data T3, mapping information M3, and synchronization information S3. The length information in the encoded data unit enables the decoding process to determine the data boundaries of the extended data area. The extended data area is suitable for storing transparency-related data when the existing encoded data structure has an extension location.

[0092] Metadata is data used to describe the attributes, relationships, or interpretation rules of other data. Metadata can describe the mapping information, synchronization information, and association relationships corresponding to throughput control data.

[0093] When encoding module 114 uses metadata, it can organize mapping information and synchronization information into structured data items and associate these structured data items with transmittance control data. Transmittance control data can also be directly encoded as part of the metadata.

[0094] For example, the metadata record includes control data fields, spatial relationship fields, and temporal relationship fields. The control data field stores transmittance control data T4, the spatial relationship field stores mapping information M4, and the temporal relationship field stores synchronization information S4. The field organization in the metadata record indicates that both mapping information M4 and synchronization information S4 correspond to transmittance control data T4.

[0095] For example, transmittance control data T5 is located in an independent data area, and metadata records store the data location of transmittance control data T5, mapping information M5, and synchronization information S5. Decoding processes locate transmittance control data T5 based on the metadata records and recover the mapping information M5 and synchronization information S5. Metadata is suitable for carrying descriptive information or structured parameters. Metadata is not limited to text format; it can use binary structures, key-value structures, or other parsable data structures.

[0096] Transparent display encoded data can also be combined using multiple bearer methods. The auxiliary bitstream can carry transmittance control data, and the metadata can carry mapping and synchronization information. The encoding module 114 establishes a correlation between the auxiliary bitstream and the metadata, enabling the decoding process to determine the transmittance control data to which the metadata applies.

[0097] The auxiliary bitstream carries transparent display-related data through independently encoded data sequences; independent data tracks carry transparent display-related data through track structures within data containers; extended data regions carry transparent display-related data through extended positions within the encoding structure; and metadata carries transparent display-related data through descriptive or structured data items. This enables transparent display encoded data to be applied to encoding structures with different scalability and data organization methods.

[0098] In addition to transmittance control content, transparent display content may also include display data for the transparent display unit to present images, videos, graphics, or text. The display data is encoded to form display encoded data.

[0099] The encoding module 114 can organize display encoded data and transparent display encoded data in the same data file, the same data stream, the same content package, or a data file with a relationship.

[0100] A data file is a data object that stores encoded data according to a file structure. When the encoding module 114 uses the same data file for organization, the encoding module 114 writes the display encoded data and the transparent display encoded data into different data areas of the same data file.

[0101] A data stream is a continuous encoded data formed by arranging multiple data units in a predetermined order. When the encoding module 114 organizes data using the same data stream, the encoding module 114 arranges the data units corresponding to the displayed encoded data and the data units corresponding to the transparently displayed encoded data in the same data stream.

[0102] A content package is a collection of data objects organized under the same transparent display content. A content package may include multiple files, multiple data directories, and content description information describing the relationships between the files.

[0103] For example, a content package may include foreground_rgb.mp4 (display data), transmission_mask.mp4 (transmittance control data), mapping.json (mapping information), and sync.json (synchronization information). It may also include background_rgb.mp4 (background display content) and display_config.json (playback parameters, etc.). The file names are for illustrative purposes only; the file content may also use other containers, binary configurations, or database records.

[0104] A data file with a relationship consists of multiple data files that separately store display-encoded data and transparent display-encoded data, and whose correspondence is established through file relationship information. File relationship information may include a common content name, a common content number, file path relationships, index records, or a content list. This file relationship information is used by the content processing device to determine the transparent display-encoded data file corresponding to a given display-encoded data file.

[0105] After encoding the transparent display encoding data, the encoding module 114 obtains the display encoding data and processes the display encoding data and transparent display encoding data according to the target content organization method.

[0106] When the target content is organized as a single data file, the encoding module 114 sets different data areas in the data file and writes display encoding data and transparent display encoding data respectively. When the target content is organized as a single data stream, the encoding module 114 forms display data units and transparent display data units, and arranges multiple data units according to the data stream structure. When the target content is organized as a single content package, the encoding module 114 forms or acquires data objects that store display encoding data and transparent display encoding data, and adds multiple data objects to the same content package. When the target content is organized as data files with a relationship, the encoding module 114 forms display data files and transparent display data files respectively, and establishes file relationship information to indicate the file correspondence.

[0107] This embodiment enables display encoding data and transparent display encoding data to be stored and retrieved within the same content organization system, thereby reducing the possibility of the two types of encoding data losing their content correspondence due to independent management.

[0108] The data processing apparatus of this application also includes a data adaptation module, which can be a content-side data adaptation module in the content side 100 or a playback data adaptation module in the playback side 200.

[0109] Specifically, after acquiring the transmittance control data and the hardware capability parameters of the target transmittance modulation hardware, the data adaptation module adapts the transmittance control data according to the hardware capability parameters to form adapted transmittance control data. The encoding module 114 encodes the adapted transmittance control data, mapping information, and synchronization information.

[0110] The target transmittance modulation hardware is the hardware used to perform transmittance modulation during the transparent display playback phase; that is, the hardware that actually carries the transmittance modulation object 410. For example, the transparent display device includes a liquid crystal dimming layer. The liquid crystal dimming layer includes multiple transmittance modulation objects capable of changing transmittance, and the liquid crystal dimming layer and the corresponding driving circuit can constitute the target transmittance modulation hardware.

[0111] Hardware capability parameters describe the types of transmittance control data that the target transmittance modulation hardware can receive, interpret, or execute. These parameters can reflect the spatial, numerical, or temporal capabilities of the target transmittance modulation hardware.

[0112] Spatial capability describes the spatial granularity and arrangement of the transmittance control that the target transmittance modulation hardware can perform. Numerical capability describes the precision and range of control values ​​that the target transmittance modulation hardware can recognize. Temporal capability describes the frequency and response characteristics of the transmittance state updates that the target transmittance modulation hardware can perform. For example, hardware capability parameters could indicate that the target transmittance modulation hardware includes 120×72 transmittance modulation objects 410, each receiving an 8-bit control value, and that the target transmittance modulation hardware supports a data update frequency of 60 Hz.

[0113] Adaptation is the process of converting transmittance control data from the source data specification to a data specification that the target transmittance modulation hardware can receive or execute. For example, the transmittance control data uses a 1920×1080 control position arrangement, while the target transmittance modulation hardware uses a 120×72 transmittance modulation object arrangement. The data adaptation module can merge the control values ​​corresponding to multiple control positions into the control value corresponding to a single transmittance modulation object 410, thereby forming 120×72 adapted transmittance control data.

[0114] Hardware capability parameters may include the spatial resolution of the target transmittance modulation hardware, the size of the transmittance modulation object, and the arrangement of the transmittance modulation objects. Data adaptation processing refers to resampling, partition mapping, or region merging of transmittance control data based on the spatial resolution, transmittance modulation object size, or transmittance modulation object arrangement.

[0115] Spatial resolution refers to the number of transmittance modulation objects that the target transmittance modulation hardware can independently control within a predetermined spatial range. For example, if the target transmittance modulation hardware includes 120 transmittance modulation objects 410 in the horizontal direction and 72 transmittance modulation objects 410 in the vertical direction, the spatial resolution of the target transmittance modulation hardware can be expressed as 120×72.

[0116] The transmittance modulation object size is the spatial extent of a single transmittance modulation object 410 within the transmittance modulation layer. The transmittance modulation object size can include its horizontal dimension, vertical dimension, or the size of the display area covered by the transmittance modulation object 410. For example, if a single transmittance modulation object 410 occupies an area of ​​4 mm horizontally and 4 mm vertically within the transmittance modulation layer, the transmittance modulation object size can be expressed as 4 mm × 4 mm.

[0117] The arrangement of transmittance modulation objects refers to the spatial organization of multiple transmittance modulation objects 410 within the transmittance modulation layer. The arrangement of transmittance modulation objects can include matrix arrangement, staggered arrangement, strip arrangement, or irregular partitioning. For example, multiple transmittance modulation objects 410 can be arranged in a matrix of 120 columns and 72 rows. Multiple transmittance modulation objects 410 can also be arranged in multiple vertical strip regions, with each strip region independently receiving a transmittance control value.

[0118] Resampling is the process of re-determining the control value corresponding to the target control position based on the correspondence between the source and target spatial positions. Resampling can change the spatial resolution of transmittance control data.

[0119] When the spatial resolution of the transmittance control data is higher than the spatial resolution of the target transmittance modulation hardware, a target control value can be calculated based on the control values ​​of multiple source control locations. When the spatial resolution of the transmittance control data is lower than the spatial resolution of the target transmittance modulation hardware, multiple target control values ​​can be determined based on one or more source control values.

[0120] For example, the transmittance control data uses 1920×1080 grid data, and the target transmittance modulation hardware uses a 120×72 transmittance modulation object array. The 16×15 grid cells in the transmittance control data correspond to one transmittance modulation object 410 in the target transmittance modulation hardware, and a target control value can be calculated according to a predetermined merging rule. The predetermined merging rule can be an average value, a weighted average value, a maximum control value, or a minimum control value. The specific merging rule can be determined based on the numerical meaning of the transmittance control value.

[0121] Partition mapping is the process of dividing the transmittance control data space into multiple source partitions and mapping each source partition to a target partition in the target transmittance modulation hardware or a transmittance modulation object 410. Partition mapping is suitable for situations where both the transmittance control data and the target transmittance modulation hardware use region-level control, but the number of regions, region boundaries, or region arrangement in the two spaces are not exactly the same.

[0122] For example, transmittance control data includes a left control region, a middle control region, and a right control region. The target transmittance modulation hardware includes four vertical modulation partitions. The data adaptation module can map the left control region to the first modulation partition, the middle control region to the second and third modulation partitions, and the right control region to the fourth modulation partition, based on the overlap of the region boundaries. During the partition mapping process, one source partition can correspond to one target partition, or it can correspond to multiple target partitions. Multiple source partitions can also jointly correspond to one target partition.

[0123] Region merging is the process of combining multiple control locations into a target control region corresponding to the target transmittance modulation object 410, and determining the control value corresponding to the target control region. Region merging is suitable when the control granularity of the target transmittance modulation hardware is lower than the expression granularity of the transmittance control data.

[0124] For example, transmittance control data includes adjacent control regions R1, R2, and R3. Control regions R1, R2, and R3 are located within the coverage area of ​​the same transmittance modulation object 410. The data adaptation module can merge control regions R1, R2, and R3 into a single target control region and generate a target control value based on the control values ​​corresponding to the three control regions.

[0125] After the data adaptation module completes resampling, partition mapping, or region merging, it can generate adapted transmittance control data that matches the number and arrangement of the target transmittance modulation objects.

[0126] Hardware capability parameters may also include the number of transmittance levels, control value bit depth, and control value range supported by the target transmittance modulation hardware. The data adaptation module can perform quantization, grayscale mapping, range conversion, or lookup table mapping on the transmittance control data based on the transmittance levels, control value bit depth, or control value range.

[0127] The transmittance level is the number of transmittance control states that the target transmittance modulation hardware can distinguish and execute. For example, if the target transmittance modulation hardware can execute 256 different transmittance control states, the transmittance level can be represented as 256 levels. The control value bit depth represents the number of binary bits used for a transmittance control value. The control value bit depth determines the number of discrete values ​​that the control value can represent. For example, an 8-bit control value can represent 256 discrete values ​​in the range of 0 to 255, and a 10-bit control value can represent 1024 discrete values ​​in the range of 0 to 1023. The control value range is the interval of control values ​​that the target transmittance modulation hardware can receive or interpret. The control value range can be represented as an integer range, a fixed-point range, or a normalized value range. For example, the target transmittance modulation hardware supports integer control values ​​from 0 to 255. 0 can represent the first transmittance control endpoint, and 255 can represent the second transmittance control endpoint. The transmittance state corresponding to each endpoint can be determined by the data definition of the target transmittance modulation hardware.

[0128] Quantization is the process of converting source control values ​​into control values ​​within a target discrete series. When the number of representable series of source control values ​​exceeds the number of series of target transmittance values, the data adaptation module can merge multiple source control values ​​into a single target control value. For example, transmittance control data may use 10-bit control values ​​with a range of 0 to 1023. Target transmittance modulation hardware may use 8-bit control values ​​with a range of 0 to 255. The data adaptation module can convert the source control values ​​into the target control value based on the ratio between the two control value ranges.

[0129] Quantization changes the discrete precision of the control value, but it does not change the transmittance control direction corresponding to the control value.

[0130] Grayscale mapping is the process of converting the logical grayscale values ​​used in transmittance control data into the target grayscale values ​​used by the target transmittance modulation hardware. The grayscale values ​​in the transmittance control data can represent transmittance changes according to a first numerical direction, while the grayscale values ​​in the target transmittance modulation hardware can represent transmittance changes according to a second numerical direction. The first and second numerical directions can be the same or opposite.

[0131] For example, transmittance control data uses a value of 0 to represent a low transmittance state and a value of 255 to represent a high transmittance state. The target transmittance modulation hardware uses a value of 0 to represent a high transmittance state and a value of 255 to represent a low transmittance state. The data adaptation module can perform reverse grayscale mapping to convert the source control value V into the target control value 255-V. Grayscale mapping ensures that the target control value maintains the transmittance change direction corresponding to the source control requirements.

[0132] Range conversion is the process of converting control values ​​within a source control value range to a target control value range. The source control value range and the target control value range can have different upper and lower limits or numerical types.

[0133] For example, transmittance control data uses normalized floating-point control values ​​between 0 and 1, while the target transmittance modulation hardware uses integer control values ​​between 16 and 2^35. The data adaptation module can convert the normalized control values ​​into integer control values ​​within the range of 16 to 2^35 according to a linear ratio. When the normalized control value is 0, the target control value can be 16; when the normalized control value is 1, the target control value can be 2^35; when the normalized control value is 0.5, the target control value can be converted into an integer control value close to 126 according to the proportional relationship.

[0134] The data adaptation module can also apply boundary constraints to values ​​that exceed the target control value range. When the source control value transformation result is lower than the target lower limit, the data adaptation module can use the target lower limit. When the source control value transformation result is higher than the target upper limit, the data adaptation module can use the target upper limit.

[0135] Lookup table mapping is a process that converts source control values ​​into target control values ​​based on a pre-established control value mapping table. The control value mapping table includes multiple source control values ​​or ranges of source control values, and the corresponding target control values. For example, the control value mapping table might record: source control values ​​0 to 31 correspond to target control value 16; source control values ​​32 to 63 correspond to target control value 32; source control values ​​64 to 95 correspond to target control value 48; and source control values ​​96 to 127 correspond to target control value 64. Lookup table mapping can represent linear or non-linear numerical relationships.

[0136] After the data adaptation module completes quantization, grayscale mapping, range conversion, or lookup table mapping, it can generate adapted transmittance control data whose bit depth, transmittance levels, and control value range meet the hardware requirements of the target transmittance modulation.

[0137] Hardware capability parameters may also include the refresh rate and response time of the target transmittance modulation hardware. The data adaptation module can perform time filtering, update rate conversion, or latency compensation on the transmittance control data based on the refresh rate or response time.

[0138] The refresh rate is the number of times the target transmittance modulation hardware can update the transmittance control state per unit of time. For example, a refresh rate of 60 Hz means that the target transmittance modulation hardware can perform 60 transmittance control state updates per second.

[0139] Response time is the time elapsed from when the target transmittance modulation hardware receives the target control value until the transmittance state reaches the predetermined response condition. For example, if the target transmittance modulation hardware receives a new control value and reaches the vicinity of the target transmittance state after 10 milliseconds, the response time of the target transmittance modulation hardware can be expressed as 10 milliseconds.

[0140] Time filtering is a process that adjusts the current control value based on control values ​​at adjacent time points. Time filtering can limit the range of change between adjacent control values, or it can generate the current output control value based on multiple adjacent control values. For example, the transmittance control data sequence includes consecutive control values ​​of 20, 220, and 30. If the response time of the target transmittance modulation hardware is longer than the control value update interval, a short change from 20 to 220 and then to 30 may not be fully executed by the target transmittance modulation hardware.

[0141] The data adaptation module can calculate a smoothed sequence of control values ​​based on adjacent control values. For example, the data adaptation module can adjust the control value 220 to the intermediate control value calculated from adjacent control values, and the adapted sequence of control values ​​can include 20, 90, and 30.

[0142] For example, the data adaptation module can set the maximum allowable change in control values ​​within a unit update cycle. When the difference between the current control value and the previous control value exceeds the maximum change, the data adaptation module can gradually adjust the control value according to the maximum change.

[0143] Update rate conversion is the process of converting the data update frequency of transmittance control data into a refresh rate supported by the target transmittance modulation hardware.

[0144] When the update frequency of the transmittance control data is higher than the target hardware refresh rate, the data adaptation module can select or synthesize a single target control data from multiple source control data. When the update frequency of the transmittance control data is lower than the target hardware refresh rate, the data adaptation module can copy the source control data or generate intermediate control data based on adjacent source control data.

[0145] For example, the transmittance control data is updated at 120 Hz, while the target transmittance modulation hardware refreshes at 60 Hz. The data adaptation module can generate a target update cycle every two source update cycles. The data adaptation module can select one of the two source control data sets, or it can calculate a target control data set based on both source control data sets.

[0146] For example, the transmittance control data is updated at a frequency of 30 Hz, while the target transmittance modulation hardware has a refresh rate of 60 Hz. The data adaptation module can maintain each source control data for two target update cycles, or it can calculate intermediate control data based on adjacent source control data.

[0147] Delay compensation adjusts the processing of transmittance control data at the corresponding output time based on the known delay of the target transmittance modulation hardware. The known delay is a time delay that the data adaptation module can acquire or predetermine. The known delay can include the response time of the target transmittance modulation hardware, or it can include a predeterminable processing delay between the start of transmittance control data processing and the change in transmittance modulation state.

[0148] For example, the known latency of the target transmittance modulation hardware is 10 milliseconds. Display data D1 is scheduled to begin displaying at time 1.000 seconds, and the corresponding transmittance control data T1 needs to coordinate with the display data D1 when the transmittance modulation state is formed. The data adaptation module can set the target output time of the transmittance control data T1 to 0.990 seconds. After the transmittance control data T1 is output at 0.990 seconds, the target transmittance modulation hardware responds after 10 milliseconds, and the target transmittance state can be formed close to 1.000 seconds.

[0149] For example, the response time of the target transmittance modulation hardware uses multiple preset delay parameters depending on the magnitude of the change in the control value. The data adaptation module can select the corresponding preset delay parameter based on the magnitude of the change between the current control value and the previous control value, and adjust the target output time according to the preset delay parameter.

[0150] Delay compensation adjusts the predetermined output time of transmittance control data.

[0151] After completing time filtering, update rate conversion, or delay compensation, the data adaptation module can generate a transmittance control data sequence that matches the refresh rate and response characteristics of the target transmittance modulation hardware. The encoding module 114 can encode the adapted transmittance control data sequence and the corresponding synchronization information into transparent display encoded data.

[0152] The transmittance control value in transmittance control data can be a logical control value used during the content production stage. The logical control value expresses the transmittance control requirements, but it does not necessarily have to be directly equal to the device control value that the transmittance modulation hardware can receive. Different transmittance modulation hardware can employ different drive ranges, control directions, or transmittance response characteristics; the same logical control value may correspond to different actual transmittance or device control values ​​in different transmittance modulation hardware.

[0153] Transmittance control values ​​are numerical values ​​used in transmittance control data to represent transmittance control requirements. Transmittance control values ​​can be represented as integers, fixed-point numbers, floating-point numbers, or normalized values. Transmittance control values ​​can represent relative transmittance levels or logical control levels set during the content creation phase.

[0154] For example, transmittance control values ​​are represented by integers from 0 to 255. A value of 0 can represent the first logical transmittance state, a value of 255 can represent the second logical transmittance state, and values ​​between 0 and 255 can represent intermediate control levels between the two logical transmittance states.

[0155] The target transmittance is the optical transmittance that the transmittance modulation object 410 needs to achieve or approach after transmittance modulation. The target transmittance can be expressed as a percentage, a normalized value, or other numerical values ​​that can characterize the proportion of incident light passing through.

[0156] For example, a target transmittance of 0.8 indicates that the transmittance modulation object 410 needs to achieve an optical transmittance of close to 80%. A target transmittance of 0.2 indicates that the transmittance modulation object 410 needs to achieve an optical transmittance of close to 20%.

[0157] The device control value is a control value that the transmittance modulation hardware can receive and use to drive the transmittance modulation object 410. The device control value can be a drive voltage level, pulse width modulation parameters, digital drive code, grayscale code, or other values ​​that the transmittance modulation drive circuit can recognize.

[0158] For example, the target transmittance modulation hardware uses a digital drive code from 0 to 1023. Device control value 200 can cause the transmittance modulation object 410 to form a first transmittance state, and device control value 800 can cause the transmittance modulation object 410 to form a second transmittance state.

[0159] Logic control values, target transmittance, and device control values ​​belong to different processing levels. Logic control values ​​express the control intent for transparent display content, target transmittance expresses the desired optical state, and device control values ​​express the actual driving quantity received by the target transmittance modulation hardware. There can be a correspondence between logic control values ​​and target transmittance, and also between target transmittance and device control values.

[0160] In order for the playback side 200 to interpret the transmittance control value according to a clear numerical relationship, the transparent display encoded data may include or be associated with transmittance response curve information.

[0161] Transmittance response curve information is used to characterize the correspondence between transmittance control values ​​and target transmittance or equipment control values. Transmittance response curve information can describe continuous numerical relationships as well as discrete numerical relationships. It can also describe linear or nonlinear relationships.

[0162] Transmittance response curves are not limited to graphical forms. Response curves can be represented by function parameters, discrete sampling points, piecewise parameters, or lookup tables. For example, transmittance response curve information includes multiple curve sampling points, each containing a transmittance control value and the corresponding target transmittance.

[0163] The transmittance response curve information can be generated from the calibration results of the target transmittance modulation hardware. The calibration process can apply multiple device control values ​​to the transmittance modulation object 410 and measure the actual transmittance corresponding to each device control value. The calibration results can establish the correspondence between the device control values ​​and the actual transmittance.

[0164] For example, the calibration device sequentially applies device control values ​​of 100, 300, 500, 700, and 900 to the transmittance modulation object 410, and measures the actual transmittances as 0.08, 0.22, 0.43, 0.68, and 0.87, respectively. The corresponding calibration results can establish a response relationship between the target transmittance and the device control values.

[0165] The encoding module 114 acquires transmittance control data and transmittance response curve information. The encoding module 114 can write the transmittance response curve information into the transparent display encoding data, and can also establish a correlation between the transparent display encoding data and the transmittance response curve information.

[0166] When transmittance response curve information is written into the transparent display encoded data, the transparent display encoded data can simultaneously contain transmittance control data and transmittance response curve information. When the decoding process recovers the transmittance control data from the transparent display encoded data, it can also recover the transmittance response curve information.

[0167] like Figure 4 As shown, the second data processing device 220 is located on the playback side 200. The second data processing device 220 includes a decoding module 222, a spatial processing module 224, a time processing module 226, and a control output module 228. The decoding module 222 is connected to both the spatial processing module 224 and the time processing module 226, and the control output module 228 is connected to both the spatial processing module 224 and the time processing module 226.

[0168] The decoding module 222 decodes the transparent display encoded data to obtain interrelated transmittance control data, mapping information, and synchronization information.

[0169] The spatial processing module 224 maps the control positions in the transmittance control data to the target transmittance modulation object 410 according to the mapping information, thereby obtaining the target transmittance control data. The target transmittance control data is transmittance control data reorganized according to the target transmittance modulation object 410. For example, the spatial processing module 224 can convert a 1920×1080 source transmittance control chart into control data for 120×72 modulation zones according to the mapping table.

[0170] The time processing module 226 determines the output timing based on the synchronization information.

[0171] When the second data processing device 220 needs to adapt to the target transmittance modulation hardware, the playback side data adaptation module can adapt according to the relevant description of the data adaptation module, which will not be elaborated here.

[0172] The second data processing device 220 may also include a curve processing module. When the transparent display encoded data also includes transmittance response curve information, after decoding the transmittance response curve information, the curve processing module can convert the transmittance control value into a target transmittance or a device control value according to the transmittance response curve information.

[0173] The control output module 228 outputs target transmittance control data or device control values ​​through the transmittance control output interface according to the output timing sequence. The display data interface can output display data through the display data output interface and background display data through the background data output interface. Both the display data interface and the background data output interface are optional.

[0174] Display data can be displayed by the transparent display unit 300, target transmittance control data or device control values ​​can be applied to the transmittance modulation layer 400, and background display data can be displayed by the background display unit 500.

[0175] The second data processing device 220 recovers the interrelated transmittance control data, mapping information, and synchronization information from the transparent display encoded data, and performs spatial processing and temporal processing respectively, so that the transmittance control data can be output to the subsequent transmittance control stage according to the spatial and temporal correspondence established on the encoding side.

[0176] This application also provides a data processing method applied to the content side 100, such as... Figure 5 As shown, the method includes the following steps S501 to S503.

[0177] In step S501, mapping information and synchronization information are determined.

[0178] In step S502, the correlation between transmittance control data and mapping information and synchronization information is established.

[0179] In step S503, the transmittance control data, mapping information, and synchronization information are encoded to obtain transparent display encoded data.

[0180] Another data processing method provided in this application, applied to the playback side 200, includes: decoding transparent display encoded data to obtain interrelated transmittance control data, mapping information, and synchronization information; mapping the control position in the transmittance control data to the target transmittance modulation object according to the mapping information to obtain target transmittance control data; determining the output timing of the target transmittance control data according to the synchronization information; and outputting the target transmittance control data according to the output timing.

[0181] The relevant description of this data processing method can be found in the description of the content processing device, and it can achieve the same technical effect, so it will not be repeated here.

[0182] like Figure 6 The diagram shown is a structural schematic of the electronic device provided in this application. The electronic device includes a processor 610 and a memory 620. The electronic device may also include a display interface, a transmittance control interface, a clock or timing processing unit, and an external storage interface.

[0183] Processor 610 may be a general-purpose processor, a special-purpose processor, a programmable logic device, or a combination thereof. Memory 620 may include volatile memory and non-volatile memory, and stores programs or instructions as well as data used during processing.

[0184] The program or instructions may include content-side data processing programs and playback-side data processing programs. Content-side data processing programs enable the processor to perform information determination, association establishment, and encoding processing; playback-side data processing programs enable the processor to perform decoding, spatial processing, temporal processing, and control output.

[0185] The electronic device can implement either the first data processing device 110 or the second data processing device 220. The electronic device can also simultaneously implement both the first data processing device 110 and the second data processing device 220, so that the generation, reading, decoding, and application of transparent display encoded data can be completed within the same device.

[0186] The electronic device executes programs or instructions stored in the memory 620 through the processor 610, enabling it to implement data processing methods on the content side or the playback side. The direct function of the electronic device is to provide the hardware implementation carriers, such as processors, memory, and data interfaces, for the corresponding data processing flow.

[0187] This application also provides a computer-readable storage medium storing at least one instruction, which is executed by a processor to implement the content-side data processing method or the playback-side data processing method as described in the above embodiments.

[0188] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, servers, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.

[0189] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0190] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0191] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code.

[0192] Those skilled in the art will recognize that, in one or more of the examples above, the functions described in this application can be implemented using hardware, software, firmware, or any combination thereof. When implemented in software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of a computer program from one place to another. Storage media can be any available medium accessible to a general-purpose or special-purpose computer.

[0193] It should be noted that the technical solutions described in this application can be combined arbitrarily without conflict.

[0194] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A data processing apparatus, characterized in that, Applied to the content side, including: The information determination module is used to determine mapping information and synchronization information. The mapping information is used to characterize the spatial correspondence between the control position in the transmittance control data and the transmittance modulation object. The synchronization information is used to characterize the temporal correspondence between the transmittance control data and the display data. The encoding module, connected to the information determination module, is used to establish the association between the transmittance control data, the mapping information, and the synchronization information, and to encode the transmittance control data, the mapping information, and the synchronization information to obtain transparent display encoded data.

2. The data processing apparatus according to claim 1, characterized in that, The information determination module is used to acquire a first spatial parameter indicating the space where the transmittance control data is located and a second spatial parameter indicating the space where the transmittance modulation object is located, and to determine the mapping information based on the first spatial parameter, the second spatial parameter, and the spatial transformation relationship between the space where the transmittance control data is located and the space where the transmittance modulation object is located.

3. The data processing apparatus according to claim 1, characterized in that, The information determination module is used to obtain a first time identifier of the transmittance control data and a second time identifier of the display data, and determine the synchronization information based on the first time identifier and the second time identifier. The first time identifier and the second time identifier respectively include at least one of frame sequence number, timestamp, frame duration and keyframe index.

4. The data processing apparatus according to claim 1, characterized in that, The transparent display encoded data uses at least one of the following methods to carry the transmittance control data, the mapping information, and the synchronization information: auxiliary bitstream, independent data track, extended data area, and metadata.

5. The data processing apparatus according to claim 1, characterized in that, The encoding module is also used to encode the display data to obtain display encoded data, and to encapsulate the display encoded data and the transparent display encoded data into the same data file.

6. The data processing apparatus according to claim 1, characterized in that, The data processing device also includes a content-side data adaptation module; The content-side data adaptation module is used to adapt the transmittance control data according to the hardware capability parameters of the target transmittance modulation hardware to obtain the adapted transmittance control data. The encoding module is used to encode the adapted transmittance control data, the mapping information, and the synchronization information.

7. The data processing apparatus according to claim 6, characterized in that, The hardware capability parameters include at least one of the following: spatial resolution of the target transmittance modulation hardware, size of the transmittance modulation object, and arrangement of the transmittance modulation objects; The content-side data adaptation module is used to perform at least one of resampling processing, partition mapping processing, and region merging processing on the transmittance control data.

8. The data processing apparatus according to claim 6, characterized in that, The hardware capability parameters include at least one of transmittance level, control value bit depth, and control value range; The content-side data adaptation module is used to perform at least one of the following on the transmittance control data: quantization processing, grayscale mapping, numerical range conversion, and lookup table mapping.

9. The data processing apparatus according to claim 6, characterized in that, The hardware capability parameters include at least one of refresh rate and response time; The content-side data adaptation module is used to perform at least one of time filtering, update rate conversion, and latency compensation on the transmittance control data.

10. The data processing apparatus according to claim 1, characterized in that, The transparent display encoding data also includes transmittance response curve information; the transmittance response curve information is used to characterize the correspondence between the transmittance control value and the target transmittance or the device control value.

11. A data processing apparatus, characterized in that, Applied to the playback side, including: The decoding module is used to decode the transparent display encoded data to obtain interrelated transmittance control data, mapping information and synchronization information; A spatial processing module, connected to the decoding module, is used to map the control position in the transmittance control data to the target transmittance modulation object according to the mapping information, so as to obtain the target transmittance control data; A time processing module, connected to the decoding module, is used to determine the output timing of the target transmittance control data based on the synchronization information; The control output module is connected to the spatial processing module and the time processing module respectively, and is used to output the target transmittance control data according to the output timing sequence.

12. A data processing method, characterized in that, Applied to the content side, including: Determine mapping information and synchronization information, wherein the mapping information represents the spatial correspondence between the control position in the transmittance control data and the transmittance modulation object, and the synchronization information represents the temporal correspondence between the transmittance control data and the display data; Establish the association between the transmittance control data, the mapping information, and the synchronization information; The transmittance control data, the mapping information, and the synchronization information are encoded to obtain transparent display encoded data.

13. A data processing method, characterized in that, Applied to the playback side, including: The transparent display encoded data is decoded to obtain interrelated transmittance control data, mapping information, and synchronization information; Based on the mapping information, the control position in the transmittance control data is mapped to the target transmittance modulation object to obtain the target transmittance control data; The output timing of the target transmittance control data is determined based on the synchronization information; The target transmittance control data is output according to the output timing.

14. A computer-readable storage medium having a program or instructions stored thereon, characterized in that, When the program or instructions are executed by the processor, the data processing method as described in claim 12 or 13 is implemented.

Images