An image capability format conversion method, device and system for large model training

Abstract

This invention provides an image capability format conversion method, apparatus, and system for large-scale model training, relating to the field of image data format conversion. The format conversion method preserves multi-dimensional image feature information, including an image capability parsing step (S1) that extracts five-dimensional feature information, a capability adaptation strategy selection step (S2) with selectable target format conversion strategies, a multi-stage format conversion step (S3), and a format verification and output step (S4) where the conversion result is output only after both format verification and semantic consistency verification pass. The multi-stage format conversion step (S3) includes an image preprocessing stage (S31) performing format decoding and color space normalization conversion, a capability-aware block coding stage (S32) performing adaptive block segmentation and capability modulation coding, and a unified format encapsulation stage (S33) performing fusion and alignment. This method can preserve more key semantic information under the same computational budget, provide more robust quality assurance for converted data, and seamlessly integrate with large-scale model training.

Description

Technical Field

[0001] This invention relates to the field of image data format conversion, and more particularly to image format conversion methods, apparatus and systems for large model training. Background Technology

[0002] With the rapid development of large-scale pre-trained models, multimodal large models (such as GPT-4V, LLaVA, Qwen-VL, InternVL, etc.) have demonstrated powerful capabilities in tasks such as image understanding, visual question answering, and image caption generation. The training process of these multimodal large models typically requires joint input of image data and text data into the model for end-to-end training.

[0003] In the training pipeline of multimodal large models, image data preprocessing and format conversion are crucial preliminary steps. Due to the diverse sources of images in the training dataset (including online images, remote sensing images, medical images, document scans, etc.), their formats (JPEG, PNG, TIFF, WebP, HEIC, RAW, etc.), resolutions (from tens of pixels to tens of thousands of pixels), color spaces (sRGB, Adobe RGB, ProPhoto RGB, CMYK, grayscale, etc.), and metadata information vary greatly. Therefore, they must undergo unified format conversion before they can be used as input for model training.

[0004] Current mainstream multimodal large-scale models typically employ the Vision Transformer architecture, which segments the image into fixed-size patches, performs linear embedding on each patch, adds positional encoding, and then inputs it into a Transformer encoder for processing. This architecture requires the input image to be converted into a tensor format with specific dimensions and data types.

[0005] The most common preprocessing approach for large model images currently involves the following steps: reading the image file and decoding it into a pixel matrix; scaling / cropping the image to a fixed resolution (e.g., 224x224 or 336x336); normalizing the pixel values; standardizing the image according to preset mean and standard deviation; and converting the processed pixel matrix into a tensor format for output. This approach is fixed and simple, but it completely discards the image's inherent capabilities and applies the same processing parameters to all images.

[0006] The ViT model, for example, involves the following image format conversion process: scaling the input image to a fixed resolution; segmenting the image into non-overlapping blocks of a fixed size (e.g., 16x16 or 32x32 pixels); linearly projecting each block into an embedding vector; adding a fixed, learnable positional encoding; and concatenating category tokens before outputting. This approach converts the image into a token sequence, but the block size is fixed, does not adaptively adjust to the image content, and does not consider the image's capability features.

[0007] While general image format conversion methods based on OpenCV / PIL are comprehensive and versatile, their output formats and encoding methods are not optimized for the input specifications of large model training, requiring users to write their own adaptation code.

[0008] Therefore, in current large-scale multimodal training, the process of converting image data to a unified input format suffers from problems such as loss of image capability information, single conversion strategy lacking adaptability, disconnect between the converted output and the large-scale model training input specifications, and lack of effective format verification mechanism. Summary of the Invention

[0009] This invention provides an image capability format conversion method, apparatus, and system for large model training, which can retain image capability feature information, retain more key semantic information under the same computational budget, provide more robust data quality assurance for conversion, and seamlessly connect with large model training.

[0010] One aspect of the present invention provides an image capability format conversion method for large model training, characterized by comprising: an image capability parsing step S1, which acquires the original image data to be converted and performs multi-dimensional capability parsing and extracts image capability feature vectors, wherein the multi-dimensional dimensions include format dimension, resolution dimension, color space dimension, metadata dimension, and semantic dimension; a capability adaptation strategy selection step S2, which inputs the image capability feature vectors into a preset capability adaptation strategy selection model and selects and determines the target format conversion strategy from candidate conversion strategies; a multi-stage format conversion step S3, which includes an image preprocessing stage S31 performing format decoding and color space normalization conversion, a capability-aware block coding stage S32 performing adaptive block segmentation and capability modulation coding, and a unified format encapsulation stage S33 performing fusion and alignment; and a format verification and output step S4, which performs format integrity verification and semantic consistency verification, and only outputs unified format tensor data as the format conversion result if both verifications pass.

[0011] Optionally, if the format verification and output step S4 fails the verification, it returns to the capability adaptation strategy selection step S2 to reselect a strategy or directly adjusts the conversion parameters for incremental correction.

[0012] Optionally, the candidate conversion strategies include a fidelity-first conversion strategy, an efficiency-first conversion strategy, and a balanced conversion strategy. The capability adaptation strategy selection model includes a strategy evaluation network and a strategy decision network. The strategy evaluation network scores the candidate conversion strategies based on image capability features, and the strategy decision network selects the target format conversion strategy with the highest fit based on the fit score and strategy selection constraints.

[0013] Optionally, the capability-aware block encoding stage S32 includes adaptive block parameter determination and adaptive block encoding based on the determined block parameters; wherein the adaptive block parameter determination dynamically adjusts the block parameters based on the semantic complexity represented by semantic entropy so that more reliable fine-grained block encoding is performed on semantically complex regions and coarse-grained block encoding is performed on semantically simple regions; the block parameters include block size and block overlap rate, and the dynamic adjustment includes: a base block size determined based on pixel resolution; dividing the semantic complexity into discrete levels, and encoding based on the level and the base block size. The specific chunk size is determined by reducing the size to a certain extent, and the chunk overlap rate is adjusted according to the level. The reduction to a certain extent means that the chunk size with the lowest semantic complexity level is determined as the base chunk size, and the ratio of the lower-level chunk size to the higher-level chunk size in adjacent levels is a reduction factor. The adjustment of chunk overlap rate according to the level means that the overlap rate of the chunk with the lowest semantic complexity level is determined to be 0, and the overlap rate of the higher-level chunk in adjacent levels is increased by a certain percentage compared to the overlap rate of the lower-level chunk. The number of levels is 3, the reduction factor is 1.5, and the certain percentage is 10%.

[0014] Optionally, the capability-aware block coding stage S32 includes capability-aware coding that performs capability modulation coding; capability-aware coding generates an initial embedding vector for each image block data through linear projection mapping, calculates and determines modulation coefficients including scaling factors and biases using relevant image capability feature information, performs an affine transformation on the initial embedding vector, and adds position coding to generate a capability embedding vector sequence; the capability feature information used to calculate and determine the modulation coefficients is capability feature information in the format dimension and color space dimension, and the affine transformation is FiLM (Feature-Level Linear Modulation); the position coding adopts two-dimensional position coding based on block index and block size.

[0015] Optionally, in the unified format encapsulation stage S33, the image capability feature vector is linearly transformed to generate a capability condition vector; the capability condition vector is used as a prefix token and concatenated with the capability embedding vector sequence to form a fusion token sequence; it is aligned to the large model dimension and a format identifier header is added.

[0016] As another aspect of the present invention, the present invention provides an image capability format conversion device for large model training, characterized in that it includes the following modules: an image capability analysis module 601, which acquires the original image data to be converted and performs multi-dimensional capability analysis and extracts image capability feature vectors, wherein the multi-dimensional dimensions include format dimension, resolution dimension, color space dimension, metadata dimension, and semantic dimension; a capability adaptation strategy selection module 602, which includes a strategy evaluation network 6021 and a strategy decision network 6022, and determines the target format conversion strategy from candidate conversion strategies based on the image capability feature vectors; a multi-stage format conversion module 603, which includes an image preprocessing submodule 6031 for format decoding and color space conversion, a capability-aware block coding submodule 6032 for adaptive block segmentation and capability modulation coding, and a unified format encapsulation submodule 6033 for fusion and alignment; and a format verification and output module 604, which includes a format integrity verification unit 6041, a semantic consistency verification unit 6042, and an output control unit 6043, wherein the output control unit 6043... The unified format tensor data is only output as the format conversion result when both format integrity and semantic consistency checks pass; the batch conversion scheduling module 605 generates a scheduling scheme based on capability characteristics and computing resources, prioritizing the allocation to GPUs (graphics processing units) for fidelity and NPUs / TPUs (neural network processors / tensor processors) for efficiency; and the format cache management module 606 uses LSH (Local Sensitive Hash) similarity retrieval, reusing conversion parameters when the cosine similarity is >0.95.

[0017] Optionally, the capability-aware block coding submodule 6032 includes an adaptive block parameter determination unit 60321. The adaptive block parameter determination unit 60321 dynamically adjusts the block size, which is reduced to different degrees based on the basic block size determined by the pixel resolution, according to the semantic complexity level. This allows for fine-grained block division of semantically complex regions and coarse-grained block division of semantically simple regions, and dynamically adjusts the block overlap rate. The capability-aware block coding submodule 6032 also includes a capability modulation coding unit 60322. The capability modulation coding unit 60322... Based on the initial embedding vector generated by linear projection mapping, modulation coefficients including scaling factors and biases are calculated and determined according to relevant image capability feature information. An affine transformation is performed on the initial embedding vector, and a two-dimensional positional code based on block index and block size is added to generate a capability embedding vector sequence. The image preprocessing submodule 6031 calls the format decoder to obtain the original pixel matrix according to the encoding format. The color space converter converts the original pixel matrix to generate intermediate image data according to the target format conversion strategy with the help of ICC (International Color Consortium) color configuration. The unified format encapsulation submodule 6033 linearly transforms the image capability feature vector to generate a capability condition vector. The capability condition vector is used as a prefix token and concatenated with the capability embedding vector sequence to form a fusion token sequence, which is aligned to the large model dimension and a format identifier header is added.

[0018] In another aspect, the present invention provides a system, characterized in that the system is configured to implement the image capability format conversion method as described above.

[0019] Thus, this invention preserves capability information in a unified format tensor, enabling large-scale model training to perceive and utilize image capability features. Through a capability adaptation strategy, the model automatically selects the optimal strategy, enhancing the adaptability of the transformation strategy and achieving a dynamic balance between fidelity and efficiency. Based on semantic complexity, the block size is adaptively adjusted, retaining more key semantic information under the same computational budget, balancing block encoding efficiency and quality. Tensor data conforming to the input specifications of large models is directly generated, with format identifier headers and checksums added for seamless integration with large-scale model training. A dual checksum mechanism ensures output data quality; automatic backoff occurs if checksums fail, providing a closed-loop guarantee of data quality. Furthermore, the unified format encapsulation stage of this invention, through a feature alignment layer, can flexibly connect to model input specifications of different dimensions and data types. Therefore, this invention is not only applicable to current mainstream ViT architecture multimodal large models, such as LLaVA, Qwen-VL, and InternVL, but also adaptable to new architecture models that may emerge in the future. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the basic steps of the method;

[0021] Figure 2 This is a schematic diagram of the overall methodology.

[0022] Figure 3 Detailed process for image capability analysis steps;

[0023] Figure 4 Detailed process for selecting a capability adaptation strategy;

[0024] Figure 5 A detailed flowchart of the multi-stage format conversion process;

[0025] Figure 6 A schematic diagram illustrating the principle of capability-aware block coding;

[0026] Figure 7 This is a block diagram of the device structure;

[0027] Figure 8 This section details the structure of the multi-stage format conversion module. Detailed Implementation

[0028] This invention provides an image capability format conversion method, apparatus, and system for large-scale model training. Image capability in this invention refers to the feature information of an image in various dimensions and its inherent potential effects and functions. Image capability features include features in dimensions such as image format, resolution, color space, metadata, and semantics. The invention innovatively unifies these features into a unified image capability feature vector. Traditional image format conversion only focuses on the format transformation of pixel data, while this invention retains image capability features during format conversion and constructs a closed-loop technology system of capability perception → capability adaptation → capability modulation → capability fusion → capability verification. This evolves the image format conversion process from "blindly unifying processing" to "conscious capability preservation and adaptation processing."

[0029] The attached diagram below provides a detailed explanation.

[0030] like Figure 1As shown, the technical solution of the present invention includes four basic steps: image capability analysis step S1, capability adaptation strategy selection step S2, multi-stage format conversion step S3, and format verification and output step S4. Image capability analysis step S1 extracts multi-dimensional feature information; capability adaptation strategy selection step S2 selects and determines the target format conversion strategy from more than one candidate conversion strategy; multi-stage format conversion step S3 includes an image preprocessing stage S31 that performs format decoding and color space standardization conversion, a capability-aware block coding stage S32 that performs adaptive block segmentation and capability modulation coding, and a unified format encapsulation stage S33 that performs fusion and alignment; format verification and output step S4 outputs the conversion result only after both format verification and semantic consistency verification pass.

[0031] More specifically, such as Figure 2 As shown, step S1, i.e., image capability analysis, involves performing five-dimensional capability analysis on the original image data after obtaining it, including its format, resolution, color space, metadata, and semantics. The image capability features of each dimension are extracted and encoded into capability feature sub-vectors f_fmt, f_res, f_color, f_meta, and f_sem. These capability feature sub-vectors are then concatenated and fused into an image capability feature vector F_cap, i.e.:

[0032] F_cap ∈ ℝ^D [f_fmt; f_res; f_color; f_meta; f_sem]

[0033] Among them, the feature sub-vectors of each dimension are fused into image capability feature vectors. In addition to simple concatenation, attention fusion mechanism (multi-head attention or gated fusion) can also be used for weighted fusion.

[0034] Step S2 is the capability adaptation strategy selection: input F_cap into the preset capability adaptation strategy selection model, and select the target format conversion strategy from the fidelity-first conversion strategy, efficiency-first conversion strategy, or balanced conversion strategy.

[0035] Among them, the methods for selecting and determining the target format conversion strategy can include scoring the fit of the strategy parameter vector, selecting the one with the highest fit, or using strategy selection methods based on decision trees or rule engines.

[0036] Step S3 is a multi-stage format conversion, including stage S31 (image preprocessing), stage S32 (capability-aware block coding), and stage S33 (unified format encapsulation). In image preprocessing stage S31, the corresponding decoder is called to obtain the original pixel matrix from the original image data according to the encoding format, and color space normalization is performed. In capability-aware block coding stage S32, conversion parameters such as block parameters and encoding parameters are dynamically adjusted based on relevant image capability features, and adaptive block coding and capability modulation are performed. In unified format encapsulation stage S33, the image capability features and capability embedding vector sequence are fused and aligned to a larger model dimension.

[0037] Among them, dynamically adjusting the block parameter values ​​according to the relevant image capability features can be done by discrete adjustment based on semantic complexity level, or by directly mapping to continuous block parameter values ​​using a regression network; capability modulation can be done using the FiLM mechanism, or by using AdaIN, conditional batch normalization, or cross-attention mechanism.

[0038] Step S4 Format Verification and Output: Perform dual verification of format integrity and semantic consistency on the unified format tensor data. If both verifications pass, output the unified format tensor data as the format conversion result; otherwise, return to S2 to select a new strategy. The format integrity verification checks the dimensions, data type, and CRC32 checksum, while the semantic consistency verification calculates semantic similarity using a Siamese network; a similarity greater than 0.90 passes.

[0039] In addition to the Siamese network, the semantic consistency discriminator can also use FID-based statistical tests or CLIP-based cross-modal similarity; and when the verification fails, in addition to returning to S2, the transformation parameters can be directly adjusted for incremental correction.

[0040] The following is combined with Figure 3 As shown, the steps for parsing the S1 image capability are explained in detail.

[0041] S11 Format Dimension Analysis: Identifies encoding format types (such as JPEG, PNG, TIFF, BMP, WebP, HEIC, RAW, etc.), extracts format identifiers, compression algorithm types, and file header information, and encodes them into format dimension feature sub-vectors f_fmt.

[0042] S12 Resolution Dimension Analysis: Obtain pixel resolution (H, W), physical resolution (DPI / PPI), and number of channels C. Determine the level identifier L_res based on the resolution level mapping table and encode it as a resolution dimension feature sub-vector f_res. .

[0043] S13 Color Space Dimension Analysis: This function detects the color space type and bit depth, calculates the color space complexity index, and encodes it as a color space dimension feature vector f_color. The index calculation formula is: σ_color = (C × B ×|P_color|) / (C_max × B_max × |P_max|). Here, C represents chroma, B represents luminance, and P represents pixel; C / C_max represents chroma utilization, B / B_max represents luminance utilization, and |P_color| / |P_max| represents the proportion of non-gray pixels ( / color pixels).

[0044] S14 Metadata Dimension Parsing: Extract at least one of the EXIF, IPTC, and XMP information, perform structured encoding, and generate a metadata dimension feature sub-vector f_meta. In addition to structured encoding, the feature information extracted by metadata dimension parsing can also be semantically encoded using text embedding models such as BERT and Sentence-BERT.

[0045] S15 Semantic Dimension Parsing: The original image is input into a pre-trained lightweight semantic feature extraction network (based on MobileNetV3 or EfficientNet-Lite architecture) to obtain semantic feature descriptors and generate semantic dimension feature vectors f_sem. Besides MobileNetV3 / EfficientNet-Lite, semantic dimension parsing can also use ShuffleNetV2, GhostNet, or a small student network based on knowledge distillation.

[0046] After the extracted features from each dimension are encoded into feature sub-vectors, they are concatenated and fused into an image capability feature vector, i.e.:

[0047] F_cap = [f_fmt; f_res; f_color; f_meta; f_sem]

[0048] F_cap ∈ ℝ^D, D=dim(f_fmt)+dim(f_res)+dim(f_color)+dim(f_meta)+dim(f_sem)

[0049] In other words, F_cap is a D-dimensional real vector whose dimension is the sum of the dimensions of the sub-vectors involved in the concatenation.

[0050] The following is combined with Figure 4 As shown, the steps for selecting the S2 capability adaptation strategy are explained in detail.

[0051] The capability adaptation strategy selection step employs a capability adaptation strategy selection model that includes a strategy evaluation network S22 and a strategy decision network S23. The strategy evaluation network scores the candidate conversion strategies based on the image capability features, and the strategy decision network selects the target format conversion strategy with the highest fit based on the fit score and strategy selection constraints.

[0052] The S22 policy evaluation network calculates the fit score between F_cap and the policy parameter vectors of the three candidate transition policies:

[0053] s_i = softmax(F_cap · θ_i / √D)

[0054] Where θ_i is the policy parameter vector (θ_fid, θ_eff, and θ_bal represent the policy parameter vectors of the fidelity-first conversion policy, efficiency-first conversion policy, and balanced conversion policy, respectively), · represents the dot product, D represents the total (column) dimension of F_cap, and dividing by √D can control the variance of the dot product to a constant range independent of the dimension. Softmax transforms the dot product (similarity) between the image capability feature vector F_cap and the policy parameter vector into the selection probability or attention weight of the candidate policy. s_i is the fitness score of the corresponding policy, i.e., the selection probability or attention weight (s_fid, s_eff, and s_bal represent the fitness scores of the fidelity-first conversion policy, efficiency-first conversion policy, and balanced conversion policy, respectively). .

[0055] The strategy decision network outputs the target format conversion strategy based on the fitness score and strategy selection constraints: fidelity priority requires PSNR>=40dB or SSIM>=0.98; efficiency priority requires single-page conversion time<=50ms; the balanced strategy requires the optimal weighted sum of fidelity and time (weight coefficient α=0.6).

[0056] The candidate strategies can be further refined, such as adding "ultra-high definition priority" and "low latency priority". In addition to using the dot product similarity between F_cap and the strategy parameter vectors of the three candidate transformation strategies to score the fit, the strategy evaluation network can also use MLP or cross attention.

[0057] The following is combined with Figure 5 and Figure 6 As shown, the S3 multi-stage format conversion is explained in detail.

[0058] Phase S31 image preprocessing includes format decoding and color space unification. The format decoding involves calling the corresponding decoder to obtain the original pixel matrix I_raw ∈ ℝ^(H×W×C) based on the encoding format (such as JPEG / PNG / TIFF / WebDP, etc.), determining the color space standardization or unification method according to the target format conversion strategy, and using the ICC (International Color Consortium) color profile to convert the original pixel matrix into intermediate image data I_mid.

[0059] The color space unification method is determined according to the target format conversion strategy: the fidelity-first strategy preserves the original color space; the efficiency-first strategy unifies to sRGB 8-bit; the equalization strategy preserves Adobe RGB for a wide color gamut, otherwise unifies to sRGB. Intermediate image data I_mid is generated through ICC color profile conversion.

[0060] Phase S32 capability-aware block coding includes adaptive block parameter determination, block operation according to the adaptively determined parameters, and capability-aware coding.

[0061] The adaptive block parameter determination includes: S321 determining the basic block size P_base based on the pixel resolution; S322 determining the semantic complexity level L_sem (levels 1-3) based on semantic feature information; S323 dynamically adjusting the block size P_adapt = P_base / γ^(L_sem-1), where γ=1.5; and S324 determining the block overlap rate (0% when L_sem=1, 10% when L_sem=2, and 20% when L_sem=3).

[0062] Capability-aware coding performs four steps for each image block, including: S325 linear projection mapping to generate an initial embedding vector e^(0); S326 calculating capability modulation coefficients (scaling factor γ_cap and bias β_cap) based on format dimension and color space dimension features; S327 performing an affine transformation using the FiLM mechanism e^(1) = γ_cap⊙e^(0)+β_cap, where e^(0) and e^(1) are the initial embedding vector before the transformation and the transformation result, respectively, and ⊙ represents element-wise multiplication (Hadamard product).

[0063] : Initial feature embedding; S328 adds two-dimensional positional encoding based on block index and block size to generate capability embedding vectors, i.e., capability embedding vector sequence E=[c_cap; e_1; e_2; ...; e_Np].

[0064] In addition to fixed position codes, the added position codes can also be learnable position codes or relative position codes.

[0065] Phase S33 unified format encapsulation includes: S331 linearly transforming F_cap to generate a capability condition vector c_cap; S332 concatenating c_cap as a prefix token with the embedded sequence to generate a fused token sequence E_fuse, i.e., E_fuse = [c_cap; e_1; e_2; ...; e_Np]; S333 aligning to the large model dimension d_model through a feature alignment layer; S334 adding a format identifier header (version number + capability description summary + CRC32 checksum).

[0066] In addition to serving as a prefix token, the capability condition vector can also be incorporated into self-attention computation as a special separator token or attention bias term. The feature alignment layer can employ a combination of linear projection and LayerNorm, i.e., T_out = LayerNorm(E_fuse · W_align), where the fused token sequence E_fuse is linearly projected through a learnable W_align and then normalized by the LayerNorm layer; alternatively, a combination of residual connections and MLP can be used.

[0067] As another aspect of the present invention, the present invention also provides an image capability format conversion apparatus for large model training, which can be used to implement the image capability format conversion method for large model training described above.

[0068] like Figure 7 As shown, the image capability format conversion device of the present invention includes the following modules:

[0069] (1) Image capability parsing module 601: includes format dimension parsing unit 6011, resolution dimension parsing unit 6012, color space dimension parsing unit 6013, metadata dimension parsing unit 6014, semantic dimension parsing unit 6015 and feature splicing unit 6016.

[0070] (2) Capability adaptation strategy selection module 602: includes strategy evaluation network 6021 and strategy decision network 6022.

[0071] (3) Multi-stage format conversion module 603: includes image preprocessing submodule 6031, capability-aware block coding submodule 6032 (including adaptive block parameter determination unit 60321 and capability modulation coding unit 60322, and unified format encapsulation submodule 6033).

[0072] (4) Format verification and output module 604: includes format integrity verification unit 6041, semantic consistency verification unit 6042 and output control unit 6043.

[0073] Optionally, the image capability format conversion apparatus of the present invention includes one or more of the following modules:

[0074] (5) Batch conversion scheduling module 605: Generates a scheduling scheme based on capability characteristics and computing resources, prioritizing the allocation to GPU for high fidelity and NPU / TPU for high efficiency.

[0075] (6) Format cache management module 606: Uses LSH similarity retrieval, and reuses conversion parameters when cosine similarity > 0.95.

[0076] The conversion parameters include block parameters and coding parameters. Block parameters include block size, block overlap rate, etc., while coding parameters include modulation parameters and position parameters, etc.

[0077] The following is combined with Figure 8 As shown, the multi-stage format conversion module 603 is described in detail.

[0078] Among them, the image preprocessing submodule 6031 calls the format decoder to obtain the original pixel matrix according to the encoding format of the original image data, and the color space converter converts the original pixel matrix to generate intermediate image data according to the target format conversion strategy with the help of ICC (International Color Consortium) color configuration.

[0079] In the capability-aware block coding submodule 6032, the adaptive block parameter determination unit 60321 dynamically adjusts the block size to different degrees based on the basic block size determined by the pixel resolution according to the semantic complexity level. This allows for fine-grained block division of semantically complex regions and coarse-grained block division of semantically simple regions, and dynamically adjusts the block overlap rate. The capability modulation coding unit 60322, based on the initial embedding vector generated by linear projection mapping, calculates and determines the modulation coefficients, including scaling factors and biases, according to the relevant image capability feature information. It then performs an affine transformation on the initial embedding vector and adds two-dimensional positional coding based on the block index and block size to generate the capability embedding vector sequence E.

[0080] Additionally, the unified format encapsulation submodule 6033 first linearly transforms the image capability feature vector to generate a capability condition vector, then concatenates the capability condition vector as a prefix token with the capability embedding vector sequence to form a fusion token sequence, and finally aligns it to the large model dimension and adds a format identifier header.

[0081] As another aspect of the present invention, the present invention also provides an image capability format conversion system for large model training, configured to implement the image capability format conversion method for large model training as described above. For example, the system may include a computing device, a storage medium, a chip, or a larger facility incorporating the aforementioned image capability format conversion device for large model training. The computing device includes a memory and a processor, the memory storing a program or instructions executable on the processor, wherein the program or instructions, when executed, implement the image capability format conversion method for large model training as described above. The storage medium stores a readable program or instructions, wherein the program or instructions, when executed, implement the image capability format conversion method for large model training as described above. The chip includes a processor and a communication interface coupled to the processor, the processor being used to run the program or instructions, wherein the processor, when running the program or instructions, implements the image capability format conversion method as described above.

[0082] The specific embodiments of the present invention have been described above, but the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many equivalent or similar substitutions or changes without departing from the spirit of the present invention, all of which are within the protection scope of the present invention.

Claims

1. An image capability format conversion method for large model training, characterized in that, include: The image capability parsing step (S1) acquires the original image data to be converted and performs multi-dimensional capability parsing and extracts image capability feature vectors. The multi-dimensional dimensions include format dimension, resolution dimension, color space dimension, metadata dimension and semantic dimension. The capability adaptation strategy selection step (S2) inputs the image capability feature vector into a preset capability adaptation strategy selection model and selects and determines the target format conversion strategy from the candidate conversion strategies. The multi-stage format conversion step (S3) includes an image preprocessing stage that performs format decoding and color space normalization conversion (S31), a capability-aware block coding stage that performs adaptive block coding and capability modulation coding (S32), and a unified format encapsulation stage that performs fusion and alignment (S33). The format verification and output step (S4) performs format integrity verification and semantic consistency verification, and only outputs the unified format tensor data as the format conversion result if both verifications pass.

2. The image capability format conversion method according to claim 1, characterized in that, If the format verification and output step (S4) fails the verification, it returns to the capability adaptation strategy selection step (S2) to reselect a strategy or directly adjust the conversion parameters for incremental correction.

3. The image capability format conversion method according to claim 1, characterized in that, The candidate conversion strategies include fidelity-first conversion strategy, efficiency-first conversion strategy, and balanced conversion strategy; the capability adaptation strategy selection model includes a strategy evaluation network and a strategy decision network. The strategy evaluation network scores the candidate conversion strategies based on image capability features, and the strategy decision network selects the target format conversion strategy with the highest fit based on the fit score and strategy selection constraints.

4. The image capability format conversion method according to claim 1, characterized in that, The capability-aware block coding stage (S32) includes determining adaptive block parameters and performing adaptive block coding based on the determined block parameters. in The adaptive segmentation parameter determination dynamically adjusts the segmentation parameters based on the semantic complexity expressed as semantic entropy, so that more reliable fine-grained segmentation is performed on semantically complex regions and coarse-grained segmentation is performed on semantically simple regions.

5. The image capability format conversion method according to claim 4, characterized in that, The segmentation parameters include segment size and segment overlap rate. The dynamic adjustment includes: a base segment size determined based on pixel resolution; dividing semantic complexity into discrete levels, determining specific segment sizes based on the level by reducing the base segment size accordingly, and adjusting the segment overlap rate according to the level; the reduction is to determine the base segment size as the segment size with the lowest semantic complexity level, and the ratio of the lower-level segment size to the higher-level segment size in adjacent levels is a reduction factor; the adjustment of segment overlap rate according to level is to determine the overlap rate of the lowest semantic complexity level as 0, and the overlap rate of the higher-level segment in adjacent levels is increased by a certain percentage compared to the overlap rate of the lower-level segment; the number of levels is 3, the reduction factor is 1.5, and the certain percentage is 10%.

6. The image capability format conversion method according to claim 1, characterized in that, The capability-aware block coding stage (S32) includes capability-aware coding that performs capability modulation coding. Capability-aware coding generates an initial embedding vector for each image block data through linear projection mapping, calculates and determines the modulation coefficients including scaling factors and biases using relevant image capability feature information, performs an affine transformation on the initial embedding vector, and adds position coding to generate a capability embedding vector sequence. The capability feature information used to calculate and determine the modulation coefficients is capability feature information in the format dimension and color space dimension, and the affine transformation is FiLM (Feature-Level Linear Modulation). The position coding adopts two-dimensional position coding based on block index and block size.

7. The image capability format conversion method according to claim 1, characterized in that, The unified format encapsulation stage (S33) linearly transforms the image capability feature vector to generate a capability condition vector; the capability condition vector is used as a prefix token and concatenated with the capability embedding vector sequence to form a fusion token sequence; it is aligned to the large model dimension and a format identifier header is added.

8. An image capability format conversion device for training large models, characterized in that, Includes the following modules: The image capability analysis module (601) acquires the original image data to be converted and performs multi-dimensional capability analysis and extracts image capability feature vectors. The multi-dimensional dimensions include format dimension, resolution dimension, color space dimension, metadata dimension and semantic dimension. The capability adaptation strategy selection module (602) includes a strategy evaluation network (6021) and a strategy decision network (6022), which determines the target format conversion strategy from candidate conversion strategies based on the image capability feature vector; A multi-stage format conversion module (603) includes an image preprocessing submodule (6031) for format decoding and color space conversion, a capability-aware block coding submodule (6032) for adaptive block coding and capability modulation coding, and a unified format encapsulation submodule (6033) for fusion and alignment. The format verification and output module (604) includes a format integrity verification unit (6041), a semantic consistency verification unit (6042), and an output control unit (6043). The output control unit (6043) only outputs the uniform format tensor data as the format conversion result when both the format integrity verification and the semantic consistency verification pass. The batch conversion scheduling module (605) generates a scheduling scheme based on capability characteristics and computing resources, prioritizing the allocation to GPUs (graphics processing units) for fidelity and NPUs / TPUs (neural network processors / tensor processors) for efficiency. The format cache management module (606) uses LSH (Local Sensitive Hash) similarity retrieval and reuses the conversion parameters when the cosine similarity is >0.

95.

9. The image capability format conversion apparatus according to claim 8, characterized in that, The capability-aware block coding submodule (6032) includes an adaptive block parameter determination unit (60321). The adaptive block parameter determination unit (60321) dynamically adjusts the block size, which is reduced to different degrees based on the basic block size determined by the pixel resolution, according to the semantic complexity level. This allows for fine-grained block division of semantically complex regions and coarse-grained block division of semantically simple regions, and dynamically adjusts the block overlap rate. The capability-aware block coding submodule (6032) also includes a capability modulation coding unit (60322). The capability modulation coding unit (60322) calculates and determines the modulation coefficients, including scaling factors and biases, based on the initial embedding vector generated by linear projection mapping and relevant image capability feature information. It then performs an affine transformation on the initial embedding vector and adds two-dimensional positional coding based on block index and block size to generate a capability embedding vector sequence. The image preprocessing submodule (6031) calls the format decoder to obtain the original pixel matrix according to the encoding format. The color space converter converts the original pixel matrix to generate intermediate image data according to the target format conversion strategy and with the help of ICC (International Color Consortium) color configuration. The unified format encapsulation submodule (6033) linearly transforms the image capability feature vector to generate a capability condition vector. It then concatenates the capability condition vector as a prefix token with the capability embedding vector sequence to form a fusion token sequence, aligns it to the large model dimension, and adds a format identifier header.

10. A system, characterized in that, The system is configured to implement the image capability format conversion method as described in any one of claims 1-7.

Images