Data processing method and apparatus

Abstract

Provided are a data processing method and apparatus. The data processing method comprises: acquiring multimedia data of a target prediction task, and converting the multimedia data into a binary byte sequence; inputting the binary byte sequence into a multimedia processing model, wherein the multimedia processing model comprises a linear projection unit, a sequence decoding unit, and a byte decoding unit; converting the binary byte sequence into a byte sequence vector by means of the linear projection unit, and using the sequence decoding unit to process the byte sequence vector so as to obtain a sequence feature; and using the byte decoding unit to process the sequence feature so as to obtain a predicted byte, and on the basis of the predicted byte, constructing target task data corresponding to the target prediction task.

Description

Data processing method and apparatus

[0001] This application claims priority to Chinese Patent Application No. 202411191435.9, filed on August 28, 2024, entitled “Data Processing Method and Apparatus”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] The embodiments in this specification relate to the field of machine learning technology, and in particular to data processing methods and apparatus. Background Technology

[0003] With the development of computer and internet technologies, deep learning technology is being applied in an increasing number of scenarios. Deep learning typically focuses on digital media files that are easily understood by humans, such as text, audio, and images. Most existing deep learning models operate in the media data domain; for example, language models excel in text generation and understanding through token prediction; audio processing models achieve advanced speech recognition and speech-to-speech translation by combining text and speech; and image processing models generate images by predicting the next pixel. These models have achieved significant results in their respective fields. However, this overlooks the ubiquitous raw binary data in the digital world. Bytes are the foundation of all digital data, devices, and software; from computer processors to operating systems in everyday electronic products, processing relies on bytes. Existing model prediction processes ignore the most fundamental element of data, resulting in the model's predictive capabilities and speed not being fully developed. Therefore, an effective solution is urgently needed to address these issues. Summary of the Invention

[0004] In view of the above, embodiments of this specification provide a data processing method. One or more embodiments of this specification also relate to a data processing apparatus, a computing device, a computer-readable storage medium, and a computer program product, to address the technical deficiencies existing in the prior art.

[0005] According to a first aspect of the embodiments of this specification, a data processing method is provided, comprising:

[0006] Acquire multimedia data for the target prediction task and convert the multimedia data into a binary byte sequence;

[0007] The binary byte sequence is input into a multimedia processing model, wherein the multimedia processing model includes a linear projection unit, a sequence decoding unit, and a byte decoding unit;

[0008] The binary byte sequence is converted into a byte sequence vector by the linear projection unit, and the byte sequence vector is processed by the sequence decoding unit to obtain sequence features;

[0009] The sequence features are processed using the byte decoding unit to obtain predicted bytes, and target task data corresponding to the target prediction task is constructed based on the predicted bytes.

[0010] Optionally, converting the multimedia data into a binary byte sequence includes:

[0011] The code point information corresponding to the multimedia data is determined according to the conversion strategy associated with the target prediction task;

[0012] The multimedia data is processed according to the code point information and the encoding rules configured in the conversion strategy to obtain the binary byte sequence corresponding to the multimedia data.

[0013] Optionally, converting the binary byte sequence into a byte sequence vector through the linear projection unit includes:

[0014] The binary byte sequence is input to the linear projection unit, and the linear projection unit converts the binary bytes contained in the binary byte sequence into a byte vector.

[0015] Construct an initial byte sequence vector corresponding to the binary byte sequence based on the byte vector corresponding to each binary byte;

[0016] The initial byte sequence vector is converted into a one-dimensional vector, and the one-dimensional vector is mapped to the binary byte space to obtain the byte sequence vector.

[0017] Optionally, processing the byte sequence vector using the sequence decoding unit to obtain sequence features includes:

[0018] The byte sequence vector is input to the sequence decoding unit, and the sequence decoding unit performs autoregressive prediction processing on the byte sequence vector to obtain sequence features;

[0019] The step of processing the sequence features using the byte decoding unit to obtain the predicted bytes includes:

[0020] The sequence features are input into the byte decoding unit, and the byte decoding unit performs autoregressive reconstruction processing on the sequence features to obtain the predicted bytes.

[0021] Optionally, constructing the target task data corresponding to the target prediction task based on the predicted bytes includes:

[0022] Determine the inverse conversion strategy corresponding to the conversion strategy, and parse the predicted bytes according to the inverse conversion strategy;

[0023] Based on the parsing results, the prediction code point information corresponding to the prediction byte is determined, and the target task data corresponding to the target prediction task is generated based on the prediction code point information.

[0024] Optionally, before the step of inputting the binary byte sequence into the multimedia processing model is executed, the method further includes:

[0025] Obtain sample data and the corresponding sample tags, convert the sample data into a sample binary byte sequence, and convert the sample tags into tag bytes;

[0026] The sample binary byte sequence is input into the initial multimedia processing model for processing to obtain training prediction bytes;

[0027] The initial multimedia processing model is trained based on the training prediction bytes and the label bytes until a multimedia processing model that meets the training stopping condition is obtained.

[0028] According to a second aspect of the embodiments of this specification, a data processing apparatus is provided, comprising:

[0029] The acquisition module is configured to acquire multimedia data for the target prediction task and convert the multimedia data into a binary byte sequence.

[0030] An input module is configured to input the binary byte sequence into a multimedia processing model, wherein the multimedia processing model includes a linear projection unit, a sequence decoding unit, and a byte decoding unit;

[0031] The conversion module is configured to convert the binary byte sequence into a byte sequence vector through the linear projection unit, and to process the byte sequence vector using the sequence decoding unit to obtain sequence features;

[0032] The construction module is configured to process the sequence features using the byte decoding unit to obtain predicted bytes, and to construct target task data corresponding to the target prediction task based on the predicted bytes.

[0033] According to a third aspect of the embodiments of this specification, a computing device is provided, comprising:

[0034] Memory and processor;

[0035] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the above-described data processing method.

[0036] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer-executable instructions that, when executed by a processor, implement the steps of the data processing method described above.

[0037] According to a fifth aspect of the embodiments of this specification, a computer program product is provided, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described data processing method.

[0038] The data processing method provided in this embodiment aims to improve the model's comprehensive understanding and simulation of digital systems, thereby enhancing its processing capabilities and efficiency for multimedia data. After acquiring the multimedia data for the target prediction task, the multimedia data is first converted into a binary byte sequence. This binary byte sequence is then input into a multimedia processing model, which includes a linear projection unit, a sequence decoding unit, and a byte decoding unit. Based on this, the linear projection unit converts the binary byte sequence into a byte sequence vector. The sequence decoding unit then processes the byte sequence vector to obtain sequence features, enabling the mining of structural relationships at the byte sequence level. The byte decoding unit further processes the sequence features to obtain predicted bytes, further enabling the mining of byte relationships at the byte level. Finally, the target task data corresponding to the target prediction task can be constructed based on the predicted bytes. This model-based processing at the binary byte level simplifies the integration process between different data types and improves the model's applicability and performance in cross-modal tasks. Attached Figure Description

[0039] Figure 1 is a schematic diagram of a data processing method provided in one embodiment of this specification;

[0040] Figure 2 is a flowchart of a data processing method provided in one embodiment of this specification;

[0041] Figure 3 is a flowchart of a data processing method provided in one embodiment of this specification;

[0042] Figure 4 is a schematic diagram of the structure of a data processing device provided in one embodiment of this specification;

[0043] Figure 5 is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0044] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0045] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0046] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0047] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0048] This specification provides a data processing method, and also relates to a data processing apparatus, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0049] In practical applications, deep learning models primarily focus on processing digital media data that is easily understood by humans, such as text, audio, and images. While these models have achieved significant results in their respective fields, they exhibit marked limitations in processing raw binary data. Specifically: text processing—language models excel in text generation and understanding through token prediction, but they rely on the tokenization process of text and cannot directly process binary data. Audio processing—while capable of handling audio data, audio processing models are similarly limited to specific data types and cannot effectively process and simulate binary data. Image processing—while able to predict the next pixel in an image, image processing models perform poorly when processing long byte sequences and complex binary data. Therefore, an effective solution is urgently needed to address these issues.

[0050] Referring to the schematic diagram in Figure 1, the data processing method provided in this embodiment, in order to improve the model's comprehensive understanding and simulation of digital systems, thereby enhancing the processing capability and efficiency of multimedia data, can first convert the multimedia data of the target prediction task into a binary byte sequence after acquisition. This binary byte sequence can then be input into the multimedia processing model, which includes a linear projection unit, a sequence decoding unit, and a byte decoding unit. Based on this, the binary byte sequence can be converted into a byte sequence vector using the linear projection unit. Then, the sequence decoding unit processes the byte sequence vector to obtain sequence features, realizing the mining of structural relationships at the byte sequence level. The byte decoding unit then processes the sequence features to obtain predicted bytes, realizing the mining of byte relationships at the byte level. Finally, the target task data corresponding to the target prediction task can be constructed based on the predicted bytes. This allows for binary byte-level processing through the model, simplifying the integration process between different data types and improving the model's applicability and performance in cross-modal tasks.

[0051] Referring to Figure 2, which shows a flowchart of a data processing method according to an embodiment of this specification, the method includes the following steps.

[0052] Step S202: Obtain multimedia data for the target prediction task and convert the multimedia data into a binary byte sequence.

[0053] The data processing method provided in this embodiment can be applied to model applications in any scenario, such as text processing scenarios (summary extraction, text classification, translation), image processing scenarios (image cropping, image generation, image-text generation), and audio processing scenarios (noise removal, audio generation, audio conversion), etc. This embodiment does not impose any limitations. This embodiment uses the application of the data processing method in a text processing scenario as an example to illustrate the data processing method. Descriptions of other scenarios can refer to the same or corresponding descriptions in this embodiment, and this embodiment does not impose any limitations. Furthermore, the data processing method provided in this embodiment can also be applied to cross-modal scenarios.

[0054] Specifically, the target prediction task refers to the task triggered in a multimedia data processing scenario. This task can be a text processing task in a text processing scenario, such as summarization, text classification, and translation; an image processing task in an image processing scenario, such as image cropping, image generation, and image-to-text generation; or an audio processing task in an audio processing scenario, such as noise removal, audio generation, and audio conversion. Correspondingly, multimedia data refers to the data obtained when the target prediction task is triggered, which needs to be input into the model for processing, such as text, images, video, and audio. Correspondingly, the binary byte sequence specifically refers to the sequence of binary bytes obtained after byte encoding of multimedia data. It is the most basic unit of computer processing. Converting multimedia data into binary bytes for model processing can effectively improve prediction efficiency. Simultaneously, it allows for the most effective development of the model, enabling the model to perform multimedia data analysis from the most basic unit.

[0055] Therefore, to improve the model's comprehensive understanding and simulation of digital systems, thereby enhancing its multimedia data processing capabilities and efficiency, after acquiring the multimedia data for the target prediction task, the multimedia data can first be converted into a binary byte sequence. This binary byte sequence can then be input into the multimedia processing model, which includes a linear projection unit, a sequence decoding unit, and a byte decoding unit. Based on this, the linear projection unit can convert the binary byte sequence into a byte sequence vector. The sequence decoding unit then processes the byte sequence vector to obtain sequence features, enabling the mining of structural relationships along the byte sequence dimension. The byte decoding unit further processes the sequence features to obtain the predicted bytes, further enabling the mining of byte relationships along the byte dimension. Finally, the target task data corresponding to the target prediction task can be constructed based on the predicted bytes.

[0056] Furthermore, when constructing binary byte sequences from multimedia data, in order to achieve cross-modal and more accurate model prediction, a preset conversion strategy can be followed. In this embodiment, the specific implementation is as follows:

[0057] The code point information corresponding to the multimedia data is determined according to the conversion strategy associated with the target prediction task; the multimedia data is processed according to the code point information and the encoding rules configured in the conversion strategy to obtain the binary byte sequence corresponding to the multimedia data.

[0058] Specifically, the conversion strategy refers to the strategy used to convert multimedia data of the associated target prediction task into a binary byte sequence according to a specified encoding method. This strategy can determine the code point information of each character in the multimedia data, as well as the encoding rules. The encoding rules are used to encode the multimedia data to obtain a binary byte sequence composed of binary bytes.

[0059] Based on this, after obtaining the multimedia data for the target prediction task, the code point information corresponding to the multimedia data can be determined first according to the conversion strategy associated with the target prediction task. On this basis, the multimedia data can be processed according to the code point information and the encoding rules configured in the conversion strategy to convert the multimedia data into multiple binary bytes, thereby forming the binary byte sequence corresponding to the multimedia data, so that it can be input into the model for prediction processing.

[0060] For example, a user submits the question text "Who is the author of poem A?" through a client. After receiving this question text, the system can first determine the corresponding code point of each character unit in Unicode encoding. Based on this, each character unit can be encoded using UTF-8 (8-bit Unicode Transformation Format). In UTF-8 encoding, the code point corresponding to each character unit is converted into a sequence of bytes of a set length. By calculating the byte sequence of the UTF-8 encoding and allocating the bytes, the hexadecimal byte sequence {A\xe5\x8f\xa4\xe8\xaf\x97\xe7\x9a\x84\xe4\xbd\x9c\xe8\80\x85\xe6\x98\xaf\xe8\xb0\x81} corresponding to the question text "Who is the author of poem A?" can be obtained. This can then be converted into a binary byte sequence for subsequent model processing.

[0061] In summary, by employing conversion strategies to transform multimedia data, it is possible to convert multimedia data into a unified binary byte sequence according to a set standard, thereby enabling subsequent models to make accurate predictions.

[0062] Step S204: Input the binary byte sequence into the multimedia processing model, wherein the multimedia processing model includes a linear projection unit, a sequence decoding unit, and a byte decoding unit.

[0063] Step S206: The binary byte sequence is converted into a byte sequence vector by the linear projection unit, and the byte sequence vector is processed by the sequence decoding unit to obtain sequence features.

[0064] Step S208: The sequence features are processed by the byte decoding unit to obtain the predicted bytes, and the target task data corresponding to the target prediction task is constructed based on the predicted bytes.

[0065] Specifically, after obtaining the binary byte sequence corresponding to the multimedia data, the binary byte sequence can be input into a multimedia processing model that includes a linear projection unit, a sequence decoding unit, and a byte decoding unit. This allows the multimedia processing model to output predicted bytes through binary byte-level prediction, thereby constructing the target task data corresponding to the target prediction task for subsequent use.

[0066] Specifically, the linear projection unit refers to the unit used to map the binary bytes contained in a binary byte sequence into a vector representation. Correspondingly, the sequence decoding unit refers to the decoder that constructs sequence-dimensional features, and the byte decoding unit refers to the decoder that constructs byte-dimensional features, used to output predicted bytes. Correspondingly, the byte sequence vector refers to the sequence vector obtained after vector mapping of the binary byte sequence. Sequence features refer to the sequence-level features extracted from the byte sequence vector. Predicted bytes refer to the model output result obtained after processing the multimedia data corresponding to the target prediction task, and this result is in binary bytes, used to construct the target task data. Correspondingly, the target task data refers to the result obtained after model processing of the multimedia data. For example, if the target prediction task is a text generation task, then the target task data is the text generation result; if the target prediction task is an audio noise removal task, then the target task data is the noise-removed audio.

[0067] In other words, when the multimedia processing model processes binary byte sequences, in order to uncover the byte relationships at the binary byte level, the binary byte sequence can first be converted into a byte sequence vector using a linear projection unit. After obtaining the vector representation of the binary byte sequence, subsequent prediction processing can be performed. To improve the model's prediction capability and achieve sufficient data relationship analysis of the binary bytes, the byte sequence vector can be processed first using a sequence decoding unit to obtain sequence features of the corresponding sequence dimension. Then, the sequence features can be processed using the byte decoding unit to complete the prediction processing, thereby obtaining the predicted bytes. Subsequently, target task data corresponding to the target prediction task can be constructed based on the predicted bytes, which can be easily used downstream. For example, the generated text can be fed back to the user, or user questions can be answered based on the generated text. This embodiment does not impose any limitations.

[0068] In practical applications, multimedia processing models can employ a hierarchical Transformer architecture to segment byte sequences into fragments to improve computational efficiency. The model can include three main components: a linear projection unit, a sequence decoding unit, and a byte decoding unit, thereby achieving byte-level prediction.

[0069] Furthermore, after inputting the binary byte sequence into the multimedia processing model, in order to achieve prediction at the binary byte level, a byte-to-vector conversion is required. In this embodiment, the specific implementation is as follows:

[0070] The binary byte sequence is input to the linear projection unit, which converts the binary bytes contained in the binary byte sequence into byte vectors. An initial byte sequence vector corresponding to the binary byte sequence is constructed based on the byte vector corresponding to each binary byte. The initial byte sequence vector is converted into a one-dimensional vector and mapped to the binary byte space to obtain the byte sequence vector.

[0071] Specifically, a byte vector refers to the vector representation obtained by vectorizing the binary bytes contained within a binary byte. Correspondingly, the initial byte sequence vector refers to the vector representation obtained by concatenating the byte vectors corresponding to each binary byte. The binary byte space refers to mapping a one-dimensional vector to a byte space, allowing the byte sequence vectors mapped from the byte space to represent the inherent data relationships within multimedia data, thus facilitating model prediction.

[0072] Based on this, after inputting the binary byte sequence into the multimedia processing model, the sequence can first be vectorized using a linear projection unit. At this point, the binary bytes contained in the binary byte sequence can be converted into byte vectors using the linear projection unit. On this basis, an initial byte sequence vector corresponding to the binary byte sequence can be constructed based on the byte vector corresponding to each binary byte. Then, the initial byte sequence vector can be converted into a one-dimensional vector, and the one-dimensional vector can be mapped to the binary byte space to obtain the byte sequence vector of the corresponding multimedia resource for subsequent prediction.

[0073] In practical applications, when mapping binary byte sequences, the linear projection unit can treat each binary byte sequence as a matrix. Each byte can be represented as a 257-dimensional vector using one-hot encoding, including all 256 byte values ​​and a vector. <eop>The sequence is marked with an end marker. The sequence is then converted into a one-dimensional vector and mapped to the binary byte space to obtain the byte sequence vector.

[0074] In summary, by embedding linear projection units in the model to map binary byte sequences to binary byte space, it is possible to represent the data relationships in multimedia data even when obtaining byte sequence vectors, thereby enabling the model to make accurate predictions.

[0075] Furthermore, after obtaining the byte sequence vector, byte prediction can be completed by combining the sequence decoding unit and the byte decoding unit in the model. In this embodiment, the specific implementation is as follows:

[0076] The byte sequence vector is input to the sequence decoding unit, which performs autoregressive prediction processing on the byte sequence vector to obtain sequence features; the sequence features are then input to the byte decoding unit, which performs autoregressive reconstruction processing on the sequence features to obtain predicted bytes.

[0077] Based on this, after obtaining the byte sequence vector mapped by the linear projection unit, the byte sequence vector can be input into the sequence decoding unit. The sequence decoding unit performs autoregressive prediction processing on the byte sequence vector to achieve structural prediction at the sequence level, thereby obtaining sequence features. Then, the sequence features are input into the byte decoding unit, which performs autoregressive reconstruction processing on the sequence features to achieve binary byte prediction at the byte dimension, thereby obtaining predicted bytes. These predicted bytes can then be converted into target task data for downstream task processing.

[0078] In practical applications, the sequence decoding unit can autoregressively predict the features of subsequent sequences and learn the structure of the data, while the byte decoding unit can autoregressively reconstruct the byte sequence within a segment based on the segment features, thereby obtaining the predicted character for subsequent use.

[0079] In summary, by employing a sequence decoder to predict features from the sequence dimension, and then generating predicted bytes from the byte dimension based on the prediction results, the model can output more accurate prediction results from the binary dimension.

[0080] After obtaining the predicted bytes, considering that the final result needs to be fed back to the user or business unit, it is also necessary to convert the predicted bytes into target task data. In this embodiment, the specific implementation is as follows:

[0081] Determine the inverse conversion strategy corresponding to the conversion strategy, and parse the predicted byte according to the inverse conversion strategy; determine the predicted code point information corresponding to the predicted byte according to the parsing result, and generate the target task data corresponding to the target prediction task according to the predicted code point information.

[0082] Specifically, the inverse conversion strategy specifically refers to the strategy of converting the predicted byte into target task data, that is, the predicted byte can be converted into text, image, audio, etc. through the inverse conversion strategy. Correspondingly, the predicted code point information specifically refers to the code point information corresponding to the predicted byte.

[0083] Based on this, after the model outputs the predicted byte, the inverse conversion strategy corresponding to the conversion strategy can be determined first. On this basis, the predicted byte can be parsed according to the inverse conversion strategy; it is realized to determine the predicted code point information corresponding to the predicted byte according to the parsing result. After that, the target task data corresponding to the target prediction task can be generated according to the predicted code point information for downstream use.

[0084] Continuing with the above example, after obtaining the binary byte sequence corresponding to the question text "Who is the author of ancient poem A", the binary byte sequence can be input into the multimedia processing model; first, the binary byte sequence is mapped through the linear projection unit in the multimedia processing model to obtain the byte sequence vector ZLc; second, the byte sequence vector ZLc can be input into the sequence decoder in the multimedia processing model, and the sequence features are predicted autoregressively through the sequence decoder to obtain the sequence vector Lc. Finally, the sequence vector Lc is input into the byte decoder in the multimedia processing model, and the byte sequence in the sequence is reconstructed autoregressively through the byte decoder, so that the predicted byte can be generated. After obtaining the predicted byte, the predicted byte is converted according to the reverse process of the above encoding, and the answer text "A" corresponding to the question text "Who is the author of ancient poem A" can be obtained and fed back to the user.

[0085] In summary, by processing multimedia data through a multimedia processing model including a linear projection unit, a sequence decoding unit and a byte decoding unit, the model can give a prediction result from the byte dimension, thereby effectively improving the model prediction accuracy and processing efficiency. At the same time, less computing resources can be occupied.

[0086] In addition, in order to enable the multimedia processing model to complete predictions from the byte dimension, the model needs to be equipped with this ability during the training phase. In this embodiment, the training of the model includes:

[0087] Obtain sample data and corresponding sample labels; convert the sample data into a sample binary byte sequence and the sample labels into label bytes; input the sample binary byte sequence into an initial multimedia processing model for processing to obtain training prediction bytes; train the initial multimedia processing model based on the training prediction bytes and the label bytes until a multimedia processing model that meets the training stopping condition is obtained.

[0088] Specifically, sample data and sample labels refer to the samples used to train the multimedia processing model. By converting them into binary sequences, the model can learn the predictive ability of binary bytes, thereby improving its ability to mine byte-level data. Correspondingly, the label bytes are the binary bytes corresponding to the sample labels. Training prediction bytes specifically refer to the predicted bytes output by the model. The training stopping condition specifically refers to the conditions for stopping the training of the multimedia processing model, including but not limited to loss value comparison conditions, validation conditions, or iteration count conditions; this embodiment does not impose any limitations on these conditions.

[0089] Based on this, during the model training phase, sample data and corresponding sample labels can be obtained first. Then, the sample data can be converted into sample binary byte sequences, and the sample labels can be converted into label bytes. On this basis, the model can be trained by inputting the sample binary byte sequences into the initial multimedia processing model for processing, thereby obtaining training prediction bytes. On this basis, the initial multimedia processing model is trained by combining the training prediction bytes and label bytes until a multimedia processing model that meets the training stopping condition is obtained.

[0090] The data processing method provided in this embodiment aims to improve the model's comprehensive understanding and simulation of digital systems, thereby enhancing its processing capabilities and efficiency for multimedia data. After acquiring the multimedia data for the target prediction task, the multimedia data is first converted into a binary byte sequence. This binary byte sequence is then input into a multimedia processing model, which includes a linear projection unit, a sequence decoding unit, and a byte decoding unit. Based on this, the linear projection unit converts the binary byte sequence into a byte sequence vector. The sequence decoding unit then processes the byte sequence vector to obtain sequence features, enabling the mining of structural relationships at the byte sequence level. The byte decoding unit further processes the sequence features to obtain predicted bytes, further enabling the mining of byte relationships at the byte level. Finally, the target task data corresponding to the target prediction task can be constructed based on the predicted bytes. This model-based processing at the binary byte level simplifies the integration process between different data types and improves the model's applicability and performance in cross-modal tasks.

[0091] The following description, in conjunction with Figure 3, uses the application of the data processing method provided in this specification in an image data processing scenario as an example to further illustrate the data processing method. Figure 3 shows a flowchart of the processing procedure of a data processing method according to an embodiment of this specification, specifically including the following steps.

[0092] Step S302: Obtain the multimedia data of the target prediction task, and determine the code point information corresponding to the multimedia data according to the conversion strategy associated with the target prediction task.

[0093] Step S304: Process the multimedia data according to the code point information and the encoding rules configured in the conversion strategy to obtain the binary byte sequence corresponding to the multimedia data.

[0094] Step S306: Input the binary byte sequence into the multimedia processing model, wherein the multimedia processing model includes a linear projection unit, a sequence decoding unit, and a byte decoding unit.

[0095] Step S308: Convert the binary bytes contained in the binary byte sequence into a byte vector using a linear projection unit.

[0096] Step S310: Construct an initial byte sequence vector corresponding to the binary byte sequence based on the byte vector corresponding to each binary byte.

[0097] Step S312: Convert the initial byte sequence vector into a one-dimensional vector and map the one-dimensional vector to the binary byte space to obtain the byte sequence vector.

[0098] Step S314: The byte sequence vector is subjected to autoregressive prediction processing by the sequence decoding unit to obtain sequence features.

[0099] Step S316: Input the sequence features into the byte decoding unit, and perform autoregressive reconstruction processing on the sequence features through the byte decoding unit to obtain the predicted bytes.

[0100] Step S318: Determine the inverse conversion strategy corresponding to the conversion strategy, and parse the predicted bytes according to the inverse conversion strategy.

[0101] Step S320: Determine the prediction code point information corresponding to the prediction byte based on the parsing result, and generate the target task data corresponding to the target prediction task based on the prediction code point information.

[0102] The data processing method provided in this embodiment aims to improve the model's comprehensive understanding and simulation of digital systems, thereby enhancing its processing capabilities and efficiency for multimedia data. After acquiring the multimedia data for the target prediction task, the multimedia data is first converted into a binary byte sequence. This binary byte sequence is then input into a multimedia processing model, which includes a linear projection unit, a sequence decoding unit, and a byte decoding unit. Based on this, the linear projection unit converts the binary byte sequence into a byte sequence vector. The sequence decoding unit then processes the byte sequence vector to obtain sequence features, enabling the mining of structural relationships at the byte sequence level. The byte decoding unit further processes the sequence features to obtain predicted bytes, further enabling the mining of byte relationships at the byte level. Finally, the target task data corresponding to the target prediction task can be constructed based on the predicted bytes. This model-based processing at the binary byte level simplifies the integration process between different data types and improves the model's applicability and performance in cross-modal tasks.

[0103] Corresponding to the above method embodiments, this specification also provides data processing apparatus embodiments. Figure 4 shows a schematic diagram of the structure of a data processing apparatus provided in one embodiment of this specification. As shown in Figure 4, the apparatus includes:

[0104] The acquisition module 402 is configured to acquire multimedia data of the target prediction task and convert the multimedia data into a binary byte sequence;

[0105] The input module 404 is configured to input the binary byte sequence into a multimedia processing model, wherein the multimedia processing model includes a linear projection unit, a sequence decoding unit, and a byte decoding unit;

[0106] The conversion module 406 is configured to convert the binary byte sequence into a byte sequence vector through the linear projection unit, and to process the byte sequence vector using the sequence decoding unit to obtain sequence features;

[0107] The construction module 408 is configured to process the sequence features using the byte decoding unit to obtain the predicted bytes, and to construct the target task data corresponding to the target prediction task based on the predicted bytes.

[0108] In an optional embodiment, the acquisition module 402 is further configured to:

[0109] The code point information corresponding to the multimedia data is determined according to the conversion strategy associated with the target prediction task; the multimedia data is processed according to the code point information and the encoding rules configured in the conversion strategy to obtain the binary byte sequence corresponding to the multimedia data.

[0110] In an optional embodiment, the conversion module 406 is further configured to:

[0111] The binary byte sequence is input to the linear projection unit, which converts the binary bytes contained in the binary byte sequence into byte vectors. An initial byte sequence vector corresponding to the binary byte sequence is constructed based on the byte vector corresponding to each binary byte. The initial byte sequence vector is converted into a one-dimensional vector and mapped to the binary byte space to obtain the byte sequence vector.

[0112] In an optional embodiment, the conversion module 406 is further configured to:

[0113] The byte sequence vector is input to the sequence decoding unit, and the sequence decoding unit performs autoregressive prediction processing on the byte sequence vector to obtain sequence features;

[0114] The construction module 408 is further configured as follows:

[0115] The sequence features are input into the byte decoding unit, and the byte decoding unit performs autoregressive reconstruction processing on the sequence features to obtain the predicted bytes.

[0116] In an optional embodiment, the building module 408 is further configured to:

[0117] Determine the inverse conversion strategy corresponding to the conversion strategy, and parse the predicted byte according to the inverse conversion strategy; determine the prediction code point information corresponding to the predicted byte according to the parsing result, and generate the target task data corresponding to the target prediction task according to the prediction code point information.

[0118] In an optional embodiment, the apparatus further includes:

[0119] The training module is configured to acquire sample data and corresponding sample labels, convert the sample data into a sample binary byte sequence, and convert the sample labels into label bytes; input the sample binary byte sequence into an initial multimedia processing model for processing to obtain training prediction bytes; and train the initial multimedia processing model based on the training prediction bytes and the label bytes until a multimedia processing model that meets the training stopping condition is obtained.

[0120] The data processing apparatus provided in this embodiment, in order to improve the model's comprehensive understanding and simulation of digital systems, thereby enhancing the processing capability and efficiency of multimedia data, can first convert the multimedia data of the target prediction task into a binary byte sequence after acquisition. This binary byte sequence can then be input into a multimedia processing model, which includes a linear projection unit, a sequence decoding unit, and a byte decoding unit. Based on this, the linear projection unit can convert the binary byte sequence into a byte sequence vector, and then the sequence decoding unit processes the byte sequence vector to obtain sequence features, thus enabling the mining of structural relationships at the byte sequence level. The byte decoding unit then processes the sequence features to obtain predicted bytes, achieving the mining of byte relationships at the byte level. Finally, the target task data corresponding to the target prediction task can be constructed based on the predicted bytes. This allows for binary byte-level processing through the model, simplifying the integration process between different data types and improving the model's applicability and performance in cross-modal tasks.

[0121] The above is an illustrative scheme of a data processing apparatus according to this embodiment. It should be noted that the technical solution of this data processing apparatus and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the data processing apparatus, please refer to the description of the technical solution of the data processing method described above.

[0122] Figure 5 shows a structural block diagram of a computing device 500 according to one embodiment of this specification. The components of the computing device 500 include, but are not limited to, a memory 510 and a processor 520. The processor 520 is connected to the memory 510 via a bus 530, and a database 550 is used to store data.

[0123] The computing device 500 also includes an access device 540, which enables the computing device 500 to communicate via one or more networks 560. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 540 may include one or more of any type of wired or wireless network interface (e.g., a network interface controller (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0124] In one embodiment of this specification, the aforementioned components of the computing device 500, as well as other components not shown in FIG. 5, may be interconnected, for example, via a bus. It should be understood that the block diagram of the computing device shown in FIG. 5 is merely illustrative and not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0125] Computing device 500 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). Computing device 500 can also be a mobile or stationary server.

[0126] The processor 520 is configured to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the above-described data processing method.

[0127] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the data processing method described above.

[0128] An embodiment of this specification also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the above-described data processing method.

[0129] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the data processing method described above.

[0130] An embodiment of this specification also provides a computer program product, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described data processing method.

[0131] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solution of the data processing method described above.

[0132] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0133] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0134] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0135] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0136] The preferred embodiments disclosed above are merely illustrative of this specification. Optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described in this specification. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification.< / eop>

Claims

1. A data processing method, characterized in that, include: Acquire multimedia data for the target prediction task and convert the multimedia data into a binary byte sequence; The binary byte sequence is input into a multimedia processing model, wherein the multimedia processing model includes a linear projection unit, a sequence decoding unit, and a byte decoding unit; The binary byte sequence is converted into a byte sequence vector by the linear projection unit, and the byte sequence vector is processed by the sequence decoding unit to obtain sequence features; The sequence features are processed using the byte decoding unit to obtain predicted bytes, and target task data corresponding to the target prediction task is constructed based on the predicted bytes.

2. The data processing method according to claim 1, characterized in that, The step of converting the multimedia data into a binary byte sequence includes: The code point information corresponding to the multimedia data is determined according to the conversion strategy associated with the target prediction task; The multimedia data is processed according to the code point information and the encoding rules configured in the conversion strategy to obtain the binary byte sequence corresponding to the multimedia data.

3. The data processing method according to claim 1, characterized in that, The step of converting the binary byte sequence into a byte sequence vector through the linear projection unit includes: The binary byte sequence is input to the linear projection unit, and the linear projection unit converts the binary bytes contained in the binary byte sequence into a byte vector. Construct an initial byte sequence vector corresponding to the binary byte sequence based on the byte vector corresponding to each binary byte; The initial byte sequence vector is converted into a one-dimensional vector, and the one-dimensional vector is mapped to the binary byte space to obtain the byte sequence vector.

4. The data processing method according to claim 1, characterized in that, The step of processing the byte sequence vector using the sequence decoding unit to obtain sequence features includes: The byte sequence vector is input to the sequence decoding unit, and the sequence decoding unit performs autoregressive prediction processing on the byte sequence vector to obtain sequence features; The step of processing the sequence features using the byte decoding unit to obtain the predicted bytes includes: The sequence features are input into the byte decoding unit, and the byte decoding unit performs autoregressive reconstruction processing on the sequence features to obtain the predicted bytes.

5. The data processing method according to claim 2, characterized in that, The construction of the target task data corresponding to the target prediction task based on the predicted bytes includes: Determine the inverse conversion strategy corresponding to the conversion strategy, and parse the predicted bytes according to the inverse conversion strategy; Based on the parsing results, the prediction code point information corresponding to the prediction byte is determined, and the target task data corresponding to the target prediction task is generated based on the prediction code point information.

6. The data processing method according to any one of claims 1 to 5, characterized in that, Before the step of inputting the binary byte sequence into the multimedia processing model is executed, the method further includes: Obtain sample data and the corresponding sample tags, convert the sample data into a sample binary byte sequence, and convert the sample tags into tag bytes; The sample binary byte sequence is input into the initial multimedia processing model for processing to obtain training prediction bytes; The initial multimedia processing model is trained based on the training prediction bytes and the label bytes until a multimedia processing model that meets the training stopping condition is obtained.

7. A data processing apparatus, characterized in that, include: The acquisition module is configured to acquire multimedia data for the target prediction task and convert the multimedia data into a binary byte sequence. An input module is configured to input the binary byte sequence into a multimedia processing model, wherein the multimedia processing model includes a linear projection unit, a sequence decoding unit, and a byte decoding unit; The conversion module is configured to convert the binary byte sequence into a byte sequence vector through the linear projection unit, and to process the byte sequence vector using the sequence decoding unit to obtain sequence features; The construction module is configured to process the sequence features using the byte decoding unit to obtain predicted bytes, and to construct target task data corresponding to the target prediction task based on the predicted bytes.

8. A computing device, characterized in that, include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, It stores computer-executable instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that, It includes a computer program or instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 6.

Images