A multi-modal data processing method and terminal

Abstract

The present invention provides a multimodal data processing method and terminal, in which text data is converted into text code through a symbol table, and after image data is converted into fingerprint data, the fingerprint data is converted into image code through a symbol table, so that the image data is converted into a symbolic form for embedding, thereby realizing that the image data and the text data form a unified symbolic mode. At the same time, the present invention does not need to construct different data processing channels for different modal data, can realize the input of mixed data, reduce data processing costs, and adopts a perceptual hash algorithm to directly extract fingerprint data of image data without additional convolution training, realize plug-and-play, and improve pre-training efficiency.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a multimodal data processing method and terminal. Background Art

[0002] At present, there are many methods for pre-training by embedding multimodal information, such as CLIP (Contrastive Language-Image Pre-training, a pre-training model based on contrasting text-image pairs) using modality alignment to achieve pre-training; GPT (Generative Pre-Training, a natural language processing model based on deep learning) predicts the next word to achieve pre-training; BERT (Bidirectional Encoder Representations from Transformers, a language model built on bidirectional Transformer) uses random masking words for pre-training. Among them, the existing image modality embedding methods generally use convolution. The image modality is passed through the transformer model (a neural network) and the convolution method is used to extract the embedded features. However, the features embedded based on the convolution method are inconsistent with the features embedded in the symbol in the structure of the transformer model. Since the transformer model is designed to learn the relationship between words, and the image is not used as a symbol in the convolution method, the image vector features and the symbol embedding features are inconsistent. Moreover, traditional embedding methods require independent processing of input text and images, that is, the text and image are input into two different modal networks for processing and then aligned. In addition, traditional image modality embedding methods require training of the regional feature extraction network, while text modality does not require it. Symbols can be directly input into the embedding network for learning, resulting in asynchronous processing efficiency of image modality and text modality, affecting pre-training efficiency. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a multimodal data processing method and terminal to achieve consistency between image modality features and text modality features, while ensuring the unity of image modality and text modality training models.

[0004] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0005] A multimodal data processing method comprises the steps of:

[0006] Acquire mixed data, where the mixed data includes image data and text data;

[0007] Obtaining a text code corresponding to the text data according to a preset symbol table;

[0008] Inputting the image data into a perceptual hash algorithm to obtain fingerprint data, and obtaining an image code corresponding to the fingerprint data according to the symbol table;

[0009] The text code and the image code are combined and processed and then input into the embedding layer network for dimension transformation to obtain embedding features.

[0010] In order to solve the above technical problems, another technical solution adopted by the present invention is:

[0011] A multimodal data processing terminal comprises a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the following steps when executing the computer program:

[0012] Acquire mixed data, where the mixed data includes image data and text data;

[0013] Obtaining a text code corresponding to the text data according to a preset symbol table;

[0014] Inputting the image data into a perceptual hash algorithm to obtain fingerprint data, and obtaining an image code corresponding to the fingerprint data according to the symbol table;

[0015] The text code and the image code are combined and processed and then input into the embedding layer network for dimension transformation to obtain embedding features.

[0016] The beneficial effect of the present invention is that: the text data is converted into text code through a symbol table, and after the image data is converted into fingerprint data, the fingerprint data is converted into image code through a symbol table, so that the image data is converted into a symbolic form for embedding, thereby realizing that the image data and the text data form a unified symbolic modality. At the same time, the present invention does not need to construct different data processing channels for different modal data, can realize the input of mixed data, reduce data processing costs, and adopts a perceptual hash algorithm to directly extract fingerprint data of image data without additional convolution training, realize plug-and-play, and improve pre-training efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A flowchart of a multimodal data processing method provided by an embodiment of the present invention;

[0018] Figure 2 A program flow chart of a multimodal data processing method provided by an embodiment of the present invention;

[0019] Figure 3 A program flow chart of a perceptual hash algorithm provided by an embodiment of the present invention;

[0020] Figure 4 A schematic diagram of the structure of a multimodal data processing terminal provided by an embodiment of the present invention;

[0021] Description of labels:

[0022] 201. Memory; 202. Processor. DETAILED DESCRIPTION

[0023] In order to explain the technical content, achieved objectives and effects of the present invention in detail, the following is an explanation in combination with the implementation modes and the accompanying drawings.

[0024] Please refer to Figure 1 , a multimodal data processing method provided by an embodiment of the present invention comprises the steps of:

[0025] Acquire mixed data, where the mixed data includes image data and text data;

[0026] Obtaining a text code corresponding to the text data according to a preset symbol table;

[0027] Inputting the image data into a perceptual hash algorithm to obtain fingerprint data, and obtaining an image code corresponding to the fingerprint data according to the symbol table;

[0028] The text code and the image code are combined and processed and then input into the embedding layer network for dimension transformation to obtain embedding features.

[0029] From the above description, it can be seen that the beneficial effect of the present invention is that: the text data is converted into text code through the symbol table, and after the image data is converted into fingerprint data, the fingerprint data is converted into image code through the symbol table, so that the image data is converted into symbolic form for embedding, thereby realizing that the image data and the text data form a unified symbolic mode. At the same time, the present invention does not need to construct different data processing channels for different modal data, can realize the input of mixed data, reduce data processing costs, and adopts the perceptual hash algorithm to directly extract the fingerprint data of the image data, without the need for additional convolution training, to achieve plug-and-play and improve pre-training efficiency.

[0030] Further, the symbol table includes a text symbol name and a first code;

[0031] The step of obtaining the text encoding corresponding to the text data according to a preset symbol table includes:

[0032] The character symbol name is retrieved according to each character in the text data, the first code corresponding to each character is determined, and all the first codes corresponding to the text data are stored in an array format to obtain the text code.

[0033] From the above description, it can be seen that each character in the text data is encoded accordingly and converted into a text encoding, so that the original natural language is converted into a digital language that can be recognized by the computer, which facilitates the subsequent multimodal pre-training processing and enables the embedding form of the text data to be unified with the embedding form of the image data.

[0034] Further, the symbol table includes an image hash symbol name and a second encoding;

[0035] The step of inputting the image data into a perceptual hash algorithm to obtain fingerprint data, and obtaining an image code corresponding to the fingerprint data according to the symbol table includes:

[0036] Preprocessing the image data to obtain a plurality of images to be processed;

[0037] Inputting the plurality of images to be processed into a perceptual hash algorithm respectively to obtain a plurality of fingerprint data;

[0038] The multiple fingerprint data are respectively converted into character data, the image hash symbol name is retrieved according to each character in the character data, the second code corresponding to each character data is determined, and the second code is stored in an array format to obtain the image code.

[0039] From the above description, it can be seen that by inputting the image data into the perceptual hash algorithm to obtain fingerprint data, and converting the fingerprint data into symbolic data through a preset symbol table, the modalities of the image data and the text data are consistent, thereby realizing the unification of the image modality and text modality training models, providing technical support for subsequent large-scale pre-training, and also providing more complex model context capabilities for downstream tasks of the model.

[0040] Furthermore, the image data is preprocessed to obtain a plurality of images to be processed, specifically:

[0041] Scaling the image data according to a preset ratio to obtain a preprocessed image;

[0042] Segmenting the pre-processed image according to a preset size to obtain a plurality of images to be processed;

[0043] The step of inputting the plurality of images to be processed into the perceptual hash algorithm to obtain the plurality of fingerprint data is specifically as follows:

[0044] Reducing the image to be processed to a preset size and converting it into a grayscale image, and calculating a grayscale threshold and a grayscale value corresponding to each pixel in the grayscale image;

[0045] The grayscale value is compared with the grayscale threshold. If the grayscale value is greater than or equal to the grayscale threshold, the label number of the pixel corresponding to the grayscale value is 1; if the grayscale value is less than the grayscale threshold, the label number of the pixel corresponding to the grayscale value is 0, and the fingerprint data of the image to be processed is obtained.

[0046] From the above description, it can be seen that the fingerprint data of small areas in the image data, that is, the fingerprint data of multiple images to be processed, can be directly extracted through the perceptual hash algorithm. The smaller the scale of the image to be processed, the more accurate the fingerprint data obtained by the image data. The image data is scaled according to a preset ratio so that the image data can be segmented according to a preset size to avoid omissions caused by size mismatch. At the same time, the image to be processed is reduced to a preset size, which can remove the details of the image and retain only the structure and light and dark information, so as to generate a grayscale image later and avoid interference factors. The method of data embedding processing using the perceptual hash algorithm does not require additional convolution training.

[0047] Furthermore, the grayscale threshold is calculated as follows:

[0048] A=0.9A'×A'+0.99A';

[0049] Wherein, A is the grayscale threshold, and A' is the grayscale average value of the grayscale image.

[0050] From the above description, it can be seen that the perceptual hash algorithm adopted in the present invention takes into account the factor of processing speed and adopts the grayscale average value comparison method to improve the data processing speed and ensure the work efficiency of pre-training.

[0051] Please refer to Figure 4 Another embodiment of the present invention provides a multimodal data processing terminal, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the following steps when executing the computer program:

[0052] Acquire mixed data, where the mixed data includes image data and text data;

[0053] Obtaining a text code corresponding to the text data according to a preset symbol table;

[0054] Inputting the image data into a perceptual hash algorithm to obtain fingerprint data, and obtaining an image code corresponding to the fingerprint data according to the symbol table;

[0055] The text code and the image code are combined and processed and then input into the embedding layer network for dimension transformation to obtain embedding features.

[0056] From the above description, it can be seen that the beneficial effect of the present invention is that: the text data is converted into text code through the symbol table, and after the image data is converted into fingerprint data, the fingerprint data is converted into image code through the symbol table, so that the image data is converted into symbolic form for embedding, thereby realizing that the image data and the text data form a unified symbolic mode. At the same time, the present invention does not need to construct different data processing channels for different modal data, can realize the input of mixed data, reduce data processing costs, and adopts the perceptual hash algorithm to directly extract the fingerprint data of the image data, without the need for additional convolution training, to achieve plug-and-play and improve pre-training efficiency.

[0057] Further, the symbol table includes a text symbol name and a first code;

[0058] The step of obtaining the text encoding corresponding to the text data according to a preset symbol table includes:

[0059] The character symbol name is retrieved according to each character in the text data, the first code corresponding to each character is determined, and all the first codes corresponding to the text data are stored in an array format to obtain the text code.

[0060] From the above description, it can be seen that each character in the text data is encoded accordingly and converted into a text encoding, so that the original natural language is converted into a digital language that can be recognized by the computer, which facilitates the subsequent multimodal pre-training processing and enables the embedding form of the text data to be unified with the embedding form of the image data.

[0061] Further, the symbol table includes an image hash symbol name and a second encoding;

[0062] The step of inputting the image data into a perceptual hash algorithm to obtain fingerprint data, and obtaining an image code corresponding to the fingerprint data according to the symbol table includes:

[0063] Preprocessing the image data to obtain a plurality of images to be processed;

[0064] Inputting the plurality of images to be processed into a perceptual hash algorithm respectively to obtain a plurality of fingerprint data;

[0065] The multiple fingerprint data are respectively converted into character data, the image hash symbol name is retrieved according to each character in the character data, the second code corresponding to each character data is determined, and the second code is stored in an array format to obtain the image code.

[0066] From the above description, it can be seen that by inputting the image data into the perceptual hash algorithm to obtain fingerprint data, and converting the fingerprint data into symbolic data through a preset symbol table, the modalities of the image data and the text data are consistent, thereby realizing the unification of the image modality and text modality training models, providing technical support for subsequent large-scale pre-training, and also providing more complex model context capabilities for downstream tasks of the model.

[0067] Furthermore, the image data is preprocessed to obtain a plurality of images to be processed, specifically:

[0068] Scaling the image data according to a preset ratio to obtain a preprocessed image;

[0069] Segmenting the pre-processed image according to a preset size to obtain a plurality of images to be processed;

[0070] The step of inputting the plurality of images to be processed into the perceptual hash algorithm to obtain the plurality of fingerprint data is specifically as follows:

[0071] Reducing the image to be processed to a preset size and converting it into a grayscale image, and calculating a grayscale threshold and a grayscale value corresponding to each pixel in the grayscale image;

[0072] The grayscale value is compared with the grayscale threshold. If the grayscale value is greater than or equal to the grayscale threshold, the label number of the pixel corresponding to the grayscale value is 1; if the grayscale value is less than the grayscale threshold, the label number of the pixel corresponding to the grayscale value is 0, and the fingerprint data of the image to be processed is obtained.

[0073] From the above description, it can be seen that the fingerprint data of small areas in the image data, that is, the fingerprint data of multiple images to be processed, can be directly extracted through the perceptual hash algorithm. The smaller the scale of the image to be processed, the more accurate the fingerprint data obtained by the image data. The image data is scaled according to a preset ratio so that the image data can be segmented according to a preset size to avoid omissions caused by size mismatch. At the same time, the image to be processed is reduced to a preset size, which can remove the details of the image and retain only the structure and light and dark information, so as to generate a grayscale image later and avoid interference factors. The method of data embedding processing using the perceptual hash algorithm does not require additional convolution training.

[0074] Furthermore, the grayscale threshold is calculated as follows:

[0075] A=0.9A'×A'+0.99A';

[0076] Wherein, A is the grayscale threshold, and A' is the grayscale average value of the grayscale image.

[0077] From the above description, it can be seen that the perceptual hash algorithm adopted in the present invention takes into account the factor of processing speed and adopts the grayscale average value comparison method to improve the data processing speed and ensure the work efficiency of pre-training.

[0078] Please refer to Figures 1 to 3 , Embodiment 1 of the present invention is:

[0079] A multimodal data processing method comprises the steps of:

[0080] S1. Acquire mixed data, where the mixed data includes image data and text data.

[0081] In some embodiments, the image data and the text data in the mixed data correspond to each other, that is, the text data is used to describe the image data.

[0082] S2. Obtaining a text code corresponding to the text data according to a preset symbol table.

[0083] In some embodiments, the symbol table includes a symbol name key and a code corresponding to the symbol name key, and the code value corresponding to the symbol name key can be searched in the symbol table by a key-value search method.

[0084] Specifically, the symbol table includes text symbol names and first codes.

[0085] The S2 includes:

[0086] S21. Retrieve the text symbol name according to each character in the text data, determine the first code corresponding to each character, and store all the first codes corresponding to the text data in an array format to obtain the text code.

[0087] In some embodiments, all first codes corresponding to the text data are stored in an array format, and the obtained text code is indexID1=[1, 2, 3, 55...].

[0088] S3. Input the image data into the perceptual hash algorithm to obtain fingerprint data, and obtain the image code corresponding to the fingerprint data according to the symbol table.

[0089] The S3 includes:

[0090] S31, pre-processing the image data to obtain a plurality of images to be processed.

[0091] Wherein, the S31 is specifically:

[0092] S311, scaling the image data according to a preset ratio to obtain a preprocessed image.

[0093] In some embodiments, the image data is scaled in length and width in equal proportion, and the scaled image is embedded in a canvas of a preset size (i.e., the image data is scaled in proportion to the size of the canvas so that the image can be exactly embedded in the canvas) to obtain a pre-processed image. If the image size cannot completely match the size of the canvas during the embedding process, the insufficient length / width is filled in by padding. In an optional embodiment, the size of the canvas is 224*224 resolution.

[0094] S312: Segment the pre-processed image according to a preset size to obtain a plurality of images to be processed.

[0095] In some embodiments, the pre-processed image is segmented at a resolution of a patch (a small area) of a preset size to obtain a plurality of images to be processed. In an optional implementation, the size of the patch is a resolution of 16*16.

[0096] In some embodiments, the size of the canvas and the size of the patch affect the speed of multimodal data processing. If the canvas size is larger and the patch size is smaller, the multimodal data processing speed is slower. If the canvas size is too small, there is a problem of insufficient image fusion features.

[0097] S32: Input the multiple images to be processed into the perceptual hash algorithm respectively to obtain multiple fingerprint data.

[0098] The S32 is specifically:

[0099] S321, reducing the image to be processed to a preset size, converting it into a grayscale image, and calculating a grayscale threshold and a grayscale value corresponding to each pixel in the grayscale image;

[0100] The grayscale threshold calculation is specifically as follows:

[0101] A=0.9A'×A'+0.99A';

[0102] Wherein, A is the grayscale threshold, and A' is the grayscale average value of the grayscale image.

[0103] In some embodiments, reference Figure 3First, reduce the size of the image to be processed, that is, reduce the size of the image to be processed from 16*16 resolution to 8*8 resolution, and the image now includes 64 pixels. In this way, the details of the image are removed, only the basic information such as structure or brightness is retained, and the image differences caused by different sizes or proportions are discarded. Secondly, simplify the color of the image to be processed, that is, convert the reduced image to be processed into a 64-level grayscale image, that is, the total number of colors of all pixels in the image is 64. Finally, calculate the grayscale average value of these 64 pixels and the corresponding grayscale threshold.

[0104] S322. Compare the grayscale value with the grayscale threshold. If the grayscale value is greater than or equal to the grayscale threshold, the label number of the pixel corresponding to the grayscale value is 1; if the grayscale value is less than the grayscale threshold, the label number of the pixel corresponding to the grayscale value is 0, and the fingerprint data of the image to be processed is obtained.

[0105] Return to execute steps S321-S322 until the fingerprint data of all the images to be processed are obtained.

[0106] In some embodiments, the size of the canvas is 224*224 resolution, the size of the patch is 16*16 resolution, and the total number of fingerprint data of the image data is 14*14=196.

[0107] S33, converting the multiple fingerprint data into character data respectively, searching the image hash symbol name according to each character in the character data, determining the second code corresponding to each character data, and storing the second code in an array format to obtain the image code.

[0108] In an optional implementation manner, multiple fingerprint data are respectively converted into character data and represented in hex format.

[0109] In some embodiments, if the size of the patch is 16*16 resolution, the number of character bits contained in the character data of each image to be processed is 4 bits.

[0110] In some embodiments, all second codes corresponding to the character data are stored in an array format, and the obtained image code is indexID2=[3f3f, 3v3r, bed3, fg93...], which includes 196 image codes.

[0111] S4. After the text code and the image code are combined and processed, the combined code is input into an embedding layer network for dimensionality transformation to obtain an embedding feature.

[0112] In some embodiments, the text encoding and the image encoding are merged and processed by directly merging the extended array.

[0113] In some embodiments, if the image code indexID2==[f3fe, 34fe, 53fe] is input into the embedding layer network, the obtained embedding feature is in the array form [344, 555, 563].

[0114] It should be noted that Figure 2 The “+” shown in the figure indicates that the two tensor matrices are directly added to increase the bias term of the embedding layer network.

[0115] Please refer to Figure 4 , Embodiment 2 of the present invention is:

[0116] A multimodal data processing terminal includes a memory 201, a processor 202, and a computer program stored in the memory 201 and running on the processor 202. When the processor 202 executes the computer program, each step in a multimodal data processing method described in Example 1 is implemented.

[0117] In summary, the present invention provides a multimodal data processing method and terminal, in which text data is converted into text code through a symbol table, and after the image data is converted into fingerprint data, the fingerprint data is converted into image code through a symbol table, so that the image data is converted into a symbolic form for embedding, thereby realizing that the image data and the text data form a unified symbolic mode; through the multimodal data processing method of the present invention, the image modality symbol can be embedded in two-dimensional spatial position information, the text modality symbol can be embedded in one-dimensional spatial position information, and the two-dimensional spatial position information can be used to represent the one-dimensional spatial position information. In addition, the present invention does not need to construct different data processing channels for different modal data, does not need to train the image embedding layer, and can directly input the image modality and the text modality into the pre-training model together, realizing the input of mixed data, reducing the data processing cost, and using the perceptual hashing algorithm to directly extract the fingerprint data of the image data, without the need for additional convolution training, realizing plug-and-play, and improving the pre-training efficiency.

[0118] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent transformations made using the contents of the present invention's specification and drawings, or directly or indirectly applied in related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A multimodal data processing method, characterized in that: Includes steps: Acquire mixed data, where the mixed data includes image data and text data; Obtaining a text code corresponding to the text data according to a preset symbol table; Inputting the image data into a perceptual hash algorithm to obtain fingerprint data, and obtaining an image code corresponding to the fingerprint data according to the symbol table; The text code and the image code are combined and processed, and then input into the embedding layer network for dimension transformation to obtain embedding features; The symbol table includes a text symbol name and a first code; The step of obtaining the text encoding corresponding to the text data according to a preset symbol table includes: Retrieving the name of the character symbol according to each character in the text data, determining the first code corresponding to each character, and storing all the first codes corresponding to the text data in an array format to obtain the text code; The symbol table includes an image hash symbol name and a second code; The step of inputting the image data into a perceptual hash algorithm to obtain fingerprint data, and obtaining an image code corresponding to the fingerprint data according to the symbol table includes: Preprocessing the image data to obtain a plurality of images to be processed; Inputting the plurality of images to be processed into a perceptual hash algorithm respectively to obtain a plurality of fingerprint data; The multiple fingerprint data are respectively converted into character data, the image hash symbol name is retrieved according to each character in the character data, the second code corresponding to each character data is determined, and the second code is stored in an array format to obtain the image code.

2. A multimodal data processing method according to claim 1, characterized in that: The preprocessing of the image data to obtain a plurality of images to be processed is specifically as follows: Scaling the image data according to a preset ratio to obtain a preprocessed image; Segmenting the pre-processed image according to a preset size to obtain a plurality of images to be processed; The step of inputting the plurality of images to be processed into the perceptual hash algorithm to obtain the plurality of fingerprint data is specifically as follows: Reducing the image to be processed to a preset size and converting it into a grayscale image, and calculating a grayscale threshold and a grayscale value corresponding to each pixel in the grayscale image; Compare the grayscale value with the grayscale threshold, if the grayscale value is greater than or equal to the grayscale threshold, the label number of the pixel corresponding to the grayscale value is 1; If the grayscale value is less than the grayscale threshold, the label number of the pixel corresponding to the grayscale value is 0, and the fingerprint data of the image to be processed is obtained.

3. A multimodal data processing method according to claim 2, characterized in that: The grayscale threshold calculation is specifically as follows: A=0.9A'×A'+0.99A'; Wherein, A is the grayscale threshold, and A' is the grayscale average value of the grayscale image.

4. A multimodal data processing terminal, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the following steps are implemented: Acquire mixed data, where the mixed data includes image data and text data; Obtaining a text code corresponding to the text data according to a preset symbol table; Inputting the image data into a perceptual hash algorithm to obtain fingerprint data, and obtaining an image code corresponding to the fingerprint data according to the symbol table; The text code and the image code are combined and processed, and then input into the embedding layer network for dimension transformation to obtain embedding features; The symbol table includes a text symbol name and a first code; The step of obtaining the text encoding corresponding to the text data according to a preset symbol table includes: Retrieving the name of the character symbol according to each character in the text data, determining the first code corresponding to each character, and storing all the first codes corresponding to the text data in an array format to obtain the text code; The symbol table includes an image hash symbol name and a second code; The step of inputting the image data into a perceptual hash algorithm to obtain fingerprint data, and obtaining an image code corresponding to the fingerprint data according to the symbol table includes: Preprocessing the image data to obtain a plurality of images to be processed; Inputting the plurality of images to be processed into a perceptual hash algorithm respectively to obtain a plurality of fingerprint data; The multiple fingerprint data are respectively converted into character data, the image hash symbol name is retrieved according to each character in the character data, the second code corresponding to each character data is determined, and the second code is stored in an array format to obtain the image code.

5. A multimodal data processing terminal according to claim 4, characterized in that: The preprocessing of the image data to obtain a plurality of images to be processed is specifically as follows: Scaling the image data according to a preset ratio to obtain a preprocessed image; Segmenting the pre-processed image according to a preset size to obtain a plurality of images to be processed; The step of inputting the plurality of images to be processed into the perceptual hash algorithm to obtain the plurality of fingerprint data is specifically as follows: Reducing the image to be processed to a preset size and converting it into a grayscale image, and calculating a grayscale threshold and a grayscale value corresponding to each pixel in the grayscale image; Compare the grayscale value with the grayscale threshold, if the grayscale value is greater than or equal to the grayscale threshold, the label number of the pixel corresponding to the grayscale value is 1; If the grayscale value is less than the grayscale threshold, the label number of the pixel corresponding to the grayscale value is 0, and the fingerprint data of the image to be processed is obtained.

6. A multimodal data processing terminal according to claim 5, characterized in that: The grayscale threshold calculation is specifically as follows: A=0.9A'×A'+0.99A'; Wherein, A is the grayscale threshold, and A' is the grayscale average value of the grayscale image.

Images