Unstructured data classification method and device based on deep learning
Through deep learning-based methods, unstructured data are processed and classified, and problems of increasing data volume, irregular structure and difficult to guarantee security compliance in image data processing are solved, and efficient and accurate image data classification and security management are achieved.
Patent Information
- Application Number
- CN202411964289.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-30
- Publication Date
- 2025-04-08
AI Technical Summary
When processing unstructured data, especially image data, the prior art faces problems such as rapid growth in data volume, irregular data structure, and difficult to guarantee data security and compliance.
Using a deep learning-based method, the original data is screened, normalized, manually annotated and data augmented, and the learning model is constructed and efficient classification of image data is achieved through pre-training and fine-tuning.
It improves the classification efficiency and accuracy of image data, reduces manual intervention, ensures data security and compliance, and has important application value in the field of network security.
Smart Images
Figure CN119785109A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of nuclear power technology, and specifically relates to a method and device for classifying unstructured data based on deep learning. Background Art
[0002] In the wave of digital transformation, the popularization of various information system equipment has generated a large amount of unstructured data, and the surge in these unstructured data has also brought unprecedented opportunities and challenges to enterprises. Although unstructured data is diverse in form and structure, it does contain rich information. Through effective classification and management, the potential value of these data can be better explored, thereby promoting business growth and innovation-driven. Unstructured data classification not only helps to improve the retrievability and availability of data, but also strengthens data security and compliance. Through intelligent classification, key information can be quickly identified and accessed, thereby improving the speed and quality of decision-making. At the same time, this also provides a basis for implementing data governance, protecting sensitive information, and complying with laws and regulations.
[0003] Deep learning technology plays an important role in the field of image processing. Early image processing could only perform tasks such as image enhancement or compression. The emergence of deep learning has brought revolutionary changes to the field of image processing. The advancement of deep learning algorithms and computing power has enhanced people's understanding of image information. Deep learning models can automatically learn and extract features from raw pixels, and then apply the features to more complex tasks. Summary of the invention
[0004] In order to overcome the problems existing in the related art, a method and device for unstructured data classification based on deep learning are provided.
[0005] According to one aspect of an embodiment of the present disclosure, a method for classifying unstructured data based on deep learning is provided, the method comprising:
[0006] Step 1: Process the original data. First, the data will be screened and normalized to obtain a data set.
[0007] Step 2: Process part of the data by manual classification, classify and label the data by manual recognition, and thus obtain a data set containing labeling information;
[0008] Step 3: The remaining data that is not manually processed will be expanded through data augmentation, thereby obtaining a dataset based on image masking and a dataset based on image rotation as the dataset for model pre-training;
[0009] Step 4: Based on the model pre-training data set, the learning model will learn feature information from the image by completing two image discrimination tasks until the pre-training of the learning model is completed, and the current model and related parameters are temporarily stored;
[0010] Step 5: After the pre-training of the learning model is completed, the pre-training model and its weights are loaded, and the learning effect of the learning model on the image classification task is further adjusted based on the manually annotated data set until the learning model is fine-tuned to obtain the final learning model;
[0011] Step 6: For the image sample to be classified input into the learning model, the learning model will output its corresponding learning features and compare them with the known image samples to calculate the similarity between the image to be detected and the target image. The similarity result will be compared with the preset threshold to obtain the classification result.
[0012] Step 7: Display the image recognition results.
[0013] In a possible implementation, in step 3, for the remaining data set that has not been manually annotated, data augmentation technology is used to increase the scale and diversity of the training data set, and the data augmentation technology includes image masking and image rotation.
[0014] The data augmentation method based on image masking randomly masks image pixels in proportion according to preset parameters and uses them as new samples. At the same time, the data after image masking are annotated. The two samples obtained from the same image with and without image masking are used as positive sample pairs, while the two samples from different images are used as negative sample pairs.
[0015] The data augmentation method based on image rotation rotates the image at random angles and generates new samples while maintaining the normalized scale. The rotated samples and the original samples from the same sample are used as positive sample pairs, and the two samples from different images are used as negative sample pairs.
[0016] In a possible implementation, in step 4, task one and task two are set for two data enhancement methods;
[0017] Task 1: Based on the image masking data set, determine whether a pair of samples in the input model are positive sample pairs from the same image; or determine whether a pair of samples in the input model are negative sample pairs from different images;
[0018] Task 2 is: based on the image rotation processing dataset, a pair of samples input to the model is a positive sample pair from the same image; or a pair of samples input to the model is judged to be a negative sample pair from different images.
[0019] In a possible implementation, in step 4, the learning model is composed of multiple Transformer models and CNN models.
[0020] In a possible implementation, step 5 includes the following steps:
[0021] Step 51, feature extraction, when the training sample G = (i1, i2, ..., i n ), and then transform it into a vector e(G) through embedding operation:
[0022] e(G)=E[i1,i2,…,i n ]
[0023] Use the Transformer Encoder model to capture the global feature te(G):
[0024] te(G)=[h1,…,h n ]=TE(e(G))
[0025] where h i is the hidden state of the Transformer Encoder output; CNN is then used to capture the local feature information c in the image k :
[0026] c k =ReLU(W·(h i , ..., h i+k-1 ))
[0027] Where Relu is the activation function, W is the parameter matrix, c k It represents the features obtained when the CNN convolution kernel is set to k; finally, c k Max pooling is performed to obtain a fixed-length vector as the feature vector f(G) of the image:
[0028] f(G)=maxpooling([c1,c2,…,c n ])
[0029] Step 52, result discrimination, the discrimination part is implemented using a feedforward neural network FNN, which discriminates the sample pair according to the feature vector f(G) to obtain a prediction result, which is 0 or 1, 0 represents a negative sample pair, and 1 represents a positive sample pair:
[0030] y predict =FNN(f1(G), f2(G),…,f n (G)
[0031] The binary cross entropy loss is used as the loss for each training task, as shown in the following formula:
[0032] Loss=-(y label logy predict )+(1-y label )log(1-y predict )
[0033] And use the weighted loss function as the common loss when completing the two tasks, as shown in the following formula:
[0034]
[0035] Among them, σ is the learning parameter, which is initialized to 1 here. After completing the pre-training, the current optimal parameter is obtained and saved;
[0036] Step 53, model fine-tuning, load the pre-trained training model and its weights, freeze some Transformer and CNN parts, and only train the last fully connected layer; the training data set used is the manually labeled data in step 2, and the loss function adopts the cross entropy loss function.
[0037] In a possible implementation, after the image data to be classified is input into the model, a feature vector containing the feature information of the image will be output, and the feature vector will be matched with the feature vector of the classified image information in the existing feature library to complete the classification; the cosine similarity calculation method is used in the matching process, as shown in the following formula:
[0038] sim(G A , G B )=Cosine_Similary(f(G A ), f(G B )
[0039] Among them, G A and G B The image feature vectors are f(G A ) and f(G B ), according to the preset threshold, the closest image type will be matched based on similarity calculation and the classification result of the image will be output.
[0040] According to another aspect of an embodiment of the present disclosure, a deep learning-based unstructured data classification device is provided, the device comprising:
[0041] The preprocessing module is used to process the raw data. First, the data will be filtered and normalized to obtain a data set;
[0042] The manual labeling module is used to process part of the data through manual classification, classify and label the data through manual recognition, and thus obtain a data set containing labeling information;
[0043] The remaining data for non-manual labeling module will be expanded through data enhancement, thereby obtaining a dataset based on image masking and a dataset based on image rotation as the dataset for model pre-training;
[0044] Pre-training module, used for data sets based on model pre-training. The learning model will learn feature information from the image by completing two image discrimination tasks until the pre-training of the learning model is completed, and the current model and related parameters are temporarily stored;
[0045] The fine-tuning module is used to load the pre-trained model and its weights after completing the pre-training of the learning model, and further adjust the learning effect of the learning model on the image classification task based on the manually annotated data set until the learning model is fine-tuned to obtain the final learning model;
[0046] The classification module is used for inputting the image samples to be classified into the learning model. The learning model will output the corresponding learning features and compare them with the known image samples to calculate the similarity between the image to be detected and the target image. The similarity result will be compared with the preset threshold to obtain the classification result.
[0047] The display module is used to display the image recognition results.
[0048] According to another aspect of an embodiment of the present disclosure, a deep learning-based unstructured data classification device is provided, the device comprising:
[0049] processor;
[0050] a memory for storing processor-executable instructions;
[0051] Wherein, the processor is configured to execute the above method.
[0052] According to another aspect of an embodiment of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor.
[0053] The beneficial effect of the present disclosure is that image data is an important part of unstructured data, and image recognition and classification have a very far-reaching significance in the field of network security. Image data contains rich information, and the information extracted by image recognition and classification technology can be effectively used to monitor and filter out images containing malicious links, phishing information or illegal content, thereby protecting the target system from the threat of network attacks.
[0054] The present disclosure provides a classification system for image-type unstructured data, which aims to solve the challenges faced by existing technologies in processing unstructured data, especially image data. These challenges include the explosive growth of data volume, the irregularity of data structure, and data security and compliance issues. The system aims to improve the classification efficiency and accuracy of image data through automated technical means, while reducing manual factors in the work and ensuring data security and compliance. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] Figure 1 It is a flowchart of an unstructured data classification method based on deep learning shown in an embodiment of the present disclosure.
[0056] Figure 2 It is a block diagram of an unstructured data classification device based on deep learning shown in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0057] The present disclosure is further described in detail below with reference to the accompanying drawings and specific embodiments.
[0058] Unless otherwise defined, the technical and scientific terms used in the present disclosure have the same meanings as those generally understood by those skilled in the art to which the present disclosure belongs; the terms used in the present disclosure are only for the purpose of describing specific embodiments and are not intended to limit the present disclosure; the term "including" and any variations thereof in the present disclosure are intended to cover non-exclusive inclusions. Obviously, the embodiments described in the present disclosure are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative efforts are within the scope of protection of the present disclosure.
[0059] Reference to "embodiments" in this disclosure means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the disclosure. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.
[0060] Figure 1 is a flowchart of a method for unstructured data classification based on deep learning shown in an embodiment of the present disclosure. The method can be executed by a terminal device, wherein the terminal device can be a server, a desktop computer, a laptop computer, a tablet computer, etc. The terminal device can also be, for example, a user device, a vehicle-mounted device or a wearable device, etc. The embodiment of the present disclosure does not limit the type of the terminal device. Figure 1 As shown, the method includes:
[0061] Step 1: Process the original data. First, the data will be screened and normalized to obtain a data set. In step 1, for the original data to be used for training, the data with poor display effect and severe noise that cannot be manually identified are manually eliminated, and then the normalization scale is determined according to the basic information of the image. The image is normalized based on the normalization scale to obtain a data set that can be used for input into the model.
[0062] Step 2: Process part of the data through manual classification. The data will be classified and labeled through manual recognition, thereby obtaining a data set containing labeled information, which will be used as the data set for model fine-tuning. In order to minimize the reliance on manual operations, only part of the data set will be manually labeled. The manually labeled part of the data set will be used for the final model tuning work.
[0063] In step 3, the remaining data that is not manually processed will be expanded in size and diversity through data augmentation, thereby obtaining a dataset based on image masking and a dataset based on image rotation as datasets for model pre-training.
[0064] In step 3, for the remaining data sets that have not been manually annotated, data enhancement technology is used to expand the scale and diversity of the training data set. Data enhancement technology includes two methods: image masking and image rotation. The data enhancement method based on image masking randomly masks the image pixels in proportion according to preset parameters (such as mask rate = 15%) and uses them as new samples, and annotates the data after image masking. The two samples obtained from the same image with and without image masking are used as positive sample pairs, and the two samples from different images are used as negative sample pairs. The data enhancement method based on image rotation rotates the image at random angles, and generates new samples while maintaining the normalized scale. The rotated samples and original samples from the same sample are used as positive sample pairs, and the two samples from different images are used as negative sample pairs.
[0065] Step 4: Based on the dataset for model pre-training, the learning model will learn feature information from the image by completing two image discrimination tasks until the pre-training of the learning model is completed, and the current model and related parameters are temporarily stored. In step 4, two tasks for pre-training are set for the two data enhancement methods. Task 1 is: based on the dataset processed by image masking, a pair of samples of the input model is judged to be a positive sample pair from the same image; or a pair of samples of the input model is judged to be a negative sample pair from different images. Task 2 is: based on the dataset processed by image rotation, a pair of samples of the input model is a positive sample pair from the same image; or a pair of samples of the input model is judged to be a negative sample pair from different images.
[0066] Step 5: After completing the pre-training of the learning model, load the pre-trained model and its weights, and further adjust the learning effect of the learning model on the image classification task based on the manually annotated data set until the learning model is fine-tuned to obtain the final learning model.
[0067] Among them, the learning model can be, for example, a Transformer model, which is good at capturing global features with long-distance dependency information in samples, or it can be, for example, a CNN model, which is good at obtaining local features. In order to enable the learning model to obtain as rich feature information as possible from the input samples, the learning model used in the present invention is composed of multiple Transformer models and CNN models.
[0068] In a possible implementation, step 5 includes the following steps.
[0069] Step 51, feature extraction, when the training sample G = (i1, i2, ..., i n ), and then transform it into a vector e(G) through embedding operation:
[0070] e(G)=E[i1,i2,…,i n ]
[0071] Use the Transformer Encoder model to capture the global feature te(G):
[0072] te(G)=[h1,…,h n ]=TE(e(G))
[0073] where h i is the hidden state of the Transformer Encoder output. Then CNN is used to capture the local feature information c in the image. k :
[0074] c k =ReLU(W·(h i , ..., h i+k-1 ))
[0075] Where Relu is the activation function, W is the parameter matrix, c k It represents the features obtained when the CNN convolution kernel is set to k. k Max pooling is performed to obtain a fixed-length vector as the feature vector f(G) of the image:
[0076] f(G)=maxpooling([c1,c2,…,c n ]).
[0077] Step 52, result discrimination, the discrimination part is implemented using a feedforward neural network FNN, which discriminates the sample pair according to the feature vector f(G) to obtain a prediction result, which is 0 or 1, 0 represents a negative sample pair, and 1 represents a positive sample pair:
[0078] y predict =FNN(f1(G), f2(G),…,f n (G)
[0079] The binary cross entropy loss is used as the loss for each training task, as shown in the following formula:
[0080] Loss=-(y label logy predict )+(1-y label )log(1-y predict )
[0081] And use the weighted loss function as the common loss when completing the two tasks, as shown in the following formula:
[0082]
[0083] Among them, σ is a learnable parameter, which is initialized to 1 here. After completing pre-training, the current optimal parameter is obtained and saved.
[0084] Step 53, model fine-tuning, load the pre-trained training model and its weights, freeze some Transformer and CNN parts, and only train the last fully connected layer. The training dataset used is the manually annotated data in step 2, and the loss function adopts the cross entropy loss function.
[0085] Step 6: For the image samples to be classified that are input into the learning model, the learning model will output their corresponding learning features and compare them with the known image samples to calculate the similarity between the image to be detected and the target image. The similarity result will be compared with the preset threshold to obtain the classification result.
[0086] After the image data to be classified is input into the model, a feature vector containing the feature information of the image will be output. This feature vector will be matched with the feature vector of the classified image information in the existing feature library to complete the classification. The cosine similarity calculation method is used in the matching process, as shown in the following formula:
[0087] sim(G A , G B )=Cosine_Similary(f(G A ), f(G B )
[0088] Among them, G A and GB The image feature vectors are f(G A ) and f(G B ).
[0089] According to the preset threshold, the closest image category will be matched based on similarity calculation and the classification result of the image will be output.
[0090] Step 7: Display the image recognition results.
[0091] The present disclosure provides a classification system for image-type unstructured data, which aims to solve the challenges faced by existing technologies in processing unstructured data, especially image data. These challenges include the explosive growth of data volume, the irregularity of data structure, and data security and compliance issues. The system aims to improve the classification efficiency and accuracy of image data through automated technical means, while reducing manual factors in the work and ensuring data security and compliance.
[0092] According to another aspect of an embodiment of the present disclosure, a deep learning-based unstructured data classification device is provided, the device comprising:
[0093] The preprocessing module is used to process the raw data. First, the data will be filtered and normalized to obtain a data set;
[0094] The manual labeling module is used to process part of the data through manual classification, classify and label the data through manual recognition, and thus obtain a data set containing labeling information;
[0095] The remaining data for non-manual labeling module will be expanded through data enhancement, thereby obtaining a dataset based on image masking and a dataset based on image rotation as the dataset for model pre-training;
[0096] Pre-training module, used for data sets based on model pre-training. The learning model will learn feature information from the image by completing two image discrimination tasks until the pre-training of the learning model is completed, and the current model and related parameters are temporarily stored;
[0097] The fine-tuning module is used to load the pre-trained model and its weights after completing the pre-training of the learning model, and further adjust the learning effect of the learning model on the image classification task based on the manually annotated data set until the learning model is fine-tuned to obtain the final learning model;
[0098] The classification module is used for inputting the image samples to be classified into the learning model. The learning model will output the corresponding learning features and compare them with the known image samples to calculate the similarity between the image to be detected and the target image. The similarity result will be compared with the preset threshold to obtain the classification result.
[0099] The display module is used to display the image recognition results.
[0100] The description of the above-mentioned device has been explained in detail in the description of the above-mentioned method, and will not be repeated here.
[0101] Figure 2 1900 is a block diagram of an unstructured data classification device based on deep learning shown in an embodiment of the present disclosure. For example, the device 1900 can be provided as a server. Figure 2 , the apparatus 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions, such as an application, that can be executed by the processing component 1922. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.
[0102] The device 1900 may also include a power supply component 1926 configured to perform power management of the device 1900, a wired or wireless network interface 1950 configured to connect the device 1900 to a network, and an input / output (I / O) interface 1958. The device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server™, MacOS X™, Unix™, Linux™, FreeBSD™, or the like.
[0103] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the device 1900 to perform the above method.
[0104] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.
[0105] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.
[0106] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.
[0107] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.
[0108] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.
[0109] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0110] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0111] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.
[0112] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.
Claims
1. A method for classifying unstructured data based on deep learning, characterized in that: The method comprises: Step 1: Process the original data. First, the data will be screened and normalized to obtain a data set. Step 2: Process part of the data by manual classification, classify and label the data by manual recognition, and thus obtain a data set containing labeling information; Step 3: The remaining data that is not manually processed will be expanded through data augmentation, thereby obtaining a dataset based on image masking and a dataset based on image rotation as the dataset for model pre-training; Step 4: Based on the model pre-training data set, the learning model will learn feature information from the image by completing two image discrimination tasks until the pre-training of the learning model is completed, and the current model and related parameters are temporarily stored; Step 5: After the pre-training of the learning model is completed, the pre-training model and its weights are loaded, and the learning effect of the learning model on the image classification task is further adjusted based on the manually annotated data set until the learning model is fine-tuned to obtain the final learning model; Step 6: For the image sample to be classified input into the learning model, the learning model will output its corresponding learning features and compare them with the known image samples to calculate the similarity between the image to be detected and the target image. The similarity result will be compared with the preset threshold to obtain the classification result. Step 7: Display the image recognition results.
2. The method according to claim 1, characterized in that In step 3, for the remaining data sets that have not been manually annotated, data augmentation techniques are used to increase the size and diversity of the training data sets. The data augmentation techniques include image masking and image rotation. The data augmentation method based on image masking randomly masks image pixels in proportion according to preset parameters and uses them as new samples. At the same time, the data after image masking are annotated. The two samples obtained from the same image with and without image masking are used as positive sample pairs, while the two samples from different images are used as negative sample pairs. The data augmentation method based on image rotation rotates the image at random angles and generates new samples while maintaining the normalized scale. The rotated samples and the original samples from the same sample are used as positive sample pairs, and the two samples from different images are used as negative sample pairs.
3. The method according to claim 1, characterized in that In step 4, task 1 and task 2 are set for two data enhancement methods; Task 1: Based on the image masking data set, determine whether a pair of samples in the input model are positive sample pairs from the same image; or determine whether a pair of samples in the input model are negative sample pairs from different images; Task 2 is: based on the image rotation processing dataset, a pair of samples input to the model is a positive sample pair from the same image; or a pair of samples input to the model is judged to be a negative sample pair from different images.
4. The method according to claim 1, characterized in that: In step 4, the learning model is composed of multiple Transformer models and CNN models.
5. The method according to claim 1, characterized in that Step 5 includes the following steps: Step 51, feature extraction, when the training sample G=(i1,i2,…,i n ), and then transform it into a vector e(G) through embedding operation: e(G)=E[i1,i2,…,i n ] Use the Transformer Encoder model to capture the global feature te(G): te(G)=[h1,…,h n ]=TE(e(G)) where h i is the hidden state of the Transformer Encoder output; CNN is then used to capture the local feature information c in the image k : c k =ReLU(W·(h i ,…, h i+k-1 )) Where Relu is the activation function, W is the parameter matrix, c k It represents the features obtained when the CNN convolution kernel is set to k; finally, c k Max pooling is performed to obtain a fixed-length vector as the feature vector f(G) of the image: f(G)=maxpooling([c1,c2,…,c n ]) Step 52, result discrimination, the discrimination part is implemented using a feedforward neural network FNN, which discriminates the sample pair according to the feature vector f(G) to obtain a prediction result, which is 0 or 1, 0 represents a negative sample pair, and 1 represents a positive sample pair: y predict =FNN(f1(G),f2(G),…,f n (G)) The binary cross entropy loss is used as the loss for each training task, as shown in the following formula: Loss=-(y label logy predict )+(1-y label )log(1-y predict ) And use the weighted loss function as the common loss when completing the two tasks, as shown in the following formula: Among them, σ is the learning parameter, which is initialized to 1 here. After completing the pre-training, the current optimal parameter is obtained and saved; Step 53, model fine-tuning, load the pre-trained training model and its weights, freeze some Transformer and CNN parts, and only train the last fully connected layer; the training data set used is the manually labeled data in step 2, and the loss function adopts the cross entropy loss function.
6. The method according to claim 1, characterized in that After the image data to be classified is input into the model, a feature vector containing the feature information of the image will be output. This feature vector will be matched with the feature vector of the classified image information in the existing feature library to complete the classification. The cosine similarity calculation method is used in the matching process, as shown in the following formula: sim(G A ,G B )=Cosine_Similary(f(G A ),f(G B ) Among them, G A and G B The image feature vectors are f(G A ) and f(G B ), according to the preset threshold, the closest image type will be matched based on similarity calculation and the classification result of the image will be output.
7. A deep learning-based unstructured data classification device, characterized in that: The device comprises: The preprocessing module is used to process the raw data. First, the data will be filtered and normalized to obtain a data set; The manual labeling module is used to process part of the data through manual classification, classify and label the data through manual recognition, and thus obtain a data set containing labeling information; The remaining data for non-manual labeling module will be expanded through data enhancement, thereby obtaining a dataset based on image masking and a dataset based on image rotation as the dataset for model pre-training; Pre-training module, used for data sets based on model pre-training. The learning model will learn feature information from the image by completing two image discrimination tasks until the pre-training of the learning model is completed, and the current model and related parameters are temporarily stored; The fine-tuning module is used to load the pre-trained model and its weights after completing the pre-training of the learning model, and further adjust the learning effect of the learning model on the image classification task based on the manually annotated data set until the learning model is fine-tuned to obtain the final learning model; The classification module is used for inputting the image samples to be classified into the learning model. The learning model will output the corresponding learning features and compare them with the known image samples to calculate the similarity between the image to be detected and the target image. The similarity result will be compared with the preset threshold to obtain the classification result. The display module is used to display the image recognition results.
8. A deep learning-based unstructured data classification device, characterized in that: The device comprises: processor; a memory for storing processor-executable instructions; The processor is configured to execute the method according to any one of claims 1 to 6.
9. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.