Picture data enhancement method and device based on semantic conversion, equipment and medium

Abstract

The application relates to an artificial intelligence technology and discloses a picture data enhancement method based on semantic conversion, which comprises the following steps: text coding is performed on the picture categories in a financial training picture set to obtain a category semantic feature vector of each financial picture; picture features of each financial picture are extracted, and a predicted financial category semantic vector corresponding to the picture features is calculated; a semantic conversion module is parameter-adjusted according to the predicted semantic feature vector and the category semantic feature vector to obtain an optimized semantic conversion module; target financial enhancement pictures are data-coded to obtain picture coding features; the picture coding features are subjected to semantic conversion by using the optimized semantic conversion module to obtain a semantic conversion vector, and the semantic conversion vector is decoded to obtain financial enhancement data. The application further discloses a picture data enhancement device based on semantic conversion, equipment and a medium. The application can improve the data enhancement effect of financial related pictures.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device, and medium for image data enhancement based on semantic transformation. Background Technology

[0002] Currently, with the development of science and technology, deep learning has been widely used in the financial field. The training of deep learning models (such as neural networks) requires a large amount of financial training data to learn in order to detect and recognize information in financial images. However, in actual business, the amount of financial image data required for training is insufficient and cannot be applied to all financial scenarios. Therefore, it is necessary to augment the financial image data. Image data augmentation is to generate more data value from the existing limited financial image data without substantially increasing the image data, thereby obtaining a more accurate deep learning model in the financial field.

[0003] Currently, the industry generally uses traditional image processing methods such as cropping, flipping, and adding noise to augment original images, and then uses the augmented images for training models with few samples. However, this traditional method has limited effectiveness in model training. For example, in financial image classification tasks, if the model can already distinguish between expense reports and invoices well, applying traditional image augmentation methods to expense report and invoice images will not significantly improve the model's performance. However, if the model's accuracy in classifying expense reports and invoices is low, data augmentation can improve the accuracy of some models, but it only works for images with high discriminative power; it still fails to classify expense report images that closely resemble invoices. This is because processing the original image only changes its shape features; the model can learn more shapes but lacks semantic features, resulting in poor image augmentation effects. Summary of the Invention

[0004] This invention provides a method, apparatus, device, and medium for image data enhancement based on semantic transformation, with the main purpose of solving the problem of poor results when performing data enhancement on financial images.

[0005] To achieve the above objectives, the present invention provides an image data enhancement method based on semantic transformation, comprising:

[0006] Obtain a set of financial training images, and perform text encoding on the categories of financial images in the set to obtain the category semantic feature vector of each financial image in the set.

[0007] Extract the image features of each financial image in the financial training image set, and use a preset semantic conversion module to calculate the predicted financial category semantic vector corresponding to the image features;

[0008] The parameter values ​​in the semantic conversion module are adjusted based on the predicted financial category semantic vector and the category semantic feature vector to obtain an optimized semantic conversion module;

[0009] Obtain the target financial enhancement image, encode the target financial enhancement image into data, and obtain the image encoding features of the target financial enhancement image;

[0010] The optimized semantic transformation module is used to perform semantic transformation on the image encoding features to obtain the semantic transformation vector of the target financial enhancement image. The semantic transformation vector is then decoded to obtain the financial enhancement data of the target financial enhancement image.

[0011] Optionally, the step of text encoding the financial image categories in the financial training image set to obtain the category semantic feature vector of each financial image in the financial training image set includes:

[0012] Extract the financial category label of each financial image in the financial training image set to obtain the financial image category in the financial training image set;

[0013] The financial image category is segmented into words to obtain financial category words. The financial category words are then vectorized to obtain financial category word vectors.

[0014] The financial category word segmentation vectors are encoded using a preset attention mechanism to obtain the category semantic feature vector of each financial image in the financial training image set.

[0015] Optionally, extracting image features from each financial image in the financial training image set includes:

[0016] The shallow convolutional layers in the preset residual network are used to perform convolution and pooling operations on each financial image in the financial training image set to obtain a shallow feature map of each financial image in the financial training image set.

[0017] The shallow feature map is convolutionally and pooled using the deep convolutional layers in the residual network to obtain the deep feature map of each financial image in the financial training image set.

[0018] The shallow feature map is downsampled to the size of the deep feature map to obtain a shallow sampled feature map. The shallow sampled feature map and the deep feature map are then residually connected to obtain a fused feature map.

[0019] Activation calculations are performed on the fused feature map to obtain the image features of each financial image in the financial training image set.

[0020] Optionally, the step of calculating the predicted financial category semantic vector corresponding to the image features using a preset semantic transformation module includes:

[0021] The first semantic transformation matrix in the semantic transformation module is used to perform vector mapping on the image features to obtain the first predicted financial category semantic vector.

[0022] The semantic transformation network in the semantic transformation module is used to calculate the image features to obtain a virtual category semantic feature vector;

[0023] The second semantic transformation matrix in the semantic transformation module is used to perform vector mapping on the virtual category semantic feature vector to obtain the second predicted financial category semantic vector;

[0024] By combining the first predicted financial category semantic vector, the virtual category semantic feature vector, and the second predicted financial category semantic vector, the predicted financial category semantic vector corresponding to the image feature is obtained.

[0025] Optionally, the step of using the first semantic transformation matrix in the semantic transformation module to perform vector mapping on the image features to obtain the first predicted financial category semantic vector includes:

[0026] The first semantic transformation matrix is ​​multiplied by the image features to obtain the dot product feature vector;

[0027] The dot product feature vector is multiplied by the transpose of the first semantic transformation matrix to obtain the transpose feature vector.

[0028] By combining the dot product feature vector and the transposed feature vector, a first predicted financial category semantic vector is obtained.

[0029] Optionally, adjusting the parameter values ​​in the semantic transformation module based on the predicted financial category semantic vector and the category semantic feature vector includes:

[0030] Calculate the first error value between the first predicted financial category semantic vector and the category semantic feature vector in the predicted financial category semantic vector, and adjust the parameters in the first semantic transformation matrix in the semantic transformation module according to the first error value until the first error value is less than a preset error value threshold to obtain the optimized first semantic transformation matrix;

[0031] Calculate the second error value between the virtual category semantic feature vector and the category semantic feature vector in the predicted financial category semantic vector, and adjust the parameters in the semantic conversion network in the semantic conversion module according to the second error value until the second error value is less than a preset error value threshold, thereby obtaining an optimized semantic conversion network;

[0032] Calculate the third error value between the second predicted financial category semantic vector and the virtual category semantic feature vector in the predicted financial category semantic vector. Adjust the parameters in the second semantic transformation matrix in the semantic transformation module according to the third error value until the third error value is less than a preset error value threshold to obtain the optimized second semantic transformation matrix.

[0033] Optionally, the step of data encoding the target financial enhancement image to obtain the image encoding features of the target financial enhancement image includes:

[0034] Obtain the pixel values ​​corresponding to the pixels in the target financial enhancement image, and encode the pixel values ​​into octets to obtain the binary dataset of the target financial enhancement image;

[0035] The binary dataset is grouped to obtain binary group sequences, and the corresponding encoding value of the binary group sequence is found using a preset encoding table;

[0036] The image coding features of the target financial enhancement image are obtained by concatenating the encoded values ​​corresponding to the binary group sequence.

[0037] To address the aforementioned problems, the present invention also provides an image data enhancement device based on semantic transformation, the device comprising:

[0038] The financial image category encoding module is used to acquire a set of financial training images, perform text encoding on the financial image categories in the set of financial training images, and obtain the category semantic feature vector of each financial image in the set of financial training images.

[0039] The semantic transformation module is used to extract the image features of each financial image in the financial training image set, and to calculate the predicted financial category semantic vector corresponding to the image features using the preset semantic transformation module.

[0040] The parameter optimization module is used to adjust the parameter values ​​in the semantic conversion module according to the predicted financial category semantic vector and the category semantic feature vector, so as to obtain the optimized semantic conversion module;

[0041] The image data encoding module is used to acquire the target financial enhancement image, encode the target financial enhancement image, and obtain the image encoding features of the target financial enhancement image.

[0042] The data augmentation module is used to perform semantic transformation on the image encoding features using the optimized semantic transformation module to obtain the semantic transformation vector of the target financial augmentation image, and to decode the semantic transformation vector to obtain the financial augmentation data of the target financial augmentation image.

[0043] To address the above problems, the present invention also provides an electronic device, the electronic device comprising:

[0044] At least one processor; and,

[0045] A memory communicatively connected to the at least one processor; wherein,

[0046] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the semantic transformation-based image data enhancement method described above.

[0047] To address the aforementioned problems, the present invention also provides a computer-readable storage medium storing at least one computer program, which is executed by a processor in an electronic device to implement the semantic transformation-based image data enhancement method described above.

[0048] This invention, through encoding the financial image categories in a financial training image set, generates deep bidirectional language representations, thereby obtaining a category semantic feature vector for each financial image. Then, it extracts the image features of each financial image and uses a pre-defined semantic conversion module to calculate the predicted financial category semantic vector corresponding to the image features. Based on the predicted financial category semantic vector and the category semantic feature vector, the parameter values ​​in the semantic conversion module are adjusted to obtain an optimized semantic conversion module. This allows image data enhancement based on semantic conversion to improve the effect of financial image data enhancement from a semantic feature perspective, resulting in more accurate semantic conversion vectors. Finally, the semantic conversion vectors are decoded, effectively improving the data enhancement effect of the target financial image. Therefore, the image data enhancement method, apparatus, device, and medium based on semantic conversion proposed in this invention can solve the problem of poor performance when enhancing financial images. Attached Figure Description

[0049] Figure 1 This is a flowchart illustrating an image data enhancement method based on semantic transformation according to an embodiment of the present invention.

[0050] Figure 2 This is a schematic diagram of a process for extracting image features according to an embodiment of the present invention;

[0051] Figure 3 This is a flowchart illustrating parameter optimization of a semantic conversion module according to an embodiment of the present invention;

[0052] Figure 4A functional block diagram of an image data enhancement device based on semantic transformation provided in an embodiment of the present invention;

[0053] Figure 5 This is a schematic diagram of the structure of an electronic device that implements the semantic transformation-based image data enhancement method according to an embodiment of the present invention.

[0054] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0055] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0056] This application provides a semantic transformation-based image data enhancement method. The executing entity of the semantic transformation-based image data enhancement method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the semantic transformation-based image data enhancement method can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0057] Reference Figure 1 The diagram shown is a flowchart illustrating an image data enhancement method based on semantic transformation according to an embodiment of the present invention. In this embodiment, the image data enhancement method based on semantic transformation includes the following steps S1-S5:

[0058] S1. Obtain a financial training image set, and perform text encoding on the financial image categories in the financial training image set to obtain the category semantic feature vector of each financial image in the financial training image set.

[0059] In this embodiment of the invention, the financial training image set contains a large number of images of different financial categories. For example, the financial training image set includes 1,000 images, each corresponding to a different financial image category label. For instance, the financial training image set contains financial training images with financial category labels such as bills, expense reports, and different types of invoices.

[0060] In this embodiment of the invention, text encoding encodes each financial image category in the financial training image set, converting each image category into a computer-recognizable language, and identifying the category semantic feature vector corresponding to each image category through text encoding.

[0061] In this embodiment of the invention, the step of text encoding the financial image categories in the financial training image set to obtain the category semantic feature vector of each financial image in the financial training image set includes:

[0062] Extract the financial category label of each financial image in the financial training image set to obtain the financial category in the financial training image set;

[0063] The financial category is segmented into words to obtain financial category words. The financial category words are then vectorized to obtain financial category word vectors.

[0064] The financial category word segmentation vectors are encoded using a preset attention mechanism to obtain the category semantic feature vector of each financial image in the financial training image set.

[0065] In this embodiment of the invention, the preset attention mechanism can be the attention mechanism in the Bidirectional Encoder Representation from Transformers (BERT) based on the attention mechanism to encode the financial category word segmentation vector, which can generate a deep bidirectional language representation, thereby obtaining the category semantic feature vector of each financial image.

[0066] S2. Extract the image features of each financial image in the financial training image set, and use the preset semantic conversion module to calculate the predicted financial category semantic vector corresponding to the image features;

[0067] In this embodiment of the invention, the image features are the most basic features that distinguish each financial training image from other images, and can be used as the distinctive attributes of each financial training image. This embodiment of the invention can use a preset residual network (ResNet) to extract the image features of each financial image.

[0068] In the embodiments of the present invention, see Figure 2 As shown, the step of extracting image features from each financial image in the financial training image set includes S21-S24:

[0069] S21. Using the shallow convolutional layers in the preset residual network, perform convolution and pooling operations on each financial image in the financial training image set to obtain the shallow feature map of each financial image in the financial training image set.

[0070] S22. The shallow feature map is convolutionally and pooled using the deep convolutional layers in the residual network to obtain the deep feature map of each financial image in the financial training image set.

[0071] S23. Downsample the shallow feature map to the size of the deep feature map to obtain a shallow sampled feature map. Perform residual connection between the shallow sampled feature map and the deep feature map to obtain a fused feature map.

[0072] S24. Perform activation calculation on the fused feature map to obtain the image features of each financial image in the financial training image set.

[0073] In this embodiment of the invention, the preset residual network includes multiple convolutional layers. For example, the residual network includes 5 convolutional layers, with the first and second layers being shallow convolutional layers and the fourth and fifth layers being deep convolutional layers. The shallow feature maps contain the overall information of each training image, but do not describe the detailed features of the image in enough detail. The deep feature maps obtained by the deep convolutional layers contain more detailed features but lack the overall features of each training image. Therefore, by downsampling the shallow feature maps to the size of the deep feature maps, the shallow and deep feature maps can be residually connected, so that the shallow sampled feature maps and the deep feature maps are superimposed to obtain a fused feature map.

[0074] In this embodiment of the invention, the ReLU activation function (Rectified Linear Unit) can be used to activate the fused feature map to obtain the image features of each financial image in the financial training image set.

[0075] In another embodiment of the present invention, the preset semantic transformation module includes two semantic transformation matrices and a semantic transformation network g([H,z]) with weight H and bias z. The semantic transformation matrices and the semantic transformation network are used to predict the image category semantics of the image features to obtain the predicted financial category semantic vector corresponding to the image features.

[0076] In this embodiment of the invention, the step of calculating the predicted financial category semantic vector corresponding to the image features using a preset semantic transformation module includes:

[0077] The first semantic transformation matrix in the semantic transformation module is used to perform vector mapping on the image features to obtain the first predicted financial category semantic vector.

[0078] The semantic transformation network in the semantic transformation module is used to calculate the image features to obtain a virtual category semantic feature vector;

[0079] The second semantic transformation matrix in the semantic transformation module is used to perform vector mapping on the virtual category semantic feature vector to obtain the second predicted financial category semantic vector;

[0080] By combining the first predicted financial category semantic vector, the virtual category semantic feature vector, and the second predicted financial category semantic vector, the predicted financial category semantic vector corresponding to the image feature is obtained.

[0081] In this embodiment of the invention, the corresponding predicted financial category semantic vector is obtained by performing a dot product of the image features, the semantic transformation matrix, and the transpose of the semantic transformation matrix.

[0082] In this embodiment of the invention, the step of using the first semantic transformation matrix in the semantic transformation module to perform vector mapping on the image features to obtain the first predicted financial category semantic vector includes:

[0083] The first semantic transformation matrix is ​​multiplied by the image features to obtain the dot product feature vector;

[0084] The dot product feature vector is multiplied by the transpose of the first semantic transformation matrix to obtain the transpose feature vector.

[0085] By combining the dot product feature vector and the transposed feature vector, a first predicted financial category semantic vector is obtained.

[0086] In this embodiment of the invention, image features are mapped using a first semantic transformation matrix and its transpose to obtain a first predicted financial category semantic vector.

[0087] In this embodiment of the invention, the image features are calculated using the following formula:

[0088]

[0089] in, For virtual category semantic feature vectors, The weights in the semantic transformation network, The image features, This is the bias in the semantic transformation network.

[0090] S3. Adjust the parameter values ​​in the semantic conversion module according to the predicted financial category semantic vector and the category semantic feature vector to obtain an optimized semantic conversion module;

[0091] In this embodiment of the invention, the parameter values ​​include the parameter values ​​in the first semantic transformation matrix, the semantic transformation network, and the second semantic transformation matrix in the semantic transformation module. Thus, by predicting the error value between the financial category semantic vector and the category semantic feature vector, the parameters of the semantic transformation module are adjusted to obtain an optimized semantic transformation module.

[0092] In the embodiments of the present invention, see Figure 3 As shown, the adjustment of parameter values ​​in the semantic transformation module based on the predicted financial category semantic vector and the category semantic feature vector includes S31-S33:

[0093] S31. Calculate the first error value between the first predicted financial category semantic vector and the category semantic feature vector in the predicted financial category semantic vector, and adjust the parameters in the first semantic transformation matrix in the semantic transformation module according to the first error value until the first error value is less than the preset error value threshold, so as to obtain the optimized first semantic transformation matrix.

[0094] S32. Calculate the second error value between the virtual category semantic feature vector and the category semantic feature vector in the predicted financial category semantic vector, and adjust the parameters in the semantic conversion network in the semantic conversion module according to the second error value until the second error value is less than the preset error value threshold, thereby obtaining the optimized semantic conversion network.

[0095] S33. Calculate the third error value between the second predicted financial category semantic vector and the virtual category semantic feature vector in the predicted financial category semantic vector. Adjust the parameters in the second semantic transformation matrix in the semantic transformation module according to the third error value until the third error value is less than the preset error value threshold to obtain the optimized second semantic transformation matrix.

[0096] In this embodiment of the invention, calculating the first error value between the first predicted financial category semantic vector and the category semantic feature vector in the predicted financial category semantic vector includes:

[0097] The first error value is calculated using the following formula:

[0098]

[0099] in, This represents the first error value. This represents the dot product feature vector in the semantic vector of the first predicted financial category. Represents the semantic feature vector of the category. Representing image features, This represents the transposed feature vector in the semantic vector of the first predicted financial category.

[0100] In this embodiment of the invention, the parameters of the first semantic transformation matrix of the semantic transformation module are optimized by the minimum mean square error calculation method. When the first error is less than a preset error threshold, the optimized first semantic transformation matrix is ​​obtained. The error threshold can be set to 0.1. The parameters of the semantic transformation module are optimized by the error threshold to obtain the optimized semantic transformation module.

[0101] In another optional embodiment of the present invention, the semantic transformation network and the second semantic transformation matrix in the semantic transformation module can be optimized by the above-mentioned minimum mean square error calculation method to obtain an optimized semantic transformation network and an optimized second semantic transformation matrix.

[0102] In this embodiment of the invention, by optimizing the parameter values ​​in the semantic conversion module, the accuracy of semantic conversion can be improved, thereby enabling more accurate data augmentation of the target image.

[0103] S4. Obtain the target financial enhancement image, perform data encoding on the target financial enhancement image, and obtain the image encoding features of the target financial enhancement image;

[0104] In this embodiment of the invention, the target financial enhancement image is a financial image that needs to be data enhanced. By encoding the image data in the target financial enhancement image, image encoding features are obtained. Specifically, data encoding involves encoding the image pixels in the target financial enhancement image into a string, thereby obtaining the image encoding features.

[0105] In this embodiment of the invention, the step of data encoding the target financial enhancement image to obtain the image encoding features of the target financial enhancement image includes:

[0106] Obtain the pixel values ​​corresponding to the pixels in the target financial enhancement image, and encode the pixel values ​​into octets to obtain the binary dataset of the target financial enhancement image;

[0107] The binary dataset is grouped to obtain binary group sequences, and the corresponding encoding value of the binary group sequence is found using a preset encoding table;

[0108] The image coding features of the target financial enhancement image are obtained by concatenating the encoded values ​​corresponding to the binary group sequence.

[0109] In this embodiment of the invention, the octet encoding is a variable-length character encoding format (Universal Character Set / Unicode Transformation Format, UTF-8). Octet encoding can convert pixel values ​​into binary, and then encode the binary dataset of the target financial enhancement image to obtain the image encoding features of the target financial enhancement image.

[0110] In this embodiment of the invention, the preset encoding table can be a base64 encoding table (based on 64 printable characters to represent binary data). Base64 converts every three 8-bit bytes into four 6-bit bytes, and then adds two high-order 0 bits to the 6-bit bytes to form four 8-bit bytes. Therefore, the binary dataset needs to be grouped to obtain an 8-bit binary group sequence, and then obtain the encoding value corresponding to each binary group. For example, 00101011,00011011 corresponds to the encoding values ​​r and b.

[0111] S5. Use the optimized semantic conversion module to perform semantic conversion on the image encoding features to obtain the semantic conversion vector of the target financial enhancement image, and decode the semantic conversion vector to obtain the financial enhancement data of the target financial enhancement image.

[0112] In this embodiment of the invention, the semantic transformation of the image encoding features using the optimized semantic transformation module is performed sequentially by calculating the optimized first semantic transformation matrix, the optimized semantic transformation network, and the optimized second semantic transformation matrix in the optimized semantic transformation module to obtain the semantic transformation vector of the target financial enhancement image.

[0113] In this embodiment of the invention, the decoding of the semantic transformation vector is the reverse process of the above data encoding, that is, the semantic transformation vector is first converted into binary, and then the financial enhancement data of the target financial enhancement image is generated by octet decoding.

[0114] In this embodiment of the invention, a semantic transformation module is used to perform semantic transformation on the image encoding features. By adding two semantic transformation matrices and a semantic transformation network, the input image data samples are augmented, thereby increasing the data augmentation effect from the perspective of semantic features, obtaining more accurate semantic transformation vectors, and thus obtaining more accurate financial augmentation data for the target financial augmentation image, effectively improving the data augmentation effect of financial images.

[0115] This invention, through encoding the financial image categories in a financial training image set, generates deep bidirectional language representations, thereby obtaining a category semantic feature vector for each financial image. Then, it extracts the image features of each financial image and uses a pre-defined semantic transformation module to calculate the predicted financial category semantic vector corresponding to the image features. Based on the predicted financial category semantic vector and the category semantic feature vector, the parameter values ​​in the semantic transformation module are adjusted to obtain an optimized semantic transformation module. This optimizes the semantic transformation-based image data augmentation, enhancing the effect from a semantic feature perspective and yielding more accurate semantic transformation vectors. Finally, the semantic transformation vectors are decoded, effectively improving the financial data augmentation effect of the target financial image. Therefore, the semantic transformation-based image data augmentation method proposed in this invention can solve the problem of poor performance when augmenting financial images.

[0116] like Figure 4 The diagram shown is a functional block diagram of an image data enhancement device based on semantic conversion provided in an embodiment of the present invention.

[0117] The semantic conversion-based image data enhancement device 100 of this invention can be installed in an electronic device. Depending on the functions implemented, the semantic conversion-based image data enhancement device 100 may include a financial image category encoding module 101, a semantic conversion module 102, a parameter optimization module 103, an image data encoding module 104, and a data enhancement module 105. The module described in this invention can also be called a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and are stored in the memory of the electronic device.

[0118] In this embodiment, the functions of each module / unit are as follows:

[0119] The financial image category encoding module 101 is used to acquire a financial training image set, perform text encoding on the financial image categories in the financial training image set, and obtain the category semantic feature vector of each financial image in the financial training image set.

[0120] The semantic conversion module 102 is used to extract the image features of each financial image in the financial training image set, and to calculate the predicted financial category semantic vector corresponding to the image features using a preset semantic conversion module.

[0121] The parameter optimization module 103 is used to adjust the parameter values ​​in the semantic conversion module according to the predicted financial category semantic vector and the category semantic feature vector, so as to obtain an optimized semantic conversion module.

[0122] The image data encoding module 104 is used to acquire the target financial enhancement image, encode the target financial enhancement image, and obtain the image encoding features of the target financial enhancement image.

[0123] The data augmentation module 105 is used to perform semantic transformation on the image encoding features using the optimized semantic transformation module to obtain the semantic transformation vector of the target financial augmentation image, and to decode the semantic transformation vector to obtain the financial augmentation data of the target financial augmentation image.

[0124] In detail, the modules in the semantic conversion-based image data enhancement device 100 described in this embodiment of the invention employ the same methods as described above. Figures 1 to 3 The image data enhancement method based on semantic transformation described herein uses the same technical means and can produce the same technical effect, so it will not be repeated here.

[0125] like Figure 5 The diagram shown is a structural schematic of an electronic device that implements a semantic transformation-based image data enhancement method according to an embodiment of the present invention.

[0126] The electronic device 1 may include a processor 10, a memory 11, a communication bus 12, and a communication interface 13. It may also include a computer program stored in the memory 11 and capable of running on the processor 10, such as a semantically transformed image data enhancement program.

[0127] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules stored in the memory 11 (e.g., executing image data enhancement programs based on semantic transformation) and calls data stored in the memory 11 to perform various functions of the electronic device and process data.

[0128] The memory 11 includes at least one type of readable storage medium, including flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of an electronic device, such as a portable hard drive. In other embodiments, the memory 11 can be an external storage device of the electronic device, such as a plug-in portable hard drive, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc. Furthermore, the memory 11 can include both internal and external storage units of the electronic device. The memory 11 can be used not only to store application software and various types of data installed on the electronic device, such as code for image data enhancement programs based on semantic translation, but also to temporarily store data that has been output or will be output.

[0129] The communication bus 12 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.

[0130] The communication interface 13 is used for communication between the aforementioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, Bluetooth interface, etc.), typically used to establish communication connections between the electronic device and other electronic devices. The user interface may be a display, an input unit (such as a keyboard), or optionally, a standard wired or wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device and to display a visual user interface.

[0131] The figure only shows an electronic device with components. Those skilled in the art will understand that the structure shown in the figure does not constitute a limitation on the electronic device and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0132] For example, although not shown, the electronic device may also include a power supply (such as a battery) to power the various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0133] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in the scope of the patent application.

[0134] The image data enhancement program based on semantic transformation stored in the memory 11 of the electronic device 1 is a combination of multiple instructions. When run in the processor 10, it can implement the following methods:

[0135] Obtain a set of financial training images, and perform text encoding on the categories of financial images in the set to obtain the category semantic feature vector of each financial image in the set.

[0136] Extract the image features of each financial image in the financial training image set, and use a preset semantic conversion module to calculate the predicted financial category semantic vector corresponding to the image features;

[0137] The parameter values ​​in the semantic conversion module are adjusted based on the predicted financial category semantic vector and the category semantic feature vector to obtain an optimized semantic conversion module;

[0138] Obtain the target financial enhancement image, encode the target financial enhancement image into data, and obtain the image encoding features of the target financial enhancement image;

[0139] The optimized semantic transformation module is used to perform semantic transformation on the image encoding features to obtain the semantic transformation vector of the target financial enhancement image. The semantic transformation vector is then decoded to obtain the financial enhancement data of the target financial enhancement image.

[0140] Specifically, the specific implementation method of the processor 10 for the above instructions can be referred to the description of the relevant steps in the corresponding embodiment of the accompanying drawings, and will not be repeated here.

[0141] Furthermore, if the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0142] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor of an electronic device, can implement the following methods:

[0143] Obtain a set of financial training images, and perform text encoding on the categories of financial images in the set to obtain the category semantic feature vector of each financial image in the set.

[0144] Extract the image features of each financial image in the financial training image set, and use a preset semantic conversion module to calculate the predicted financial category semantic vector corresponding to the image features;

[0145] The parameter values ​​in the semantic conversion module are adjusted based on the predicted financial category semantic vector and the category semantic feature vector to obtain an optimized semantic conversion module;

[0146] Obtain the target financial enhancement image, encode the target financial enhancement image into data, and obtain the image encoding features of the target financial enhancement image;

[0147] The optimized semantic transformation module is used to perform semantic transformation on the image encoding features to obtain the semantic transformation vector of the target financial enhancement image. The semantic transformation vector is then decoded to obtain the financial enhancement data of the target financial enhancement image.

[0148] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0149] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0150] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0151] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0152] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0153] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0154] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any specific order.

[0155] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for image data enhancement based on semantic transformation, characterized in that, The method includes: Obtain a set of financial training images, and perform text encoding on the categories of financial images in the set to obtain the category semantic feature vector of each financial image in the set. Extract the image features of each financial image in the financial training image set, and use a preset semantic conversion module to calculate the predicted financial category semantic vector corresponding to the image features; The parameter values ​​in the semantic conversion module are adjusted based on the predicted financial category semantic vector and the category semantic feature vector to obtain an optimized semantic conversion module; Obtain the target financial enhancement image, encode the target financial enhancement image into data, and obtain the image encoding features of the target financial enhancement image; The optimized semantic transformation module is used to perform semantic transformation on the image encoding features to obtain the semantic transformation vector of the target financial enhancement image. The semantic transformation vector is then decoded to obtain the financial enhancement data of the target financial enhancement image.

2. The image data enhancement method based on semantic transformation as described in claim 1, characterized in that, The step of text encoding the financial image categories in the financial training image set to obtain the category semantic feature vector of each financial image in the financial training image set includes: Extract the financial category label of each financial image in the financial training image set to obtain the financial image category in the financial training image set; The financial image category is segmented into words to obtain financial category words. The financial category words are then vectorized to obtain financial category word vectors. The financial category word segmentation vectors are encoded using a preset attention mechanism to obtain the category semantic feature vector of each financial image in the financial training image set.

3. The image data enhancement method based on semantic transformation as described in claim 1, characterized in that, The extraction of image features for each financial image in the financial training image set includes: The shallow convolutional layers in the preset residual network are used to perform convolution and pooling operations on each financial image in the financial training image set to obtain a shallow feature map of each financial image in the financial training image set. The shallow feature map is convolutionally and pooled using the deep convolutional layers in the residual network to obtain the deep feature map of each financial image in the financial training image set. The shallow feature map is downsampled to the size of the deep feature map to obtain a shallow sampled feature map. The shallow sampled feature map and the deep feature map are then residually connected to obtain a fused feature map. Activation calculations are performed on the fused feature map to obtain the image features of each financial image in the financial training image set.

4. The image data enhancement method based on semantic transformation as described in claim 1, characterized in that, The step of calculating the predicted financial category semantic vector corresponding to the image features using a preset semantic transformation module includes: The first semantic transformation matrix in the semantic transformation module is used to perform vector mapping on the image features to obtain the first predicted financial category semantic vector. The semantic transformation network in the semantic transformation module is used to calculate the image features to obtain a virtual category semantic feature vector; The second semantic transformation matrix in the semantic transformation module is used to perform vector mapping on the virtual category semantic feature vector to obtain the second predicted financial category semantic vector; By combining the first predicted financial category semantic vector, the virtual category semantic feature vector, and the second predicted financial category semantic vector, the predicted financial category semantic vector corresponding to the image feature is obtained.

5. The image data enhancement method based on semantic transformation as described in claim 4, characterized in that, The step of using the first semantic transformation matrix in the semantic transformation module to perform vector mapping on the image features to obtain the first predicted financial category semantic vector includes: The first semantic transformation matrix is ​​multiplied by the image features to obtain the dot product feature vector; The dot product feature vector is multiplied by the transpose of the first semantic transformation matrix to obtain the transpose feature vector. By combining the dot product feature vector and the transposed feature vector, a first predicted financial category semantic vector is obtained.

6. The image data enhancement method based on semantic transformation as described in claim 1, characterized in that, The step of adjusting the parameter values ​​in the semantic transformation module based on the predicted financial category semantic vector and the category semantic feature vector includes: Calculate the first error value between the first predicted financial category semantic vector and the category semantic feature vector in the predicted financial category semantic vector, and adjust the parameters in the first semantic transformation matrix in the semantic transformation module according to the first error value until the first error value is less than a preset error value threshold to obtain the optimized first semantic transformation matrix; Calculate the second error value between the virtual category semantic feature vector and the category semantic feature vector in the predicted financial category semantic vector, and adjust the parameters in the semantic conversion network in the semantic conversion module according to the second error value until the second error value is less than a preset error value threshold, thereby obtaining an optimized semantic conversion network; Calculate the third error value between the second predicted financial category semantic vector and the virtual category semantic feature vector in the predicted financial category semantic vector. Adjust the parameters in the second semantic transformation matrix in the semantic transformation module according to the third error value until the third error value is less than a preset error value threshold to obtain the optimized second semantic transformation matrix.

7. The image data enhancement method based on semantic transformation as described in claim 1, characterized in that, The step of encoding the target financial enhancement image to obtain its image encoding features includes: Obtain the pixel values ​​corresponding to the pixels in the target financial enhancement image, and encode the pixel values ​​into octets to obtain the binary dataset of the target financial enhancement image; The binary dataset is grouped to obtain binary group sequences, and the corresponding encoding value of the binary group sequence is found using a preset encoding table; The image coding features of the target financial enhancement image are obtained by concatenating the encoded values ​​corresponding to the binary group sequence.

8. An image data enhancement device based on semantic transformation, characterized in that, The device includes: The financial image category encoding module is used to acquire a set of financial training images, perform text encoding on the financial image categories in the set of financial training images, and obtain the category semantic feature vector of each financial image in the set of financial training images. The semantic transformation module is used to extract the image features of each financial image in the financial training image set, and to calculate the predicted financial category semantic vector corresponding to the image features using the preset semantic transformation module. The parameter optimization module is used to adjust the parameter values ​​in the semantic conversion module according to the predicted financial category semantic vector and the category semantic feature vector, so as to obtain the optimized semantic conversion module; The image data encoding module is used to acquire the target financial enhancement image, encode the target financial enhancement image, and obtain the image encoding features of the target financial enhancement image. The data augmentation module is used to perform semantic transformation on the image encoding features using the optimized semantic transformation module to obtain the semantic transformation vector of the target financial augmentation image, and to decode the semantic transformation vector to obtain the financial augmentation data of the target financial augmentation image.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the image data enhancement method based on semantic transformation as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the image data enhancement method based on semantic transformation as described in any one of claims 1 to 7.

Images