Image processing method and device, storage medium and electronic equipment

Abstract

The invention discloses an image processing method and device, a storage medium and electronic equipment. Relates to the field of artificial intelligence. The target image is processed through the image processing model, a target table corresponding to the image content of the target image is obtained, in the training process of the image processing model, model parameters are optimized based on a training sample and target loss, the training sample comprises a sample image and a sample table, and the sample table comprises a target image; the target loss comprises similarity loss between the sample image and the numeric type attribute and between the sample image and the category type attribute in the sample table. Through the method and the device, the problem of low table generation accuracy when a table is generated based on image content in the prior art is solved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and more specifically, to an image processing method, apparatus, storage medium, and electronic device. Background Technology

[0002] In today's information-intensive society, image content recognition and analysis technology has become a bridge connecting the physical and digital worlds, playing a crucial role in business processes, particularly in fields such as finance, healthcare, and education. With advancements in deep learning and computer vision, developing technologies for automatic image-to-table conversion has become possible. This technology aims to identify structured data from images and convert it into spreadsheet format for rapid data entry and efficient management. While traditional image information extraction techniques have achieved significant results in image recognition and classification tasks, their feature processing capabilities remain limited when handling image-to-table conversion, resulting in low accuracy in generated tables.

[0003] There is currently no effective solution to the aforementioned problems in the relevant technologies. Summary of the Invention

[0004] The main objective of this application is to provide an image processing method, apparatus, storage medium, and electronic device to solve the problem of low accuracy in generating tables based on image content in related technologies.

[0005] To achieve the above objectives, according to one aspect of this application, an image processing method is provided. The method includes: acquiring a target image; processing the target image using an image processing model to obtain a target table corresponding to the image content of the target image, wherein, during the training process of the image processing model, model parameters are optimized based on training samples and a target loss. The training samples include sample images and sample tables, and the target loss includes similarity losses between the sample images and numerical attributes and categorical attributes in the sample tables, respectively.

[0006] Optionally, the image processing method further includes: acquiring a training sample set and an initial image processing model, wherein the training sample set includes training samples, and the initial image processing model includes a similarity measurement module and a decoder; determining the similarity loss between the sample image and the numerical attributes and categorical attributes in the sample table through the similarity measurement module; processing the sample image through the decoder to obtain a prediction table, and determining the generation loss between the prediction table and the sample table; determining the target loss based on the similarity loss and the generation loss, and optimizing the parameters of the initial image processing model based on the target loss until a preset iteration condition is met to obtain the image processing model.

[0007] Optionally, the image processing method further includes: determining the mutual information loss between the sample image and the numerical and categorical attributes in the sample table through a feature distribution alignment module; determining the alignment loss between the sample image and the sample table through an alignment relationship optimization module; and determining the target loss based on the similarity loss, mutual information loss, alignment loss, and generation loss.

[0008] Optionally, the image processing method further includes: performing feature transformation on the sample image to obtain image features; performing feature transformation on the numerical attributes in the sample table to obtain numerical attribute features, and performing feature transformation on the categorical attributes in the sample table to obtain categorical attribute features; calculating a first similarity between the image features and the numerical attribute features based on a first similarity formula, and determining a first similarity loss based on the first similarity; calculating a second similarity between the image features and the categorical attribute features based on a second similarity formula, and determining a second similarity loss based on the second similarity.

[0009] Optionally, the image processing method further includes: calculating the mutual information loss between the sample image features of the sample image and the numerical attribute features of the sample table to obtain a first mutual information loss; and calculating the mutual information loss between the sample image features and the categorical attribute features of the sample table to obtain a second mutual information loss.

[0010] Optionally, the image processing method further includes: determining a cost matrix based on similarity loss, wherein the cost matrix represents the cost of aligning a sample image with a sample table; solving for a transfer matrix based on the cost matrix with the objective of minimizing the function value of the target alignment function, wherein the target alignment function is determined based on the cost matrix and the transfer matrix, and the transfer matrix represents the matching weight between the sample image and the sample table; and determining an alignment loss based on the target alignment function, the transfer matrix, and the cost matrix.

[0011] Optionally, the image processing method further includes: obtaining the loss calculation weights corresponding to the similarity measurement module, feature distribution alignment module, alignment relationship optimization module and decoder respectively; and calculating the target loss based on the loss calculation weights corresponding to the similarity measurement module, feature distribution alignment module, alignment relationship optimization module and decoder respectively, as well as the similarity loss, mutual information loss, alignment loss and generation loss.

[0012] Optionally, the image processing method further includes: performing feature transformation on the target image through the similarity measurement module in the image processing model to obtain target image features; and processing the target image features through the decoder in the image processing model to obtain a target table.

[0013] To achieve the above objectives, according to another aspect of this application, an image processing apparatus is provided. The apparatus includes: a first acquisition module for acquiring a target image; and a first processing module for processing the target image using an image processing model to obtain a target table corresponding to the image content of the target image. During the training process of the image processing model, model parameters are optimized based on training samples and a target loss. The training samples include sample images and sample tables, and the target loss includes similarity losses between the sample images and numerical attributes and categorical attributes in the sample tables, respectively.

[0014] Optionally, the image processing device further includes: a second acquisition module for acquiring a training sample set and an initial image processing model, wherein the training sample set includes training samples, and the initial image processing model includes a similarity measurement module and a decoder; a first determination module for determining the similarity loss between the sample image and the numerical attribute and categorical attribute in the sample table respectively through the similarity measurement module; a second determination module for processing the sample image through the decoder to obtain a prediction table, and determining the generation loss between the prediction table and the sample table; and a second processing module for determining the target loss based on the similarity loss and the generation loss, and optimizing the parameters of the initial image processing model based on the target loss until a preset iteration condition is met to obtain the image processing model.

[0015] Optionally, the second processing module further includes: a first determining submodule, used to determine the mutual information loss between the sample image and the numerical attributes and categorical attributes in the sample table respectively through the feature distribution alignment module; a second determining submodule, used to determine the alignment loss between the sample image and the sample table through the alignment relationship optimization module; and a third determining submodule, used to determine the target loss based on the similarity loss, mutual information loss, alignment loss and generation loss.

[0016] Optionally, the first determining module further includes: a first processing submodule, used to perform feature transformation on the sample image to obtain image features; a second processing submodule, used to perform feature transformation on the numerical attributes in the sample table to obtain numerical attribute features, and to perform feature transformation on the categorical attributes in the sample table to obtain categorical attribute features; a first calculation submodule, used to calculate a first similarity between the image features and the numerical attribute features based on a first similarity formula, and to determine a first similarity loss based on the first similarity; and a second calculation submodule, used to calculate a second similarity between the image features and the categorical attribute features based on a second similarity formula, and to determine a second similarity loss based on the second similarity.

[0017] Optionally, the first determining submodule further includes: a first calculation unit, used to calculate the mutual information loss between the sample image features of the sample image and the numerical attribute features of the sample table, to obtain a first mutual information loss; and a second calculation unit, used to calculate the mutual information loss between the sample image features and the categorical attribute features of the sample table, to obtain a second mutual information loss.

[0018] Optionally, the second determining submodule further includes: a first determining unit, used to determine a cost matrix based on similarity loss, wherein the cost matrix represents the cost of aligning the sample image with the sample table; a processing unit, used to solve for the target alignment function based on the cost matrix with the objective of minimizing the function value of the target alignment function, to obtain a transfer matrix, wherein the target alignment function is determined based on the cost matrix and the transfer matrix, and the transfer matrix represents the matching weight between the sample image and the sample table; and a second determining unit, used to determine the alignment loss based on the target alignment function, the transfer matrix, and the cost matrix.

[0019] Optionally, the third determining submodule further includes: an acquisition unit, used to acquire the loss calculation weights corresponding to the similarity measurement module, the feature distribution alignment module, the alignment relationship optimization module, and the decoder respectively; and a third calculation unit, used to calculate the target loss based on the loss calculation weights corresponding to the similarity measurement module, the feature distribution alignment module, the alignment relationship optimization module, and the decoder respectively, as well as the similarity loss, mutual information loss, alignment loss, and generation loss.

[0020] Optionally, the first processing module further includes: a third processing submodule, used to perform feature transformation on the target image through the similarity measurement module in the image processing model to obtain target image features; and a fourth processing submodule, used to process the target image features through the decoder in the image processing model to obtain a target table.

[0021] To achieve the above objectives, according to another aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is executed, it controls the device where the computer-readable storage medium is located to perform the above-described image processing method.

[0022] To achieve the above objectives, according to another aspect of this application, an electronic device is provided, the electronic device including a memory storing an executable program; and a processor for running the program, wherein the program executes the above-described image processing method when it runs.

[0023] To achieve the above objectives, according to another aspect of this application, a computer program product is provided, including computer instructions that, when executed by a processor, implement the steps of the image processing method described above.

[0024] In this embodiment, the target loss for training the image processing model is designed to include the similarity loss between the sample image and the numerical and categorical attributes in the sample table, respectively. This achieves the optimization of the similarity loss based on multiple scales during the model training stage, taking into account the differences between numerical and categorical attributes. This ensures the robustness and generalization ability of the model when processing complex heterogeneous data. As a result, in the image processing stage of practical applications, the model can intelligently identify the structured data in the target image, generate a clear and complete spreadsheet, and improve the accuracy of the generated spreadsheet.

[0025] Therefore, the method provided in this application achieves the goal of optimizing the model based on multi-scale similarity loss during the model training process and performing image-to-table conversion based on the trained model, thereby improving the accuracy of the generated table and solving the technical problem of low accuracy in generating tables based on image content in related technologies. Attached Figure Description

[0026] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0027] Figure 1 This is a hardware structure block diagram of a computer terminal provided according to an embodiment of this application;

[0028] Figure 2 This is a flowchart of an image processing method provided according to an embodiment of this application;

[0029] Figure 3 This is a schematic diagram of an image processing apparatus provided according to an embodiment of this application;

[0030] Figure 4 This is a structural block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0031] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0032] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0033] It should be noted that the information collected in this application (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) are information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of this data all comply with relevant laws, regulations, and standards, necessary confidentiality measures have been taken, and they do not violate public order and good morals. Corresponding access points are provided for users to choose to authorize or refuse. For example, interfaces are set up between this system and relevant users or organizations, providing users with corresponding access points to choose to agree to or refuse automated decision-making results; if the user chooses to refuse, the process proceeds to the expert decision-making stage.

[0034] Example 1

[0035] According to an embodiment of this application, an embodiment of an image processing method is also provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0036] The method embodiment provided in Embodiment 1 of this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 A hardware block diagram of a computer terminal (or mobile device) for implementing an image processing method is shown. Figure 1As shown, the computer terminal 10 (or mobile device) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor (MCU) or a field-programmable gate array (FPGA) or similar processing device), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output (I / O) interface, a Universal Serial Bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0037] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0038] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the image processing method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the above-mentioned image processing method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0039] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0040] The display can be, for example, a touchscreen liquid crystal display (LCD), which allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0041] Under the aforementioned operating environment, this application provides the following: Figure 2 The image processing method shown. Figure 2 This is a flowchart of the image processing method according to Embodiment 1 of this application.

[0042] Step S201: Obtain the target image.

[0043] Optionally, electronic devices, application systems, servers, or other similar devices can be used as the execution subject of this application. In this embodiment, the target processing system is used as the execution subject to perform the image processing method described above.

[0044] Optionally, the target image can be an image from which information is to be extracted. These images may originate from daily business operations or data collection processes, containing rich unstructured and / or structured information that can be used for subsequent business processing or data statistical analysis. For example, in a financial scenario, the target image may be an image of a financial statement, an invoice, a risk assessment report, or a statistical table.

[0045] Step S202: The target image is processed by the image processing model to obtain the target table corresponding to the image content of the target image. During the training process of the image processing model, the model parameters are optimized based on the training samples and the target loss. The training samples include sample images and sample tables. The target loss includes the similarity loss between the sample images and the numerical attributes and categorical attributes in the sample tables, respectively.

[0046] After obtaining the target image, an image processing model is used to process the target image to obtain a target table corresponding to the image content. The image processing model is a pre-trained deep learning model that can parse the large amount of unstructured data contained in the image into structured data and construct a spreadsheet according to the logical relationships between the data, thus achieving digital extraction of information.

[0047] In the training process of an image processing model, the training samples consist of sample images and corresponding sample tables, serving as the foundational data for model learning. The target loss includes the similarity loss between the sample image and the numerical attributes and categorical attributes in the sample table, respectively, as well as a metric defining the difference between the model's predictions and the actual table data. The aforementioned numerical attributes refer to the numerical data in the cells (such as amounts, dates, etc.), while categorical attributes refer to the labels or classification information in the table (such as item categories, region codes, etc.). Minimizing the target loss optimizes the model parameters, ensuring that the model can make accurate predictions on different types of data.

[0048] In this embodiment, the target loss for training the image processing model is designed to include the similarity loss between the sample image and the numerical and categorical attributes in the sample table, respectively. This achieves the optimization of the similarity loss based on multiple scales during the model training stage, taking into account the differences between numerical and categorical attributes. This ensures the robustness and generalization ability of the model when processing complex heterogeneous data. As a result, in the image processing stage of practical applications, the model can intelligently identify the structured data in the target image, generate a clear and complete spreadsheet, and improve the accuracy of the generated spreadsheet.

[0049] Therefore, the method provided in this application achieves the goal of optimizing the model based on multi-scale similarity loss during the model training process and performing image-to-table conversion based on the trained model, thereby improving the accuracy of the generated table and solving the technical problem of low accuracy in generating tables based on image content in related technologies.

[0050] Optionally, in the image processing method provided in this application embodiment, the image processing model is obtained in the following manner: acquiring a training sample set and an initial image processing model, wherein the training sample set includes training samples, and the initial image processing model includes a similarity measurement module and a decoder; determining the similarity loss between the sample image and the numerical attribute and categorical attribute in the sample table respectively through the similarity measurement module; processing the sample image through the decoder to obtain a prediction table, and determining the generation loss between the prediction table and the sample table; determining the target loss based on the similarity loss and the generation loss, and optimizing the parameters of the initial image processing model based on the target loss until a preset iteration condition is met to obtain the image processing model.

[0051] The sample image and the target image have the same image format, so they will not be described again here. The sample table corresponds to the image content of the sample image. For example, the sample image could be a loan approval statistics chart used internally by a financial institution. This chart might present the distribution of loan applications in 2022 by age group, occupation type, and approval results. For example, the chart might use a bar chart to show the loan approval rate for applicants in different age groups, or a pie chart to break down the approval and rejection ratios of loan applications for different occupation types. The sample table could use "Age Group," "Occupation Type," and "Approval Result" as headers, with specific age range values, occupations, etc., as attribute values.

[0052] During model training, a sample table is used as a labeling reference, and sample images are used as input data. First, the similarity measurement module in the initial image processing model attempts to identify key information in the image and match it with the numerical and categorical attributes in the sample table, calculating the similarity loss. Next, the decoder module in the initial image processing model converts the image features into a prediction table, compares it with the sample table, and determines the generation loss. Finally, the model combines the similarity loss and the generation loss, continuously optimizing the parameters until training is complete. This allows the model to extract and convert structured data from the sample images with high accuracy, forming a prediction table that matches the information in the sample table.

[0053] It should be noted that the above method combines the similarity loss between images and tables with the model's generation loss for model training, which means that model training is based on multi-dimensional loss, thereby effectively improving the model training effect.

[0054] Optionally, in the image processing method provided in the embodiments of this application, the initial image processing model further includes a feature distribution alignment module and an alignment relationship optimization module. The target loss is determined based on similarity loss and generation loss, including: determining the mutual information loss between the sample image and the numerical attributes and categorical attributes in the sample table through the feature distribution alignment module; determining the alignment loss between the sample image and the sample table through the alignment relationship optimization module; and determining the target loss based on similarity loss, mutual information loss, alignment loss, and generation loss.

[0055] Optionally, the initial image processing model also includes a feature distribution alignment module and an alignment relationship optimization module.

[0056] In an optional embodiment, the similarity measurement module, the feature distribution alignment module, the alignment relationship optimization module, and the decoder are executed sequentially.

[0057] During model training, the feature distribution alignment module is used to optimize the distribution consistency between features in the sample images and numerical and categorical attributes in the sample table. Mutual information (MI) is a statistical concept used to measure the degree of interdependence between two random variables. In this context, it is used as an indicator to measure how much information is shared between sample image features and sample table attributes. Specifically, by calculating the mutual information between the feature vectors obtained after processing the sample images and the attribute vectors in the sample table, the mutual information loss can be obtained. This loss reflects the alignment between the feature space and the attribute space, that is, whether the image features extracted by the model match the data features in the table. By minimizing this loss, the model can learn more effective and representative image features, enabling it to more accurately map to the table data during the transformation process.

[0058] For example, mutual information can be estimated using Kullback-Leibler (KL) divergence or a similar method, thereby defining the mutual information loss function. During the training loop, the mutual information loss of each pair of sample images and their corresponding table attributes is calculated and added to the target loss.

[0059] The alignment optimization module adjusts and optimizes the mapping relationship between image features and table attributes, ensuring their semantic consistency. Alignment loss is defined here as a quantitative metric of the alignment between the model-predicted image features and the true table attributes. The alignment loss can be calculated based on feature distance, attribute matching degree, or specific alignment criteria to evaluate the difference between the model-predicted features and the true attributes.

[0060] For example, Optimal Transport (OT) techniques can be used to optimize the alignment between features and attributes. These methods can find the optimal feature mapping path while preserving the inherent properties of image features and table attributes, thereby calculating the alignment loss.

[0061] After determining the similarity loss, mutual information loss, alignment loss, and generation loss, the target loss is determined based on these losses. For example, the target loss is obtained by summing the aforementioned losses.

[0062] It should be noted that by designing a feature distribution alignment module, information sharing between cross-modal features can be enhanced, and the joint distribution between image features and table features can be optimized, enabling the two types of features to share information better. By designing an alignment relationship optimization module, the alignment relationship between image channels and table attributes can be optimized, thereby further improving the model training effect.

[0063] Optionally, in the image processing method provided in this application embodiment, the similarity measurement module determines the similarity loss between the sample image and the numerical attributes and categorical attributes in the sample table, respectively, including: performing feature transformation on the sample image to obtain image features; performing feature transformation on the numerical attributes in the sample table to obtain numerical attribute features, and performing feature transformation on the categorical attributes in the sample table to obtain categorical attribute features; calculating a first similarity between the image features and the numerical attribute features based on a first similarity formula, and determining a first similarity loss based on the first similarity; calculating a second similarity between the image features and the categorical attribute features based on a second similarity formula, and determining a second similarity loss based on the second similarity.

[0064] In an optional embodiment, numerical attributes include the name and value of the numerical attribute, and categorical attributes include the name and value of the categorical attribute. The value of the numerical attribute can be normalized to convert it into a numerical feature within a uniform range for easier measurement; while the names of each attribute and the values ​​of the categorical attribute can be converted into dense vector representations using one-hot encoding or an embedding layer to reflect the semantic relationships between attributes.

[0065] To address the different characteristics of numerical and categorical table attributes, a multi-scale similarity measurement function is designed to handle continuous numerical data and discrete categorical data respectively. For numerical attributes, a kernel density-based similarity measurement method, such as the Gaussian kernel function, is used to capture the continuous variation relationships between data. For categorical attributes, a measurement method based on information entropy or Jaccard similarity coefficient is used to reflect the degree of overlap and differences between categories.

[0066] For example, let the image features of the sample image be represented as X = {x1, x2, ..., x...} n}, where one element represents the feature vector corresponding to a single pixel or image patch in the sample image, and the table of numerical attribute vectors (i.e., numerical attribute features) is as follows. One element represents the feature vector of a single numerical attribute in the sample table, and the categorical attribute vector (i.e., categorical attribute features) is... Each element in the formula represents a feature vector of a single categorical attribute in the sample table. For example, the first similarity formula can be expressed as follows:

[0067]

[0068] in, σ represents the similarity between the i-th element in the image features and the j-th element in the numerical attribute vector of the table, where σ is the bandwidth parameter of the Gaussian kernel.

[0069] In an optional embodiment, a similarity matrix (equivalent to a first similarity) is determined based on the similarity between each element in the image features and each element in the tabular numerical attribute vector. Then, the complement number (or opposite number) of each element in the similarity matrix is ​​taken to obtain a first similarity loss matrix, which is then used as the first similarity loss.

[0070] For example, the second similarity formula can be expressed in the following form:

[0071]

[0072] in, This represents the similarity between the i-th element in the image feature and the j-th element in the categorical attribute vector. Here, the image feature x can be represented as... i Converting to a category representation can be achieved through methods such as clustering.

[0073] In an optional embodiment, a similarity matrix (equivalent to a second similarity) is determined based on the similarity between each element in the image features and each element in the categorical attribute vector. Then, the complement number of each element in the similarity matrix is ​​taken to obtain a second similarity loss matrix, which is then used as the second similarity loss.

[0074] It should be noted that by classifying the numerical and categorical attributes in the sample images and sample tables and introducing different similarity measurement mechanisms, the accuracy and robustness of the model in cross-modal data conversion are effectively improved.

[0075] Optionally, in the image processing method provided in this application embodiment, the mutual information loss between the sample image and the numerical attributes and categorical attributes in the sample table is determined by the feature distribution alignment module, including: calculating the mutual information loss between the sample image features of the sample image and the numerical attribute features of the sample table to obtain a first mutual information loss; and calculating the mutual information loss between the sample image features and the categorical attribute features of the sample table to obtain a second mutual information loss.

[0076] Optionally, the joint distribution between image features and table features can be optimized by maximizing mutual information (MI) to maximize information sharing between the two while preserving their respective feature space characteristics.

[0077] During training, the goal is to maximize mutual information, that is... Where θ represents the model parameters. For example, through -MI(X;Y) mum ) Calculate the mutual information loss between the sample image features and the numerical attribute features of the sample table to obtain the first mutual information loss, and then use -MI(X;Y) to calculate the mutual information loss between the sample image features and the numerical attribute features of the sample table. catCalculate the mutual information loss between the sample image features and the categorical attribute features of the sample table to obtain the second mutual information loss.

[0078] It should be noted that by introducing the concept of mutual information loss, a quantitative assessment of the consistency of information between sample image features and numerical and categorical attribute features in the sample table is achieved. This method not only improves the model's information retention ability during feature transformation but also ensures that the model can accurately understand and transform different types of attribute information, thus performing excellently in cross-modal data processing.

[0079] Optionally, in the image processing method provided in this application embodiment, the alignment loss between the sample image and the sample table is determined by the alignment relationship optimization module, including: determining a cost matrix based on the similarity loss, wherein the cost matrix represents the cost of aligning the sample image and the sample table; solving the target alignment function based on the cost matrix with the objective of minimizing the function value of the target alignment function to obtain the transfer matrix, wherein the target alignment function is determined based on the cost matrix and the transfer matrix, and the transfer matrix represents the matching weight between the sample image and the sample table; and determining the alignment loss based on the target alignment function, the transfer matrix, and the cost matrix.

[0080] After completing multi-scale similarity measurement and feature distribution alignment, a transfer optimization strategy can be employed to further adjust the alignment relationship between image channels and table attributes. The cost matrix is ​​a two-dimensional array that measures the "cost" required to align sample image features with sample table attributes. Here, "cost" refers to the expense incurred when the model maps image features to specific table attributes due to insufficient similarity between features and attributes.

[0081] In an optional embodiment, the first similarity loss matrix (e.g., an M*N matrix) and the second similarity loss matrix (e.g., an M*P matrix) can be directly merged to obtain the target similarity loss matrix (e.g., an M*(N+P) matrix), thereby determining the target similarity loss matrix as the cost matrix. The element in the i-th row and j-th column of the cost matrix represents the cost required to align the i-th element in the image features with the j-th element in the table attribute features (including numerical attribute features and categorical attribute features).

[0082] Optionally, the minimized target alignment function can be expressed as follows:

[0083]

[0084] in,<C,T> Let C be the target alignment function, C be the cost matrix, T be the transfer matrix, T be the term to be solved, and Π(μ,v) be the set of all possible migrations, satisfying the edge distribution constraints μ and v. μ is the edge distribution on one side of the image, usually set as the uniform weight vector of each image patch, and v is the edge distribution on one side of the table, usually set as the uniform weight vector of each attribute.

[0085] The objective is to minimize the function value of the target alignment function. The target alignment function is solved based on the cost matrix to obtain the transfer matrix. The element in the i-th row and j-th column of the transfer matrix represents how much "quality" the i-th element in the image feature is assigned to the j-th element in the table attribute features (including numerical and categorical attribute features). In other words, the element in the i-th row and j-th column of the transfer matrix represents the matching weight between the i-th element in the image feature and the j-th element in the table attribute features (including numerical and categorical attribute features).

[0086] After obtaining the cost matrix, the transfer matrix and the cost matrix are substituted into the target alignment function to obtain the alignment loss.

[0087] In an optional embodiment, the solved transfer matrix can also be used as explicit attention supervision. That is, the transfer matrix is ​​added to the cross-attention in the decoder: L_attn = KL(softmax(QK) T / √d)||T), where L_attn is the attention alignment loss during training, Q represents the query vector of the table token in the decoder, K represents the key vector of the image token, d represents the vector dimension, and √d is used for scaling to prevent softmax saturation. This forces the attention distribution of the decoder to be close to the optimal allocation, thereby accurately projecting image information to the corresponding attributes.

[0088] It should be noted that by constructing and optimizing the cost matrix and transfer matrix, effective control over the alignment relationship between sample image features and sample table attributes is achieved. This can improve the model training effect.

[0089] Optionally, in the image processing method provided in this application embodiment, the target loss is determined based on similarity loss, mutual information loss, alignment loss, and generation loss, including: obtaining the loss calculation weights corresponding to the similarity measurement module, feature distribution alignment module, alignment relationship optimization module, and decoder respectively; and calculating the target loss based on the loss calculation weights corresponding to the similarity measurement module, feature distribution alignment module, alignment relationship optimization module, and decoder respectively, as well as the similarity loss, mutual information loss, alignment loss, and generation loss.

[0090] Optionally, the loss calculation weights corresponding to the similarity measurement module, feature distribution alignment module, alignment relationship optimization module, and decoder can be preset.

[0091] The target loss can be a weighted sum of the losses of each module according to their respective loss calculation weights, and it directly guides the updating and optimization of model parameters. For example, multiplying the loss calculation weight of the similarity measurement module by its corresponding similarity loss yields the corresponding weighted similarity loss. The same applies to other modules, so it will not be elaborated here.

[0092] Optionally, the first similarity loss and the second similarity loss can be additive, and the first mutual information loss and the second mutual information loss can also be additive.

[0093] It should be noted that the above method further improves the accuracy of the calculated target loss.

[0094] Optionally, in the image processing method provided in this application embodiment, the target image is processed by an image processing model to obtain a target table corresponding to the image content of the target image, including: performing feature transformation on the target image through a similarity measurement module in the image processing model to obtain target image features; and processing the target image features through a decoder in the image processing model to obtain the target table.

[0095] In practical applications, the feature distribution alignment module and alignment relationship optimization module do not participate in feature processing. That is, the similarity measurement module directly performs feature transformation on the target image to obtain the target image features, which are then processed by the decoder to obtain the target table. The aforementioned decoder can also be called a generator.

[0096] Optionally, the resulting target table can be used for business processing or data statistics.

[0097] It should be noted that the above method effectively extracts information from the target image, improving the accuracy of the obtained target table.

[0098] Therefore, the method provided in this application achieves the goal of optimizing the model based on multi-scale similarity loss during the model training process and performing image-to-table conversion based on the trained model, thereby improving the accuracy of the generated table and solving the technical problem of low accuracy in generating tables based on image content in related technologies.

[0099] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0100] Example 2

[0101] This application also provides an image processing apparatus. It should be noted that the image processing apparatus of this application can be used to execute the image processing method provided in this application. The image processing apparatus provided in this application will be described below.

[0102] According to embodiments of this application, an apparatus for implementing the above-described image processing method is also provided, such as... Figure 3 As shown, the device includes:

[0103] The first acquisition module 301 is used to acquire the target image;

[0104] The first processing module 302 is used to process the target image through an image processing model to obtain a target table corresponding to the image content of the target image. During the training process of the image processing model, the model parameters are optimized based on training samples and target loss. The training samples include sample images and sample tables, and the target loss includes the similarity loss between the sample images and the numerical attributes and categorical attributes in the sample tables, respectively.

[0105] In this embodiment, the target loss for training the image processing model is designed to include the similarity loss between the sample image and the numerical and categorical attributes in the sample table, respectively. This achieves the optimization of the similarity loss based on multiple scales during the model training stage, taking into account the differences between numerical and categorical attributes. This ensures the robustness and generalization ability of the model when processing complex heterogeneous data. As a result, in the image processing stage of practical applications, the model can intelligently identify the structured data in the target image, generate a clear and complete spreadsheet, and improve the accuracy of the generated spreadsheet.

[0106] Therefore, the method provided in this application achieves the goal of optimizing the model based on multi-scale similarity loss during the model training process and performing image-to-table conversion based on the trained model, thereby improving the accuracy of the generated table and solving the technical problem of low accuracy in generating tables based on image content in related technologies.

[0107] Optionally, in the image processing apparatus provided in the embodiments of this application, the image processing apparatus further includes: a second acquisition module, used to acquire a training sample set and an initial image processing model, wherein the training sample set includes training samples, and the initial image processing model includes a similarity measurement module and a decoder; a first determination module, used to determine the similarity loss between the sample image and the numerical attribute and the categorical attribute in the sample table respectively through the similarity measurement module; a second determination module, used to process the sample image through the decoder to obtain a prediction table, and determine the generation loss between the prediction table and the sample table; and a second processing module, used to determine the target loss based on the similarity loss and the generation loss, and optimize the parameters of the initial image processing model based on the target loss until a preset iteration condition is met to obtain the image processing model.

[0108] Optionally, in the image processing apparatus provided in this application embodiment, the second processing module further includes: a first determining submodule, used to determine the mutual information loss between the sample image and the numerical attribute and the categorical attribute in the sample table respectively through the feature distribution alignment module; a second determining submodule, used to determine the alignment loss between the sample image and the sample table through the alignment relationship optimization module; and a third determining submodule, used to determine the target loss based on the similarity loss, mutual information loss, alignment loss and generation loss.

[0109] Optionally, in the image processing apparatus provided in this application embodiment, the first determining module further includes: a first processing submodule, used to perform feature transformation on the sample image to obtain image features; a second processing submodule, used to perform feature transformation on the numerical attributes in the sample table to obtain numerical attribute features, and to perform feature transformation on the categorical attributes in the sample table to obtain categorical attribute features; a first calculation submodule, used to calculate a first similarity between the image features and the numerical attribute features based on a first similarity formula, and to determine a first similarity loss based on the first similarity; and a second calculation submodule, used to calculate a second similarity between the image features and the categorical attribute features based on a second similarity formula, and to determine a second similarity loss based on the second similarity.

[0110] Optionally, in the image processing apparatus provided in the embodiments of this application, the first determining submodule further includes: a first calculation unit, used to calculate the mutual information loss between the sample image features of the sample image and the numerical attribute features of the sample table, to obtain a first mutual information loss; and a second calculation unit, used to calculate the mutual information loss between the sample image features and the categorical attribute features of the sample table, to obtain a second mutual information loss.

[0111] Optionally, in the image processing apparatus provided in this application embodiment, the second determining submodule further includes: a first determining unit, configured to determine a cost matrix based on similarity loss, wherein the cost matrix represents the cost of aligning a sample image with a sample table; a processing unit, configured to solve for a target alignment function based on the cost matrix with the objective of minimizing the function value of the target alignment function, thereby obtaining a transfer matrix, wherein the target alignment function is determined based on the cost matrix and the transfer matrix, and the transfer matrix represents the matching weight between the sample image and the sample table; and a second determining unit, configured to determine an alignment loss based on the target alignment function, the transfer matrix, and the cost matrix.

[0112] Optionally, in the image processing apparatus provided in this application embodiment, the third determining submodule further includes: an acquisition unit, used to acquire the loss calculation weights corresponding to the similarity measurement module, the feature distribution alignment module, the alignment relationship optimization module and the decoder respectively; and a third calculation unit, used to calculate the target loss based on the loss calculation weights corresponding to the similarity measurement module, the feature distribution alignment module, the alignment relationship optimization module and the decoder respectively, as well as the similarity loss, mutual information loss, alignment loss and generation loss.

[0113] Optionally, in the image processing apparatus provided in the embodiments of this application, the first processing module further includes: a third processing submodule, used to perform feature transformation on the target image through the similarity measurement module in the image processing model to obtain target image features; and a fourth processing submodule, used to process the target image features through the decoder in the image processing model to obtain a target table.

[0114] It should be noted that the first acquisition module 301 and the first processing module 302 mentioned above correspond to steps S201 to S202 in Embodiment 1. The two modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules or units can be hardware components or software components stored in memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n). The above modules can also be part of the device and run in the computer terminal 10 provided in Embodiment 1.

[0115] Example 3

[0116] Embodiments of this application may provide an electronic device. Figure 4 This is a structural block diagram of an electronic device according to an embodiment of this application. Figure 4 As shown, the electronic device may include: one or more ( Figure 4(Only one is shown) processor 1002, memory 1004, memory controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0117] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and apparatus in the embodiments of this application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the above-described methods. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0118] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: acquire the target image; process the target image through the image processing model to obtain the target table corresponding to the image content of the target image. In the training process of the image processing model, the model parameters are optimized based on the training samples and the target loss. The training samples include sample images and sample tables. The target loss includes the similarity loss between the sample images and the numerical attributes and categorical attributes in the sample tables, respectively.

[0119] The processor can also access information and applications stored in the memory via a transmission device to perform the following steps: acquiring a training sample set and an initial image processing model, wherein the training sample set includes training samples, and the initial image processing model includes a similarity measurement module and a decoder; determining the similarity loss between the sample image and the numerical and categorical attributes in the sample table through the similarity measurement module; processing the sample image through the decoder to obtain a prediction table, and determining the generation loss between the prediction table and the sample table; determining the target loss based on the similarity loss and the generation loss, and optimizing the parameters of the initial image processing model based on the target loss until the preset iteration conditions are met to obtain the image processing model.

[0120] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: determine the mutual information loss between the sample image and the numerical attributes and categorical attributes in the sample table through the feature distribution alignment module; determine the alignment loss between the sample image and the sample table through the alignment relationship optimization module; and determine the target loss based on the similarity loss, mutual information loss, alignment loss and generation loss.

[0121] The processor can also call the information and application program stored in the memory through the transmission device to perform the following steps: perform feature transformation on the sample image to obtain image features; perform feature transformation on the numerical attributes in the sample table to obtain numerical attribute features, and perform feature transformation on the categorical attributes in the sample table to obtain categorical attribute features; calculate the first similarity between the image features and the numerical attribute features based on the first similarity formula, and determine the first similarity loss based on the first similarity; calculate the second similarity between the image features and the categorical attribute features based on the second similarity formula, and determine the second similarity loss based on the second similarity.

[0122] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: calculate the mutual information loss between the sample image features of the sample image and the numerical attribute features of the sample table to obtain the first mutual information loss; calculate the mutual information loss between the sample image features and the categorical attribute features of the sample table to obtain the second mutual information loss.

[0123] The processor can also invoke information and applications stored in the memory via the transmission device to perform the following steps: determining a cost matrix based on similarity loss, wherein the cost matrix represents the cost of aligning a sample image with a sample table; solving for the target alignment function based on the cost matrix with the objective of minimizing the function value of the target alignment function to obtain a transmission matrix, wherein the target alignment function is determined based on the cost matrix and the transmission matrix, and the transmission matrix represents the matching weight between the sample image and the sample table; and determining the alignment loss based on the target alignment function, the transmission matrix, and the cost matrix.

[0124] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: obtain the loss calculation weights corresponding to the similarity measurement module, feature distribution alignment module, alignment relationship optimization module and decoder respectively; calculate the target loss based on the loss calculation weights corresponding to the similarity measurement module, feature distribution alignment module, alignment relationship optimization module and decoder respectively, as well as the similarity loss, mutual information loss, alignment loss and generation loss.

[0125] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: perform feature transformation on the target image through the similarity measurement module in the image processing model to obtain the target image features; process the target image features through the decoder in the image processing model to obtain the target table.

[0126] In this embodiment, the target loss for training the image processing model is designed to include the similarity loss between the sample image and the numerical and categorical attributes in the sample table, respectively. This achieves the optimization of the similarity loss based on multiple scales during the model training stage, taking into account the differences between numerical and categorical attributes. This ensures the robustness and generalization ability of the model when processing complex heterogeneous data. As a result, in the image processing stage of practical applications, the model can intelligently identify the structured data in the target image, generate a clear and complete spreadsheet, and improve the accuracy of the generated spreadsheet.

[0127] Therefore, the method provided in this application achieves the goal of optimizing the model based on multi-scale similarity loss during the model training process and performing image-to-table conversion based on the trained model, thereby improving the accuracy of the generated table and solving the technical problem of low accuracy in generating tables based on image content in related technologies.

[0128] Those skilled in the art will understand that Figure 4 The structure shown is for illustrative purposes only. Electronic devices can also be smartphones, tablets, handheld computers, mobile internet devices (MIDs), PADs, and other terminal devices. Figure 4 This does not limit the structure of the aforementioned electronic device. For example, electronic devices may also include components that are more... Figure 4 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 4 The different configurations shown.

[0129] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0130] Example 4

[0131] Embodiments of this application also provide a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the image processing method provided in Embodiment 1.

[0132] Optionally, in this embodiment, the storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0133] This application also provides a computer program product, which, when executed on a data processing device, is adapted to perform image processing method steps.

[0134] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0135] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0136] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0137] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0138] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0139] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0140] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. An image processing method, characterized in that, include: Acquire the target image; The target image is processed by an image processing model to obtain a target table corresponding to the image content of the target image. During the training process of the image processing model, the model parameters are optimized based on training samples and target loss. The training samples include sample images and sample tables. The target loss includes the similarity loss between the sample images and the numerical attributes and categorical attributes in the sample tables, respectively.

2. The method according to claim 1, characterized in that, The image processing model is obtained in the following way: Obtain a training sample set and an initial image processing model, wherein the training sample set includes the training samples, and the initial image processing model includes a similarity measurement module and a decoder; The similarity measurement module determines the similarity loss between the sample image and the numerical and categorical attributes in the sample table, respectively. The sample images are processed by a decoder to obtain a prediction table, and the generation loss between the prediction table and the sample table is determined. The target loss is determined based on the similarity loss and the generation loss, and the parameters of the initial image processing model are optimized based on the target loss until the preset iteration conditions are met, thus obtaining the image processing model.

3. The method according to claim 2, characterized in that, The initial image processing model further includes a feature distribution alignment module and an alignment relationship optimization module, wherein determining the target loss based on the similarity loss and the generation loss includes: The feature distribution alignment module determines the mutual information loss between the sample image and the numerical and categorical attributes in the sample table, respectively. The alignment loss between the sample image and the sample table is determined by the alignment optimization module. The target loss is determined based on the similarity loss, the mutual information loss, the alignment loss, and the generation loss.

4. The method according to claim 2, characterized in that, The similarity measurement module determines the similarity loss between the sample image and the numerical and categorical attributes in the sample table, including: The sample image is subjected to feature transformation to obtain image features; The numerical attributes in the sample table are transformed to obtain numerical attribute features, and the categorical attributes in the sample table are transformed to obtain categorical attribute features. The first similarity between the image features and the numerical attribute features is calculated based on the first similarity formula, and the first similarity loss is determined based on the first similarity. The second similarity between the image features and the categorical attribute features is calculated based on the second similarity formula, and the second similarity loss is determined based on the second similarity.

5. The method according to claim 3, characterized in that, The feature distribution alignment module determines the mutual information loss between the sample image and the numerical and categorical attributes in the sample table, including: Calculate the mutual information loss between the sample image features of the sample image and the numerical attribute features of the sample table to obtain the first mutual information loss; The mutual information loss between the sample image features and the categorical attribute features of the sample table is calculated to obtain the second mutual information loss.

6. The method according to claim 3, characterized in that, The alignment loss between the sample image and the sample table is determined by the alignment optimization module, including: A cost matrix is ​​determined based on the similarity loss, wherein the cost matrix represents the cost of aligning the sample image with the sample table; With the objective of minimizing the function value of the target alignment function, the target alignment function is solved based on the cost matrix to obtain the transfer matrix, wherein the target alignment function is determined based on the cost matrix and the transfer matrix, and the transfer matrix represents the matching weight between the sample image and the sample table; The alignment loss is determined based on the target alignment function, the transfer matrix, and the cost matrix.

7. The method according to claim 3, characterized in that, The target loss is determined based on the similarity loss, the mutual information loss, the alignment loss, and the generation loss, including: Obtain the loss calculation weights corresponding to the similarity measurement module, feature distribution alignment module, alignment relationship optimization module, and decoder, respectively; The target loss is calculated based on the loss calculation weights corresponding to the similarity measurement module, feature distribution alignment module, alignment relationship optimization module and decoder, as well as the similarity loss, mutual information loss, alignment loss and generation loss.

8. The method according to claim 2, characterized in that, The target image is processed using an image processing model to obtain a target table corresponding to the image content of the target image, including: The target image features are obtained by performing feature transformation on the target image through the similarity measurement module in the image processing model. The target image features are processed by the decoder in the image processing model to obtain the target table.

9. An image processing apparatus, characterized in that, include: The first acquisition module is used to acquire the target image; The first processing module is used to process the target image through an image processing model to obtain a target table corresponding to the image content of the target image. During the training process of the image processing model, the model parameters are optimized based on training samples and target loss. The training samples include sample images and sample tables, and the target loss includes the similarity loss between the sample images and numerical attributes and categorical attributes in the sample tables, respectively.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program, wherein, when the executable program is executed, it controls the device on which the computer-readable storage medium is located to perform the image processing method according to any one of claims 1 to 8.

11. An electronic device, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program, when running, performs the image processing method according to any one of claims 1 to 8.

Images