A Selected Method for Chinese and Graph Matching in Artificial Intelligence
By calculating the text word vector and projecting it into the image space, and selecting the image corresponding to the projection vector with the shortest distance as the output, the problem of low matching between text and image in the prior art is solved, and accurate matching between text and image is achieved.
Patent Information
- Application Number
- CN202410713126.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-04
- Publication Date
- 2025-03-11
- Estimated Expiration
- 2044-06-04
AI Technical Summary
When converting text into images, it is difficult to accurately capture the full meaning of the text, resulting in the low matching degree of the generated image with the text.
By obtaining text information, the text word vector is calculated, and multiple images are generated. Convert the image into a design matrix, calculate the projection matrix, and project the text word vector into the image space to obtain the projection vector. Select the image corresponding to the projection vector of the shortest distance as the final output.
The precise text-to-image matching is achieved to ensure that the generated image best matches the semantic features of the original text.
Smart Images

Figure CN118674801B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image-text matching, and in particular to a method for selecting Chinese and image matching in artificial intelligence. Background Art
[0002] In the field of artificial intelligence, large language models have made significant progress, capable of performing tasks from text input to text output, and ensuring the accuracy and coherence of the output results. These models learn the grammar, semantics, and contextual relationships of the language by training massive amounts of text data, so that they can generate high-quality text responses. However, relying solely on text output is not enough to meet people's expectations for artificial intelligence, because information in the real world includes not only text, but also multimodal data such as images and videos.
[0003] In order to extend the capabilities of large language models to the field of image generation, researchers proposed a new method that combines deep learning image generation models, such as generative adversarial networks (GANs), variational autoencoders (VAEs), and Stable Diffusion, and uses the text output of large language models as the input of these image generation models. This method takes advantage of the powerful language understanding and generation capabilities of large language models, and also takes advantage of the advantages of image generation models in the visual field to achieve cross-modal conversion from text to image.
[0004] Specifically, the input text description is first processed using a large language model to generate a high-quality text output that contains a detailed description of the image content, style, composition, etc. Then, this text output is used as the input of the image generation model. Through the learning and generation process of the model, an image corresponding to the text description is finally obtained. The advantage of this method is that it can make full use of the semantic information in the text and generate images that are highly relevant to the text content, rather than just randomly generated images.
[0005] However, there are some differences between text descriptions and images. Text descriptions are usually more abstract and indirect, while images are more intuitive and specific. A text description may correspond to multiple different image representations, because different people may have different understandings and imaginations of the same text. Therefore, in practical applications, this method can generate multiple images based on text descriptions.
[0006] Of course, if you need to generate a specific image, you can also control the output of the image generation model by setting the corresponding parameters. For example, you can specify the style, color, composition and other attributes of the image to generate an image that meets specific requirements. However, this approach may lose some details and richness in the text description and fail to accurately capture the full meaning of the text. Summary of the invention
[0007] In view of this, the present invention provides a selection method for matching Chinese and images in artificial intelligence. On the basis of generating multiple images, according to the high correlation between text and image, an image that best matches the text is selected as the final single output. This method effectively achieves accurate matching from text to image.
[0008] The technical purpose of the present invention is achieved in this way:
[0009] The present invention provides a method for selecting Chinese characters and images in artificial intelligence matching, comprising the following steps:
[0010] S1 obtains text information and calculates the text word vector of the text information, where the dimension of the text word vector is N;
[0011] S2 generates multiple images based on the text information and numbers the images;
[0012] S3 converts the image into a design matrix, each design matrix is in the form of N×Q, calculates a projection matrix based on the design matrix, projects the text word vector into the image space based on the projection matrix, obtains the projection vector, and calculates the distance between each projection vector and the text word vector;
[0013] S4 selects the projection vector with the shortest distance and outputs the corresponding image as the matching image.
[0014] Based on the above solution, preferably, step S3 includes:
[0015] S31 sets two temporary variables tmpd and tmpnum, which are used to store the currently calculated shortest distance and the image number corresponding to the distance respectively;
[0016] S32 selects an image;
[0017] S33 checks the resolution of the current image. If the resolution does not meet the requirements, the image is skipped and the process returns to step S32. If the resolution meets the requirements, step S34 is executed.
[0018] S34 calculates the Q value by dimensionality calculation;
[0019] S35 converts the current image into grayscale format;
[0020] S36 determines the construction method of the design matrix, and constructs the design matrix X according to the construction method according to the N value and the Q value. i ;
[0021] S37 uses the projection matrix calculation formula to calculate the projection matrix P of the current image i , based on the projection matrix P iCalculate the projection vector Y of the text word vector Y in the current image space i ;
[0022] S38 Calculate the distance d between the text word vector Y and the projection vector Y i If d i , if d i <tmpd or tmpd < 0, then assign the calculated d i and the current image serial number to the temporary variables tmpd and tmpnum respectively;
[0023] S39 Repeat steps S32 - S38 to traverse all images and obtain the final temporary variables tmpd and tmpnum. On the basis of the above solution, preferably, the dimension calculation formula is:
[0024] Q = (H × V) / N
[0025] In the formula, H and V are the number of pixels in the horizontal and vertical directions of the image respectively, and the value of N is the same as the dimension of the text word vector.
[0026] On the basis of the above solution, preferably, the rounding method is rounding down or rounding up, where:
[0027] Rounding down means truncating the part of the image that exceeds the Q value;
[0028] Rounding up means filling the part of the image that is less than the Q value with 0 or 255.
[0029] On the basis of the above solution, preferably, step S36 includes:
[0030] Determine the size of the design matrix, the number of its rows N is the same as the dimension of the text word vector, and the number of its columns is Q;
[0031] Select the construction method of the design matrix, including filling each pixel point of the image into the design matrix row by row from top to bottom and from left to right in sequence, or, filling each pixel point of the image into the design matrix row by row from bottom to top and from right to left in sequence;
[0032] Fill the pixel values of the image into the design matrix according to the construction method;
[0033] Process the pixels that exceed the size of the design matrix. If the Q value is obtained by rounding down, truncate the pixels that exceed the size of the design matrix. If the Q value is obtained by rounding up, after filling all the pixels, fill the remaining part of the design matrix with 0 or 255.
[0034] On the basis of the above solution, preferably, the projection matrix P i The calculation formula of is:
[0035]
[0036] Where, X i is the design matrix.
[0037] Based on the above solution, preferably, the projection vector Y i The calculation formula is:
[0038] Y i =P i Y
[0039] Where P i is the projection matrix, and Y is the text word vector.
[0040] Based on the above solution, preferably, the text word vector Y and the projection vector Y i The distance d i The calculation formula is:
[0041]
[0042] In the formula, y k and ik They represent the coordinate values of the text word vector Y and the projection vector Yi in the kth dimension respectively, and N is the dimension of the text word vector.
[0043] Based on the above scheme, preferably, the dimension of the text word vector is 768, 1024 or 1536.
[0044] Based on the above solution, preferably, in step S35, after converting the current image into a grayscale format, each pixel is normalized and the value of the pixel is normalized to between 0 and 1.
[0045] The method of the present invention has the following beneficial effects compared with the prior art:
[0046] The present invention first generates a series of images from text, and then uses these images to construct a projection matrix to generate projection vectors corresponding to the text. By comparing the proximity between these projection vectors and the original text, the image with the highest correlation with the original text is selected. This process achieves the goal of accurately selecting the image that best represents the semantics of the original text in the text-to-image generation model. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0048] Figure 1 is a flow chart of a method according to an embodiment of the present invention;
[0049] Figure 2 It is a projection schematic diagram of an embodiment of the present invention;
[0050] Figure 3 It is a technical implementation diagram of an embodiment of the present invention;
[0051] Figure 4 This is a diagram of the implementation results of an embodiment of the present invention. DETAILED DESCRIPTION
[0052] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0053] like Figure 1 As shown, the present invention provides a method for selecting Chinese and image matching in artificial intelligence, comprising the following steps:
[0054] S1 obtains text information and calculates the text word vector of the text information, where the dimension of the text word vector is N;
[0055] S2 generates multiple images based on the text information and numbers the images;
[0056] S3 converts the image into a design matrix, each design matrix is in the form of N×Q, calculates a projection matrix based on the design matrix, projects the text word vector into the image space based on the projection matrix, obtains the projection vector, and calculates the distance between each projection vector and the text word vector;
[0057] S4 selects the projection vector with the shortest distance and outputs the corresponding image as the matching image.
[0058] Specifically, the present invention explores the strength of the correlation between text and image by evaluating the projection of text on the image, rather than attempting to establish a direct prediction model, thus effectively revealing the connection between text and image even in the face of complex nonlinear relationships.
[0059] In one embodiment of the present invention, the text data is first mapped to the space created by the image to obtain its mapping data in the space. Then, by comparing the distance or similarity between these mapping data and the original text data, multiple mapping results can be obtained for the same text in the space constructed by multiple images. Finally, the mapping result with the closest distance or the highest similarity to the original text is selected, and the corresponding image is regarded as the image most relevant to the original text. This method can effectively identify the image most relevant to a specific text from a large number of images.
[0060] For example, let the original text be Y, and the two images generated by Y be X1 and X2. Further assume that the projections of Y in the image spaces represented by X1 and X2 are Y1 and Y2 respectively. By comparison, if it is found that the degree of closeness between Y1 and Y is higher than that between Y2 and Y, it can be inferred that in the image generated by Y, the correlation between X1 and the original text Y is higher than that between X2. Figure 2 shown.
[0061] In short, by comparing how close the original text is projected in the space of different images it generates, we are able to assess which images have a higher relevance to the original text.
[0062] Specifically, in one embodiment of the present invention, the text is first converted into a word vector Y, which represents the representation of the text information in a certain vector space. The dimension of the word vector is usually set to N, where typical values of N include 768, 1024 or 1536. The selection of these dimensions is mainly related to the design of large language models. In the present invention, N dimension 768 is taken as an example.
[0063] Then, the image generated by the text is converted into a design matrix, each design matrix is in the form of N×Q. The conversion method first includes converting the image into a grayscale image, and then normalizing the value of each pixel point, each value corresponds to an element of the matrix, and filling it into the N×Q format in the order of characteristics. The Q value here is directly related to the pixel resolution (H×V) of the generated image, that is, it is related to the dimension, and the dimension calculation formula is: Q=(H×V) / N. Since Q must be a positive integer, the excess part is processed by rounding down, or rounding up and filling the missing data with 0 or 255 to adapt to the matrix size. The minimum resolution of the generated image should exceed 128×128. The present invention takes the resolution of 256×256 as an example, and selects the interception method to determine the Q value, so Q=85. This processing method ensures that the image generated from the text can be effectively converted into a design matrix for subsequent analysis and processing. m pictures are generated by text, which can constitute m design matrices (X1, X2, ..., X m ).
[0064] Next, the projection matrix is obtained by designing the matrix. For m design matrices, m corresponding projection matrices (P1, P2, ..., P m ), each projection matrix is in the form of N×N, and its calculation formula is as follows:
[0065]
[0066] Where i=1~m.
[0067] Then, the projection of the original word vector in the design matrix is calculated to obtain m different projection matrices. For each projection matrix, a corresponding projection vector is obtained. These projection vectors are all N-dimensional and keep the same dimension as the original word vector Y. In this way, through m different projection matrices, m N-dimensional projection vectors (Y1, Y2, ..., Y m ). Its calculation formula is as follows:
[0068] Y i =P i Y
[0069] Where i=1~m.
[0070] Then compare the word vector Y with each projection vector Y i The distance or similarity between them, the present invention chooses to calculate the distance between them, that is, the Euclidean distance calculation method. The calculation formula is as follows:
[0071]
[0072] Among them, d i Represents word vector Y and projection vector Y i The Euclidean distance between k and ik They represent the coordinate values of the text word vector Y and the projection vector Yi in the kth dimension respectively, and N is the dimension of the text word vector.
[0073] Finally, by comparing the projection vectors Y i The distance between the original word vector Y and the word vector with the smallest distance d is selected. i The corresponding image is taken as the generated image that best matches the text. Image design matrix X i After a specific projection matrix P i The projection vector obtained after projection is highly consistent with the original text information. This consistency highlights the i Compared with other projection matrices, it has higher efficiency and effectiveness in extracting image features that are highly relevant to this paragraph of text information.
[0074] In the text-to-image generation model, the comparison of the proximity between the original word vector corresponding to the text and its projection vector in the generated image can be used as an important indicator to measure the accuracy of the generated image. The higher this proximity, the higher the degree of match between the image generated by the text and the original text content, which indirectly indicates that the accuracy of the generated image is higher. Therefore, this indicator can be used to verify the performance of the text-to-image generation model. Specifically, a threshold can be set to define the standard of proximity, and this standard can be used to guide the further training of the text-to-image generation model. In this way, the model can adjust its parameters according to the proximity between the generated image and the original text to improve the accuracy of the generated image. In this way, not only can the performance of the model be effectively evaluated and optimized, but also the quality and relevance of the generated image can be improved in practical applications.
[0075] Specifically, in one embodiment of the present invention, step S3 includes:
[0076] S31 sets two temporary variables tmpd and tmpnum, which are used to store the currently calculated shortest distance and the image number corresponding to the distance respectively.
[0077] S32 selects an image.
[0078] S33 checks the resolution of the current image. If the resolution does not meet the requirements, the image is skipped and the process returns to step S32. If the resolution meets the requirements, step S34 is executed.
[0079] S34 calculates the Q value using the dimension calculation formula and rounding method; the rounding method is rounding down or rounding up, where:
[0080] Rounding down means truncating the part of the image that exceeds the Q value;
[0081] Rounding up means filling the part of the image that is less than the Q value with 0 or 255.
[0082] S35 converts the current image into a grayscale format and normalizes each pixel point to a value between 0 and 1.
[0083] S36 determines the construction method of the design matrix, and constructs the design matrix X according to the construction method according to the N value and the Q value. i ; Specifically include:
[0084] Determine the size of the design matrix, whose number of rows N is the same as the dimension of the text word vector, and whose number of columns is Q;
[0085] Select the construction method of the design matrix, including filling each pixel of the image into the design matrix row by row from top to bottom and from left to right in sequence, or filling each pixel of the image into the design matrix row by row from bottom to top and from right to left in sequence;
[0086] Fill the pixel values of the image into the design matrix according to the construction method;
[0087] Process the pixels exceeding the size of the design matrix. If the Q value is obtained by rounding down, truncate the pixels exceeding the size of the design matrix. If the Q value is obtained by rounding up, after filling all the pixels, fill the empty part of the design matrix with 0 or 255.
[0088] S37 Use the projection matrix calculation formula to calculate the projection matrix P of the current image i , based on the projection matrix P i Calculate the projection vector Y of the text word vector Y in the current image space i .
[0089] S38 Calculate the distance d between the text word vector Y and the projection vector Y i If d i , if d i <tmpd or tmpd < 0, then assign the calculated d i and the current image serial number to the temporary variables tmpd and tmpnum respectively.
[0090] S39 Repeat steps S32 - S38 to traverse all images and obtain the final temporary variables tmpd and tmpnum.
[0091] Please refer to Figure 3 , and illustrate the implementation process of the present invention with a specific embodiment:
[0092] Step 1: Obtain text information. First, extract the required text information from the text - to - image generation model. These text information are the inputs of the model and are used for the subsequent image generation process. After completion, proceed to Step 2.
[0093] Step 2: Calculate the text word vector Y. Use the extracted text information to calculate the word vector Y corresponding to the text. In this embodiment, Y is a one - dimensional vector with 768 dimensions, which will be used as the key basis for evaluating image relevance in the subsequent steps. After completion, proceed to Step 3.
[0094] Step 3: Get all images generated by the text. In this step, the number of images generated is set by adjusting the picture output control parameters in the text-to-image generation model. In this embodiment, 24 images are set to be generated. It should be noted that the more the number of generated images is set, the longer it takes to select the best image. However, in order to ensure effective image selection, the number of generated images should not be set to 1, because this will make the subsequent selection process impossible. After completion, proceed to step 4.
[0095] Step 4: Number all images. In the generated image collection, assign a unique serial number to each image starting from 1 so that each image can be easily identified and referenced. Once completed, proceed to step 5.
[0096] Step 5: Set tmpd = -1, tmpnum = -1. Set two temporary variables tmpd and tmpnum to store the currently calculated shortest distance and the image number corresponding to the distance. Initialize tmpd to -1, indicating that no distance has been calculated yet; similarly, set tmpnum to -1, indicating that there is currently no image corresponding to the shortest distance. After completion, proceed to step 6.
[0097] Step 6: Select an image. Select an image from the generated image set for processing, including its sequence number and the image itself. If it is the first selection, select the image with sequence number 1; if it is not the first selection, select the image with the next sequence number to continue processing. After completion, proceed to step 7.
[0098] Step 7: Check the resolution. Check whether the resolution of the currently selected image is higher than 128×128. This is because low-resolution images may cause the dimensions of the constructed design matrix Q to be insufficient, which in turn affects the dimensions of the image eigenvalues extracted from the projection matrix. If the image resolution meets the requirements, proceed to step 8; otherwise, the image will not be used for subsequent calculations and will proceed to step 16 for processing.
[0099] Step 8: Calculate Q. In this step, the Q value is calculated by using a formula determined by the resolution of the image, and the rounding down method is used, which means that a small part of the image is truncated in the design matrix constructed based on the current image. You can also choose to round up and fill the insufficient part with 0 or 255 to construct the design matrix. This embodiment selects the rounding down calculation method. After completion, proceed to step 9.
[0100] Step 9: Grayscale processing. Since the generated image is an RGB color image, to simplify the complexity of image processing and reduce the computational amount, and at the same time facilitate more accurate extraction of image features such as edges and textures, the image is converted to grayscale format. Color information may affect the recognition of these features. After grayscaling, each pixel is normalized, and the value of each pixel is between 0 and 1. After completion, proceed to Step 10.
[0101] Step 10: Construct design matrix X i . This step generates the design matrix from the grayscaled image. There are various ways to construct the matrix, including filling the design matrix sequentially from each pixel of the image in ways such as row by row from top to bottom and from left to right, and row by row from bottom to top and from right to left. In these methods, each pixel corresponds to an element in the design matrix. After determining the specific way to construct the matrix, the design matrices of all subsequent images need to be constructed in this way to ensure the unity and standardization of the processing flow. The size of the design matrix is N×Q. If the floor function is used to calculate Q, the pixels exceeding the size of the design matrix will be truncated; if the ceiling function is used, after all images are filled into the design matrix, the insufficient part will be filled with 0 or 255 to complete the construction of the design matrix. After completion, proceed to Step 11.
[0102] Step 11: Calculate projection matrix P i . Calculate the corresponding projection matrix P for each image by applying the formula of the projection matrix i . The obtained projection matrix is a square matrix of N×N. After completion, proceed to Step 12.
[0103] Step 12: Calculate projection vector Y i . Calculate the projection vector Y of the word vector Y in the current image space using a specific formula i . The dimension of this projection vector is the same as that of the word vector Y, both being N. After completion, proceed to Step 13.
[0104] Step 13: Calculate distance d i . Calculate d using the Euclidean distance formula i , that is, calculate the distance between the two by comparing the differences of the corresponding elements in Y and Y i . After completion, proceed to Step 14.
[0105] Step 14: d i <tmpd or tmpd < 0. If d i is less than tmpd, this indicates that the current image performs better in reflecting the semantic features of the text, so the serial number of this image needs to be recorded. If tmpd is less than 0, this usually occurs when calculating d for the first time iIn short, if any of the above conditions is met, proceed to step 15; if not, proceed to step 16.
[0106] Step 15: tmpd = d i , tmpnum = image number. i and the current image serial number are assigned to temporary variables tmpd and tmpnum respectively. After completion, proceed to step 16.
[0107] Step 16: Check whether all images have been traversed. The purpose of this step is to confirm whether the image set has been completely traversed. If the traversal is complete, it means that all calculations have been completed and an image closest to the target has been selected, and its index number is saved in the variable tmpnum. At this point, you should go to step 17. If the traversal is not complete, you need to go back to step 6 and continue to traverse the next image.
[0108] Step 17: Get the corresponding image according to tmpnum. In this step, the image number saved in tmpnum is used to retrieve and find the corresponding image in the image collection. This image is considered to be the image that best represents the required text content after the selection process.
[0109] Specifically, the present invention provides an implementation case to illustrate the above process:
[0110] In this example, a large language model for agricultural question answering was used to process the query "How to prevent rapeseed clubroot disease". The model output was "Spraying quicklime on the land can prevent rapeseed clubroot disease." A text-to-image generation model was further used to take the sentence "Spraying quicklime on the land can prevent rapeseed clubroot disease" as input and generate two images as examples. Figure 4 Picture 1 and Picture 2 in the.
[0111] In order to evaluate the correlation between the two images and the input text, the images were first grayscaled and normalized, and then the design matrix was constructed and the projection matrix was calculated. Next, the projection vectors of the word vectors of the input text on the two images were calculated. By calculating the Euclidean distance between these projection vectors and the original word vectors, the distance between the word vector and the projection of image 1 is 8.3114, and the distance between the word vector and the projection of image 2 is 7.9234.
[0112] Based on these calculation results, it is concluded that the correlation between Image 2 and the input text "Spraying quicklime on the land can prevent clubroot in rapeseed" is higher. This indicates that Image 2 more accurately represents the given text content.
[0113] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for matching Chinese characters with images in artificial intelligence, characterized in that: The steps include: S1 obtains text information and calculates the text word vector of the text information, where the dimension of the text word vector is N; S2 generates multiple images based on the text information and numbers the images; S3 converts the image into a design matrix, each design matrix is in the form of N×Q, calculates a projection matrix based on the design matrix, projects the text word vector into the image space based on the projection matrix, obtains the projection vector, and calculates the distance between each projection vector and the text word vector; S4 selects the projection vector with the shortest distance and outputs its corresponding image as a matching image; Step S3 includes: S31 sets two temporary variables tmpd and tmpnum, which are used to store the currently calculated shortest distance and the image number corresponding to the distance respectively; S32 selects an image; S33 checks the resolution of the current image. If the resolution does not meet the requirements, the image is skipped and the process returns to step S32. If the resolution meets the requirements, step S34 is executed. S34 calculates the Q value by using the dimension calculation formula and rounding method; S35 converts the current image into grayscale format; S36 determines the construction method of the design matrix, and constructs the design matrix X according to the construction method according to the N value and the Q value. i ; S37 uses the projection matrix calculation formula to calculate the projection matrix P of the current image i , based on the projection matrix P i Calculate the projection vector Y of the text word vector Y in the current image space i ; S38 Calculate the distance d between the text word vector Y and the projection vector Y i of d i , if d i <tmpd or tmpd < 0, then assign the calculated d i and the current image sequence number to the temporary variables tmpd and tmpnum respectively; S39 repeats steps S32-S38, traverses all images, and obtains the final temporary variables tmpd and tmpnum; The dimension calculation formula is: Q=(H×V) / N In the formula, H and V are the number of pixels in the horizontal and vertical directions of the image respectively, and the value of N is the same as the dimension of the text word vector; The rounding method is floor or ceiling, where: Rounding down means truncating the part of the image that exceeds the Q value; Rounding up means filling the part of the image that is less than the Q value with 0 or 255; Step S36 includes: Determine the size of the design matrix, whose number of rows N is the same as the dimension of the text word vector, and whose number of columns is Q; Selecting a construction method of the design matrix includes filling the design matrix with each pixel point of the image row by row from top to bottom and from left to right, or filling the design matrix with each pixel point of the image row by row from bottom to top and from right to left; Fill the pixel values of the image into the design matrix according to the construction method; When processing pixels that exceed the design matrix size, if the Q value is obtained by rounding down, the pixels that exceed the design matrix size will be truncated. If the Q value is obtained by rounding up, after all pixels are filled, the remaining part of the design matrix will be filled with 0 or 255.
2. A method for selecting Chinese and image matching in artificial intelligence as claimed in claim 1, characterized in that: Projection Matrix P i The calculation formula is: Where, X i is the design matrix.
3. A method for selecting Chinese and image matching in artificial intelligence as claimed in claim 2, characterized in that: Projection vector Y i The calculation formula is: AND i =P i AND Where P i is the projection matrix, and Y is the text word vector.
4. A method for selecting Chinese and image matching in artificial intelligence as claimed in claim 1, characterized in that: Text word vector Y and projection vector Y i The distance d i The calculation formula is: In the formula, y k and ik They represent the coordinate values of the text word vector Y and the projection vector Yi in the kth dimension respectively, and N is the dimension of the text word vector.
5. A method for selecting Chinese and image matching in artificial intelligence as claimed in claim 1, characterized in that: The dimensions of text word vectors are 768, 1024, or 1536.
6. A method for selecting Chinese and image matching in artificial intelligence as claimed in claim 1, characterized in that: In step S35, after the current image is converted into a grayscale format, each pixel is normalized and the value of the pixel is normalized to between 0 and 1.
Citation Information
Patent Citations
Text correlation measurement method and system based on fine-grained matching signal
CN109753649A
An identified full-similarity preserving Hash cross-mode retrieval method
CN109766455A