Image processing method, device, electronic device and storage medium

Abstract

The present disclosure provides an image processing method, device, electronic device and storage medium, which relate to artificial intelligence fields such as computer vision, natural language processing, image retrieval and deep learning. The method may include: obtaining the image vector of the image in the database, grouping each image according to the image vector, and the images in the same group are the same image; for the obtained groups, respectively perform the following processing: using the original text descriptions corresponding to the images in the group to form a first text set, based on the text vectors of the original text descriptions in the first text set, filtering out the determined low-quality text descriptions from the first text set, and generating the text descriptions corresponding to the images in the group according to the remaining original text descriptions. Applying the scheme described in the present disclosure can achieve the supplementation of missing text descriptions and the improvement of low-quality text descriptions.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to image processing methods, devices, electronic devices, and storage media in the fields of computer vision, natural language processing, image retrieval, and deep learning. Background Art

[0002] In the image retrieval scenario, the text description of the image plays an indispensable role. The text description refers to some textual description information about the image. The text description can be used to assist in image retrieval, that is, to help the corresponding image be correctly retrieved. Summary of the Invention

[0003] The present disclosure provides an image processing method, an apparatus, an electronic device, and a storage medium.

[0004] A method for processing an image, comprising:

[0005] Obtaining picture vectors of pictures in a database, and grouping the pictures in the database according to the picture vectors, where pictures in the same group are the same pictures;

[0006] For the obtained groups, the following processing is performed respectively: the original text descriptions corresponding to the pictures in the group are used to form a first text set; based on the text vectors of the original text descriptions in the first text set, the determined low-quality text descriptions are filtered out from the first text set; and the text descriptions corresponding to the pictures in the group are generated based on the remaining original text descriptions.

[0007] An image processing device, comprising: an image grouping module and a text optimization module;

[0008] The picture grouping module is used to obtain picture vectors of pictures in the database, and group the pictures in the database according to the picture vectors, so that pictures in the same group are the same pictures;

[0009] The text optimization module is used to perform the following processing on the obtained groups: using the original text descriptions corresponding to the pictures in the group to form a first text set, filtering out the determined low-quality text descriptions from the first text set based on the text vectors of the original text descriptions in the first text set, and generating a text description corresponding to the pictures in the group based on the remaining original text descriptions.

[0010] An electronic device, comprising:

[0011] at least one processor; and

[0012] a memory communicatively connected to the at least one processor; wherein,

[0013] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method as described above.

[0014] A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the method as described above.

[0015] A computer program product comprises a computer program / instruction, which implements the above method when executed by a processor.

[0016] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0018] Figure 1 This is a flowchart of an embodiment of the image processing method disclosed herein;

[0019] Figure 2 Schematic diagram of the implementation process of the image processing method disclosed in the present invention;

[0020] Figure 3 Schematic diagram of the composition structure of the first embodiment 300 of the image processing device described in the present disclosure;

[0021] Figure 4 Schematic diagram of the structure of the second embodiment 400 of the image processing device described in the present disclosure;

[0022] Figure 5 FIG. 5 is a schematic block diagram of an electronic device 500 that can be used to implement an embodiment of the present disclosure. DETAILED DESCRIPTION

[0023] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0024] Furthermore, it should be understood that the term "and / or" as used herein simply describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " as used herein generally indicates that the associated objects are in an "or" relationship.

[0025] Figure 1 This is a flow chart of an embodiment of the image processing method disclosed in the present invention. Figure 1 As shown, the following specific implementation methods are included.

[0026] In step 101, the picture vectors of the pictures in the database are obtained, and the pictures in the database are grouped according to the picture vectors. Pictures in the same group are the same pictures.

[0027] In step 102, the following processing is performed on the obtained groups: the original text descriptions (text) corresponding to the pictures in the group are used to form a first text set, based on the text vectors of the original text descriptions in the first text set, the determined low-quality text descriptions are filtered out from the first text set, and the text descriptions corresponding to the pictures in the group are generated based on the remaining original text descriptions.

[0028] In the traditional method, when a user performs an image search, the corresponding text vector can be obtained for the query text entered by the user, and the similarity between the text vector and the image vector and text vector of each image in the database can be obtained respectively. Furthermore, the two similarities can be fused to determine the overall score of each image, that is, the "text-image" and "text-text" similarities can be fused to determine the overall score of each image, and then the search results can be returned based on the overall score of each image.

[0029] The database can store multiple images and their corresponding text descriptions. These images can originate from different web pages, and the text descriptions can refer to all or part of the content on the web page. Furthermore, an image vector is a vector generated based on the image itself, while an image text vector is a vector generated based on the text description of the image.

[0030] Since the images in the database come from different sources, some images may be missing text descriptions, which may result in such images being unable to be correctly retrieved, thus reducing the accuracy of the retrieval results.

[0031] By adopting the scheme described in the embodiment of the method disclosed herein, the images in the database can be identified and grouped as identical images, and the images in the same group can be transmitted with text descriptions. In this way, the missing text description of an image can be supplemented by the text description of an image that is visually identical to it. In addition, low-quality text descriptions can be filtered out and improved / optimized by high-quality text descriptions within the group, thereby increasing the possibility of these images being correctly retrieved, that is, improving the accuracy of the image retrieval results.

[0032] For each image in the database, its image vector can be obtained. There are no restrictions on how to obtain the image vector. For example, traditional implementation methods can be used, such as using a deep neural network model as an image encoder to encode each image into an image vector. The similarity between different image vectors can then be used to measure the similarity between different images.

[0033] Afterwards, the images in the database can be grouped according to the image vectors. Preferably, the images in the database can first be clustered according to the image vectors, with different images clustered to the closest cluster center. Each cluster center can then be processed as follows: for each image belonging to the cluster center, the similarity between its image vector and the image vectors of other images belonging to the cluster center is obtained. The images belonging to the cluster center can then be grouped according to the similarity, wherein the similarity between any two images in the same group is greater than a first threshold. The specific value of the first threshold can be determined according to actual needs.

[0034] For example, we can calculate the cosine similarity between image vectors pairwise, treating images with high similarity as identical and grouping them together. Identical images are transitive; that is, if images a and b are identical, and images b and c are identical, then images a and c are identical.

[0035] In the above processing method, different images can first be clustered to the nearest cluster center through clustering operation, and then the similarity of each image belonging to each cluster center can be calculated pairwise to determine and group the same images. Compared with the method of directly calculating the similarity of each image in the database pairwise, it can significantly reduce the amount of calculation, reduce resource consumption, and improve processing efficiency.

[0036] For example: Assume that there are 10,000 pictures in the database (these numbers are only for example and the actual number may be much larger than this), and these pictures belong to 100 different cluster centers. Then, for each picture belonging to each cluster center, it can be further divided into multiple groups. For example, if 100 pictures belong to the same cluster center, these 100 pictures can be further divided into 5 groups, and the pictures in each group are considered to be the same pictures.

[0037] For each group obtained, the original text descriptions corresponding to each image in the group can be used to form a first text set, and based on the text vectors of the original text descriptions in the first text set, the determined low-quality text descriptions can be filtered out from the first text set, and the text descriptions corresponding to the images in the group can be generated based on the remaining original text descriptions.

[0038] Preferably, before filtering out the determined low-quality text descriptions from the first text collection based on the text vectors of the original text descriptions in the first text collection, the original text descriptions in the first text collection may be deduplicated.

[0039] Since the pictures in the same group are the same pictures, the corresponding text descriptions may also be the same. In order to reduce the workload of subsequent processing, the original text descriptions in the first text set can be deduplicated, that is, for two or more completely identical text descriptions, only one of them can be retained.

[0040] Preferably, the method of filtering out the determined low-quality text descriptions from the first text set may include: clustering the text vectors of each original text description in the first text set, determining outliers based on the clustering results, and using the original text descriptions corresponding to the outliers as low-quality text descriptions.

[0041] There is no restriction on how to obtain the text vector. For example, a traditional implementation method can be adopted, such as using a deep neural network model as a text encoder, and using the text encoder to encode each text description into a text vector.

[0042] For each original text description in the first text set, after obtaining the corresponding text vector, each text vector can be clustered, and outliers, that is, points far away from the large cluster, can be determined based on the clustering results. Among them, the outliers can be identified using the currently available mature outlier identification method, and then the original text description corresponding to the outlier can be used as the required low-quality text description.

[0043] Alternatively, preferably, filtering out the determined low-quality text descriptions from the first text set may include: in response to determining that any original text description in the first text set meets the following condition: the similarity between a text vector of the original text description and an image vector of an image corresponding to the original text description is less than a second threshold, treating the original text description as a low-quality text description. The specific value of the second threshold may be determined based on actual needs.

[0044] In practical applications, there is no restriction on which of the above methods is used to determine low-quality text descriptions, and it can be determined according to actual needs. Preferably, the above two methods can be used at the same time. Among them, the former method is mainly suitable for screening out original text descriptions with poor text quality, such as incoherent sentences, etc., and the latter method is mainly suitable for screening out original text descriptions with "inconsistency between pictures and texts".

[0045] The selected original text descriptions can be filtered out from the first text set as low-quality text descriptions. By filtering, the negative impact of low-quality text descriptions on subsequent image retrieval can be avoided.

[0046] For the convenience of distinction, the first text set after filtering may be referred to as the second text set, and text descriptions corresponding to the pictures in the group may be generated based on the original text descriptions in the second text set.

[0047] Preferably, a secondary filtering process may be performed on the second text set. The secondary filtering process may include: for any two original text descriptions in the second text set whose content repetition rate exceeds a third threshold, deleting the duplicate content in one of the original text descriptions, and using the content in the second text set after the secondary filtering process as the common text description corresponding to the images in the group. The specific value of the third threshold may be determined according to actual needs.

[0048] In actual applications, it is possible that two original text descriptions are not exactly the same, but there is a large amount of repeated content, such as long sentences that are repeated, and only differ at the word or phrase level. These words and phrases may be synonyms, etc. Accordingly, word segmentation and other methods can be used to delete repeated content while retaining different word and phrase information in order to simplify the final generated text description, thereby facilitating subsequent retrieval operations and ensuring retrieval results.

[0049] The content in the second text set after the secondary filtering process can be used as the common text description corresponding to the pictures in the group. For example, assuming that a group includes 10 pictures, 8 of which have corresponding original text descriptions. After deduplication, 4 original text descriptions remain. Assuming that 2 of the remaining 4 original text descriptions are low-quality text descriptions, then these 2 low-quality text descriptions can be filtered out to obtain the remaining 2 original text descriptions. Assuming that there is a large amount of repeated content in the remaining 2 original text descriptions, then the repeated content in one of the original text descriptions can be deleted, and only the different content can be retained. Accordingly, one original text description and the retained different content can be finally obtained, and these contents can be used as the common text description corresponding to each picture in the group. These contents are all high-quality text descriptions. Accordingly, the missing original text descriptions of the pictures are supplemented and the low-quality text descriptions of the pictures are improved.

[0050] Based on the above introduction, Figure 2 Schematic diagram of the implementation process of the image processing method disclosed in this disclosure. Figure 2 As shown, the pictures in the database can be grouped, that is, the same pictures can be grouped, so as to obtain group 1, group 2 and group 3 shown in the figure. Taking group 3 as an example, in order to simplify the accompanying drawings, only 3 pictures are shown. The original text description of each picture (that is, text1, text2 and text3 shown in the figure) may be empty (that is, missing) or may not be empty. For group 3, the deduplication processing of the original text description, the filtering processing of low-quality text descriptions and the secondary filtering processing can be performed in the manner described in the present disclosure, so as to obtain the text description corresponding to each picture, that is, text123 shown in the figure. Subsequently, picture retrieval can be performed based on text123.

[0051] Preferably, when a user performs an image search, in response to determining that an image in any group is recalled, the group can be used as a target group, and the web page with the best quality can be selected from the web pages corresponding to the images in the target group, and the images corresponding to the web page with the best quality and the web page with the best quality can be returned to the user as search results, wherein the images in the same group come from different web pages.

[0052] When performing image retrieval, in addition to presenting the retrieved images to the user as search results, the image source information is usually also displayed, such as the subject and link of the corresponding web page. After the user clicks the link, the corresponding web page can be further displayed.

[0053] Assume that a group includes 10 pictures. Since these 10 pictures correspond to the same text description after being processed in accordance with the method described in the present disclosure, these 10 pictures will be recalled at the same time, but only one duplicate picture will be selected and returned to the user. Accordingly, this involves the problem of selecting multiple web pages containing the same picture, that is, selecting the web page with the best quality, thereby improving the accuracy of the search results and optimizing the user's search experience.

[0054] Preferably, the method of selecting the web page with the best quality from the web pages corresponding to the pictures in the target group may include: performing the following processing on each web page: obtaining the correlation score between the web page and the corresponding picture, and obtaining the authority score of the web page, determining the quality score of the web page based on the obtained correlation score and authority score, and selecting the web page with the highest quality score as the web page with the best quality.

[0055] The relevance and authority of the web page can be used as selection indicators, and the web page with the best quality can be selected by combining the two, thereby improving the accuracy of the selection results.

[0056] Preferably, for each web page, the similarity between the image vector of the image corresponding to the web page and the text vector of the original text description of the image corresponding to the web page, such as cosine similarity, can be obtained, and the relevance score of the web page can be determined based on the similarity, wherein the greater the similarity, the higher the relevance score can be. For example, the similarity can be directly used as the relevance score.

[0057] Through the above processing, the correlation between the web page and the corresponding image can be evaluated efficiently and accurately, thus laying a good foundation for subsequent processing.

[0058] Preferably, the text vectors of the original text descriptions corresponding to each image in the target group can be clustered. For any web page, the distance between the text vector corresponding to the web page and the cluster center obtained by clustering can be obtained, and the authority score of the web page can be determined based on the distance, wherein the closer the distance, the higher the authority score. For example, the authority score can be determined based on the distance according to a pre-set authority score generation rule.

[0059] The cluster center can be obtained by using the K-means clustering algorithm (K-means) or the mean.

[0060] That is, for a web page corresponding to any image in the target group, the authority of the web page can be measured by using the degree of clustering of the text vector corresponding to the web page among the text vectors corresponding to each image in the target group.

[0061] Preferably, the sum of the relevance score and the authority score may be obtained, and the obtained sum may be used as the quality score of the webpage.

[0062] It should be noted that, for the sake of simplicity, the aforementioned method embodiments are described as a series of action combinations. However, those skilled in the art should be aware that the present disclosure is not limited by the order of the actions described, because according to the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present disclosure.

[0063] In summary, by adopting the scheme described in the embodiment of the method disclosed herein, it is possible to supplement the missing text description of the image and improve the low-quality text description of the image, thereby increasing the possibility of the image being correctly retrieved, that is, improving the image retrieval effect. In addition, for different web pages corresponding to the recalled same image, the web page with the best quality can be selected and returned to the user, thereby further improving the retrieval effect.

[0064] The above is an introduction to the method embodiment. The following is a further explanation of the solution disclosed in the present disclosure through an apparatus embodiment.

[0065] Figure 3 FIG. 3 is a schematic diagram of the composition structure of the first embodiment 300 of the image processing device described in the present disclosure. Figure 3 As shown, it includes: a picture grouping module 301 and a text optimization module 302.

[0066] The picture grouping module 301 is used to obtain picture vectors of pictures in the database, and group the pictures in the database according to the picture vectors. Pictures in the same group are the same pictures.

[0067] The text optimization module 302 is used to perform the following processing on the obtained groups: using the original text descriptions corresponding to the pictures in the group to form a first text set, filtering out the determined low-quality text descriptions from the first text set based on the text vectors of the original text descriptions in the first text set, and generating a text description corresponding to the pictures in the group based on the remaining original text descriptions.

[0068] By adopting the scheme described in the above-mentioned device embodiment, the images in the database can be identified and grouped as identical images, and the images in the same group can transmit text descriptions. In this way, the missing text description of the image can be supplemented by the text description of the image that is visually identical to it. In addition, low-quality text descriptions can be filtered out and improved / optimized by high-quality text descriptions within the group, thereby increasing the possibility of these images being correctly retrieved, that is, improving the accuracy of the image retrieval results.

[0069] After the picture grouping module 301 obtains the picture vectors of each picture in the database, it can preferably first cluster the pictures in the database according to the picture vectors, and cluster different pictures to the nearest cluster center. Then, the following processing can be performed for each cluster center: for each picture belonging to the cluster center, the similarity between its picture vector and the picture vectors of other pictures belonging to the cluster center is obtained respectively, and then the pictures belonging to the cluster center can be grouped according to the similarity, wherein the similarity between any two pictures in the same group is greater than the first threshold.

[0070] For each group obtained, the text optimization module 302 can use the original text descriptions corresponding to each image in the group to form a first text set, and can filter out the determined low-quality text descriptions from the first text set based on the text vectors of the original text descriptions in the first text set, and generate a text description corresponding to the images in the group based on the remaining original text descriptions.

[0071] Preferably, the text optimization module 302 may first perform deduplication processing on the original text descriptions in the first text set before filtering out the determined low-quality text descriptions from the first text set based on the text vectors of the original text descriptions in the first text set.

[0072] Preferably, the text optimization module 302 filters out the determined low-quality text descriptions from the first text set, which may include: clustering the text vectors of the original text descriptions in the first text set, determining outliers based on the clustering results, and treating the original text descriptions corresponding to the outliers as low-quality text descriptions.

[0073] And / or, preferably, the way in which the text optimization module 302 filters out the determined low-quality text descriptions from the first text set may include: in response to determining that any original text description in the first text set meets the following conditions: the similarity between the text vector of the original text description and the image vector of the image corresponding to the original text description is less than a second threshold, treating the original text description as a low-quality text description.

[0074] Preferably, the text optimization module 302 can use the remaining original text descriptions to form a second text set, and can perform a secondary filtering process on the second text set. The secondary filtering process may include: for any two original text descriptions in the second text set whose content repetition rate is greater than a third threshold, deleting the repeated content in one of the original text descriptions, and using the content in the second text set after the secondary filtering process as the text description corresponding to the pictures in the group.

[0075] Figure 4FIG. 4 is a schematic diagram of the structure of the second embodiment 400 of the image processing device described in the present disclosure. Figure 4 As shown, it includes: a picture grouping module 301, a text optimization module 302 and a picture retrieval module 303.

[0076] Among them, the picture grouping module 301 and the text optimization module 302 are Figure 3 The same is true in the illustrated embodiment.

[0077] The image retrieval module 303 is used to, when a user performs an image retrieval, in response to determining that an image in any group is recalled, use the group as a target group, and select the web page with the best quality from the web pages corresponding to the images in the target group, and return the image corresponding to the web page with the best quality and the web page with the best quality as the retrieval results to the user, wherein the images in the same group come from different web pages.

[0078] Preferably, the image retrieval module 303 selects the web page with the best quality from the web pages corresponding to the images in the target group, which may include: performing the following processing on each web page: obtaining the correlation score between the web page and the corresponding image, and obtaining the authority score of the web page, determining the quality score of the web page based on the obtained correlation score and authority score, and selecting the web page with the highest quality score as the web page with the best quality.

[0079] Preferably, the image retrieval module 303 can obtain the similarity between the image vector of the image corresponding to the web page and the text vector of the original text description of the image corresponding to the web page, such as cosine similarity, for each web page, and can determine the relevance score of the web page based on the similarity, wherein the greater the similarity, the higher the relevance score can be.

[0080] Preferably, the image retrieval module 303 can cluster the text vectors of the original text descriptions corresponding to each image in the target group. For any web page, the distance between the text vector corresponding to the web page and the cluster center obtained by clustering can be obtained, and then the authority score of the web page can be determined based on the distance, wherein the closer the distance, the higher the authority score can be.

[0081] In addition, preferably, the image retrieval module 303 may obtain the sum of the relevance score and the authority score, and use the obtained sum as the quality score of the webpage.

[0082] Figure 3 and Figure 4 The specific working process of the device embodiment shown can refer to the relevant description in the aforementioned method embodiment and will not be repeated here.

[0083] In summary, by adopting the scheme described in the embodiment of the device disclosed herein, it is possible to supplement the missing text description of the image and improve the low-quality text description of the image, thereby increasing the possibility of the image being correctly retrieved, that is, improving the image retrieval effect. In addition, for different web pages corresponding to the recalled same image, the web page with the best quality can be selected and returned to the user, thereby further improving the retrieval effect.

[0084] The solutions described in this disclosure can be applied to the field of artificial intelligence, particularly in areas such as computer vision, natural language processing, image retrieval, and deep learning. Artificial intelligence is the study of how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It involves both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing. Artificial intelligence software technologies mainly include computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graph technologies.

[0085] In addition, the images and text descriptions in the embodiments of this disclosure are not targeted at any specific user and do not reflect the personal information of any specific user. The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solutions of this disclosure comply with relevant laws and regulations and do not violate public order and good morals.

[0086] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0087] Figure 5 A schematic block diagram of an electronic device 500 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0088] like Figure 5As shown, the device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. Various programs and data required for the operation of the device 500 can also be stored in the RAM 503. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0089] Various components in device 500 are connected to I / O interface 505, including: an input unit 506, such as a keyboard, mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a magnetic disk, optical disk, etc.; and a communication unit 509, such as a network card, modem, wireless communication transceiver, etc. The communication unit 509 allows device 500 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0090] The computing unit 501 can be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 501 performs the various methods and processes described above, such as the methods described in the present disclosure. For example, in some embodiments, the methods described in the present disclosure can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the method described in the present disclosure can be performed. Alternatively, in other embodiments, the computing unit 501 can be configured to perform the method described in the present disclosure in any other appropriate manner (e.g., by means of firmware).

[0091] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0092] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0093] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0094] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0095] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0096] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0097] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0098] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A method for processing an image, comprising: Obtaining image vectors of images in a database, clustering the images in the database according to the image vectors, clustering different images to the nearest cluster center, and performing the following processing on each cluster center: for each image belonging to the cluster center, obtaining the similarity between its image vector and the image vectors of other images belonging to the cluster center, and grouping the images belonging to the cluster center according to the similarity, wherein if the similarity between any two images in the same group is greater than a first threshold, the images in the same group are the same image; For the obtained groups, the following processing is performed respectively: using the original text descriptions corresponding to the pictures in the group to form a first text set, clustering the text vectors of the original text descriptions in the first text set, determining outliers based on the clustering results, determining the original text descriptions corresponding to the outliers as low-quality text descriptions and filtering them out, and, in response to determining that any original text description in the first text set meets the following conditions: the similarity between the text vector of the original text description and the picture vector of the picture corresponding to the original text description is less than a second threshold, determining the original text description as the low-quality text description and filtering it out; generating a text description corresponding to the pictures in the group based on the remaining original text descriptions; The method further includes: when a user performs an image search, the recalled group is used as a target group; for the web pages corresponding to each image in the target group, the following processing is performed respectively: a correlation score between the web page and the corresponding image is obtained, and the text vectors of the original text descriptions corresponding to the images in the target group are clustered, and the distance between the text vector corresponding to the web page and the cluster center obtained by clustering is obtained, and the authority score of the web page is determined according to the distance, the closer the distance between the text vector corresponding to the web page and the cluster center, the higher the authority score, and the quality score of the web page is determined according to the correlation score and the authority score, and the web page with the highest quality score and its corresponding image are returned to the user as a search result, wherein the images in the same group come from different web pages.

2. The method according to claim 1, further comprising: Before filtering out the low-quality text descriptions, duplicate removal is performed on the original text descriptions in the first text set.

3. The method according to claim 1, wherein Generating a text description corresponding to the pictures in the group according to the remaining original text descriptions includes: The remaining original text descriptions are used to form a second text set, and the second text set is subjected to a secondary filtering process, wherein the secondary filtering process includes: for any two original text descriptions in the second text set whose content repetition rate is greater than a third threshold, deleting the repetitive content in one of the original text descriptions; The content in the second text set after the secondary filtering process is used as the text description corresponding to the pictures in the group.

4. The method according to any one of claims 1 to 3, wherein Obtaining the correlation score between the web page and the corresponding image includes: obtaining the similarity between the image vector of the image corresponding to the web page and the text vector of the original text description of the image corresponding to the web page, and determining the correlation score based on the similarity, wherein the greater the similarity, the higher the correlation score.

5. The method according to any one of claims 1 to 3, wherein Determining the quality score of the webpage according to the relevance score and the authority score includes: obtaining a sum of the relevance score and the authority score, and using the obtained sum as the quality score.

6. An image processing device, comprising: Image grouping module, text optimization module and image retrieval module; The picture grouping module is configured to obtain picture vectors of pictures in a database, cluster the pictures in the database according to the picture vectors, cluster different pictures to the nearest cluster center, and perform the following processing on each cluster center: for each picture belonging to the cluster center, obtain the similarity between its picture vector and the picture vectors of other pictures belonging to the cluster center, and group the pictures belonging to the cluster center according to the similarity, wherein if the similarity between any two pictures in the same group is greater than a first threshold, the pictures in the same group are the same pictures; The text optimization module is configured to perform the following processing on the obtained groups: forming a first text set using the original text descriptions corresponding to the pictures in the group, clustering the text vectors of the original text descriptions in the first text set, determining outliers based on the clustering results, determining the original text descriptions corresponding to the outliers as low-quality text descriptions and filtering them out, and, in response to determining that any original text description in the first text set meets the following conditions: the similarity between the text vector of the original text description and the picture vector of the picture corresponding to the original text description is less than a second threshold, determining the original text description as the low-quality text description and filtering it out; and generating a text description corresponding to the pictures in the group based on the remaining original text descriptions; The image retrieval module is used to use the recalled group as the target group when the user performs an image retrieval; and perform the following processing on the web pages corresponding to each image in the target group: obtain the correlation score between the web page and the corresponding image, cluster the text vectors of the original text descriptions corresponding to the images in the target group, obtain the distance between the text vector corresponding to the web page and the cluster center obtained by clustering, and determine the authority score of the web page based on the distance. The closer the distance between the text vector corresponding to the web page and the cluster center, the higher the authority score. The quality score of the web page is determined based on the correlation score and the authority score, and the web page with the highest quality score and its corresponding image are returned to the user as retrieval results, wherein the images in the same group come from different web pages.

7. The device according to claim 6, wherein The text optimization module is further configured to perform deduplication processing on the original text descriptions in the first text set before filtering out the low-quality text descriptions.

8. The device according to claim 6, wherein The text optimization module uses the remaining original text descriptions to form a second text set and performs a secondary filtering process on the second text set. The secondary filtering process includes: for any two original text descriptions in the second text set whose content repetition rate is greater than a third threshold, deleting the repeated content in one of the original text descriptions, and using the content in the second text set after the secondary filtering process as the text description corresponding to the pictures in the group.

9. The device according to any one of claims 6 to 8, wherein: The image retrieval module obtains the similarity between the image vector of the image corresponding to the web page and the text vector of the original text description of the image corresponding to the web page, and determines the relevance score based on the similarity, wherein the greater the similarity, the higher the relevance score.

10. The device according to any one of claims 6 to 8, wherein: The image retrieval module obtains the sum of the relevance score and the authority score, and uses the obtained sum as the quality score.

11. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 5.

13. A computer program product comprising a computer program / instructions, wherein when the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images