A multi-functional data retrieval system and method

Abstract

This invention discloses a multifunctional data retrieval system and method, relating to the field of data retrieval technology. It includes a text data analysis module that analyzes keywords in user-uploaded text content and categorizes the text data according to keywords; an image data analysis module that saves images of the same category according to user-submitted images; and a neural network model that identifies subcategories of image categories by using CNN neural network technology as input. The neural network model is then used to analyze the subcategories of each image. By classifying images, the required training set of image data is significantly reduced during neural network training, thus reducing the computational power and time required for training.

Description

Technical Field

[0001] This invention belongs to the field of data retrieval and relates to neural network technology. Specifically, it is a multifunctional data retrieval system and method. Background Technology

[0002] Data retrieval involves extracting data stored in a database according to user needs. The results of the data retrieval will generate a data table, which can be either put back into the database or used for further processing.

[0003] Currently, data retrieval is mainly used to retrieve text information, and the methods used to retrieve text data are mainly character or keyword matching technology; and for image data, due to the huge amount of image uploads and the lack of tags on the uploaded images, it is difficult to use deep learning methods for intelligent recognition.

[0004] To address this, a multifunctional data retrieval system and method are proposed. Summary of the Invention

[0005] This invention aims to at least solve one of the technical problems existing in the prior art. To this end, this invention proposes a multifunctional data retrieval system and method. This system and method includes a text data analysis module that analyzes keywords in user-uploaded text content and categorizes the text data according to keywords; an image data analysis module that saves images of the same category according to the user-submitted image category; and a neural network model that identifies subcategories of image categories by using CNN neural network technology as input; the neural network model is used to analyze the subcategories of each image; by classifying images, the required training set of image data is greatly reduced during the training of the neural network, thus reducing the computational power and time required for training.

[0006] To achieve the above objectives, according to an embodiment of the first aspect of the present invention, a multifunctional data retrieval system is provided, including a data uploading module, a text data analysis module, an image data analysis module, a data storage module, a data retrieval module, and a data display module; wherein the data uploading module, text data analysis module, image data analysis module, data storage module, data retrieval module, and data display module are electrically connected;

[0007] The data upload module includes a text upload unit and an image upload unit; the text upload unit is mainly used by users to upload text data; the image upload unit is mainly used by users to upload image data.

[0008] When users upload text data through the text upload unit, they need to upload both the text data title and the content summary; the text upload unit then sends the uploaded text data to the text data analysis module.

[0009] When a user uploads image data through the image upload unit, they must also upload the image title and image category. The image category can be a multi-level category set based on practical experience. The image upload unit then sends the uploaded image data to the image processing module.

[0010] The text data analysis module is mainly used to extract keywords from the uploaded text data;

[0011] The text data analysis module uses keyword extraction technology to extract keyword phrases from the text data content summary; the text data analysis module sends the title of the text content and the extracted keyword phrases to the data storage module;

[0012] The data storage module divides the text storage space into two areas, labeled W1 and W2 respectively. Storage space W1 stores all the text data, while storage space W2 stores the text data categorized by the same keywords.

[0013] The image data analysis module is mainly used to analyze the categories of images;

[0014] The image data analysis module determines whether the category of each received image is specific to the last level under that category; the last level is when there are no further subcategories under that category; if the image category is specific to the last level under that category, then the image and image category are sent to the data storage module; the data storage module divides the image according to the last level of category; otherwise, it analyzes the category of the image.

[0015] The data storage module allocates a block of image storage space; the image storage space is marked as P; the images are stored uniformly in a tree diagram format according to image categories;

[0016] Specifically, the image data analysis module analyzes image categories by including the following steps:

[0017] Step S1: Mark the category of the uploaded image as Sij; where S is the top-level category of the image's category; i is the level of the current category; and j is the image category number in the current category.

[0018] It is understandable that the uploaded image belongs to the i-th level of category S. Therefore, what needs to be analyzed is the category at a more specific level to which the image belongs; that is, the category under a category with a level greater than i.

[0019] Step S2: In a preferred embodiment, the image data analysis module obtains the image set of each subclass of level i+1 under all categories S from the data storage module; obtains the category S(i+1)h of each image; marks each image with its corresponding number h; and inputs the image set into the CNN neural network model to train a CNN neural network model that recognizes each subclass of level i+1 under category S.

[0020] The image Sij is input into the trained CNN neural network model; the probability of image Sij belonging to each subclass is obtained; if the maximum probability among all subclasses is greater than the probability threshold p, the category j1 of that subclass is obtained; i is updated to i+1; j is updated to h; step S2 is repeated; the category of the subclass of image Sij under category j1 is obtained; otherwise, the category of image Sij is the category corresponding to category number h; where the probability threshold p is set according to practical experience;

[0021] The data storage module sends the image Sij to the data storage module and saves it together with the image of the category corresponding to the category number h;

[0022] In another preferred embodiment, the image data analysis module obtains a set of images from level i+1 to level M under all categories S from the data storage module; marks each image in the image set as the last level of its category; and inputs the image set into a CNN neural network model to train a CNN neural network model that can identify the image categories after level i+1 under category S.

[0023] The image Sij is input to the trained CNN neural network model; the probability of image Sij belonging to each type is obtained; all categories with probabilities greater than the probability threshold q are selected; the level of each category is determined, and the category with the highest level is selected as the final category of image Sij; the category with the highest level is labeled as h; where the probability threshold q is set according to practical experience.

[0024] The data storage module sends the image Sij to the data storage module and saves it together with the image of the category corresponding to the category number h;

[0025] The data retrieval module is mainly used to retrieve the required data from the data storage module.

[0026] Users input the text to be searched through the data retrieval module; the data retrieval module searches data based on the keywords to be searched, including the following methods:

[0027] Method 1: The data retrieval module uses keyword extraction technology to extract keywords from the text to be retrieved; the data retrieval module retrieves text data containing the keywords of the text to be retrieved from all text data in the storage space W1 of the data storage module; simultaneously, the data retrieval module searches for text data corresponding to the keywords of the text to be retrieved from the storage space W2 of the data storage module; the data retrieval module merges and deduplicates the text data obtained from the storage space W1 and storage space W2, and sends the merged and deduplicated text data set to the data display module;

[0028] Method 2: The data retrieval module uses keyword extraction technology to extract keywords from the text to be retrieved; the data retrieval module uses the keywords of the text to be retrieved as image categories, and retrieves the image set corresponding to the image category from the image storage space P; further, it can determine whether the keywords of the text to be retrieved contain time, location, or event; and search for images related to the time, location, or event based on the time, location, or event in the keywords; the data retrieval module sends the retrieved images to the data display module;

[0029] The data display module is mainly used to display the retrieved text data and image data to the user;

[0030] In a preferred embodiment, the data display module displays text data and image data to the user in text sections and image sections respectively; wherein the display order of text data and image data can be lexicographical or click-through rate.

[0031] According to an embodiment of a second aspect of the present invention, a multifunctional data retrieval method is provided, comprising the following steps:

[0032] Step 1: Users upload text and image data through the data upload module;

[0033] Step 2: The text data analysis module analyzes the keywords in the text data and saves the text data in the data storage module according to the keyword classification.

[0034] Step 3: The image data analysis module uses a deep learning model to analyze the sub-category of the uploaded images based on the categories entered by the user; and then saves the images in the data storage module according to the analyzed image categories.

[0035] Step 4: The data retrieval module extracts keywords based on the user's input search text using keyword extraction technology; it then retrieves text and image data matching the keywords from the data storage module and sends the text and image data to the data display module.

[0036] Step 5: The data display module shows the retrieved text and image data to the user.

[0037] Compared with the prior art, the beneficial effects of the present invention are:

[0038] This invention includes a text data analysis module that analyzes keywords in user-uploaded text content and categorizes the text data by keywords; an image data analysis module that saves images of the same category according to the user-submitted image category; and a neural network model that identifies subcategories of image categories by using CNN neural network technology as input. The neural network model is then used to analyze the subcategories of each image. By classifying the images, the required training set of image data is significantly reduced during neural network training, thus reducing the computational power and time required for training. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of the present invention;

[0040] Figure 2 This is a flowchart of the present invention. Detailed Implementation

[0041] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] like Figure 1 As shown, a multifunctional data retrieval system includes a data upload module, a text data analysis module, an image data analysis module, a data storage module, a data retrieval module, and a data display module; wherein the data upload module, text data analysis module, image data analysis module, data storage module, data retrieval module, and data display module are electrically connected;

[0043] The data upload module includes a text upload unit and an image upload unit; the text upload unit is mainly used by users to upload text data; the image upload unit is mainly used by users to upload image data.

[0044] When users upload text data through the text upload unit, they need to upload both the text data title and the content summary; the text upload unit then sends the uploaded text data to the text data analysis module.

[0045] When users upload image data through the image upload unit, they need to upload both the image title and the image category. The image category can be a multi-level category set based on practical experience; for example: real-world image - plant - flower - peach blossom. Users can choose any level of the category for the image they upload.

[0046] Furthermore, when users upload images through the image upload unit, they can choose whether to upload the image's creation date, shooting location, and specific event; the image upload unit then sends the uploaded image data to the image processing module.

[0047] The text data analysis module is mainly used to extract keywords from the uploaded text data;

[0048] In a preferred embodiment, the text data analysis module uses keyword extraction technology to extract keyword phrases from the text data content summary; the text data analysis module sends the title of the text content and the extracted keyword phrases to the data storage module;

[0049] The data storage module divides the text storage space into two areas, labeled W1 and W2 respectively. Storage space W1 stores all the text data, while storage space W2 stores the text data categorized by the same keywords.

[0050] The image data analysis module is mainly used to analyze the categories of images;

[0051] In a preferred embodiment, the image data analysis module determines whether the category of each received image is specific to the last level under that category; the last level is when there are no further subcategories under that category; if the image category is specific to the last level under that category, then the image and image category are sent to the data storage module; the data storage module divides the image according to the last level of category; otherwise, it analyzes the category of the image.

[0052] In a preferred embodiment, the data storage module allocates a block of image storage space; the image storage space is labeled P; the images are stored in a tree diagram format according to image category; for example: images labeled as "Image-Plant" are stored uniformly, and images labeled as "Image-Plant-Flower" are stored uniformly.

[0053] Specifically, the image data analysis module analyzes image categories by including the following steps:

[0054] Step S1: Mark the category of the uploaded image as Sij; where S is the top-level category of the image's category; i is the current category level; j is the image category number in the current category; for example: in Reality Images - Plants - Flowers, S represents Reality Images, i is 3, and j is the number of Flowers in all plant categories under the parent category Plants; mark the total number of category levels under category S as M;

[0055] It is understandable that the uploaded image belongs to the i-th level of category S. Therefore, what needs to be analyzed is the category at a more specific level to which the image belongs; that is, the category under a category with a level greater than i.

[0056] Step S2: In a preferred embodiment, the image data analysis module obtains the image set of each subclass of level i+1 under all categories S from the data storage module; obtains the category S(i+1)h of each image; marks each image with its corresponding number h; and inputs the image set into the CNN neural network model to train a CNN neural network model that recognizes each subclass of level i+1 under category S.

[0057] The image Sij is input into the trained CNN neural network model; the probability of image Sij belonging to each subclass is obtained; if the maximum probability among all subclasses is greater than the probability threshold p, the category j1 of that subclass is obtained; i is updated to i+1; j is updated to h; step S2 is repeated; the category of the subclass of image Sij under category j1 is obtained; otherwise, the category of image Sij is the category corresponding to category number h; where the probability threshold p is set according to practical experience;

[0058] The data storage module sends the image Sij to the data storage module and saves it together with the image of the category corresponding to the category number h;

[0059] In another preferred embodiment, the image data analysis module obtains a set of images from level i+1 to level M under all categories S from the data storage module; each image is labeled with the last level of its category; for example, a real-world image-plant-flower is labeled as flower, and a real-world image-plant-flower-peach blossom is labeled as peach blossom; the image set is used as input to a CNN neural network model to train a CNN neural network model that can identify the image categories after level i+1 under category S;

[0060] The image Sij is input to the trained CNN neural network model; the probability of image Sij belonging to each type is obtained; all categories with probabilities greater than the probability threshold q are selected; the level of each category is determined, and the category with the highest level is selected as the final category of image Sij; the category with the highest level is labeled as h; where the probability threshold q is set according to practical experience.

[0061] The data storage module sends the image Sij to the data storage module and saves it together with the image of the category corresponding to the category number h;

[0062] The data retrieval module is mainly used to retrieve the required data from the data storage module.

[0063] In a preferred embodiment, the user inputs the text to be searched through the data retrieval module; the data retrieval module searches data based on the keywords to be searched in the following ways:

[0064] Method 1: The data retrieval module uses keyword extraction technology to extract keywords from the text to be retrieved; the data retrieval module retrieves text data containing the keywords of the text to be retrieved from all text data in the storage space W1 of the data storage module; simultaneously, the data retrieval module searches for text data corresponding to the keywords of the text to be retrieved from the storage space W2 of the data storage module; the data retrieval module merges and deduplicates the text data obtained from the storage space W1 and storage space W2, and sends the merged and deduplicated text data set to the data display module;

[0065] Method 2: The data retrieval module uses keyword extraction technology to extract keywords from the text to be retrieved; the data retrieval module uses the keywords of the text to be retrieved as image categories, and retrieves the image set corresponding to the image category from the image storage space P; further, it can determine whether the keywords of the text to be retrieved contain time, location, or event; and search for images related to the time, location, or event based on the time, location, or event in the keywords; the data retrieval module sends the retrieved images to the data display module;

[0066] The data display module is mainly used to display the retrieved text data and image data to the user;

[0067] In a preferred embodiment, the data display module displays text data and image data to the user in text sections and image sections respectively; wherein the display order of text data and image data can be lexicographical or click-through order.

[0068] like Figure 2 As shown, a multifunctional data retrieval method includes the following steps:

[0069] Step 1: Users upload text and image data through the data upload module;

[0070] Step 2: The text data analysis module analyzes the keywords in the text data and saves the text data in the data storage module according to the keyword classification.

[0071] Step 3: The image data analysis module uses a deep learning model to analyze the sub-category of the uploaded images based on the categories entered by the user; and then saves the images in the data storage module according to the analyzed image categories.

[0072] Step 4: The data retrieval module extracts keywords based on the user's input search text using keyword extraction technology; it then retrieves text and image data matching the keywords from the data storage module and sends the text and image data to the data display module.

[0073] Step 5: The data display module shows the retrieved text and image data to the user.

[0074] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A multifunctional data retrieval system, characterized in that, It includes a data upload module, a text data analysis module, an image data analysis module, a data storage module, a data retrieval module, and a data display module; the various modules are electrically connected. The data upload module includes a text upload unit and an image upload unit; the text upload unit is used for users to upload text data; the image upload unit is used for users to upload image data; the text upload unit and the image upload unit send the uploaded text data and image data to the text data analysis module; The text data analysis module is used to extract keywords from the uploaded text data; the text data analysis module sends the title and keywords of the text content to the data storage module; The image data analysis module is used to analyze the subcategories of an image using a CNN neural network model and a set of subcategories of images under the category to which each image belongs; the image data analysis module sends the image and its subcategories to the data storage module for storage; The data retrieval module is used to retrieve the required data from the data storage module; the data retrieval module sends the retrieved text data and image data to the data display module; The data display module is used to display the retrieved text data and image data to the user; The image data analysis module analyzes image subcategories through the following steps: Step S1: Mark the category of the uploaded image as Sij; where S is the top-level category of the image's category; i is the level of the current category; and j is the image category number in the current category. Step S2: The image data analysis module obtains the image set of each subclass of level i+1 under all categories S from the data storage module; obtains the category S(i+1)h of each image; marks each image with its corresponding number h; and uses the image set as input to the CNN neural network model to train a CNN neural network model that can identify each subclass of level i+1 under category S. The image Sij is input into the trained CNN neural network model; the probability of image Sij belonging to each subclass is obtained; if the maximum probability among all subclasses is greater than the probability threshold p, the category j1 of that subclass is obtained; i is updated to i+1; j is updated to h; step S2 is repeated; the category of the subclass of image Sij under category j1 is obtained; otherwise, the category of image Sij is the category corresponding to category number h; where the probability threshold p is set according to practical experience; The data storage module sends the image Sij to the storage space P of the data storage module and saves it together with the image of the category corresponding to the category number h; or: The image data analysis module obtains the image set from level i+1 to level M under all categories S from the data storage module; marks each image as the last level of its category; and inputs the image set into the CNN neural network model to train a CNN neural network model that can identify the image category after level i+1 under category S. The image Sij is input to the trained CNN neural network model; the probability of image Sij belonging to each type is obtained; all categories with probabilities greater than the probability threshold q are selected; the level of each category is determined, and the category with the highest level is selected as the final category of image Sij; the category with the highest level is labeled as h; where the probability threshold q is set according to practical experience. The data storage module sends the image Sij to the storage space P of the data storage module and saves it together with the images of the category corresponding to the category number h.

2. The multifunctional data retrieval system according to claim 1, characterized in that, When a user uploads text data through the text upload unit, they also upload the text data title and content summary; when a user uploads image data through the image upload unit, they also upload the image title and image category; the image category is a multi-level category set based on practical experience.

3. The multifunctional data retrieval system according to claim 1, characterized in that, The image data analysis module determines whether the category of each received image is specific to the last level under that category; if the image category is specific to the last level under that category, the image and image category are sent to the data storage module; otherwise, the subcategories of the image are analyzed.

4. The multifunctional data retrieval system according to claim 3, characterized in that, The last level refers to a category under which there are no further subcategories.

5. A multifunctional data retrieval system according to claim 3, characterized in that, The data storage module divides the images according to the last level of category; the data storage module allocates a block of image storage space; the image storage space is marked as P; the images are stored uniformly in a tree diagram format according to the image category.

6. The multifunctional data retrieval system according to claim 1, characterized in that, The data retrieval module searches for data based on the keywords to be retrieved, including the following methods: Method 1: The data retrieval module uses keyword extraction technology to extract keywords from the text to be retrieved; based on the keywords, the data retrieval module retrieves text data containing the keywords of the text to be retrieved from all text data in storage space W1 of the data storage module; simultaneously, the data retrieval module searches for text data corresponding to the keywords of the text to be retrieved from storage space W2 of the data storage module; the data retrieval module merges and deduplicates the text data obtained from storage space W1 and storage space W2, and sends the merged and deduplicated text data set to the data display module; storage space W1 and storage space W2 are two text storage spaces divided by the data storage module; Method 2: The data retrieval module uses keyword extraction technology to extract keywords from the text to be retrieved; the data retrieval module uses the keywords of the text to be retrieved as image categories, and retrieves the image set corresponding to the image category from the image storage space P.

7. A retrieval method for a multifunctional data retrieval system as described in any one of claims 1-6, characterized in that, Includes the following steps: Step 1: Users upload text and image data through the data upload module; Step 2: The text data analysis module analyzes the keywords in the text data and saves the text data in the data storage module according to the keyword classification. Step 3: The image data analysis module uses a deep learning model to analyze the sub-category of the uploaded images based on the categories entered by the user; and then saves the images in the data storage module according to the analyzed image categories. Step 4: The data retrieval module extracts keywords based on the user's input search text using keyword extraction technology; it then retrieves text and image data matching the keywords from the data storage module and sends the text and image data to the data display module. Step 5: The data display module shows the retrieved text and image data to the user.

Images