Scientific and technological information resource retrieval and query system based on big data

Abstract

The invention discloses a scientific and technological information resource retrieval and query system based on big data, and relates to the technical field of internet resource retrieval and query, and the scientific and technological information resource retrieval and query system comprises the following steps: S1, user identity verification; s2, inputting retrieval information; s3, performing data retrieval; s4, retrieval data downloading: downloading the retrieval data through a data downloading module according to the science and technology information resources retrieved in the step S3, and S5, retrieval data displaying. In the invention, the member website library can provide login authority for members needing to be registered in the retrieval process, and a user behavior analysis library is added; the number of member websites in the member website library is increased through user authorization of the system, and the retrieval and query practicability of the system is improved; and the data downloading module performs screen capture and screen recording on the web page through a computer terminal by opening the web page, so that the situation that part of web pages cannot be copied and pasted on the web page to perform data retrieval and downloading is avoided, and the retrieval and query practicability of the system is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of Internet resource retrieval and query, and in particular to a scientific and technological information resource retrieval and query system based on big data. Background Art

[0002] Information resource retrieval refers to obtaining relevant information needed by users from a large amount of information through certain methods and tools. It mainly includes two parts: information organization and information retrieval. Information organization is to describe and organize data and its characteristics for the convenience of information retrieval; information retrieval is to obtain relevant information needed by users from a large set of documents.

[0003] The existing information resource retrieval has been significantly improved on the basis of big data retrieval. According to the patent number: CN111382341B, the patent name: a scientific and technological information resource retrieval query system and method based on big data, hereinafter collectively referred to as the reference patent, which records that "the scientific and technological information resource retrieval query system based on big data includes: main control module, user identity authentication module, user access behavior analysis module, keyword input module, query preprocessing module, retrieval statistics module, data acquisition module, data processing module, data temporary storage module, information recommendation module, information retrieval query module, retrieval report generation module, information terminal module, information storage module, display module", those skilled in the art can clearly know that the reference patent does not have the function of processing some special data. For example, some websites cannot copy information, so they cannot obtain the relevant data of the webpage. In addition, the website requires members to browse, which further limits the difficulty of retrieval.

[0004] In summary, a scientific and technological information resource retrieval and query system based on big data was designed. Summary of the invention

[0005] In order to overcome the above-mentioned deficiencies, the present invention provides a scientific and technological information resource retrieval query system based on big data.

[0006] The present invention achieves the above-mentioned purpose through the following technical solutions:

[0007] A scientific and technological information resource retrieval query system based on big data includes the following steps:

[0008] S1. User identity verification: the user enters his account and password to log into the system;

[0009] S2, input search information, the user inputs the information to be searched;

[0010] S3: Perform data retrieval, based on the information input by the user, using Solr search engine technology to retrieve scientific and technological information resources related to the search information input by the user from the large database;

[0011] S4, downloading the search data, downloading the search data through the data download module according to the scientific and technological information resources retrieved in step S3;

[0012] The data download module includes

[0013] 1) Search the website and find the corresponding website according to the search data.

[0014] 2) Determine whether it is a member website; if it is not a member website, directly go to step 4) download website search information, if it is a member website, start searching the member website library;

[0015] 3) Log in to the member website. If the member website in step 2) is in the member website library, the system will log in to the member website through the account and password. If the member website in step 2) is not in the member website library, the system will search in the user behavior analysis library. If the member website is not in the user behavior analysis library, the system will register the website as a member and store the account and password in the member website library and log in to the member website. If the member website is in the user behavior analysis library, the system will store the account and password in the member website library and log in to the member website.

[0016] 4) Download website retrieval information;

[0017] S5. Retrieve data and display the retrieved data information on a display screen.

[0018] Among them, the member website is a website that requires registered members to log in and download information.

[0019] Preferably, in step S1, the user is authenticated by an identity verification module, and then the user behavior analysis module analyzes and records the user's website access information, which includes website security monitoring, logged-in member websites, and member website login data.

[0020] Preferably, in the step S2, the retrieval information that the user can input includes text, voice, picture, video and website, text information recognition is performed by a text recognition module, the text recognition module includes a Tesseract OCR engine, voice information recognition is performed by a voice recognition module, the voice recognition module includes a voice recognition chip and some other auxiliary circuits, the voice recognition module communicates with the main control chip to achieve the purpose of voice interaction, and picture information recognition is performed by a picture recognition module, which is based on a convolutional neural network in a deep learning algorithm. First, the image data is input into the network, and the image is divided into a plurality of small pixel areas, and these pixel values ​​are used as the initial input of the network. Different convolution kernels can detect different features, such as horizontal edges, vertical edges, etc. Then, the convolution layer is usually followed by an activation function, such as a ReLU function, which performs a nonlinear transformation on the convolution result to increase the expression ability of the network. Then there is a pooling layer, and the pooling operation can reduce the dimension of the feature map, reduce the amount of data, and retain the main feature information. After alternating processing of multiple convolution layers, activation functions and pooling layers, the features of the image are gradually extracted and abstracted. Finally, the fully connected layer maps these abstract features to specific classification labels or prediction results, and determines the category to which the image belongs by calculating the probabilities of different categories; video information recognition is performed through the video recognition module, which recognizes text, pictures and voice in the video, and then uses the text recognition module, voice recognition module and picture recognition module to identify related information.

[0021] Preferably, in steps 2) and 3), the member website library is a collection of member websites involved in user search by the user behavior analysis module, and the user behavior analysis library is a collection of member accounts and passwords of member websites involved in user search by the user behavior analysis module, and the member accounts and passwords of member websites are saved with the user's consent and consent for use.

[0022] Preferably, the step 4) comprises the following steps:

[0023] Step 1: Open the website page;

[0024] Step 2: Take screenshots or record screens. On the website page, take screenshots of pictures or texts on the website page, or record screens of videos on the website page through a computer terminal.

[0025] Step 3: Extracting text information from the image, identifying and extracting text from the image through image information recognition;

[0026] Step 4: Extract video information, identify and extract text, pictures and voice in the video through the video recognition module.

[0027] Preferably, in step S5, a report is generated for the retrieved data information.

[0028] The beneficial effects of the present invention are as follows: In the scientific and technological information resource retrieval and query system based on big data:

[0029] 1. The member website library can provide login permissions for registered members encountered during the search process, and join the user behavior analysis library. The user authorization of the system increases the number of member websites in the member website library, and improves the practicality of system search and query;

[0030] 2. The data download module opens the website page and directly takes screenshots and records the website page through the computer terminal, avoiding the situation where some website pages cannot be copied and pasted on their web pages for data retrieval and downloading, thereby improving the practicality of system retrieval and query. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The present invention will now be described by way of example with reference to the accompanying drawings, in which:

[0032] Figure 1 It is a step diagram of the present invention;

[0033] Figure 2 It is a step diagram of the data download module of the present invention;

[0034] Figure 3 It is a step diagram of step 4) of the present invention. DETAILED DESCRIPTION

[0035] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0036] like Figure 1 and Figure 2 As shown, a scientific and technological information resource retrieval query system based on big data includes the following steps:

[0037] S1. User identity verification: the user enters his account and password to log into the system;

[0038] S2, input search information, the user inputs the information to be searched;

[0039] S3: Perform data retrieval, based on the information input by the user, using Solr search engine technology to retrieve scientific and technological information resources related to the search information input by the user from the large database;

[0040] S4, downloading the search data, downloading the search data through the data download module according to the scientific and technological information resources retrieved in step S3;

[0041] The data download module includes

[0042] 1) Search the website and find the corresponding website according to the search data.

[0043] 2) Determine whether it is a member website; if it is not a member website, directly go to step 4) download website search information, if it is a member website, start searching the member website library;

[0044] 3) Log in to the member website. If the member website in step 2) is in the member website library, the system will log in to the member website through the account and password. If the member website in step 2) is not in the member website library, the system will search in the user behavior analysis library. If the member website is not in the user behavior analysis library, the system will register the website as a member and store the account and password in the member website library and log in to the member website. If the member website is in the user behavior analysis library, the system will store the account and password in the member website library and log in to the member website.

[0045] 4) Download website retrieval information;

[0046] S5. Retrieve data and display the retrieved data information on a display screen. Specific embodiment 1:

[0048] In step S1, the user authenticates his / her identity through the identity authentication module, and then the user behavior analysis module analyzes and records the user's website access information, which includes website security monitoring, logged-in member websites, and member website login data.

[0049] In step S2, the retrieval information that the user can input includes text, voice, picture, video and URL. Text information recognition is performed through a text recognition module, which includes a Tesseract OCR engine. Voice information recognition is performed through a voice recognition module, which includes a voice recognition chip and some other auxiliary circuits. The voice recognition module communicates with the main control chip to achieve the purpose of voice interaction. Image information recognition is performed through an image recognition module, which is based on a convolutional neural network in a deep learning algorithm. First, the image data is input into the network, and the image is divided into multiple small pixel areas, and these pixel values ​​are used as the initial input of the network. Different convolution kernels can detect different features, such as horizontal edges, vertical edges, etc. Next, the convolution layer is usually followed by an activation function, such as a ReLU function, which performs a nonlinear transformation on the convolution result to increase the network's expressive power. Then comes the pooling layer, which can reduce the dimension of the feature map, reduce the amount of data, and retain the main feature information. After alternating multiple convolution layers, activation functions, and pooling layers, the features of the image are gradually extracted and abstracted. Finally, the fully connected layer maps these abstract features to specific classification labels or prediction results, and determines the category to which the image belongs by calculating the probabilities of different categories; video information recognition is performed through the video recognition module, which recognizes text, pictures and voice in the video, and then uses the text recognition module, voice recognition module and picture recognition module to identify related information.

[0050] In the steps 2) and 3), the member website library is a collection of member websites involved in user search by the user behavior analysis module. The user behavior analysis library is a collection of member accounts and passwords of member websites involved in user search by the user behavior analysis module. When the member accounts and passwords of the member websites are saved, they will be subject to the user's consent and consent for use. Specific embodiment 2:

[0052] like Figure 3 As shown, the step 4) comprises the following steps:

[0053] Step 1: Open the website page;

[0054] Step 2: Take screenshots or record screens. On the website page, take screenshots of pictures or texts on the website page, or record screens of videos on the website page through a computer terminal.

[0055] Step 3: Extracting text information from the image, identifying and extracting text from the image through image information recognition;

[0056] Step 4: Extract video information, identify and extract text, pictures and voice in the video through the video recognition module.

[0057] In step S5, a report is generated for the retrieved data information.

[0058] The above is based on the present invention as an inspiration. Through the above description, relevant staff can make various changes and modifications without departing from the technical idea of ​​this invention. The technical scope of this invention is not limited to the content in the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. A scientific and technological information resource retrieval and query system based on big data, characterized in that: The following steps are involved: S1. User identity verification: the user enters his account and password to log into the system; S2, input search information, the user inputs the information to be searched; S3: Perform data retrieval, based on the information input by the user, using Solr search engine technology to retrieve scientific and technological information resources related to the search information input by the user from the large database; S4, downloading the search data, downloading the search data through the data download module according to the scientific and technological information resources retrieved in step S3; The data download module includes 1) Search the website and find the corresponding website according to the search data. 2) Determine whether it is a member website; if it is not a member website, directly go to step 4) download website search information, if it is a member website, start searching the member website library; 3) Log in to the member website. If the member website in step 2) is in the member website library, the system will log in to the member website through the account and password. If the member website in step 2) is not in the member website library, the system will search in the user behavior analysis library. If the member website is not in the user behavior analysis library, the system will register the website as a member and store the account and password in the member website library and log in to the member website. If the member website is in the user behavior analysis library, the system will store the account and password in the member website library and log in to the member website. 4) Download website retrieval information; S5. Retrieve data and display the retrieved data information on a display screen.

2. A scientific and technological information resource retrieval and query system based on big data according to claim 1, characterized in that: In the step S1, the user is authenticated by the identity authentication module, and then the user behavior analysis module analyzes and records the user's website access information.

3. A scientific and technological information resource retrieval and query system based on big data according to claim 1, characterized in that: In the step S2, the search information that the user can input includes text, voice, picture, video and website.

4. A scientific and technological information resource retrieval and query system based on big data according to claim 2, characterized in that: In the steps 2) and 3), the member website library is a collection of member websites involved in user search by the user behavior analysis module, and the user behavior analysis library is a collection of member accounts and passwords of member websites involved in user search by the user behavior analysis module.

5. A scientific and technological information resource retrieval and query system based on big data according to claim 3, characterized in that: The step 4) comprises the following steps: Step 1: Open the website page; Step 2: Take screenshots or record screens. On the website page, take screenshots of pictures or texts on the website page, or record screens of videos on the website page through a computer terminal. Step 3: Extracting text information from the image, identifying and extracting text from the image through image information recognition; Step 4: Extract video information, identify and extract text, pictures and voice in the video through the video recognition module.

6. A scientific and technological information resource retrieval and query system based on big data according to claim 1, characterized in that: In the step S5, a report is generated for the retrieved data information.

Images