Tourism information data management method and system

Abstract

The invention discloses a data management method and system for tourism information, belongs to the technical field of big data application and analysis, and aims to solve the technical problem of how to quickly and accurately manage tourism information from various channels and meet the requirements of tourism industry for tourism information. According to the technical scheme, the method comprises the following steps: formulating an acquisition strategy and regularly executing acquisition: formulating a corresponding acquisition strategy according to an information source channel, and setting an acquisition period; data preprocessing and cleaning: performing repeated data removal, missing value processing, error data correction and formatting processing on the acquired tourism information data to obtain preprocessed data; data storage and management: establishing a tourism information database, and storing the preprocessed tourism information data into the tourism information database according to a set data structure; and data updating and maintenance: setting a data updating period, and determining a corresponding updating frequency according to the real-time requirement of the tourism information and the characteristics of different information types.

Description

Technical Field

[0001] The present invention relates to the technical field of big data application and analysis, and particularly to a data management method and system for tourism information. Background Art

[0002] With the booming development of the tourism industry, the acquisition of tourism information is crucial for tourism practitioners, tourism planners, and the general public. However, currently, the sources of tourism information are extensive and scattered, including tourism websites, social media, tourism forums, offline tourism agencies, etc. These information are often in inconsistent formats and not updated in a timely manner, bringing great difficulties to the collection and collation of tourism information. Traditional tourism information collection methods mainly rely on manual collection, which is inefficient, prone to omissions and errors, and difficult to ensure the timeliness and accuracy of information.

[0003] Therefore, how to quickly and accurately manage tourism information from various channels to meet the needs of the tourism industry for tourism information is a technical problem that urgently needs to be solved at present. Summary of the Invention

[0004] The technical task of the present invention is to provide a data management method and system for tourism information to solve the problem of how to quickly and accurately manage tourism information from various channels to meet the needs of the tourism industry for tourism information.

[0005] The technical task of the present invention is achieved in the following manner. A data management method for tourism information is as follows:

[0006] Formulate a collection strategy and perform collection regularly: According to the information source channels, formulate corresponding collection strategies and set the collection period;

[0007] Data preprocessing and cleaning: Remove duplicate data, handle missing values, correct incorrect data, and format the collected tourism information data to obtain preprocessed data;

[0008] Data storage and management: Establish a tourism information database and store the preprocessed tourism information data into the tourism information database according to the set data structure;

[0009] Data update and maintenance: Set the data update period and determine the corresponding update frequency according to the real-time requirements of tourism information and the characteristics of different information types.

[0010] Preferably, formulating a collection strategy and performing collection regularly is as follows:

[0011] For travel websites and the official websites of offline travel agencies, by writing crawlers to simulate user access behaviors, obtain travel data information in web pages; among them, in the crawler program, set request header information, access frequency control parameters, and exception handling mechanisms to avoid causing excessive access pressure on the target website or being blocked by the website.

[0012] For social media platforms and travel forums, adopt a combination of keyword search and data mining techniques to obtain corresponding travel information.

[0013] More preferably, for social media platforms and travel forums, the specific method of adopting a combination of keyword search and data mining techniques to obtain corresponding travel information is as follows:

[0014] By setting travel-related keywords, search in social media platforms and travel forums to obtain posts and comment information containing travel-related keywords; among them, travel-related keywords include scenic spot names, hotel names, and travel routes.

[0015] Apply data mining algorithms to extract valuable travel information from posts and comment information containing travel-related keywords; among them, valuable travel information includes tourists' evaluations of scenic spots and reasons for recommending hotels.

[0016] Set the collection scope, collection start interval, and retention duration of historical data for each website as needed.

[0017] Preferably, data preprocessing and cleaning are as follows:

[0018] Remove duplicate data by setting data deduplication rules: Deduplicate according to the unique identifiers of scenic spot names and hotel names to ensure the uniqueness of data.

[0019] For the situation of missing data, adopt data filling methods: Fill with mean, median, or mode, or make reasonable speculation and filling according to context information.

[0020] For incorrect data, identify and correct it through data verification rules: Check whether the opening hours of scenic spots are reasonable and whether the hotel prices are within a reasonable range.

[0021] Perform formatting processing on the data, convert data from different sources and different formats into a unified format for subsequent data storage and analysis: Unify the date format to "YYYY-MM-DD" and the price format to "X yuan".

[0022] Preferably, data storage and management are as follows:

[0023] Select to use a relational database (such as MySQL) or a non-relational database (such as MongoDB) according to the characteristics and requirements of the data;

[0024] In the database, design the data table structure, store different types of data in different data tables respectively, and establish the association relationship between the data tables by setting foreign keys to achieve the effective organization and management of data;

[0025] To facilitate data query and update, establish a data index mechanism: create corresponding indexes in the database according to the common data query conditions such as scenic spot names, hotel names and tourist routes to improve data query efficiency;

[0026] Regularly maintain and update the database, and timely delete expired or invalid data to ensure the accuracy and timeliness of the data in the database.

[0027] Preferably, the data update and maintenance are as follows:

[0028] For relatively stable information such as scenic spot information and hotel information, set to update once a week or once a month;

[0029] For information with strong real-time nature such as tourist route information and tourist evaluation information, set to update daily or in real time.

[0030] More preferably, in the data update process, adopt the incremental update method, only collect newly added or changed data, avoid repeated collection and storage of a large amount of existing data, and improve data update efficiency;

[0031] At the same time, when updating data, remove duplicate data, process missing values, correct error data and format the newly collected data to ensure the quality of the updated data.

[0032] A data management system for tourism information, which is used to implement the data management method of tourism information as described above; the system includes:

[0033] A data collection module, which is used to formulate corresponding collection strategies according to the information source channels and set the collection period;

[0034] A data preprocessing and cleaning module, which is used to remove duplicate data, process missing values, correct error data and format the collected tourism information data to obtain preprocessed data;

[0035] A data storage and management module, which is used to establish a tourism information database and store the preprocessed tourism information data into the tourism information database according to the set data structure;

[0036] A data update and maintenance module is used to set the data update period and determine the corresponding update frequency according to the real-time requirements of tourism information and the characteristics of different information types.

[0037] An electronic device includes: a memory and at least one processor;

[0038] Wherein, a computer program is stored on the memory;

[0039] The at least one processor executes the computer program stored in the memory, so that the at least one processor executes the data management method of tourism information as described above.

[0040] A computer-readable storage medium stores a computer program, and the computer program can be executed by a processor to implement the data management method of tourism information as described above.

[0041] The data management method and system of tourism information of the present invention have the following advantages:

[0042] (1) By collecting the data open interfaces of tourism information publicity websites in real time, the present invention realizes the dynamic traversal of tourism information data, especially focuses on the collection of scenic spot information, including scenic spot tickets, group tours, free independent travel and other tourism product information, which is efficient, accurate and real, reduces the cost of data collection, and thus provides a data basis for the application of big data technologies such as subsequent data processing, data analysis, modeling and simulation, greatly improving the utilization efficiency of Internet data;

[0043] (2) The present invention solves the problems of low efficiency, poor accuracy, insufficient real-time performance and incomplete information existing in the existing tourism information collection process, can quickly and accurately collect tourism information from multiple channels, and effectively integrate and store it, providing reliable data support for all aspects of the tourism industry;

[0044] (3) The present invention has high efficiency: by combining web crawler technology, keyword search technology and data mining technology, it can quickly collect tourism information from multiple channels, greatly improving the data collection efficiency, and saving a large amount of time and labor costs compared with traditional manual collection methods;

[0045] (4) The present invention has accuracy: in the data collection process, through data preprocessing and cleaning steps, operations such as duplicate removal, missing value processing, error data correction and formatting processing are performed on the collected data, ensuring the accuracy and consistency of the data, and providing a reliable data basis for subsequent data applications;

[0046] (5) The present invention has real-time performance: By setting a reasonable data update period and adopting an incremental update method, the latest tourism information can be obtained in a timely manner, ensuring the real-time performance of the data. This enables tourism practitioners, tourism planners, and tourists to obtain the latest tourism trends and make more reasonable decisions;

[0047] (6) The present invention has comprehensiveness: The collection method of the present invention covers various types of tourism information and multiple information source channels, and can comprehensively collect various types of information in the tourism industry, providing rich data support for all aspects of the tourism industry, and helping to deeply understand the tourism market situation, tourist needs, and tourism product characteristics. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The present invention will be further described below in conjunction with the accompanying drawings.

[0049] Appendix Figure 1 is a flowchart of the data management method for tourism information. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0050] The data management method and system for tourism information of the present invention will be described in detail below with reference to the accompanying drawings of the specification and specific embodiments.

[0051] Embodiment 1:

[0052] As shown in the appendix Figure 1 This embodiment provides a data management method for tourism information, and the method is as follows:

[0053] S1. Formulate a collection strategy and perform collection regularly: According to the information source channels, formulate corresponding collection strategies and set the collection period;

[0054] S2. Data preprocessing and cleaning: Remove duplicate data, handle missing values, correct incorrect data, and format the collected tourism information data to obtain preprocessed data;

[0055] S3. Data storage and management: Establish a tourism information database and store the preprocessed tourism information data into the tourism information database according to the set data structure;

[0056] S4. Data update and maintenance: Set the data update period, and determine the corresponding update frequency according to the real-time requirements of tourism information and the characteristics of different information types.

[0057] Before formulating a collection strategy and regularly performing collection in this embodiment, the collection target and scope are first determined, specifically as follows: According to the needs of tourism information, clarify the types of target information to be collected, such as scenic spot information (including scenic spot names, locations, scenic spot introductions, opening hours, ticket prices, etc.), hotel information (including hotel names, addresses, room types, prices, facilities, user reviews, etc.), tourism route information (including route names, itinerary arrangements, prices, included scenic spots, etc.), tourist evaluation information (including evaluation content, scores, etc. of scenic spots, hotels, tourism routes, etc.), and other relevant tourism information. Determine the information source channels for collection, including but not limited to major tourism websites (such as Ctrip, Qunar, Mafengwo, etc.), social media platforms (such as Weibo, Xiaohongshu, etc.), tourism forums (such as Qyer.com forum, etc.), and official websites of offline tourism agencies.

[0058] The formulation of the collection strategy and the regular execution of collection in step S1 of this embodiment are specifically as follows:

[0059] For tourism websites and official websites of offline tourism agencies, by writing crawlers to simulate user access behaviors, obtain tourism data information in web pages; among them, in the crawler program, set request header information, access frequency control parameters, and exception handling mechanisms to avoid causing excessive access pressure on the target website or being blocked by the website;

[0060] For social media platforms and tourism forums, a combination of keyword search and data mining techniques is used to obtain corresponding tourism information.

[0061] In this embodiment, for social media platforms and tourism forums, the combination of keyword search and data mining techniques to obtain corresponding tourism information is specifically as follows:

[0062] ① By setting tourism-related keywords, search in social media platforms and tourism forums to obtain posts and comment information containing tourism-related keywords; among them, tourism-related keywords include scenic spot names, hotel names, and tourism routes;

[0063] ② Use data mining algorithms to extract valuable tourism information from the posts and comment information containing tourism-related keywords; among them, valuable tourism information includes tourists' evaluations of scenic spots and recommended reasons for hotels;

[0064] ③ Set the collection scope, collection start interval, and retention duration of historical data for each website as needed; for example, collect popular scenic spots, tourism routes, etc., collect three times a day (such as 6:00, 12:00, 20:00), collect information published on the same day each time, and save the collected data for one year, etc.

[0065] The data preprocessing and cleaning in step S2 of this embodiment are specifically as follows:

[0066] S201. Remove duplicate data by setting data deduplication rules: Deduplicate based on the unique identifiers of scenic spot names and hotel names to ensure data uniqueness;

[0067] S202. For cases of missing data, adopt data filling methods: Fill with mean, median or mode, or make reasonable inferences and fill based on context information;

[0068] S203. For incorrect data, identify and correct it through data verification rules: Check whether the opening hours of scenic spots are reasonable and whether the hotel prices are within a reasonable range;

[0069] S204. Format the data, convert data from different sources and in different formats into a unified format for subsequent data storage and analysis: Unify the date format to "YYYY - MM - DD" and the price format to "X yuan".

[0070] The data storage and management in step S3 of this embodiment are specifically as follows:

[0071] S301. Select to use a relational database (such as MySQL) or a non - relational database (such as MongoDB) according to the characteristics and requirements of the data;

[0072] S302. In the database, design the data table structure, store different types of data in different data tables respectively, and establish the association relationship between data tables by setting foreign keys to achieve effective organization and management of data;

[0073] S303. To facilitate data query and update, establish a data indexing mechanism: Create corresponding indexes in the database according to the common data query conditions of scenic spot names, hotel names and travel routes to improve data query efficiency;

[0074] S304. Regularly maintain and update the database, and timely delete expired or invalid data to ensure the accuracy and timeliness of the data in the database.

[0075] The data update and maintenance in step S4 of this embodiment are specifically as follows:

[0076] For relatively stable information such as scenic spot information and hotel information, set to update once a week or a month;

[0077] For information with strong real - time nature such as travel route information and tourist evaluation information, set to update daily or in real - time.

[0078] In this embodiment, during the data update process, adopt the incremental update method, only collect newly added or changed data, avoid repeated collection and storage of a large amount of existing data, and improve data update efficiency;

[0079] Meanwhile, when updating data, duplicate data removal, missing value handling, error data correction, and formatting are performed on the newly collected data to ensure the quality of the updated data.

[0080] Embodiment 2:

[0081] This embodiment provides a data management system for tourism information, which is used to implement the data management method of tourism information in Embodiment 1; the system includes:

[0082] A data collection module, which is used to formulate corresponding collection strategies according to the information source channels and set the collection period;

[0083] A data preprocessing and cleaning module, which is used to remove duplicate data, handle missing values, correct error data, and format the collected tourism information data to obtain preprocessed data;

[0084] A data storage and management module, which is used to establish a tourism information database and store the preprocessed tourism information data into the tourism information database according to the set data structure;

[0085] A data update and maintenance module, which is used to set the data update period and determine the corresponding update frequency according to the real-time requirements of tourism information and the characteristics of different information types.

[0086] The working process of the system is specifically as follows:

[0087] (1) Determine the collection target and scope, and clarify the types of tourism information and information source channels to be collected;

[0088] (2) Design a data collection strategy through the data collection module, and adopt a combination of web crawler technology, keyword search technology, and data mining technology to collect tourism information for different information source channels;

[0089] (3) Perform preprocessing and cleaning on the collected tourism information data through the data preprocessing and cleaning module, including removing duplicate data, handling missing values, correcting error data, and formatting;

[0090] (4) Establish a tourism information database through the data storage and management module, store the preprocessed tourism information data according to a certain data structure, and establish a data indexing mechanism;

[0091] (5) Set the data update period through the data update and maintenance module, and use the incremental update method to update and maintain the tourism information data.

[0092] Among them, the types of tourism information include scenic spot information, hotel information, tourism route information, tourist evaluation information, and other relevant tourism information; the information source channels include major tourism websites, social media platforms, tourism forums, and the official websites of offline tourism agencies.

[0093] In the data acquisition module of this embodiment, for tourism websites and the official websites of offline tourism agencies, by writing a crawler program to simulate user access behavior, tourism information data in web pages is obtained; for social media platforms and tourism forums, a combination of keyword search and data mining techniques is adopted to obtain information such as posts and comments containing tourism-related keywords, and valuable tourism information is extracted from them.

[0094] In the data preprocessing and cleaning module of this embodiment, by setting reasonable data deduplication rules, data filling methods, and data verification rules, the collected data is deduplicated, missing value processing, incorrect data correction, and formatting processing are performed.

[0095] In this embodiment, the tourism information database can use a relational database or a non-relational database, which is selected according to the characteristics and requirements of the data, and a reasonable data table structure is designed, and the association relationship between data tables is established by setting foreign keys and other methods.

[0096] In the data update and maintenance module of this embodiment, according to the real-time requirements of tourism information and the characteristics of different information types, the corresponding update frequency is set, and the incremental update method is adopted to only collect newly added or changed data, and at the same time, the newly collected data is processed in the same process as data preprocessing and cleaning.

[0097] Embodiment 3:

[0098] This embodiment also provides an electronic device, including: a memory and at least one processor;

[0099] Wherein, the memory stores computer execution instructions;

[0100] The at least one processor executes the computer execution instructions stored in the memory, so that the at least one processor executes the data management method of tourism information in any embodiment of the present invention.

[0101] The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0102] The memory can be used to store computer programs and / or modules. By running or executing the computer programs and / or modules stored in the memory, and by invoking the data stored in the memory, the processor can implement various functions of the electronic device. The memory mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function, etc.; the data storage area can store data created according to the use of the terminal, etc. In addition, the memory can also include high-speed random access memory, and can also include non-volatile memory, such as a hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash memory card, at least one magnetic disk storage period, flash memory device, or other volatile solid-state storage devices.

[0103] Embodiment 4:

[0104] This embodiment also provides a computer-readable storage medium, which stores multiple instructions. The instructions are loaded by the processor to cause the processor to execute the data management method of travel information in any embodiment of the present invention. Specifically, a system or device equipped with a storage medium can be provided. On this storage medium, software program codes for implementing the functions of any one of the above embodiments are stored, and the computer (or CPU or MPU) of the system or device reads and executes the program codes stored in the storage medium.

[0105] In this case, the program code read from the storage medium itself can implement the functions of any one of the above embodiments. Therefore, the program code and the storage medium storing the program code constitute a part of the present invention.

[0106] Embodiments of the storage medium for providing program codes include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Optionally, the program code can be downloaded from a server computer via a communication network.

[0107] In addition, it should be clear that not only can the functions of any one of the above embodiments be realized by executing the program code read by the computer, but also by causing the operating system etc. operating on the computer based on the instructions of the program code to complete part or all of the actual operations.

[0108] In addition, it can be understood that the program code read from the storage medium is written into the memory provided in the expansion board inserted into the computer or into the memory provided in the expansion unit connected to the computer. Subsequently, based on the instructions of the program code, the CPU etc. installed on the expansion board or expansion unit execute part and all of the actual operations, thereby realizing the functions of any one of the above embodiments.

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data management method for tourism information, characterized in that The method is as follows: Formulate a collection strategy and execute the collection regularly: According to the information source channels, formulate corresponding collection strategies and set the collection cycle; Data preprocessing and cleaning: Remove duplicate data, handle missing values, correct incorrect data, and format the collected tourism information data to obtain the preprocessed data; Data storage and management: Establish a tourism information database and store the preprocessed tourism information data into the tourism information database according to the set data structure; Data update and maintenance: Set the data update cycle and determine the corresponding update frequency according to the real-time requirements of tourism information and the characteristics of different information types.

2. The data management method for tourism information according to claim 1, characterized in that, Formulating a collection strategy and executing the collection regularly are as follows: For tourism websites and official websites of offline tourism agencies, simulate user access behavior by writing crawlers to obtain tourism data information on web pages; among them, in the crawler program, set the request header information, access frequency control parameters, and exception handling mechanism; For social media platforms and tourism forums, adopt a combination of keyword search and data mining techniques to obtain corresponding tourism information.

3. The data management method for tourism information according to claim 2, wherein Adopting a combination of keyword search and data mining techniques to obtain corresponding tourism information for social media platforms and tourism forums is as follows: Search in social media platforms and tourism forums by setting tourism-related keywords to obtain posts and comment information containing tourism-related keywords; among them, tourism-related keywords include scenic spot names, hotel names, and tourism routes; Use data mining algorithms to extract valuable tourism information from the posts and comment information containing tourism-related keywords; among them, valuable tourism information includes tourists' evaluations of scenic spots and reasons for recommending hotels; Set the collection scope, collection start interval, and retention duration of historical data for each website as needed.

4. The data management method for tourism information according to claim 1, characterized in that, Data preprocessing and cleaning are as follows: Remove duplicate data by setting data deduplication rules: Deduplicate according to the unique identifiers of scenic spot names and hotel names to ensure data uniqueness; For data missing situations, adopt data filling methods: Fill with mean, median, or mode, or make reasonable speculation and filling according to context information; For incorrect data, identify and correct it through data verification rules: Check whether the opening hours of scenic spots are reasonable and whether the hotel prices are within a reasonable range; Format the data to convert data from different sources and different formats into a unified format: Unify the date format to "YYYY-MM-DD" and the price format to "X yuan".

5. The data management method for tourism information according to claim 1, wherein Data storage and management are as follows: Select a relational database or a non-relational database according to the characteristics and requirements of the data; In the database, design the data table structure, store different types of data in different data tables respectively, and establish the association relationship between data tables by setting foreign keys to achieve effective organization and management of data; Establish a data indexing mechanism: Create corresponding indexes in the database according to the commonly used data query conditions of scenic spot names, hotel names, and tourism routes; Regularly maintain and update the database, and promptly delete expired or invalid data.

6. The data management method for tourism information according to claim 1, wherein The data update and maintenance are as follows: For relatively stable information such as scenic spot information and hotel information, set to update once a week or once a month; For information with strong real-time requirements such as travel route information and tourist evaluation information, set to update daily or in real time.

7. The data management method for tourism information according to claim 6, characterized in that, During the data update process, adopt the incremental update method, and only collect newly added or changed data; At the same time, when updating the data, remove duplicate data, handle missing values, correct incorrect data, and perform formatting processing on the newly collected data to ensure the quality of the updated data.

8. A data management system for tourism information, characterized in that, This system is used to implement the data management method of tourism information described in any one of claims 1 to 7; this system includes: A data collection module, used to formulate corresponding collection strategies according to the information source channels and set the collection period; A data preprocessing and cleaning module, used to remove duplicate data, handle missing values, correct incorrect data, and perform formatting processing on the collected tourism information data to obtain preprocessed data; A data storage and management module, used to establish a tourism information database and store the preprocessed tourism information data into the tourism information database according to the set data structure; A data update and maintenance module, used to set the data update period and determine the corresponding update frequency according to the real-time requirements of tourism information and the characteristics of different information types.

9. An electronic device, characterized in that, Including: A memory and at least one processor; Wherein, a computer program is stored on the memory; The at least one processor executes the computer program stored in the memory, so that the at least one processor executes the data management method of tourism information described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A computer program is stored in the computer-readable storage medium, and the computer program can be executed by a processor to implement the data management method of tourism information described in any one of claims 1 to 7.

Images