File travel big data acquisition and treatment method, system and device and storage medium

Abstract

The invention provides a text travel big data collection and treatment method, system and device and a storage medium, and the method comprises the steps: obtaining multi-source unstructured text travel data, carrying out the recognition of the unstructured text travel data based on a text travel field knowledge base, and generating structured text travel data according to a data structure defined by a DSL; inputting the structured text travel data into a data processing rule base to determine a data processing logic, generating a regular expression corresponding to the data processing logic by using regular expression middleware, performing pre-compiling, analyzing the structured text travel data by using a finite automaton algorithm, and generating to-be-classified text travel data; and setting a composite logic gate according to a classification requirement, inputting the to-be-classified text travel data into the composite logic gate, performing classification processing according to a corresponding logic expression, and displaying a classification result. According to the method, comprehensive and accurate big data support can be provided for the text travel industry, and the collection, treatment and application capabilities of the text travel big data are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and more specifically to a method, system, device and storage medium for collecting and managing cultural and tourism big data. Background Art

[0002] As the cultural and tourism industry is booming, big data has become a key force driving innovation and change in the industry. However, traditional big data collection and governance methods have gradually revealed their inherent limitations when dealing with the complexity and diversity of the cultural and tourism industry.

[0003] First, traditional methods are unable to cope with multi-source unstructured cultural and tourism data. Cultural and tourism data often come from multiple channels, including but not limited to SQL databases, NoSQL databases, and various file systems, and most of these data exist in unstructured forms, such as text descriptions, pictures, videos, etc. Traditional methods often rely on manual intervention or simple rule matching to extract information, which is not only inefficient, but also prone to errors, making it difficult to fully capture valuable information in the data.

[0004] Secondly, traditional methods lack effective support from domain knowledge in the process of data structuring. The cultural and tourism industry has its own unique business logic and domain knowledge, such as scenic spot information, travel routes, tourist behavior, etc. These data need to be fully considered in the process of structuring. However, traditional methods often use general data structure templates, which are difficult to accurately reflect the essential characteristics of cultural and tourism data, resulting in a significant reduction in the effectiveness of subsequent data analysis and application.

[0005] Furthermore, traditional methods lack flexibility and intelligence in data processing and classification. Traditional methods usually rely on preset algorithms and rules to process data, which makes it difficult to adapt to the rapidly changing needs and diverse classification standards of the cultural and tourism industry. At the same time, traditional methods often lack intelligent decision-making support in the classification process, making it difficult to dynamically adjust the classification logic according to actual needs, resulting in inaccurate classification results or failure to meet specific needs.

[0006] In addition, traditional methods also have shortcomings in data storage and sharing. Cultural and tourism data involves multiple departments and institutions, and it is necessary to achieve cross-platform and cross-system data sharing and exchange. However, traditional methods often use isolated data storage methods and lack unified data standards and interface specifications, resulting in serious data islands and difficulty in achieving effective data integration and utilization. Summary of the invention

[0007] In response to the above problems, the purpose of the present invention is to provide a method, system, device and storage medium for collecting and managing cultural and tourism big data, which can provide comprehensive and accurate big data support for the cultural and tourism industry and significantly improve the collection, management and application capabilities of cultural and tourism big data.

[0008] In order to achieve the above object, the present invention is implemented through the following technical solutions: In a first aspect, the present invention discloses a method for collecting and managing cultural and tourism big data, comprising the following steps: S1: Obtain unstructured cultural and tourism data from multiple sources, identify the unstructured cultural and tourism data based on the cultural and tourism field knowledge base, and generate structured cultural and tourism data according to the data structure defined by DSL; S2: Input the structured cultural and tourism data into the data processing rule library to determine the data processing logic, use the regular expression middleware to generate the regular expression corresponding to the data processing logic, and use the finite automaton algorithm to analyze the structured cultural and tourism data after pre-compilation to generate the cultural and tourism data to be classified; S3: Obtain classification requirements, set a compound logic gate according to the classification requirements, input the cultural and tourism data to be classified into the compound logic gate, perform classification processing according to the corresponding logical expression, and display the classification results.

[0009] Further, the step S1 comprises: Obtain unstructured cultural and travel data from SQL databases, NoSQL databases, and file systems; Classify unstructured cultural and tourism data based on the cultural and tourism knowledge base, and generate DSL data with cultural and tourism characteristics according to the data structure defined by DSL; The DSL data with cultural and tourism characteristics is processed by Kafka streaming, the corresponding DDL statements are generated, and stored in the database.

[0010] Furthermore, the cultural and tourism field knowledge base represents entities, concepts, attributes and the relationships between them in the cultural and tourism field in the form of a graph.

[0011] Furthermore, the data structure defined by the DSL includes: basic data types, complex data types and application-specific structural information; basic data types include integers, strings, and floating-point numbers; complex data types include lists, dictionaries, and sets; application-specific structural information includes attraction ticket sales information and weather.

[0012] Further, the step S2 comprises: Input DSL statements into the data processing rule base to match the corresponding data processing logic; Push DSL statements and corresponding data processing logic to the regular expression middleware to generate corresponding regular expressions; Perform JIT precompilation on regular expressions to generate corresponding machine code; A deterministic finite automaton algorithm is used to generate cultural and tourism data to be classified based on machine code and DSL statements. Further, the step S3 comprises: Obtain classification requirements, use logic algebra and Boolean logic principles to set up complex logic gates or logic networks, and generate logic expressions based on the classification requirements; The cultural and tourism data to be classified are input into a composite logic gate or a logic network, and the cultural and tourism data to be classified are classified and processed according to the logical expression to generate a classification result or a final decision.

[0013] Furthermore, the step S3 further includes: According to the data format requirements of the target data warehouse, set the data format of the classification results or final decisions output by the compound logic gate or logic network; The classification results or final decisions are stored in the target data warehouse.

[0014] In a second aspect, the present invention further discloses a cultural tourism big data collection and management system, including: The data structure introduction generator is used to obtain unstructured cultural and tourism data from multiple sources, identify the unstructured cultural and tourism data based on the cultural and tourism field knowledge base, and generate structured cultural and tourism data according to the data structure defined by DSL; The intelligent data processing engine is used to input the structured cultural and tourism data into the data processing rule library to determine the data processing logic, use the regular expression middleware to generate the regular expression corresponding to the data processing logic, and use the finite automaton algorithm to analyze the structured cultural and tourism data after pre-compilation to generate the cultural and tourism data to be classified; The data classification module is used to obtain classification requirements, set compound logic gates according to the classification requirements, input the cultural and tourism data to be classified into the compound logic gates, perform classification processing according to the corresponding logical expressions, and display the classification results.

[0015] In a third aspect, the present invention further discloses a cultural tourism big data collection and management device, comprising: Storage, used to store cultural and tourism big data collection and management programs; A processor is used to implement the steps of the cultural and tourism big data collection and management method as described in any of the above items when executing the cultural and tourism big data collection and management program.

[0016] In a fourth aspect, the present invention also discloses a readable storage medium, on which a cultural and tourism big data collection and management program is stored. When the cultural and tourism big data collection and management program is executed by a processor, the steps of the cultural and tourism big data collection and management method as described in any one of the above items are implemented.

[0017] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention processes cultural and tourism big data efficiently through predefined calculation rules and dynamically generated conversion logic, ensuring the accuracy and availability of data, and providing a reliable basis for subsequent data analysis and application. At the same time, it realizes the standardization and normalization of data, eliminates data silos, enables data from different sources to be integrated into a unified structure, and greatly improves the consistency and quality of data.

[0018] 2. The present invention introduces deep learning and NLP technologies for metadata analysis, realizes the intelligent generation and standardized processing of data structures, and greatly improves the efficiency and quality of cultural and tourism big data integration.

[0019] 3. The present invention integrates regular calculation recognition and dynamic rule engine, realizes the automation and intelligence of cultural and tourism big data processing, and enhances the accuracy and flexibility of data calculation.

[0020] 4. The present invention applies circuit logic ideas to the field of data processing, realizes intuitive configuration and efficient execution of cultural and tourism big data classification and processing, and further improves the flexibility and scalability of data processing.

[0021] 5. This paper realizes the effective collection and structured processing of multi-source unstructured cultural and tourism data by building a knowledge base in the field of cultural and tourism and a data structure defined by DSL. It combines the data processing rule base and regular expression middleware, and uses the finite automaton algorithm to improve the efficiency and accuracy of data processing. At the same time, through the flexible setting of compound logic gates, it meets the diverse classification needs, and finally efficiently stores the classification results or decisions in the target data warehouse, providing comprehensive and accurate big data support for the cultural and tourism industry, and significantly improving the collection, governance and application capabilities of cultural and tourism big data.

[0022] It can be seen that compared with the prior art, the present invention has outstanding substantive features and significant progress, and the beneficial effects of its implementation are also obvious. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0024] Figure 1 It is a method flow chart of a specific implementation mode of the present invention.

[0025] Figure 2 It is a data collection flow chart of a specific implementation mode of the present invention.

[0026] Figure 3 It is a data processing flow chart of a specific implementation mode of the present invention.

[0027] Figure 4 It is a system structure diagram of a specific implementation mode of the present invention.

[0028] In the figure, 1. Data structure introduction generator; 2. Intelligent data processing engine; 3. Data classification module. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0030] See also Figure 1 As shown, this embodiment provides a method for collecting and managing cultural and tourism big data, including the following steps: S1: Obtain unstructured cultural and tourism data from multiple sources, identify the unstructured cultural and tourism data based on the cultural and tourism field knowledge base, and generate structured cultural and tourism data according to the data structure defined by DSL.

[0031] In a specific implementation method, first, unstructured cultural and tourism data is obtained from SQL databases, NoSQL databases and file systems; then, the unstructured cultural and tourism data is classified based on the cultural and tourism field knowledge base, and DSL data with cultural and tourism characteristics is generated according to the data structure defined by DSL; finally, the DSL data with cultural and tourism characteristics is processed by Kafka streaming, corresponding DDL statements are generated, and stored in the database.

[0032] The knowledge base in the field of culture and tourism represents entities, concepts, attributes and their relationships in the field of culture and tourism in the form of graphs. The data structures defined by DSL include: basic data types, complex data types and application-specific structural information; basic data types include integers, strings, and floating-point numbers; complex data types include lists, dictionaries, and sets; application-specific structural information includes information on ticket sales for scenic spots and weather.

[0033] As an example, step S1 converts a large amount of multi-dimensional unstructured cultural and tourism data into structured data by executing a data collection process, so that the obtained data is a unified format data that has undergone data preprocessing. Figure 2As shown, the data collection process realizes data standardization and normalization, as well as data conversion. At the same time, after the DSL definition data structure is generated, the user can also fine-tune the DSL definition data structure; and according to the table structure, field type, and field value, the processing logic can be set for the table and field, including data desensitization, data verification, missing value filling, etc. After the setting is completed, in the process of converting the DSL definition data structure into a DDL statement, the data will be pre-processed accordingly through Kafka streaming calculation, so that the underlying data can be realized, data cleaning can be achieved, and data security and privacy protection are processed at the bottom. Among them, the Kafka streaming processing process includes desensitization, missing value filling, and automatic field coverage.

[0034] See also Figure 2 It can be seen that natural language processing is combined with the knowledge base in the field of culture and tourism, and the acquired information is combined with the knowledge base in the field of culture and tourism. After identifying the information, the data structure defined by the DSL of the cultural and tourism big data platform is generated. The knowledge base in the field of culture and tourism represents entities, concepts, attributes and the relationships between them in the field in the form of graphs. It covers many aspects such as tourist attractions, hotels, catering, transportation, weather, etc.

[0035] In this step, the DSL design in the field of culture and tourism is aimed at the unified characteristics of the cultural and tourism market. Because there are many similarities between different attractions in the cultural and tourism data, the structured data designed by this method is as follows: {Table name (data acquisition date - data acquisition method - scenic spot / city name)}: { [Field name: Field constraint: Field type]: [Field value], ... } Based on the above data collection process, the following example is used to illustrate the process of natural language processing combined with the knowledge base in the field of culture and tourism: First, the knowledge base in the field of culture and tourism will classify nouns. For example, T Mountain is a scenic spot, ZT Gate is a scenic spot affiliated to T Mountain, and 200 is the ticket price of the cable car. So if there is a text like this: You can take the cable car to climb T Mountain, and we take the cable car to ZT Gate, which is only 200 yuan. Then NLP (natural language processing) combined with the knowledge base in the field of culture and tourism can obtain information through fuzzy matching and exact matching, and can obtain keywords such as "T Mountain", "T Mountain Cableway", "ZT Gate", and "200". By combining keywords with the knowledge base in the field of culture and tourism, the following corresponding relationships can be obtained: Scenic spot: T Mountain; Ticket: T Mountain Cableway; Price: 200 -> (mapped to the number 200); Scenic spot: ZT Gate. This completes the initial data conversion.

[0036] Then, convert the above content into the following DSL statement: {20240918-NLP-T-Mountain}: { [attractions:nonempty:string]:[ZT gate], [product:nonempty:string]: [ropeway], [price: non-empty: float]:

[200] } This is a DSL (domain-specific language) with effective cultural and tourism characteristics. The statements are mapped to the NoSQL / SQL database through program recognition. The table name reflects the cultural and tourism characteristics and usage needs, and the fields are also the data required by cultural and tourism. The table name, field name and various constraints of the generated NoSQL / SQL database are consistent with the DSL requirements.

[0037] S2: Input the structured cultural and tourism data into the data processing rule library to determine the data processing logic, use the regular expression middleware to generate the regular expression corresponding to the data processing logic, and use the finite automaton algorithm to analyze the structured cultural and tourism data after pre-compilation to generate the cultural and tourism data to be classified.

[0038] In a specific implementation, the DSL statement is first input into the data processing rule library to match the corresponding data processing logic; then the DSL statement and the corresponding data processing logic are pushed to the regular expression middleware to generate the corresponding regular expression; at this time, the regular expression is JIT precompiled to generate the corresponding machine code; finally, a deterministic finite automaton algorithm is used to generate the cultural and tourism data to be classified based on the machine code and DSL statements. Specifically, if Figure 3 As shown, this step uses a data processing flow to introduce structured cultural and tourism data to query the data processing rule base to find out what logic the corresponding data needs to be processed through and process it. Among them, the data processing rule base is generated by the user through DSL (domain-specific language), and the DSL statement is recognized to generate the rule base. After the rule base matches the corresponding processing logic, the processing logic and the data to be processed are pushed to the regular expression middleware according to the selected processing logic. The middleware first takes out the corresponding regular expression rules and processes the data through the generated JIT pre-compiled results. The data processing process uses a deterministic finite automaton algorithm to generate the processed results. Both pre-compilation and deterministic finite automata are aimed at improving the data processing speed. In principle, the data results generated in this way are data that are close to business and usage needs, and no further data processing is required. They can be directly used by the presentation layer (that is, users), which is high-quality data. Moreover, the data processing process can be fully reflected in the software log, realizing data lineage tracking; data cleaning and conversion are realized, solving the problem of underlying data quality, because a set of high-quality data will be generated after processing, which can be written back to the storage and can be directly obtained and used by the business, and naturally data calculation is also realized.

[0039] Based on the above data processing flow, the following example shows the actual data processing process: First, through the processing logic specific to culture and tourism, it is set what kind of processing needs to be done for the data in what table. For example, if we input a DSL like this: {'%T Mountain%'}: {[Scenic Spot: Non-empty: String]}, and input it into the data processing rule library, then for any data whose table name contains 'T Mountain' and whose fields meet the [Scenic Spot: Non-empty: String] fields, they will be matched by the rules. Such data will be matched by the data rules and thus pushed to the regular expression middleware.

[0040] Since the data processing rules in the data processing rule library can be associated with multiple regular expression processing logics. The data processing rule {'%T Mountain%'}: {[Scenic Spot: Non-empty: String]} is associated with two regular expression rules: data desensitization and data encryption. Then such a piece of data will be automatically desensitized and encrypted. Moreover, the regular expression rule library covers common regular expression rules, such as data desensitization, data encryption, data formatting, etc., and also includes regular expression rules for processing data that are relatively specific to the culture and tourism field, such as ticket floor price filtering, praise degree encryption, etc., to meet the special requirements of scenic spots. The expression rules can be added at any time.

[0041] S3: Obtain the classification requirements, set a composite logic gate according to the classification requirements, input the culture and tourism data to be classified into the composite logic gate, and perform classification processing according to the corresponding logical expression, and display the classification result.

[0042] In the specific implementation, first, obtain the classification requirements, and use the principles of logic algebra and Boolean logic to set a composite logic gate or a logic network according to the classification requirements to generate a logical expression; then input the culture and tourism data to be classified into the composite logic gate or the logic network, and perform classification operation processing on the culture and tourism data to be classified according to the logical expression to generate a classification result or a final decision.

[0043] Among them, the data format of the classification result or the final decision output by the composite logic gate or the logic network can be set according to the data format requirements of the target data warehouse; thus, the classification result or the final decision that meets the format requirements can be stored in the target data warehouse.

[0044] As an example, first of all, based on business needs, clarify the specific requirements of logical operations, such as judgment, selection, exclusion, etc., and select the appropriate logic gate type (such as AND, OR, NOT, etc.). Then, use the principles of logic algebra and Boolean logic to design compound logic gates or logic networks. The logical expression is presented in a tree structure through a visual page, which simplifies the design process and supports users to complete the design through fool-proof operations such as dragging and dropping and configuration. At this time, the input data is processed according to the designed logical expression to generate intermediate results or final decisions.

[0045] In addition, in this step, the output format of the logic gate can be adjusted according to the requirements of the target data warehouse to ensure compatibility. It supports relational databases, NoSQL databases, and semi-structured data storage.

[0046] When storing data, store the processed data in the specified format in the corresponding data warehouse. Use database management tools or automated scripts to ensure the accuracy and efficiency of the data.

[0047] Moreover, for real-time processing scenarios, it is also necessary to consider timely feedback of processing results to upstream systems or users to form a closed-loop data processing process.

[0048] Through the above steps, the idea of ​​logic gate circuits can be effectively utilized to implement complex data classification and processing tasks efficiently and accurately.

[0049] The present invention provides a method for collecting and managing cultural and tourism big data. By building a cultural and tourism knowledge base and a data structure defined by DSL, the efficient collection and structured processing of multi-source unstructured cultural and tourism data is realized; the accuracy and flexibility of data processing are improved through intelligent data processing and classification algorithms; and the effective integration and utilization of data are realized through a unified data storage and sharing mechanism. These innovations will effectively promote the digital transformation and intelligent upgrading of the cultural and tourism industry.

[0050] See also Figure 4 As shown, the present invention also discloses a cultural and tourism big data collection and management system, including: a data structure introduction generator 1, an intelligent data processing engine 2 and a data classification module 3. The data structure introduction generator 1 is used to obtain unstructured cultural and tourism data from multiple sources, identify the unstructured cultural and tourism data based on the cultural and tourism field knowledge base, and generate structured cultural and tourism data according to the data structure defined by DSL.

[0051] The intelligent data processing engine 2 is used to input the structured cultural and tourism data into the data processing rule library to determine the data processing logic, use the regular expression middleware to generate the regular expression corresponding to the data processing logic, and use the finite automaton algorithm to analyze the structured cultural and tourism data after pre-compilation to generate the cultural and tourism data to be classified.

[0052] The data classification module 3 is used to obtain classification requirements, set a compound logic gate according to the classification requirements, input the cultural and tourism data to be classified into the compound logic gate, perform classification processing according to the corresponding logical expression, and display the classification results.

[0053] The specific implementation methods of the cultural and tourism big data collection and management system of this embodiment are basically the same as the specific implementation methods of the above-mentioned cultural and tourism big data collection and management method, and will not be repeated here.

[0054] The present invention also discloses a cultural and tourism big data collection and management device, comprising a processor and a memory; wherein, when the processor executes the cultural and tourism big data collection and management program stored in the memory, the steps of the cultural and tourism big data collection and management method as described in any one of the above items are implemented.

[0055] Furthermore, the cultural and tourism big data collection and management device in this embodiment may also include: The input interface is used to obtain the cultural and tourism big data collection and management program imported from the outside, and save the obtained cultural and tourism big data collection and management program to the memory. It can also be used to obtain various instructions and parameters transmitted by external terminal devices and transmit them to the processor so that the processor can use the above various instructions and parameters to carry out corresponding processing. In this embodiment, the input interface can specifically include but is not limited to a USB interface, a serial interface, a voice input interface, a fingerprint input interface, a hard disk reading interface, etc.

[0056] The output interface is used to output various data generated by the processor to the terminal device connected thereto, so that other terminal devices connected to the output interface can obtain various data generated by the processor. In this embodiment, the output interface may specifically include but is not limited to a USB interface, a serial interface, etc.

[0057] The communication unit is used to establish a remote communication connection between the cultural and tourism big data collection and management device and the external server so that the cultural and tourism big data collection and management device can mount the image file to the external server. In this embodiment, the communication unit may specifically include but is not limited to a remote communication unit based on wireless communication technology or wired communication technology.

[0058] The keyboard is used to obtain various parameter data or instructions input by the user by tapping the keycaps in real time.

[0059] The display is used to display the relevant information of the cultural and tourism big data collection and management process in real time.

[0060] The mouse can be used to assist users in inputting data and simplify user operations.

[0061] The present invention also discloses a readable storage medium, wherein the readable storage medium includes a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable hard disk, a CD-ROM or any other form of storage medium known in the technical field. The readable storage medium stores a cultural tourism big data collection and management program, and when the cultural tourism big data collection and management program is executed by a processor, the steps of the cultural tourism big data collection and management method described in any one of the above items are implemented.

[0062] In summary, the present invention can provide comprehensive and accurate big data support for the cultural and tourism industry, and significantly improve the collection, management and application capabilities of cultural and tourism big data.

[0063] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. As for the method disclosed in the embodiment, since it corresponds to the system disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0064] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0065] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or units, which can be electrical, mechanical or other forms.

[0066] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0067] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, or each module may exist physically separately, or two or more modules may be integrated into one unit.

[0068] Similarly, each processing unit in each embodiment of the present invention may be integrated into one functional module, or each processing unit may exist physically, or two or more processing units may be integrated into one functional module.

[0069] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0070] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0071] The above is a detailed introduction to the cultural and tourism big data collection and management method, system, device and readable storage medium provided by the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, the present invention can also be improved and modified in several ways, and these improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. A method for collecting and managing cultural and tourism big data, characterized in that: The steps include: S1: Obtain unstructured cultural and tourism data from multiple sources, identify the unstructured cultural and tourism data based on the cultural and tourism field knowledge base, and generate structured cultural and tourism data according to the data structure defined by DSL; S2: Input the structured cultural and tourism data into the data processing rule library to determine the data processing logic, use the regular expression middleware to generate the regular expression corresponding to the data processing logic, and use the finite automaton algorithm to analyze the structured cultural and tourism data after pre-compilation to generate the cultural and tourism data to be classified; S3: Obtain classification requirements, set a compound logic gate according to the classification requirements, input the cultural and tourism data to be classified into the compound logic gate, perform classification processing according to the corresponding logical expression, and display the classification results.

2. The method for collecting and managing cultural and tourism big data according to claim 1 is characterized in that: The step S1 comprises: Obtain unstructured cultural and travel data from SQL databases, NoSQL databases, and file systems; Classify unstructured cultural and tourism data based on the cultural and tourism knowledge base, and generate DSL data with cultural and tourism characteristics according to the data structure defined by DSL; The DSL data with cultural and tourism characteristics is processed by Kafka streaming, the corresponding DDL statements are generated, and stored in the database.

3. The method for collecting and managing cultural and tourism big data according to claim 2 is characterized in that: The cultural tourism field knowledge base represents entities, concepts, attributes and the relationships between them in the cultural tourism field in the form of a graph.

4. The method for collecting and managing cultural and tourism big data according to claim 3 is characterized in that: The data structure defined by the DSL includes: basic data types, complex data types and application-specific structure information; Basic data types include integers, strings, and floating-point numbers; complex data types include lists, dictionaries, and sets; application-specific structured information includes attraction ticket sales information and weather.

5. The method for collecting and managing cultural and tourism big data according to claim 4 is characterized in that: The step S2 comprises: Input DSL statements into the data processing rule base to match the corresponding data processing logic; Push DSL statements and corresponding data processing logic to the regular expression middleware to generate corresponding regular expressions; Perform JIT precompilation on regular expressions to generate corresponding machine code; A deterministic finite automaton algorithm is used to generate cultural and tourism data to be classified based on machine code and DSL statements.

6. The method for collecting and managing cultural and tourism big data according to claim 5 is characterized in that: The step S3 comprises: Obtain classification requirements, use logic algebra and Boolean logic principles to set up complex logic gates or logic networks, and generate logic expressions based on the classification requirements; The cultural and tourism data to be classified are input into a composite logic gate or a logic network, and the cultural and tourism data to be classified are classified and processed according to the logical expression to generate a classification result or a final decision.

7. The method for collecting and managing cultural and tourism big data according to claim 6 is characterized in that: The step S3 further comprises: According to the data format requirements of the target data warehouse, set the data format of the classification results or final decisions output by the compound logic gate or logic network; The classification results or final decisions are stored in the target data warehouse.

8. A cultural tourism big data collection and management system, characterized by: The system adopts the cultural and tourism big data collection and management method according to any one of claims 1 to 7, and the system includes: The data structure introduction generator is used to obtain unstructured cultural and tourism data from multiple sources, identify the unstructured cultural and tourism data based on the cultural and tourism field knowledge base, and generate structured cultural and tourism data according to the data structure defined by DSL; The intelligent data processing engine is used to input the structured cultural and tourism data into the data processing rule library to determine the data processing logic, use the regular expression middleware to generate the regular expression corresponding to the data processing logic, and use the finite automaton algorithm to analyze the structured cultural and tourism data after pre-compilation to generate the cultural and tourism data to be classified; The data classification module is used to obtain classification requirements, set compound logic gates according to the classification requirements, input the cultural and tourism data to be classified into the compound logic gates, perform classification processing according to the corresponding logical expressions, and display the classification results.

9. A device for collecting and managing cultural and tourism big data, characterized in that: include: Storage, used to store cultural and tourism big data collection and management programs; A processor is used to implement the steps of the cultural and tourism big data collection and management method as described in any one of claims 1 to 7 when executing the cultural and tourism big data collection and management program.

10. A readable storage medium, characterized in that: The readable storage medium stores a cultural and tourism big data collection and management program, and when the cultural and tourism big data collection and management program is executed by the processor, the steps of the cultural and tourism big data collection and management method as described in any one of claims 1 to 7 are implemented.

Images