Authorized intelligent search method and system based on controlled system

Abstract

The invention belongs to the field of artificial intelligence and information retrieval, and relates to an intelligent search method and system with authority based on a controlled system, and the method comprises the following steps: 1, building an in-platform field in intelligent knowledge; 2, reporting structured and unstructured data with security levels and permission marks to the intelligent knowledge middle table; step 3, processing the structured data; step 4, analyzing the unstructured data; step 5, processing before searching; step 6, searching through a search engine; according to the method, on the premise of guaranteeing information safety and privacy, intelligent and personalized search service is provided, and safe and reliable information retrieval experience is provided for users.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence and information retrieval, and relates to technical fields such as natural language processing, data mining, and information retrieval algorithms, and in particular to a permissioned intelligent search method and system based on a controlled system. Background Art

[0002] With the acceleration of digital transformation of enterprises, the concept of data middle platform has gradually become popular. It integrates various business data within the enterprise, builds a unified data platform, and realizes efficient management and intelligent application of data. When generating data query requests, users often search for relevant business data through the search engine in the data middle platform.

[0003] However, some of the business data stored in the data center and its search engine are sensitive information. If sensitive information is not properly protected, it will face serious information leakage problems, which will bring about many adverse effects.

[0004] Therefore, sensitive information can be protected, and only authorized users can access and search related content, effectively protecting data security and privacy; the system allows administrators to manage the permissions of users or user groups in a refined manner, ensuring that each user can only view content for which they have permission, thereby ensuring the security and controllability of information; the intelligent search function based on artificial intelligence technology can provide personalized and accurate search results based on the user's query intention, improving user experience and work efficiency; it can achieve an accurate understanding of the user's search intention, and provide more accurate search results through technologies such as deep learning to meet user needs.

[0005] In view of this, it is necessary to propose an authorized intelligent search method for controlled systems based on artificial intelligence technology. Summary of the invention

[0006] Technical problem solved by the present invention: The present invention is used to solve the problem in the prior art that part of the business data stored in the data middle platform and its search engine is sensitive information. If the sensitive information is not properly protected, it will face serious information leakage problems. The purpose of the present invention is to propose a permissioned intelligent search method and system based on a controlled system, aiming to achieve secure management and intelligent search of sensitive information.

[0007] Technical solution of the present invention

[0008] A permissioned intelligent search method based on a controlled system comprises the following steps:

[0009] Step 1: Build domains in the intelligent knowledge platform;

[0010] Step 2: Report structured and unstructured data with confidentiality and permission marks to the intelligent knowledge center;

[0011] Step 3: Process the structured data;

[0012] Step 4: Parse the unstructured data;

[0013] Step 5: Pre-search processing;

[0014] Step 6: Search through search engines.

[0015] Further, step 1 includes the following steps:

[0016] Step 1.1 Investigate user needs in specific areas, conduct business scenario analysis, collect needs and business information, and extract key needs and scenario features;

[0017] Step 1.2: In the intelligent knowledge platform, create a specific domain based on the information obtained in step 1.1, allocate a dedicated storage area for the domain, and configure corresponding system resources;

[0018] Step 1.3 generates a unique work scenario identifier for the field.

[0019] Further, step 2 includes:

[0020] When choosing to upload data locally, develop an upload interface. In the interface design, use an encrypted transmission protocol to ensure the security of data transmission. For the uploaded data, perform preliminary encryption processing on the client side and convert the data into ciphertext before uploading. At the same time, embed confidentiality and permission mark information in the uploaded data. The mark information is stored in a specific data structure, which contains the confidentiality level of the data and the permission information of different user roles.

[0021] When reporting data through the ETL data synchronization tool, set up data encryption and logo embedding links in the ETL process. In the data extraction stage, the data in the data source is encrypted, and the encryption method is consistent with local upload; in the data conversion stage, the confidentiality level and permission logo information are added to the data to form structured data with security attributes; in the data loading stage, ensure that the data is accurately transmitted to the intelligent knowledge center, and perform integrity verification during the transmission process. At the same time, in the ETL process, the performance of data synchronization is optimized.

[0022] Further, step 3 includes:

[0023] (1) The data verification module in the intelligent knowledge center uses a multi-level verification algorithm to verify structured data, including:

[0024] Verify the format of structured data: Check whether the data conforms to the specified structure according to the predefined data model;

[0025] Verify the integrity of structured data: Verify the integrity of structured data by comparing the checksum of the calculated data with the checksum of the original data.

[0026] For key data fields, perform logical checks;

[0027] If the structured data verification fails, the structured data will be marked as an error state, and detailed error information will be recorded. At the same time, the alarm mechanism will be triggered to notify the administrator;

[0028] (2) The data improvement module in the intelligent knowledge center performs data cleaning operations

[0029] Identify and process duplicate records, erroneous data, and incomplete data in structured data. For duplicate records, use a deduplication algorithm based on data features to remove completely duplicate records. For erroneous data, repair or delete it based on the data type and predefined error handling strategy. For incomplete data, complete it through data association and filling algorithms.

[0030] (3) Perform data standardization: For structured data from different sources, unify the data representation format, unify the units for numerical data represented by different units, and ensure the consistency of data within the knowledge platform by establishing data mapping tables and conversion rules. During the data standardization process, monitor data quality indicators in real time.

[0031] Further, step 4 includes the following steps:

[0032] Step 4.1 After the unstructured data is uploaded to the intelligent knowledge platform, it is sent to the file parsing module in the intelligent knowledge platform; the file parsing module identifies the unstructured data, determines the type of the file, and then selects the corresponding parsing strategy based on the file type:

[0033] For image files or PDF files containing text, optical character recognition is used to parse them;

[0034] For unstructured data such as documents, layout analysis is performed: by analyzing the structural information of the document, the document content is divided into different logical units;

[0035] Step 4.2 converts the parsed unstructured data into structured data form, including:

[0036] For text content, convert it into a format that can be stored in the database;

[0037] For documents containing tables, convert the table contents into a relational data structure;

[0038] During the conversion process, the accuracy and completeness of the data are ensured, while the confidentiality level and permission marks are retained, and the data is reported to the search engine for storage in the same way as structured data.

[0039] Furthermore, in step 4.1, an optical character recognition method is used to parse the image file or PDF file containing text, including: in the optical character recognition process, the recognition accuracy is improved by training the model, the training data includes a large number of text samples of different fonts, font sizes, and clarity, and the image is pre-processed; in the recognition process, the recognized characters are contextually analyzed in combination with the language model to correct possible recognition errors;

[0040] For unstructured document data, layout analysis is performed by using a rule-based method to determine the layout structure according to the format characteristics and semantic characteristics of the document. At the same time, a machine learning method is used to train the layout analysis model and learn the characteristics of the layout structure through a large amount of annotated document data.

[0041] Furthermore, step 5 is that when the user enters the search page of the intelligent knowledge platform and enters keywords, the intelligent knowledge platform first obtains the user's basic information, and then interacts with the enterprise's organizational structure database and authority management system to obtain the user's complete authority information;

[0042] Based on the obtained user authority information, the user's data visibility range is calculated. For departmental authority, the departmental authority configuration table is queried to determine the data range that the department can access. For personal authority, the data visibility range is further refined in combination with the user role and special authorization conditions. For personal confidentiality, it is compared with the confidentiality mark of the data. Only when the user confidentiality level is higher than or equal to the data confidentiality level, the user has the right to access the data. By combining the above authority data, the user's data visibility range in this search is determined.

[0043] Verify the permission information associated with the query request according to the verification items: first, check whether the organization permissions in the query object permission information are within the data permissions of the data to be queried; second, check whether the object permissions in the query object permission information are within the data permissions of the data to be queried; finally, check whether the object confidentiality level in the query object permission information is higher than part or all of the data confidentiality level of the data to be queried.

[0044] Further, the permission information associated with the query request is verified according to the verification item, including:

[0045] First, check whether the organization authority in the query object authority information is within the data authority of the data to be queried: by comparing the authority identifier of the user's department with the authority identifier of the department to which the data belongs, if the two match or the user's department has a higher authority, the corresponding verification item is met;

[0046] Secondly, check whether the object permissions in the query object permission information are within the data permissions of the data to be queried: including comparing the user's personal role permissions and the data access restrictions for the personal role;

[0047] Finally, check whether the object confidentiality level in the query object permission information is higher than the partial or all data confidentiality levels of the data to be queried. By comparing the user's personal confidentiality level with the confidentiality level mark of the data, if the user confidentiality level is higher than the data confidentiality level, the corresponding verification item is met.

[0048] Further, step 6 includes the following steps:

[0049] Step 6.1: construct a search request with the search keyword input by the user and the user's authority level data, send the constructed search request to the search engine, establish a connection with the search engine through the network communication protocol, and send the request to the search interface of the search engine;

[0050] Step 6.2 After receiving the search request, the search engine searches the index data based on the keywords.

[0051] Furthermore, in step 6.1, the search request adopts the HTTP-based RESTful API format. In the search request, the keywords are segmented and decomposed using Chinese word segmentation algorithms and English natural language processing tools to improve the accuracy of the search. At the same time, the user's authority level information is used as one of the request parameters for authority filtering in the search engine.

[0052] During the search process in step 6.2, the search results are filtered based on the user's authority level information, and only search results visible to the user are returned; the relevance score between the search term and the search result document is calculated, and the search results are sorted in descending order according to the relevance score, and all data with the highest relevance to the search term is returned to the user. When returning the results, the results are formatted and presented to the user in a user-friendly interface.

[0053] An authorized intelligent search system based on a controlled system is applied to the method described above, and the system comprises:

[0054] Building unit: configured to build a domain in the intelligent knowledge platform;

[0055] Reporting unit: configured to report structured and unstructured data with confidentiality and authority marks to the intelligent knowledge platform;

[0056] Processing unit A: configured to process structured data;

[0057] Parsing unit: configured to parse unstructured data;

[0058] Processing unit B: configured for pre-search processing;

[0059] Search unit: configured to perform a search through a search engine.

[0060] Beneficial Effects

[0061] The present invention provides a permissioned intelligent search method based on a controlled system, which has the following advantages:

[0062] Security assurance: The method of the present invention can ensure that sensitive information is protected, and only authorized users can access and search related content, effectively protecting data security and privacy.

[0063] Permission management: The method of the present invention allows administrators to perform fine-grained management of the permissions of users or user groups, ensuring that each user can only view content for which they have permission, thereby ensuring the security and controllability of information.

[0064] Intelligent search function: The intelligent search function based on artificial intelligence technology of the method of the present invention can provide personalized and accurate search results according to the user's query intention, thereby improving user experience and work efficiency.

[0065] Semantic understanding: The method of the present invention can achieve accurate understanding of user search intent and provide more accurate search results through deep learning and other technologies to meet user needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 It is a method flow chart of the method of the present invention. DETAILED DESCRIPTION

[0067] The features and illustrative embodiments of various aspects of the present invention will be described in detail below. In the detailed description below, many specific details are proposed to provide a comprehensive understanding of the present invention. However, it is obvious to those skilled in the art that the present invention can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present invention by illustrating examples of the present invention. The present invention is by no means limited to any specific settings and methods proposed below, but covers any improvements, replacements and modifications of structures, methods, devices without departing from the spirit of the present invention. In the accompanying drawings and the following description, known structures and technologies are not shown to avoid unnecessary ambiguity in the present invention.

[0068] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other, and the embodiments can refer to and quote each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0069] An embodiment of the present invention is to provide a permissioned intelligent search method based on a controlled system, comprising the following steps:

[0070] Step 1: Build domains in the intelligent knowledge platform

[0071] Investigate user needs in specific areas through various means, such as designing online questionnaires, which cover users' daily operations, expected functions, problems faced, etc. At the same time, conduct in-depth business scenario analysis and conduct face-to-face interviews with relevant business department personnel to learn more about business processes, involved links, data flow, etc. After collecting this information, organize and classify it to extract key needs and scenario features.

[0072] Then, in the self-developed intelligent knowledge platform, a specific domain is created based on the above information. This process involves allocating a dedicated storage area for the domain, configuring the corresponding system resources, and other operations to ensure that the domain has an independent and stable operating environment in the knowledge platform.

[0073] A unique work scenario identifier is generated through a specific algorithm. The algorithm can perform hash operations based on the key attributes of the domain (such as domain name, main business type, data type involved, etc.) to ensure that the generated identifier is unique. For example, using the SHA-256 hash algorithm, the key attributes of the domain are combined into a string as input to obtain a 256-bit hash value as an identifier. This identifier will serve as an important basis for distinguishing data processing in different domains in the subsequent data processing process.

[0074] Among them, in step 1, the needs and usage scenario information of a specific field are read, for example, by investigating user needs and analyzing business scenarios. In the process of investigating user needs, a variety of methods can be used, such as group discussions, convening experts and typical users in the field, discussing around specific topics, and collecting opinions and suggestions from all parties; user experience testing can also be carried out to observe users using related products or services in simulated or actual business scenarios, and record user behavior, feedback and problems. For business scenario analysis, in addition to interviewing business department personnel, business processes can also be modeled, and business process diagrams can be drawn using tools such as business process modeling notation (BPMN), clearly showing each link, role, data flow, etc. of the business, providing comprehensive and in-depth information for domain construction.

[0075] Among them, the tag information in step 1 allocates a unique work scenario identifier for the domain construction, which is used to distinguish different data processing processes in subsequent steps. The identifier generation algorithm must ensure its uniqueness and stability, and a conflict check and update mechanism can be performed on the identifier regularly. For example, a global scan of identifiers is performed once a month to check whether there are any duplications. If found, the affected identifiers are regenerated and the relevant data association information is updated.

[0076] Step 2: Report structured and unstructured data with confidentiality and permission marks to the intelligent knowledge platform

[0077] When choosing to upload data locally, develop a dedicated upload interface. In the interface design, use an encrypted transmission protocol (such as HTTPS) to ensure the security of data transmission. For the uploaded data, preliminary encryption processing is performed on the client, and the encryption algorithm uses the AES (Advanced Encryption Standard) algorithm to convert the data into ciphertext before uploading. At the same time, the confidentiality level and permission flag information are embedded in the uploaded data. These flag information are stored through a specific data structure, such as using the JSON format, which contains the confidentiality level of the data (such as top secret, confidential, secret, internally disclosed, etc.) and the permission information of different user roles (such as read-write permission, read-only permission, etc.).

[0078] For data reported through the ETL data synchronization tool, set up data encryption and logo embedding links in the ETL process. In the data extraction stage, the data in the data source is encrypted, and the encryption method is consistent with local upload. In the conversion stage, the confidentiality level and permission logo information are added to the data to form structured data with security attributes. In the loading stage, ensure that the data is accurately transmitted to the knowledge center, and perform integrity verification during the transmission process. The CRC (cyclic redundancy check) algorithm is used to ensure that the data is not damaged during the transmission process. At the same time, in the entire ETL process, the performance of data synchronization is optimized. By monitoring data traffic, adjusting synchronization frequency, etc., ensure that the data synchronization process is efficient and meets the expected time and cost requirements.

[0079] In step 2, upload locally or use the ETL data synchronization tool to report structured and unstructured data with confidentiality and permission marks to ensure data security and compliance. During the data transmission process, in addition to encryption and mark embedding, a data audit mechanism should be established to record data upload time, source, operator and other information. Strict audits should also be conducted on data access and use to ensure that all operations meet security and compliance requirements. When an abnormal situation is found (such as illegal access, data tampering, etc.), the security incident response process is triggered in a timely manner, such as notifying the security administrator, performing data backup and recovery, and other operations.

[0080] Step 3: Process structured data

[0081] The data verification module uses a multi-level verification algorithm. First, the format of the structured data is verified to check whether the data conforms to the specified structure according to the predefined data mode (JSON mode). The integrity of the data is verified by comparing the calculated data checksum with the checksum of the original data to ensure that the data is not lost or tampered with. For key data fields, logical verification is performed, such as checking whether the date field conforms to the date format specification, whether the numeric field is within a reasonable value range, etc. If the data verification fails, the data is marked as an error state, and detailed error information is recorded, and the alarm mechanism is triggered to notify the administrator.

[0082] The data improvement module first performs data cleaning operations. By writing data cleaning rules, duplicate records, erroneous data, and incomplete data in the data are identified and processed. For duplicate records, a deduplication algorithm based on data characteristics (such as the same key field values) can be used, such as using the method of adjacent comparison after sorting to remove completely duplicate records. For erroneous data, repair or delete it according to the data type and predefined error handling strategy. For example, for outliers in numerical data, a reasonable range can be determined through statistical analysis, and values ​​outside the range can be corrected or marked. For incomplete data, it is improved through data association and filling algorithms, such as using information in other related data tables to fill missing values.

[0083] Then, perform data standardization. For structured data from different sources, unify the data representation. For example, unify the date format to "YYYY-MM-DD", and unify the units for numerical data represented by different units. Ensure the consistency of data within the knowledge platform by establishing data mapping tables and conversion rules. During the data improvement process, monitor the changes in data quality indicators (such as data accuracy, completeness, consistency, etc.) in real time to ensure that the quality and availability of processed data are significantly improved.

[0084] Among them, the data verification module in step 3 is used to verify the integrity and accuracy of structured data to ensure that the data quality meets the standards. A data quality monitoring dashboard can be established to display various indicators of data verification (such as error rate, integrity ratio, etc.) in real time. For frequently occurring error types, in-depth analysis is carried out to find the causes and optimize the data source or processing flow. At the same time, a feedback mechanism is established with data providers to promptly feedback data quality issues to relevant parties and jointly improve data quality.

[0085] Among them, the data improvement module in step 3 improves data quality and availability through data cleaning, standardization and other processing. In the data cleaning process, machine learning algorithms are introduced to automatically identify and process abnormal data. For example, algorithms are used to detect outliers in the data, and whether the data is abnormal is determined based on the distribution characteristics and distance metrics of the data. For data standardization, a standardized rule base is established to uniformly process data according to industry standards and internal corporate specifications, and the rule base is regularly updated to adapt to business changes.

[0086] When creating the data synchronization task in step 3, ensure that the data synchronization process is efficient and meets the expected time and cost requirements. Use performance testing tools to continuously monitor and optimize the data synchronization process. For example, use tools such as JMeter to simulate synchronization scenarios of large amounts of data, analyze bottlenecks in the synchronization process (such as network bandwidth limitations, slow database write speeds, etc.), and take targeted optimization measures, such as increasing network bandwidth and optimizing database indexes.

[0087] Among them, in step 3, when the processing of structured data is completed, ensure that the data is accurately stored in the knowledge center and reported to the search engine in a timely manner for subsequent retrieval. Establish a monitoring mechanism for data storage and reporting to track the storage status and reporting progress of data in real time. If data storage fails or reporting is not timely, the retry mechanism is automatically triggered and a detailed error log is recorded. At the same time, sample checks are performed on the stored and reported data to ensure the accuracy and completeness of the data.

[0088] In addition, in steps 1, 2, and 3, create a new data control process, including data approval, access rights management, etc., to ensure data security and compliance. For the data approval process, establish an approval workflow system to clarify the approval roles, approval steps, and approval conditions. For example, the upload or modification of high-level data requires multiple levels of approval, including review by department heads, security administrators, etc. For access rights management, adopt a role-based access control (RBAC) model, combining factors such as the user's department, role, and confidentiality level to accurately control the user's access rights to data. At the same time, audit and clean up permissions regularly to prevent abuse of permissions and data leakage.

[0089] Step 4: Parse the unstructured data

[0090] After unstructured data is uploaded to the knowledge center, it is sent to the file parsing module. The file parsing module is a system that integrates multiple parsing technologies. First, the file type is identified and the file type (such as PDF, JPEG, Word document, etc.) is determined through multiple methods such as file header information and extension. Then, the corresponding parsing strategy is selected according to the file type.

[0091] For image files containing text (such as scanned document images, screenshots containing text, etc.) or PDF files, advanced OCR (optical character recognition) technology is used. In the OCR process, the recognition accuracy is improved by training the model. The training data includes a large number of text samples with different fonts, font sizes, and clarity. The image is preprocessed, such as grayscale, binarization, and denoising, to improve the OCR effect. For example, the median filter algorithm is used to remove salt and pepper noise in the image, and the adaptive threshold algorithm is used for binarization. In the recognition process, the recognized characters are analyzed in context in combination with the language model to correct possible recognition errors.

[0092] For document-type unstructured data (such as PDF, Word documents, etc.), layout parsing is performed. By analyzing the structural information of the document (such as paragraphs, titles, tables, lists, etc.), the document content is divided into different logical units. A rule-based method can be used to determine the layout structure based on the document's format features (such as the position of the text box in PDF, font size changes, etc.) and semantic features (such as the format characteristics of the title, the indentation of the paragraph, etc.). At the same time, a machine learning method is used to train the layout parsing model, and the characteristics of the layout structure are learned through a large amount of annotated document data. For example, a convolutional neural network (CNN) is used to identify the table structure in the document, and the table is accurately divided by learning features such as table borders and cell contents.

[0093] Convert the parsed unstructured data into structured data. For text content, convert it into a format that can be stored in the database, such as dividing the text into paragraphs, sentences, words, and other levels, and adding corresponding labels to each level. For documents containing tables, convert the table content into a relational data structure, such as treating the rows of the table as records and the columns as fields. During the conversion process, ensure the accuracy and completeness of the data, while retaining the confidentiality level and permission marks, and report it to the search engine for storage in the same way as structured data.

[0094] Step 5: Pre-search processing

[0095] When a user enters the search page of the knowledge platform system and enters keywords, the system first obtains the user's basic information, including department, personal role, personal confidentiality level, etc. It then interacts with the enterprise's organizational structure database and authority management system to obtain the user's complete authority information.

[0096] Based on the obtained user authority information, including department authority, personal authority, personal confidentiality, etc., calculate the user's data visibility range. For department authority, by querying the department authority configuration table, determine the data range that the department can access. For example, a department can only access data related to a specific project. For personal authority, the data visibility range is further refined in combination with user roles (such as administrators, ordinary users, etc.) and special authorization situations (such as some users have additional access rights to specific data). For personal confidentiality, it is compared with the confidentiality mark of the data. Only when the user confidentiality level is higher than or equal to the data confidentiality level, the user has the right to access the data. By combining these authority data, the authority calculation algorithm is used to determine the user's data visibility range in this search.

[0097] Verify the permission information associated with the query request according to the verification items. First, check whether the organization permissions in the query object permission information are within the data permissions of the data to be queried. By comparing the permission identifier of the user's department with the permission identifier of the department to which the data belongs (stored in the metadata of the data), if the two match or the user's department has a higher permission, the corresponding verification item is satisfied. Secondly, check whether the object permissions in the query object permission information are within the data permissions of the data to be queried, which involves comparing the user's personal role permissions and the data access restrictions for personal roles. For example, a normal user role may only be able to read part of the data, while an administrator role can read and write all data. Finally, check whether the object confidentiality in the query object permission information is higher than part or all of the data confidentiality of the data to be queried. By comparing the user's personal confidentiality and the confidentiality mark of the data, if the user confidentiality is higher than the data confidentiality, the corresponding verification item is satisfied.

[0098] Step 6: Search via search engines

[0099] The search keywords entered by the user and the user's authority level and other data are constructed into a search request. The search request uses a specific protocol format, based on the HTTP RESTful API format. In the request, the keywords are segmented, and the keywords are decomposed using mature Chinese word segmentation algorithms and English natural language processing tools to improve the accuracy of the search. At the same time, the user's authority level information is used as one of the parameters of the request for authority filtering in the search engine. The constructed search request is sent to the search engine, a connection is established with the search engine through the network communication protocol, and the request is sent to the search interface of the search engine.

[0100] After receiving the search request, the search engine searches the index data based on the keywords. During the search process, the search results are filtered based on the user's authority level information, and only the search results visible to the user are returned. For the relevance sorting of the search results, the relevance score between the search term and the search result document is calculated based on algorithms such as the vector space model and TF-IDF. The search results are sorted in descending order according to the relevance score, and all the data with the highest relevance to the search term is returned to the user. When returning the results, the results are formatted and presented to the user in a user-friendly interface. For example, the results are presented in a list format, including information such as the title, abstract, and data source, so that users can quickly browse and select.

[0101] A second embodiment of the present invention is:

[0102] A permissioned intelligent search method based on a controlled system is proposed, which includes the following steps:

[0103] Domain Construction

[0104] Create a new domain of "aircraft fault handling experience" in the intelligent knowledge middle platform system. Allocate independent server resources and storage areas to this domain, and configure a database system suitable for processing large-scale fault data.

[0105] According to the results of preliminary investigation and analysis, detailed attribute information is set for the domain, including department (such as maintenance department, flight department), author (personnel who record fault experience), rank (to distinguish the weight of information provided by personnel with different experience levels), role (such as maintenance engineer, pilot), profession (such as mechanical engineering, avionics, etc.), model (aircraft model), confidentiality level (confidentiality level is divided according to the degree of impact of the fault on flight safety, such as high confidentiality level for failures involving critical flight systems), etc. These attributes will be used for subsequent strict permission filtering.

[0106] A unique work scenario identifier for a field is generated through a specific algorithm, which performs a hash operation (such as using the SHA-256 algorithm) based on the field name, main business type (aircraft fault handling), and data type involved (fault description, handling method, related pictures, etc.) to ensure the uniqueness of the identifier in the entire knowledge middle platform system. This identifier will run through the subsequent data processing process to distinguish different data processing paths and associate related data.

[0107] Reporting structured and unstructured data

[0108] Enter the knowledge platform backend management page and first select the data confidentiality level.

[0109] In addition to the confidentiality level, fill in other business data, including the aircraft model where the fault occurred, the time of the fault, the stage of the fault (take-off, cruising, landing, etc.), the role of the person who discovered the fault, etc. For structured data (such as fault code, fault parameters, etc.), fill in according to the predefined data format to ensure the integrity and accuracy of the data. For unstructured data (such as photos of the fault site, maintenance personnel's fault description notes, etc.), upload through a special file upload interface, and embed confidentiality and permission mark information during the upload process.

[0110] When uploading unstructured data, the data is encrypted locally and the AES encryption algorithm is used to convert the file into ciphertext before uploading to ensure the security of the data during transmission. At the same time, a data audit mechanism is established to record the person who uploaded the data, time, data source and other information for subsequent tracking and auditing.

[0111] For large amounts of structured and unstructured data, ETL tools are used for reporting. In the ETL data extraction phase, data is obtained from data sources (such as aircraft fault monitoring system database, maintenance record database, etc.) and encrypted (the encryption method is the same as manual uploading).

[0112] In the conversion phase, confidentiality and authority mark information is added to the data, and the data format is converted and cleaned (such as unifying the date format and processing outliers in the data) to ensure data quality. For unstructured data, such as electronic documents of aircraft maintenance manuals, preprocessing is performed, including extracting key information from the document (such as the content of the fault chapter) and converting it into semi-structured data.

[0113] In the loading phase, the processed structured and unstructured data are accurately transmitted to the knowledge center. During the transmission process, the CRC algorithm is used to perform integrity verification to ensure that the data is not damaged during the transmission process. In addition, in the entire ETL process, the data flow and synchronization frequency are monitored, and the data synchronization performance is optimized to ensure that the data synchronization is efficient and meets the cost requirements.

[0114] Reporting data to search engines

[0115] Data uploaded by users (whether manually uploaded or synchronized through ETL) needs to be fully integrated before being reported to the search engine. For structured data, data from different sources are merged and associated, such as aircraft fault codes and corresponding fault parameters, fault occurrence time, and other information are associated and stored.

[0116] For unstructured data, after being parsed (such as extracting and structuring text information from photos of the fault site through OCR text extraction, layout analysis and format conversion), it is integrated with structured data. At the same time, business data (such as aircraft model, fault occurrence stage), confidentiality level, authority data and fault-related data are integrated to form complete search index data.

[0117] During the integration process, the data is rechecked and cleaned to ensure data accuracy and consistency. The data quality monitoring dashboard displays the quality indicators (such as error rate and integrity ratio) of the data integration process in real time, and any problems that arise are handled in a timely manner.

[0118] The integrated data set is reported to the search engine through a dedicated interface. During the reporting process, the data is converted and packaged according to the search engine's index format requirements. For example, the data is converted into a document format that the search engine can recognize, and key information of the data (such as fault keywords, aircraft models, confidentiality levels, permission flags, etc.) is extracted as index fields to establish an efficient index structure for subsequent rapid retrieval. At the same time, a monitoring mechanism for data storage and reporting is established to track the progress of data reporting in real time. If a problem occurs, the retry mechanism is automatically triggered and a detailed error log is recorded.

[0119] Obtaining data through search engines

[0120] When a user enters a keyword (such as "landing gear rear slide movement blockage") on the search page, the system first obtains the user's basic information, including the user's department (such as the landing gear maintenance team of the maintenance department), major (such as mechanical maintenance major), authority (such as the read and write authority for landing gear fault data of a specific aircraft model), confidentiality level (determined according to the user's role and the level of fault handling), etc.

[0121] By interacting with the organizational structure database and permission management system, the user's permission information is further refined.

[0122] According to the user's permission information and search keywords, a search request is constructed. In the process of constructing the request, the keywords are segmented and the Jieba segmentation algorithm is used to improve the accuracy of the search. At the same time, the user's permission level information is used as an important parameter of the request for strict permission filtering in the search engine.

[0123] After receiving the search request, the search engine searches the index data based on the keywords. During the search process, the search results are filtered based on the user's authority level information, and only the results within the user's visible range are returned. For the relevance sorting of the search results, the relevance score between the search term and the search result document is calculated using algorithms such as the vector space model and TF-IDF. The search results are sorted in descending order based on the relevance score.

[0124] The data most relevant to the search term is returned to the user. When the results are returned, they are formatted and presented to the user in a user-friendly interface. For example, the results are presented in a list format, including the fault title, fault summary, aircraft model where the fault occurred, data source (such as which maintenance personnel provided it), etc., to facilitate users to quickly browse and select. At the same time, a feedback mechanism for search results is established to allow users to evaluate the accuracy and relevance of search results, so as to optimize the search engine algorithm later.

[0125] The method of the present invention has the following characteristics:

[0126] Security and privacy protection: Through a controlled system, ensure that users' sensitive information is protected to avoid information leakage and unauthorized access.

[0127] Permission management: Implement permission management for different users or user groups to ensure that only users with corresponding permissions can access specific sensitive information.

[0128] Intelligent Search: Uses artificial intelligence technology to provide more accurate and personalized search results to meet users' needs for sensitive information, and combines it with permission management to ensure that users can only see information they are authorized to view.

[0129] In summary, the "authorized intelligent search method based on a controlled system" of the present invention provides intelligent and personalized search services while ensuring information security and privacy, and provides users with a safe and reliable information retrieval experience.

[0130] The third embodiment of the present invention provides a permissioned intelligent search system based on a controlled system, which is applied to the method described above, and the system includes:

[0131] Building unit: configured to build a domain in the intelligent knowledge platform;

[0132] Reporting unit: configured to report structured and unstructured data with confidentiality and authority marks to the intelligent knowledge platform;

[0133] Processing unit A: configured to process structured data;

[0134] Parsing unit: configured to parse unstructured data;

[0135] Processing unit B: configured for pre-search processing;

[0136] Search unit: configured to perform a search through a search engine.

[0137] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the field can easily think of various equivalent modifications or substitutions within the technical scope disclosed by the present invention, and these modifications or substitutions should all be covered within the protection scope of the present invention.

Claims

1. A permissioned intelligent search method based on a controlled system, characterized in that: The following steps are involved: Step 1: Build domains in the intelligent knowledge platform; Step 2: Report structured and unstructured data with confidentiality and permission marks to the intelligent knowledge center; Step 3: Process the structured data; Step 4: Parse the unstructured data; Step 5: Pre-search processing; Step 6: Search through search engines.

2. The method for intelligent search with authority based on a controlled system according to claim 1, characterized in that: Step 1 includes the following steps: Step 1.1 Investigate user needs in specific areas, conduct business scenario analysis, collect needs and business information, and extract key needs and scenario features; Step 1.2: In the intelligent knowledge platform, create a specific domain based on the information obtained in step 1.1, allocate a dedicated storage area for the domain, and configure corresponding system resources; Step 1.3 generates a unique work scenario identifier for the field.

3. The permission-based intelligent search method based on a controlled system according to claim 2 is characterized in that: Step 2 includes: When choosing to upload data locally, develop an upload interface. In the interface design, use an encrypted transmission protocol to ensure the security of data transmission. For the uploaded data, perform preliminary encryption processing on the client side and convert the data into ciphertext before uploading. At the same time, embed confidentiality and permission mark information in the uploaded data. The mark information is stored in a specific data structure, which contains the confidentiality level of the data and the permission information of different user roles. When reporting data through the ETL data synchronization tool, set up data encryption and logo embedding links in the ETL process. In the data extraction stage, the data in the data source is encrypted, and the encryption method is consistent with local upload; in the data conversion stage, the confidentiality level and permission logo information are added to the data to form structured data with security attributes; in the data loading stage, ensure that the data is accurately transmitted to the intelligent knowledge center, and perform integrity verification during the transmission process. At the same time, in the ETL process, the performance of data synchronization is optimized.

4. The method for intelligent search with authority based on a controlled system according to claim 3 is characterized in that: Step 3 includes: (1) The data verification module in the intelligent knowledge center uses a multi-level verification algorithm to verify structured data, including: Verify the format of structured data: Check whether the data conforms to the specified structure according to the predefined data model; Verify the integrity of structured data: Verify the integrity of structured data by comparing the checksum of the calculated data with the checksum of the original data. For key data fields, perform logical checks; If the structured data verification fails, the structured data will be marked as an error state, and detailed error information will be recorded. At the same time, the alarm mechanism will be triggered to notify the administrator; (2) The data improvement module in the intelligent knowledge center performs data cleaning operations Identify and process duplicate records, erroneous data, and incomplete data in structured data. For duplicate records, use a deduplication algorithm based on data features to remove completely duplicate records. For erroneous data, repair or delete it based on the data type and predefined error handling strategy. For incomplete data, complete it through data association and filling algorithms. (3) Perform data standardization: For structured data from different sources, unify the data representation format, unify the units for numerical data represented by different units, and ensure the consistency of data within the knowledge platform by establishing data mapping tables and conversion rules. During the data standardization process, monitor data quality indicators in real time.

5. The method for intelligent search with authority based on a controlled system according to claim 4 is characterized in that: Step 4 includes the following steps: Step 4.1 After the unstructured data is uploaded to the intelligent knowledge platform, it is sent to the file parsing module in the intelligent knowledge platform; the file parsing module identifies the unstructured data, determines the type of the file, and then selects the corresponding parsing strategy based on the file type: For image files or PDF files containing text, optical character recognition is used to parse them; For unstructured data such as documents, layout analysis is performed: by analyzing the structural information of the document, the document content is divided into different logical units; Step 4.2 converts the parsed unstructured data into structured data form, including: For text content, convert it into a format that can be stored in the database; For documents containing tables, convert the table contents into a relational data structure; During the conversion process, the accuracy and completeness of the data are ensured, while the confidentiality level and permission marks are retained, and the data is reported to the search engine for storage in the same way as structured data.

6. The method for intelligent search with authority based on a controlled system according to claim 5, characterized in that: In step 4.1, an optical character recognition method is used to parse the image file or PDF file containing text, including: in the optical character recognition process, the recognition accuracy is improved by training the model, the training data includes a large number of text samples of different fonts, font sizes, and clarity, and the image is pre-processed; in the recognition process, the recognized characters are analyzed in context in combination with the language model to correct possible recognition errors; For unstructured document data, layout analysis is performed by using a rule-based method to determine the layout structure according to the format characteristics and semantic characteristics of the document. At the same time, a machine learning method is used to train the layout analysis model and learn the characteristics of the layout structure through a large amount of annotated document data.

7. The method for intelligent search with authority based on a controlled system according to claim 6, characterized in that: Step 5 is that when the user enters the search page of the intelligent knowledge platform and enters keywords, the intelligent knowledge platform first obtains the user's basic information, and then interacts with the enterprise's organizational structure database and permission management system to obtain the user's complete permission information; Based on the obtained user authority information, the user's data visibility range is calculated. For departmental authority, the departmental authority configuration table is queried to determine the data range that the department can access. For personal authority, the data visibility range is further refined in combination with the user role and special authorization conditions. For personal confidentiality, it is compared with the confidentiality mark of the data. Only when the user confidentiality level is higher than or equal to the data confidentiality level, the user has the right to access the data. By combining the above authority data, the user's data visibility range in this search is determined. Verify the permission information associated with the query request according to the verification items: first, check whether the organization permissions in the query object permission information are within the data permissions of the data to be queried; second, check whether the object permissions in the query object permission information are within the data permissions of the data to be queried; finally, check whether the object confidentiality level in the query object permission information is higher than part or all of the data confidentiality level of the data to be queried.

8. The method for intelligent search with authority based on a controlled system according to claim 7, characterized in that: Verify the permission information associated with the query request based on the verification items, including: First, check whether the organization authority in the query object authority information is within the data authority of the data to be queried: by comparing the authority identifier of the user's department with the authority identifier of the department to which the data belongs, if the two match or the user's department has a higher authority, the corresponding verification item is met; Secondly, check whether the object permissions in the query object permission information are within the data permissions of the data to be queried: including comparing the user's personal role permissions and the data access restrictions for the personal role; Finally, check whether the object confidentiality level in the query object permission information is higher than the partial or all data confidentiality levels of the data to be queried. By comparing the user's personal confidentiality level with the confidentiality level mark of the data, if the user confidentiality level is higher than the data confidentiality level, the corresponding verification item is met.

9. The method for intelligent search with authority based on a controlled system according to claim 8, characterized in that: Step 6 includes the following steps: Step 6.1: construct a search request with the search keyword input by the user and the user's authority level data, send the constructed search request to the search engine, establish a connection with the search engine through the network communication protocol, and send the request to the search interface of the search engine; In this step, the search request adopts the HTTP-based RESTful API format. In the search request, the keywords are segmented and decomposed using the Chinese word segmentation algorithm and the English natural language processing tool to improve the accuracy of the search. At the same time, the user's authority level information is used as one of the request parameters for authority filtering in the search engine. Step 6.2 After receiving the search request, the search engine searches the index data based on the keywords; During the search process of this step, the search results are filtered based on the user's permission level information, and only search results visible to the user are returned; the relevance score between the search term and the search result document is calculated, and the search results are sorted in descending order according to the relevance score, and all data with the highest relevance to the search term is returned to the user. When returning the results, the results are formatted and presented to the user in a user-friendly interface.

10. An authorized intelligent search system based on a controlled system, characterized in that: The method as applied to claim 9, characterized in that the system comprises: Building unit: configured to build a domain in the intelligent knowledge platform; Reporting unit: configured to report structured and unstructured data with confidentiality and authority marks to the intelligent knowledge platform; Processing unit A: configured to process structured data; Parsing unit: configured to parse unstructured data; Processing unit B: configured for pre-search processing; Search unit: configured to perform a search through a search engine.

Images