Unified search and page configuration rendering method and system based on data model

Abstract

The invention discloses a unified search and page configuration rendering method and system based on a data model. The method comprises the following steps: S1, data acquisition and model mapping; s2, indexing and synchronizing data; s3, unified search request processing; s4, processing and returning a result set; and S5, dynamically rendering the page. The system comprises a data source layer, a data unification layer, an index layer, an application service layer and a display layer. The invention provides a set of end-to-end integrated solution for solving the problems that multi-source heterogeneous data is difficult to integrate, the cross-system search efficiency is low, page rendering lacks flexibility and the like. According to the scheme, by constructing a standardized data model, a unified search engine index system and a dynamic page rendering mechanism driven by configuration, a complete technical link from data source access to front-end display is achieved, and the efficiency and expansibility of the system in the aspects of data management, information retrieval and page display are remarkably improved.

Description

Technical Field

[0001] This invention relates to a page configuration rendering method and system, and more particularly to a unified search and page configuration rendering method and system based on a data model. Background Technology

[0002] With the accelerating pace of informatization, the number of business systems relied upon by enterprises and institutions continues to increase, and data sources are becoming increasingly diversified, forming a vast and complex multi-source heterogeneous data environment. Data sources include relational databases, NoSQL databases, log files, document repositories, and search engine indexes, with significant differences in data formats, structures, and storage methods. How to efficiently integrate this heterogeneous data and achieve unified retrieval and consistent display across systems has become a core technical challenge in the fields of information management and big data governance.

[0003] However, traditional multi-system data processing solutions have the following shortcomings: 1) Database Join Queries: Traditional methods typically employ cross-database join queries, using SQL to link multiple structured databases and achieve a certain degree of cross-system access. However, this method is limited to structured data and is almost powerless for semi-structured and unstructured data. Furthermore, as the data scales up, the efficiency of join queries drops sharply, making it difficult to support the needs of large-scale heterogeneous data governance.

[0004] 2) Middleware-based data integration: Another common approach is to build a middleware layer, which uses data extraction, cleaning, and transformation (ETL) processes to centrally store multi-source data in a data warehouse for unified access. While this method addresses heterogeneous data access to some extent, it has significant drawbacks: firstly, data synchronization is delayed, making real-time searching difficult; secondly, when the source data model changes, extensive manual maintenance is required, resulting in insufficient flexibility and adaptability.

[0005] 3) Distributed Search Engine Solution: In recent years, distributed search technologies, such as distributed search engines, have been widely used. These solutions support full-text retrieval of both structured and unstructured data and offer strong scalability. However, in practice, each business system often deploys its own independent search engine cluster, lacking a unified index architecture and model standards. This fragmentation leads to slow response times, inconsistent result formats, inconsistent sorting rules, and a poor user experience when searching across systems.

[0006] 4) Static Template Development Mode: Traditional systems commonly use static templates for page rendering at the result display level. This method heavily relies on hard coding; any adjustment to page fields or layout requires redevelopment, resulting in long modification cycles and insufficient flexibility. Furthermore, the lack of a unified configuration-driven mechanism across different systems leads to significant differences in page style and interaction, increasing maintenance costs.

[0007] To address the aforementioned issues, the Chinese patent CN110837585A, "Multi-source Heterogeneous Data Association Query Method and System," while alleviating heterogeneous data access problems to some extent, does not fully standardize and unify entity models and lacks precise processing for data standardization and verification, resulting in the need to strengthen the consistency and compatibility of the integrated data.

[0008] The Chinese patent CN106202564A, entitled "Ontology Relationship Data Search Framework Based on Distributed Search Engine", has limited support for field mapping and dynamic page display of multi-source heterogeneous data, and lacks system scalability and page configuration flexibility.

[0009] The Chinese patent CN114489610A, "Method and System for Configuring Dynamic Components for Page Visualization," fails to combine a unified global data model and unified search results, making it difficult to solve the comprehensive problem of data association retrieval across multiple systems and unified page display.

[0010] Therefore, this invention aims to solve the following technical problems commonly found in current multi-system integration and big data governance scenarios: Integrating multi-source heterogeneous data is challenging: Existing business systems are numerous, with diverse data sources, including structured databases, semi-structured documents, unstructured text, and multimedia data. Due to significant differences in format and structure, traditional database joint query methods are only applicable to structured data and cannot be compatible with or uniformly process diverse heterogeneous information, easily forming "data silos" that severely restrict data sharing, integration, and in-depth utilization.

[0011] Poor cross-system search experience: Existing multi-system search relies heavily on independent search engines, lacking a unified index and rule system, resulting in slow cross-system search response and a poor user experience. Inconsistent result formats and a lack of standardized sorting across different systems make it difficult to meet business needs for accurate and standardized retrieval.

[0012] Traditional page displays lack dynamic configuration capabilities: Current page displays mostly rely on hard-coded templates. Once business requirements change, page adjustments require developer intervention, resulting in long modification cycles, low efficiency, and an inability to flexibly adapt to rapidly changing business scenarios. Furthermore, significant differences in page styles between different systems increase later maintenance costs and reduce development efficiency.

[0013] Poor system scalability and maintainability: When adding new data sources or page requirements, the existing system often needs to make large-scale modifications to the data processing flow and front-end code, which increases development complexity and reduces the system's flexibility and maintainability in response to business changes. Summary of the Invention

[0014] To address the shortcomings of the aforementioned technologies, this invention provides a unified search and page configuration rendering method and system based on a data model. By constructing a fully automated and configurable solution that integrates backend data and dynamically displays the frontend, it achieves efficient governance, agile search, and flexible display of multi-source heterogeneous data, significantly improving the system's scalability and maintainability.

[0015] To solve the above technical problems, the technical solution adopted by this invention is: a unified search and page configuration rendering method based on a data model, comprising the following steps: Step S1: Data Acquisition and Model Mapping; Step S2: Data indexing and synchronization; Step S3: Unified search request processing; Step S4: Result set processing and return; Step S5: Dynamic page rendering. Service Preferably, it includes the following steps: Step S1: Obtain raw data from multi-source heterogeneous data sources, and perform field mapping and data type conversion on the raw data according to the pre-built unified data model UDM; Step S2: The index management module imports the standardized data output in step S1 into the distributed search engine cluster and generates index mapping configuration based on UDM. At the same time, it uses timed batch or real-time streaming methods to ensure that the index library is consistent with the source data. Step S3: After the user initiates a search request on the front-end interface, the unified search module receives the request, parses the keywords, filtering conditions and sorting parameters, and the system dynamically constructs a distributed search engine query DSL based on the parsing results and sends the query request to the distributed search engine. Step S4: After receiving the raw JSON results returned by the distributed search engine, the unified search module performs secondary processing on the result set, including field extraction, redundant information cleanup, unified sorting, and formatting encapsulation. The processed results are converted into a unified and standardized JSON format and returned to the front-end application. Step S5: After receiving the unified result set, the page rendering module automatically matches appropriate components and styles according to the preset page configuration rule library, and fills the data into the component attributes. The rendering engine then generates diverse view pages and dynamically returns them to the user.

[0016] Preferably, in step S1, constructing a unified data model includes: Define a unified data model: Use a structured description language to define the field structure of business entities, and introduce _meta metadata on this basis to extend the semantics of the behavior of fields in the process of retrieval, indexing and display; Data Mapping and Transformation Module: This module maps fields from heterogeneous source systems to UDM through configuration rules, and applies _meta rules to complete automatic standardization during the transformation process; Validation and cleaning: Before the data is written to the UDM, the system performs automatic validation and cleaning based on the constraints in _meta.

[0017] Preferably, in step S1, the behavioral semantics of the description field are expanded, including: Structure definition, index control, and presentation binding: used for component selection, formatting, and desensitization when driving page rendering; The data mapping and transformation module includes the following operations: Rule storage: Mapping rules are stored in a database or configuration file, supporting hot reloading and version management; Format conversion: Automatically selects the conversion function based on _meta.indexType; Constraint application: When _meta.isFilterable=true, its data type is automatically validated to ensure its availability for subsequent queries; Legality verification and cleanup include the following operations: Type validation: Date fields must conform to ISO standard format; Limit the range of values; Missing fields handling: For required fields that are missing or have invalid values, perform repair, desensitization, or tagging based on _meta.maskPolicy; Logs and Tracking: Non-compliant data will be recorded and tracked through the monitoring system to ensure a closed loop for data quality.

[0018] Preferably, step S2 specifically includes the following process: Automated indexing and word segmentation strategy: The index management module is automatically triggered when the UDM is updated, reads the _meta metadata in the JSON Schema, and automatically generates the index mapping for the search engine; Data synchronization module: Standardized data is imported into the search engine's index library through the synchronization module, supporting: real-time synchronization based on message queue / streaming processing, and scheduled batch synchronization suitable for low-frequency updates or offline tasks; Unified Search Module: It provides an API as the sole entry point, allowing users to initiate searches without needing to concern themselves with the underlying engine details; Result set formatting and sorting: The unified search module cleans and unifies the results according to the UDM definition, including: extracting core fields, removing redundant content, applying configurable sorting and pagination strategies, and outputting a unified result set that conforms to JSON Schema.

[0019] Preferably, the workflow of the unified search module includes: Receive user search requests: Front-end input of keywords, filtering conditions, and sorting rules; The analysis of keywords and parameters includes: 1) Parse parameters: entity type, keyword, filter condition, sort field; 2) Load UDM: Read the UDM definition of the target entity and determine which fields can be used for searching, filtering, and sorting; Construct a search engine query DSL: dynamically generate query weights based on _meta.searchWeight, and construct filtering conditions based on _meta.isFilterable; Send a query to the search engine cluster: Initiate a request to the search engine, enabling parallel cross-index queries; Receive search engine results: Retrieve the raw JSON results; Result formatting: Clean up redundant fields, sort them uniformly according to business rules, and return a result set consistent with UDM.

[0020] Preferably, in step S5, after the user initiates a search request, the unified search module returns standardized results, and the page rendering module performs component selection, data binding, and final rendering according to the page configuration rules, specifically including the following operations: Define page configuration rule library: The system provides a visual interface for administrators to define page configuration rules. The rules are stored in JSON or YAML format and include: page layout, component type, mapping relationship between fields and component attributes, and interaction behavior rules; Page rendering module: Its task is to automatically generate pages based on configuration rules and a unified search result set; Build the page layout skeleton: The final generated page can display tables, charts, and cards, and also supports interactive behaviors. When business changes occur, only the JSON configuration file needs to be modified to take effect immediately, without modifying or compiling the front-end code.

[0021] Preferably, the workflow of the page rendering module is as follows: Receive the JSON result set returned by the unified search module; Load the corresponding page configuration rules based on the entity type of the result set; Construct the page layout skeleton; Dynamically instantiate UI components according to their definitions; Populate component properties with search result data to achieve data-driven rendering; Insert the rendered component into the specified position in the DOM.

[0022] Preferably, in step S5, the dynamic rendering of the page includes the following process: Step 1: User initiates a search request: The user enters search criteria on the front-end application interface and clicks the search button; Step 2: Front-end application calls unified search module: The front-end application encapsulates the user input query parameters into a unified request format and sends it to the unified search module via REST / GraphQL protocol; Step 3: The unified search module performs the retrieval: After receiving the request, the search module parses the request parameters, loads the corresponding entity's UDM definition, constructs the search engine DSL according to the _meta rules, and sends the query to the search engine cluster in parallel. Step 4: The search engine cluster returns the raw JSON results: After each index shard is retrieved, it returns the raw JSON structure containing the hit documents and metadata to the unified search module. Step 5: Unify the formatting and merging of search results: The search module removes redundant fields, cleans the data according to the sorting and field whitelist in UDM_meta, merges the results from multiple shards, and generates a unified JSON result set; Step 6: The unified search module returns a standardized result set: The formatted unified JSON result set is returned to the front-end application via a REST response; Step 7: The page rendering module loads the page configuration rules: After receiving the result set, the rendering module retrieves the corresponding layout and component configurations from the page configuration rule library based on the entity type of the current page. Step 8: The page rendering module dynamically generates components according to rules: Build the page skeleton according to the layout defined in the configuration; For each component, instantiate the corresponding UI component and map the specified field dataIndex from the JSON result set to the component property; Step 9, Page rendering module application - meta rendering tips: Based on uiComponentHint and formatHint in UDM_meta, the fields will be automatically formatted or rendered as map markers or data anonymization will be performed. Step 10: The page rendering module completes the DOM update: After rendering, the component is inserted into the corresponding container position on the front-end page, the front-end route or view is updated, and the final search results page is displayed to the user; Step 11: User Viewing and Interacting: Users sort, filter, or click on rows on the rendered page to trigger the display of details. Such interactions will trigger corresponding client-side logic or secondary queries, and the above process loop will be entered again.

[0023] A system for a unified search and page configuration rendering method based on a data model, the system comprising: Data source layer: It contains raw data from multiple heterogeneous sources, including relational databases, log files, JSON documents, API interfaces, and various business systems; Data unification layer: includes a data acquisition module and a data model building and mapping module. The data acquisition module connects to different source systems to complete data extraction and aggregation; the data model building and mapping module maps heterogeneous data to a standardized data model to form standardized data with consistent semantics and a unified structure. The index layer consists of an index management module and a distributed search engine cluster. The index management module generates index configurations based on a unified data model and defines field mapping, word segmentation strategies, and weight rules. The search engine cluster is responsible for the index storage and query execution of the data. Application service layer: When a user initiates a search request, the unified search module parses the query conditions based on the unified data model and calls the distributed search engine cluster to perform the retrieval. After the results are returned, the unified search module formats and standardizes the results and outputs a unified result set. Presentation Layer: The presentation layer consists of a page rendering module, a page configuration rule base, and a front-end user interface. After the front-end user enters a search request, the unified result set returned by the system will be processed by the page rendering module, and the page configuration rule base will complete the automatic generation and display of the page.

[0024] Compared to existing technologies, this invention uses a unified data model (UDM) as the core driver to achieve deep decoupling and automated linkage between the data layer, service layer, and presentation layer. Its specific advantages are as follows: 1) Model-driven automated data governance: This invention not only defines a unified model, but also provides a mechanism that can automatically generate and manage the index mapping and data verification rules of the distributed search engine based on UDM, solving the problem of the disconnect between the data model and the actual index and data quality control in the existing technology, and greatly improving the accuracy and automation of data fusion.

[0025] 2) Abstraction and unification of query language: This invention designs a unified search service that dynamically compiles the general query requests from the front end (which are independent of the specific engine) into precise native query statements of the search engine. This allows upper-layer applications to be free from concern about the complexity of the underlying search engine and ensures the consistency of query logic and sorting rules across all data sources.

[0026] 3) Complete separation of data and view, and configurable rendering: A closed-loop system of page configuration rules and dynamic rendering engine is proposed. This system can not only dynamically generate page layouts and components based on configuration, but also automatically and accurately map the returned standardized data to the corresponding component properties. When business requirements change (e.g., adding a display field), only the JSON configuration needs to be modified to take effect, without any front-end code development, achieving true "what you configure is what you get".

[0027] 4) End-to-end highly scalable architecture: Through the above model-driven and service decoupling design, adding a new data source only requires defining its mapping rules to the UDM; adding a new page display style only requires creating a new page configuration. The core search and rendering services do not need any changes, which significantly enhances the scalability and maintenance efficiency of the system. Attached Figure Description

[0028] Figure 1 This is a system architecture diagram of the present invention.

[0029] Figure 2 This is a schematic diagram illustrating the construction of the unified model for multi-source heterogeneous data in this invention.

[0030] Figure 3 This is a flowchart illustrating the unified search workflow of the present invention.

[0031] Figure 4 This is a sequence diagram for the dynamic rendering of pages in this invention.

[0032] Figure 5 This is a schematic diagram illustrating the high generalization capability of the system of the present invention.

[0033] Figure 6 This is the overall flowchart of the present invention. Detailed Implementation

[0034] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0035] This invention provides a unified search and page configuration rendering method based on a data model, addressing issues such as the difficulty in integrating multi-source heterogeneous data, low cross-system search efficiency, and lack of flexibility in page rendering. It proposes an end-to-end integrated solution. This solution achieves a complete technical chain from data source access to front-end display by constructing a standardized data model, a unified search engine indexing system, and a configuration-driven dynamic page rendering mechanism, significantly improving the system's efficiency and scalability in data governance, information retrieval, and page display.

[0036] like Figure 6 The method for unified search and page configuration rendering based on a data model, as shown, includes the following five steps: Step S1, Data Acquisition and Model Mapping: The data acquisition service is responsible for obtaining raw data from multiple heterogeneous data sources, such as databases, tables, API interfaces, and log files. Subsequently, the data unification service performs field mapping and data type conversion on the raw data according to a predefined unified data model (UDM).

[0037] For example, the source field `station_nm` is mapped to `monitorSiteName`, `sample_time` is converted to a compliant `sampledAt` (DATETIME type), and validity checks and data cleaning are performed. This step ensures the consistency of naming, format, and quality of the data entering subsequent processing.

[0038] Constructing a Unified Multi-Source Heterogeneous Data Model (UDM): The Unified Multi-Source Heterogeneous Data Model (UDM) construction mechanism proposed in this invention is the core foundation for realizing unified cross-system search and dynamic page rendering. This model not only uses JSON Schema to define the data structure, but also introduces extended metadata (_meta) to drive subsequent automatic index generation, search DSL construction, and page rendering binding, thereby achieving a closed-loop end-to-end system of "one definition, three effects".

[0039] like Figure 2 The diagram illustrating the construction of a unified model for multi-source heterogeneous data demonstrates the entire process from heterogeneous source data access to model definition, mapping transformation, and validity verification.

[0040] (1) Definition of Unified Data Model (UDM): UDM uses a structured description language (such as JSON Schema) to define the field structure of business entities, and on this basis, introduces _meta metadata to extend the semantics describing the behavior of fields during retrieval, indexing, and display, including: Structure definition: field names, data types, value ranges, required fields, etc.; Index control: isSearchable, isFilterable, isSortable, searchWeight, indexType, analyzer; Display bindings: uiComponentHint, formatHint, and maskPolicy are used to select, format, and mask components during page rendering.

[0041] For example, when defining a water quality monitoring point UDM: { "$schema": "http: / / json-schema.org / draft-07 / schema#", / / Using the JSON Schema standard "title": "WaterQualityMonitoringSite", / / Entity name: Water Quality Monitoring Site "type": "object", "properties": { "siteId": { / / Field: Unique ID of the monitoring point (primary key / foreign key) "type": "string", "description": "Unique ID of the monitoring point", "_meta": { / / Extended metadata: drives indexing / querying / rendering "isSearchable": true, / / Allows keyword searching "isFilterable": true, / / Allow as a filter condition "isSortable": false, / / Does not participate in sorting "searchWeight": 1.0, / / Search weight (used for multi_match weighting) "indexType": "keyword", / / Index type: exact match "uiComponentHint": "text" / / Front-end rendering suggestion: plain text } }, "siteName": { / / Field: Name of the monitoring point (main search field) "type": "string", "description": "Monitoring point name", "_meta": { "isSearchable": true, / / Keyword search is supported "isFilterable": false, / / Not used for filtering "isSortable": true, / / Enables sorting (must be derived from a sortable field) "searchWeight": 5.0, / / Higher search weight "indexType": "text", / / Word segmentation text index "analyzer": ["chinese_smart","pinyin"], / / Word segmentation / pinyin search "uiComponentHint": "label" / / Front-end rendering suggestion: title / label } }, "location": { / / Field: geographic coordinates "type": "object", "description": "Location", "properties": { "lat": { "type": "number"}, "lon": { "type": "number"}}, "_meta": { "indexType": "geo_point", / / Spatial index type (point) "uiComponentHint": "mapPoint" / / Front-end rendering: map marker point } }, "lastSampledAt": { / / Field: most recent sampling time "type": "string", "format": "date-time", / / JSON Schema time format validation "description": "Recent sampling time", "_meta": { "isSearchable": false, / / Not included in keyword search "isFilterable": true, / / Time filtering is possible "isSortable": true, / / Sort by time "indexType": "date", / / Time index type "formatHint": "yyyy-MM-dd HH:mm" / / Front-end formatting prompt } } }, "required": ["siteId", "siteName", "lastSampledAt"] / / Collection of required fields } At this stage, the _meta metadata not only serves index management (indexType, analyzer), but also directly drives query weight (searchWeight) and page rendering (uiComponentHint, formatHint, maskPolicy), forming end-to-end consistency.

[0042] Through event mechanisms (such as SchemaChanged), when the UDM is updated, it can trigger automatic index updates and page rendering configuration synchronization.

[0043] (2) Data mapping and transformation services: This service maps fields from heterogeneous source systems to UDM through configuration rules, and applies _meta rules to automatically standardize the data during the transformation process. Specifically, it includes the following operations: Rule storage: Mapping rules are stored in a database or configuration file, supporting hot reloading and version management; Format conversion: Automatically selects the conversion function based on _meta.indexType, such as string to number, latitude and longitude concatenation to geo_point, and time field to ISO standard; Constraint application: When _meta.isFilterable=true, its data type is automatically validated to ensure that subsequent queries are available.

[0044] At this stage, the _meta metadata ensures that fields are named uniformly and formatted in a standardized manner, and prepares for subsequent index generation and search DSL construction.

[0045] Compared to traditional solutions that only perform field mapping, this invention pre-defines indexes and rendering rules within the UDM, avoiding configuration drift caused by multiple maintenance steps.

[0046] (3) Legality verification and cleaning: Before the data is written to the UDM, the system performs automatic validation and cleaning based on the constraints in _meta, including the following operations: Type validation: Date fields must conform to ISO standard format; Numerical range: For example, the water quality grade must be between 1 and 5; Missing fields handling: For required fields that are missing or have invalid values, perform repair, desensitization, or tagging based on _meta.maskPolicy; Logs and Tracking: Non-compliant data will be recorded and tracked through the monitoring system to ensure a closed loop for data quality.

[0047] At this stage, the formatHint and maskPolicy properties in the _meta metadata not only constrain data storage but also ensure that the front-end presentation layer automatically applies the same rules (e.g., uniform time formatting and automatic de-identification of sensitive information). Through these operations, data quality and page display rules are synchronized, ensuring that the results seen by the user are consistent with the data layer.

[0048] Step S2, Data Indexing and Synchronization: The index management module imports the standardized data output from step S1 into the distributed search engine cluster and generates an index mapping configuration based on the UDM. This configuration includes field types, word segmentation strategies, weighting rules, etc., enabling text fields to support multi-modal word segmentation (such as Chinese word segmentation, Pinyin word segmentation, and N-gram), and numeric and time fields to support range queries and sorting. The data synchronization service can use scheduled batch or real-time streaming methods to ensure that the index database remains consistent with the source data.

[0049] The index management module, based on UDM's automated indexing and efficient search architecture, is the core of achieving high-speed cross-system retrieval. Through UDM-driven automated index building, unified search services, and standardized result processing, this invention achieves breakthroughs in search consistency, scalability, and efficiency. Its core lies in: strong consistency between the index structure and data model, automated transformation of the query process, and a unified cross-system search experience based on _meta.

[0050] (1) Automated indexing and word segmentation strategies: The index management module is automatically triggered when the UDM is updated. It reads the _meta metadata in the JSON Schema and automatically generates the index mapping for the search engine.

[0051] For example, the mapping automatically generated by UDM is as follows: { "mappings": { "properties": { "siteId": { "type": "keyword"}, / / keyword comes from _meta.indexType "siteName": { "type": "text", / / Segmented text "analyzer": "chinese_smart", / / Default Chinese word segmenter "fields": { "pinyin": { "type": "text", "analyzer": "pinyin"} / / Pinyin subfield for easy homophonic searching } }, "location": { "type": "geo_point"}, / / Spatial retrieval "lastSampledAt": { "type": "date"}, / / Time-based search / sorting "waterQualityLevel": { "type": "integer"} / / Example of a business-extended field } } } At this stage, information such as _meta.indexType and analyzer directly drive the automatic generation of Mapping, avoiding the complexity of manually configuring the search engine index in traditional solutions.

[0052] When _meta.isSortable=true, the system automatically derives the keyword subfield to ensure that sorting is available.

[0053] The index management module supports dynamic sharding and replica count adjustment, achieving high availability and high performance under large-scale data.

[0054] (2) Data synchronization module: The standardized data is imported into the search engine's index library through the synchronization module, supporting: real-time synchronization: based on message queues / streaming processing to ensure high timeliness; scheduled batch synchronization: suitable for low-frequency updates or offline tasks.

[0055] During this phase, constraint information in the _meta metadata is reused to perform validity checks (such as date format and numerical range) during data synchronization, ensuring that the data entering the index meets expectations.

[0056] The synchronization module supports breakpoint resume and idempotency mechanisms to ensure the consistency between the index and the source system in abnormal situations.

[0057] It supports SchemaChanged event-driven updates, which automatically trigger the synchronization process when the UDM version is updated, ensuring that data and indexes always evolve in sync.

[0058] (3) Unified Search Module: The unified search module provides an API as the sole entry point, allowing users to initiate searches without needing to understand the underlying engine details. Its processing flow is as follows: Figure 3 The unified search workflow diagram shown primarily includes: Receive user search requests: The front end inputs keywords, filtering conditions, and sorting rules.

[0059] Parse keywords and parameters: Parse parameters: parse entity (entity type), keyword (keyword), filter (filtering conditions), sort (sorting fields); Load UDM: read the UDM definition of the target entity and determine which fields can be used for searching, filtering, and sorting.

[0060] Construct a search engine query DSL: dynamically generate query weights based on _meta.searchWeight, and construct filtering conditions based on _meta.isFilterable.

[0061] Send a query to the search engine cluster: Initiate a request to the search engine, enabling parallel cross-index queries.

[0062] Receive search engine results: Get the raw JSON results.

[0063] Result formatting: Clean up redundant fields, sort them uniformly according to business rules, and return a result set consistent with UDM.

[0064] At this stage, the query field whitelist, weights, and filtering conditions are all derived from UDM_meta, avoiding the inconsistencies caused by the separation of query parameters and index configuration in traditional systems.

[0065] The system supports parallel queries across indexes and result merging, and provides multi-tenant isolation and access control to ensure data boundaries and security.

[0066] (4) Result set formatting and sorting: The search engine returns results containing a large number of redundant fields (such as _index, _score). The unified search module cleans and unifies the results according to the UDM definition, including: extracting core fields, removing redundant content, applying configurable sorting and pagination strategies, and outputting a unified result set conforming to JSON Schema.

[0067] In this phase, `_meta.formatHint` and `maskPolicy` are directly applied to the result formatting stage, ensuring that the output conforms to both business display specifications and data anonymization requirements. Search results from different systems are unified into the same JSON Schema structure, improving the usability of cross-system data consumption.

[0068] Step S3: Unified search request processing: After a user initiates a search request on the front-end interface, the unified search module receives the request and parses the keywords, filtering conditions, and sorting parameters. Based on the parsed results, the system dynamically constructs a distributed search engine query DSL and sends the query request to the distributed search engine. Through this mechanism, users can complete cross-data source and cross-system search operations through a single unified entry point.

[0069] Step S4, Result Set Processing and Return: After receiving the raw JSON results from the distributed search engine, the unified search module performs secondary processing on the result set, including field extraction, redundant information cleanup, unified sorting, and formatting encapsulation. The processed results are converted into a standardized JSON format and returned to the front-end application. This process ensures consistency in structure and presentation of search results from different data sources.

[0070] Step S5: Dynamic page rendering: After receiving the unified result set, the page rendering module automatically matches appropriate components and styles based on the preset page configuration rule library and populates the data into the component attributes. The rendering engine then generates a page containing diverse views such as tables, cards, and charts, and dynamically returns it to the user. Because of the configuration-driven mechanism, the page layout and display logic can be flexibly adjusted by modifying the configuration rules without requiring secondary development of the front-end code.

[0071] This invention achieves flexible generation and automated display of front-end pages through a configuration-driven dynamic page rendering mechanism. Unlike traditional front-ends that rely on hard-coded page logic, this invention tightly integrates UDM field definitions with page configuration rules. This allows the page rendering module to automatically generate page components and populate content based on the results returned by the unified search module, thereby achieving cross-system page style consistency and rapid adaptation to business needs.

[0072] After a user initiates a search request, the unified search module returns standardized results, and the page rendering module selects components, binds data, and performs final rendering according to the page configuration rules.

[0073] (1) Page configuration rule base: The system provides a visual interface for administrators to define page configuration rules. The rules are stored in JSON or YAML format and mainly include: page layout (such as two-column and three-column layouts); component types (tables, charts, cards, maps, etc.); mapping relationship between fields and component attributes; and interactive behavior rules (such as clicking a row to expand details and linking between components).

[0074] At this stage, the dataIndex must be consistent with the UDM field name to ensure a strong binding between the page configuration and the data model; _meta.uiComponentHint and _meta.formatHint can be referenced by the rule library. For example, formatHint=date-time will automatically suggest using the date formatting rendering function. The page rule base supports versioning and canary releases to prevent business adjustments from affecting existing pages.

[0075] (2) Page rendering module (i.e., front-end engine): The page rendering module's task is to automatically generate pages based on configuration rules and a unified search result set. The process is as follows: Receive the JSON result set returned by the unified search module; Load the corresponding page configuration rules based on the entity type of the result set; Construct the page layout skeleton; Dynamically instantiate UI components (tables, maps, charts, etc.) according to component definitions; Populate component properties with search result data to achieve data-driven rendering; Insert the rendered component into the specified position in the DOM.

[0076] During this stage, the rendering engine verifies dataIndex⊆UDM.properties. If the configuration field is not in UDM, it triggers a default template fallback and logs the changes to ensure robustness. The _meta.maskPolicy is applied during rendering (e.g., displaying a phone number as ****1234) to achieve data anonymization and compliant display. _meta.uiComponentHint automatically guides component selection (e.g., geo_point type is automatically rendered as map markers), reducing manual configuration; It supports virtual DOM and lazy loading to ensure rendering performance in scenarios with large amounts of data.

[0077] (3) Constructing the page layout skeleton: The final generated page can not only display tables, charts, and cards, but also support interactive behaviors (such as clicking on rows to expand, map positioning, and component linkage). Since rendering is entirely configuration-driven, changes in business logic only require modification of the JSON configuration file to take effect immediately, without the need to modify or compile the front-end code.

[0078] At this stage, _meta.isSortable and _meta.isFilterable control whether the front end automatically displays sorting / filtering controls, realizing a data model-driven interactive UI; The rendering output follows the UDM's formatHint (such as uniform time formatting) to ensure consistency between the front-end and back-end. The rendering results support cache key hash(UDM.version, pageConfig.version) to achieve differential updates and high-performance refresh.

[0079] like Figure 4 The page dynamic rendering sequence diagram shown demonstrates how the system achieves end-to-end configuration, automation, and data model-driven rendering from unified search to front-end pages through the following steps. This eliminates the need for hard-coding front-end development and ensures that changes in business requirements only require updating the configuration for immediate effect.

[0080] Step 1: User initiates a search request: The user enters search criteria (keywords, filters, sorting, etc.) in the front-end application interface and clicks the "Search" button.

[0081] Step 2: Front-end application calls unified search module: The front-end application encapsulates the user input query parameters into a unified request format and sends it to the unified search module through protocols such as REST / GraphQL.

[0082] Step 3: The unified search module executes the retrieval: After receiving the request, the search module parses the request parameters, loads the corresponding entity's UDM definition, constructs the search engine DSL according to the _meta rules, and sends the query to the search engine cluster in parallel.

[0083] Step 4: The search engine cluster returns the raw JSON results: After each index shard is retrieved, it returns the raw JSON structure containing the hit documents and metadata (such as _hits, _score, etc.) to the unified search module.

[0084] Step 5: Unify the formatting and merging of search results: The search module removes redundant fields, cleans the data according to the sorting and field whitelist in UDM_meta, merges the results from multiple shards, and generates a unified JSON result set.

[0085] Step 6: The unified search module returns a standardized result set: The formatted unified JSON result set is returned to the front-end application via a REST response.

[0086] Step 7: The page rendering module loads the page configuration rules: After receiving the result set, the rendering module retrieves the corresponding layout and component configuration from the page configuration rule library based on the "entity" type of the current page.

[0087] Step 8: The page rendering module dynamically generates components according to rules: Build the page skeleton according to the layout defined in the configuration (such as a two-column layout); For each component (table, chart, card, map, etc.), instantiate the corresponding UI component and map the specified field (dataIndex) in the JSON result set to the component property (columns, markerField, etc.).

[0088] Step 9, Page rendering module application - meta rendering tips: Based on uiComponentHint and formatHint in UDM_meta, for example, the date field can be automatically formatted, and the geo_point field can be rendered as a map marker.

[0089] If you configure maskPolicy, you can perform data anonymization (e.g., keep the last four digits of the phone number).

[0090] Step 10: The page rendering module completes the DOM update: After rendering, the component is inserted into the corresponding container position on the front-end page, the front-end route or view is updated, and the final search results page is displayed to the user.

[0091] Step 11, User View and Interact: Users can sort, filter, or click on rows to trigger the display of details on the rendered page. Such interactions will trigger corresponding client-side logic or secondary queries, and then enter the above process loop again.

[0092] Through the above steps, the system achieves end-to-end configuration, automation, and data model-driven rendering from unified search to front-end pages, eliminating the need for hard-coding front-end development and ensuring that changes in business requirements only require updating the configuration to take effect immediately.

[0093] The system of this invention is divided into the following five core layers: Data source layer: This layer contains raw data from multiple heterogeneous sources, including relational databases, log files, JSON documents, API interfaces, and various business systems (such as source system A, source system B, source system C, etc.). Due to differences in structure, format, and granularity, direct integration and use of this data is quite difficult.

[0094] Data Unification Layer: Above the data source layer, this invention includes a data acquisition module and a data model construction and mapping module. The data acquisition module interfaces with different source systems to extract and aggregate data. Subsequently, the data model construction and mapping module maps heterogeneous data into a standardized data model, forming standardized data with consistent semantics and a unified structure. This step not only eliminates the differences between multi-source heterogeneous systems but also provides a unified foundation for subsequent index construction and searching.

[0095] The index layer consists of an index management module and a distributed search engine cluster. The index management module generates index configurations based on a unified data model, defining field mappings, word segmentation strategies, and weighting rules. The search engine cluster handles data index storage and query execution. Through this centralized indexing mechanism, the system achieves efficient retrieval and unified management of cross-system data.

[0096] Application Service Layer: The core of this layer is the unified search module. When a user initiates a search request, the search module parses the query conditions based on a unified model and calls the distributed search engine cluster to execute the retrieval. After the results are returned, the search module formats and standardizes the results, outputting a unified result set, thereby ensuring consistency in content and format of search results across systems.

[0097] Presentation Layer: The presentation layer mainly consists of a page rendering module, a page configuration rule base, and a front-end user interface. After the user enters a search request, the unified result set returned by the system is processed by the page rendering module. The page, combined with the configuration rule base, automatically generates and displays the page. This mechanism supports diverse display formats such as tables, charts, and cards, achieving configurability and unification of page style and component logic, avoiding the maintenance costs associated with hard coding.

[0098] This invention integrates data acquisition, standardized modeling, unified indexing, centralized search, and dynamic rendering into a complete unified process through the above five-layer architecture, forming an efficient, flexible, and scalable unified search and display solution.

[0099] This invention ensures the system possesses high generalization ability and scalability through modular design and a unified data model (UDM) decoupling mechanism. Unlike traditional systems with high coupling and high expansion costs, this invention emphasizes module independence, interface standardization, model decoupling, and rapid response at the architectural level, and combines _meta metadata to achieve flexible adaptation across data sources and business scenarios.

[0100] Figure 5 The system uses a modular diagram to illustrate the overall relationship between the "unified data model," the core link connecting the data source, search module, and page rendering module, demonstrating the system's high generalization and scalability. It offers the following advantages: 1) The entire system adopts a modular design, consisting of modules such as data acquisition, data standardization, index management, unified search, and page rendering. Each module communicates through a standard API.

[0101] This system has the following advantages: When upgrading, expanding, or replacing modules, interoperability is maintained by relying on the API protocol to avoid cascading modifications; The _meta metadata is consumed by all modules as a unified constraint, ensuring that different modules maintain consistent behavior after the upgrade; The system supports containerized deployment and module orchestration (such as a containerized orchestration platform), enabling on-demand scaling and canary releases.

[0102] (2) Decoupling through a data model: UDM is independent of specific business logic and front-end presentation, serving as a "contract" between the data layer and the application layer. When adding a new data source or business entity, only: Configure field mapping rules to map source system fields to UDM; automatically generate indexes and retrieval strategies based on _meta; and automatically bind fields to front-end components in the rendering module.

[0103] It has the following advantages: Through _meta.uiComponentHint, the page rendering module can automatically select the best component (such as geographic coordinates → map markers), reducing manual development; With _meta.isFilterable and _meta.isSortable, the front end can enable sorting and filtering controls without having to write logic manually. The data layer is completely decoupled from the front-end presentation layer, reducing the cost of cross-department integration.

[0104] (3) Adopting a rapid response mechanism: In scenarios involving data expansion and page expansion, this invention can be quickly deployed through configuration. Data Expansion: When adding a new data source, simply configure the mapping rules and index synchronization strategy, and the data can flow into the UDM and index database; Page expansion: When adding new page requirements, simply add a JSON configuration file to the page rule library, and the rendering engine will take effect immediately; Version evolution: UDM supports schema versioning and schemaChanged event notifications, and the search and rendering modules will automatically subscribe and update.

[0105] It has the following advantages: when adding new fields or rules, the index, search, and rendering can all be automatically updated based on _meta, avoiding manual synchronization; Page expansion does not require recompiling the front end; Multi-tenancy support: Different business units can share the same search and rendering engine, but the data and page behavior are differentiated by configuring isolation rules.

[0106] The unified search and page configuration rendering method based on a data model proposed in this invention has the following technical advantages in terms of multi-source data integration, cross-system search efficiency, page flexibility, and system scalability: (1) Data consistency and standardization effect: By leveraging the definition and mapping of the Unified Model for Multi-Source Heterogeneous Data (UDM), the problems of inconsistent field naming, significant semantic differences, and difficulty in unified integration across different systems can be solved.

[0107] The _meta metadata mechanism in UDM adds behavioral information such as retrieval, filtering, and sorting to fields, so that data has a unified standard before entering the indexing and display stages.

[0108] Compared to traditional cleaning and mapping methods that rely on manual scripts, this invention improves efficiency in data consistency verification and automated cleaning, significantly reducing retrieval and display errors caused by data inconsistency.

[0109] (2) Improved performance and accuracy of cross-system search: By using UDM-driven automated index mapping, the performance instability that may result from manually defined indexes is avoided, ensuring that the index structure is strictly consistent with the data model.

[0110] Based on the weight of the _meta field and the word segmentation strategy, the unified search module can automatically generate the native query statements of the search engine, and realize cross-index parallel search and result merging.

[0111] (3) Improved page rendering flexibility and development efficiency: This invention achieves strong consistency between the data layer and the presentation layer by directly binding UDM fields to component properties through a configuration-driven dynamic page rendering mechanism.

[0112] Page layout, field mapping, and interaction rules are all defined by a JSON configuration file, so there is no need to modify or recompile the front-end code when business needs are adjusted.

[0113] Compared to traditional front-end development methods, the configurable rendering of this invention shortens the page adjustment and deployment cycle, effectively reducing front-end development and maintenance costs.

[0114] (4) Enhanced system scalability and maintainability: The system architecture adopts a modular design, with data acquisition, standardization, indexing, search and rendering all being independent modules. The modules communicate with each other through standard APIs, and upgrades or replacements do not affect each other.

[0115] UDM is independent of specific business logic. When adding a new data source, only mapping rules and indexes need to be configured, without modifying the core module; when adding a new page requirement, only the configuration file needs to be adjusted to go live.

[0116] (5) End-to-end closed-loop consistency and user experience optimization: From data access, index building, unified search to page rendering, this invention is based on the unified definition of UDM, forming a complete closed loop of "data-index-search-display".

[0117] Regardless of which system a user searches for data from, they can obtain a result set with a unified structure, consistent field standards, and reasonable sorting, avoiding the problems of fragmented results and inconsistent display in traditional cross-system searches.

[0118] This effect directly improves the usability and user experience of data consumption across departments and business systems, and has high versatility and promotional value.

[0119] The above embodiments are not intended to limit the present invention, and the present invention is not limited to the examples given above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of the technical solution of the present invention are also within the protection scope of the present invention.

Claims

1. A unified search and page configuration rendering method based on a data model, characterized in that: Includes the following steps: Step S1: Data Acquisition and Model Mapping; Step S2: Data indexing and synchronization; Step S3: Unified search request processing; Step S4: Result set processing and return; Step S5: Dynamic rendering of the page.

2. The unified search and page configuration rendering method based on a data model according to claim 1, characterized in that: Includes the following steps: Step S1: Obtain raw data from multi-source heterogeneous data sources, and perform field mapping and data type conversion on the raw data according to the pre-built unified data model UDM; Step S2: The index management module imports the standardized data output in step S1 into the distributed search engine cluster and generates index mapping configuration based on UDM. At the same time, it uses timed batch or real-time streaming methods to ensure that the index library is consistent with the source data. Step S3: After the user initiates a search request on the front-end interface, the unified search module receives the request, parses the keywords, filtering conditions and sorting parameters, and the system dynamically constructs a distributed search engine query DSL based on the parsing results and sends the query request to the distributed search engine. Step S4: After receiving the raw JSON results returned by the distributed search engine, the unified search module performs secondary processing on the result set, including field extraction, redundant information cleanup, unified sorting, and formatting encapsulation. The processed results are converted into a unified and standardized JSON format and returned to the front-end application. Step S5: After receiving the unified result set, the page rendering module automatically matches appropriate components and styles according to the preset page configuration rule library, and fills the data into the component attributes. The rendering engine then generates diverse view pages and dynamically returns them to the user.

3. The unified search and page configuration rendering method based on a data model according to claim 2, characterized in that: In step S1, constructing a unified data model includes: Define a unified data model: Use a structured description language to define the field structure of business entities, and introduce _meta metadata on this basis to extend the semantics of the behavior of fields in the process of retrieval, indexing and display; Data Mapping and Transformation Module: This module maps fields from heterogeneous source systems to UDM through configuration rules, and applies _meta rules to complete automatic standardization during the transformation process; Validation and cleaning: Before the data is written to the UDM, the system performs automatic validation and cleaning based on the constraints in _meta.

4. The unified search and page configuration rendering method based on a data model according to claim 3, characterized in that: In step S1, expanding the behavioral semantics of the description field includes: Structure definition, index control, and presentation binding: used for component selection, formatting, and desensitization when driving page rendering; The data mapping and transformation module includes the following operations: Rule storage: Mapping rules are stored in a database or configuration file, supporting hot reloading and version management; Format conversion: Automatically selects the conversion function based on _meta.indexType; Constraint application: When _meta.isFilterable=true, its data type is automatically validated to ensure its availability for subsequent queries; Legality verification and cleanup include the following operations: Type validation: Date fields must conform to ISO standard format; Limit the range of values; Missing fields handling: For required fields that are missing or have invalid values, perform repair, desensitization, or tagging based on _meta.maskPolicy; Logs and Tracking: Non-compliant data will be recorded and tracked through the monitoring system to ensure a closed loop for data quality.

5. The unified search and page configuration rendering method based on a data model according to claim 2, characterized in that: Step S2 specifically includes the following process: Automated indexing and word segmentation strategy: The index management module is automatically triggered when the UDM is updated, reads the _meta metadata in the JSON Schema, and automatically generates the index mapping for the search engine; Data synchronization module: Standardized data is imported into the search engine's index library through the synchronization module, supporting: real-time synchronization based on message queue / streaming processing, and scheduled batch synchronization suitable for low-frequency updates or offline tasks; Unified Search Module: It provides an API as the sole entry point, allowing users to initiate searches without needing to concern themselves with the underlying engine details; Result set formatting and sorting: The unified search module cleans and unifies the results according to the UDM definition, including: extracting core fields, removing redundant content, applying configurable sorting and pagination strategies, and outputting a unified result set that conforms to JSON Schema.

6. The unified search and page configuration rendering method based on a data model according to claim 5, characterized in that: The workflow of the unified search module includes: Receive user search requests: Front-end input of keywords, filtering conditions, and sorting rules; The analysis of keywords and parameters includes: 1) Parse parameters: entity type, keyword, filter condition, sort field; 2) Load UDM: Read the UDM definition of the target entity and determine which fields can be used for searching, filtering, and sorting; Construct a search engine query DSL: dynamically generate query weights based on _meta.searchWeight, and construct filtering conditions based on _meta.isFilterable; Send a query to the search engine cluster: Initiate a request to the search engine, enabling parallel cross-index queries; Receive search engine results: Retrieve the raw JSON results; Result formatting: Clean up redundant fields, sort them uniformly according to business rules, and return a result set consistent with UDM.

7. The unified search and page configuration rendering method based on a data model according to claim 2, characterized in that: In step S5, after the user initiates a search request, the unified search module returns standardized results, and the page rendering module performs component selection, data binding, and final rendering according to the page configuration rules, specifically including the following operations: Define page configuration rule library: The system provides a visual interface for administrators to define page configuration rules. The rules are stored in JSON or YAML format and include: page layout, component type, mapping relationship between fields and component attributes, and interaction behavior rules; Page rendering module: Its task is to automatically generate pages based on configuration rules and a unified search result set; Build the page layout skeleton: The final generated page can display tables, charts, and cards, and also supports interactive behaviors. When business changes occur, only the JSON configuration file needs to be modified to take effect immediately, without modifying or compiling the front-end code.

8. The unified search and page configuration rendering method based on a data model according to claim 7, characterized in that: The workflow of the page rendering module is as follows: Receive the JSON result set returned by the unified search module; Load the corresponding page configuration rules based on the entity type of the result set; Construct the page layout skeleton; Dynamically instantiate UI components according to their definitions; Populate component properties with search result data to achieve data-driven rendering; Insert the rendered component into the specified position in the DOM.

9. The unified search and page configuration rendering method based on a data model according to claim 7, characterized in that: In step S5, the dynamic rendering of the page includes the following process: Step 1: User initiates a search request: The user enters search criteria on the front-end application interface and clicks the search button; Step 2: Front-end application calls unified search module: The front-end application encapsulates the user input query parameters into a unified request format and sends it to the unified search module via REST / GraphQL protocol; Step 3: The unified search module performs the retrieval: After receiving the request, the search module parses the request parameters, loads the corresponding entity's UDM definition, constructs the search engine DSL according to the _meta rules, and sends the query to the search engine cluster in parallel. Step 4: The search engine cluster returns the raw JSON results: After each index shard is retrieved, it returns the raw JSON structure containing the hit documents and metadata to the unified search module. Step 5: Unify the formatting and merging of search results: The search module removes redundant fields, cleans the data according to the sorting and field whitelist in UDM_meta, merges the results from multiple shards, and generates a unified JSON result set; Step 6: The unified search module returns a standardized result set: The formatted unified JSON result set is returned to the front-end application via a REST response; Step 7: The page rendering module loads the page configuration rules: After receiving the result set, the rendering module retrieves the corresponding layout and component configurations from the page configuration rule library based on the entity type of the current page. Step 8: The page rendering module dynamically generates components according to rules: Build the page skeleton according to the layout defined in the configuration; For each component, instantiate the corresponding UI component and map the specified field dataIndex from the JSON result set to the component property; Step 9, Page rendering module application - meta rendering tips: Based on uiComponentHint and formatHint in UDM_meta, the fields will be automatically formatted or rendered as map markers or data anonymization will be performed. Step 10: The page rendering module completes the DOM update: After rendering, the component is inserted into the corresponding container position on the front-end page, the front-end route or view is updated, and the final search results page is displayed to the user; Step 11: User Viewing and Interacting: Users sort, filter, or click on rows on the rendered page to trigger the display of details. Such interactions will trigger corresponding client-side logic or secondary queries, and the above process loop will be entered again.

10. A system for a unified search and page configuration rendering method based on a data model as described in any one of claims 1-9, characterized in that: The system includes: Data source layer: It contains raw data from multiple heterogeneous sources, including relational databases, log files, JSON documents, API interfaces, and various business systems; Data unification layer: includes a data acquisition module and a data model building and mapping module. The data acquisition module connects to different source systems to complete data extraction and aggregation; the data model building and mapping module maps heterogeneous data to a standardized data model to form standardized data with consistent semantics and a unified structure. The index layer consists of an index management module and a distributed search engine cluster. The index management module generates index configurations based on a unified data model and defines field mapping, word segmentation strategies, and weight rules. The search engine cluster is responsible for the index storage and query execution of the data. Application service layer: When a user initiates a search request, the unified search module parses the query conditions based on the unified data model and calls the distributed search engine cluster to perform the retrieval. After the results are returned, the unified search module formats and standardizes the results and outputs a unified result set. Presentation Layer: The presentation layer consists of a page rendering module, a page configuration rule base, and a front-end user interface. After the front-end user enters a search request, the unified result set returned by the system will be processed by the page rendering module, and the page configuration rule base will complete the automatic generation and display of the page.

Images