A method and system for constructing a digital twin model of a mobile terminal

Abstract

The present invention discloses a method and system for constructing a digital twin model of a mobile terminal, including obtaining non-relational multi-source heterogeneous mobile terminal data; constructing a three-dimensional memory mapping relational model to obtain a mobile terminal database; providing data retrieval services for the mobile terminal database to realize data retrieval of vectorized data in the mobile terminal database; constructing a virtual entity mapping space based on the vectorized data retrieved from the data; defining an application intelligent body, and obtaining a mobile terminal digital twin model for realizing real-time perception and processing of the user environment and executing RPA based on the virtual entity mapping space. This solves the problem that current information technology cannot effectively realize the collection and integration of non-relational multi-source heterogeneous mobile terminal data fragment information, cannot perform structured storage of multi-source heterogeneous mobile terminal data, and cannot provide technical support for users in managing their own data security, maintaining data rights and interests, processing and using data, and understanding data status.

Description

Technical Field

[0001] The present invention relates to the field of digital twin technology, and in particular to a method and system for constructing a digital twin model of a mobile terminal. Background Art

[0002] With the continuous advancement of big data and artificial intelligence technologies, the core value of data in the new generation of information technology is becoming increasingly important. Building a reliable data ecosystem and maximizing the value of data elements have become key guidelines for driving future data development. In this process, not only large enterprises and institutions can produce intuitive and high-value data, but individual users also generate a large amount of scattered data resources in their daily activities. However, the potential value of this data is often overlooked. This data can provide a broad and in-depth understanding of all aspects of users' lives, from daily consumption habits and social interactions to work and study. It not only contains basic user information such as age, gender, and location, but also covers more detailed information such as users' interests, emotional state, psychological activities, health status, and behavioral patterns and decision-making processes in different scenarios. The data generated by users in their daily lives is a critical window into their lifestyles, needs, preferences, and potential behavioral trends.

[0003] The current information technology service model primarily relies on vendors. The vast amounts of data generated and accessed by individual users in their daily lives, including personal information, usage history, and preferences, are almost entirely stored and maintained on these vendors' servers or data platforms. Furthermore, major application service vendors lack interoperability, fail to provide users with complete data export services, and fail to effectively translate data usage into actual user benefits. This creates significant difficulties and challenges for users in managing their data security, safeguarding their data rights, processing and using data, and understanding its status. The lack of a robust method for collecting and integrating this fragmented information hinders the full and effective realization of the value of personal life data. How to effectively collect personal data and transform it into more valuable information for use and circulation has become a pressing issue.

[0004] To unlock the value of user data while ensuring its security, data storage and usage must be completed on local mobile terminals. Existing technical solutions currently have the following problems: First, due to the diverse data types and wide range of sources, traditional single-table storage or JSON format storage methods make it difficult to effectively expand and utilize multi-source heterogeneous data. Second, when using large-model RAG technology, the word embedding model is too large to run on mobile terminals, so TF-IDF or BM25 algorithms are often used, resulting in semantic loss and affecting data utilization. Third, how to use artificial intelligence technology to ensure user data security while providing valuable feedback to users, reduce the complexity of using technologies such as AIGC, LLM, and AGENT, and build a complete personal digital twin space. Summary of the Invention

[0005] The present invention provides a method and system for constructing a digital twin model of a mobile terminal to overcome the above-mentioned technical problems.

[0006] In order to achieve the above object, the technical solution of the present invention is:

[0007] A method for constructing a digital twin model of a mobile terminal comprises the following steps:

[0008] S1: Collect and acquire non-relational multi-source heterogeneous mobile terminal data;

[0009] Furthermore, multi-source heterogeneous mobile terminal data includes mobile terminal application data, hardware sensor data, and user private data;

[0010] S2: Construct a three-dimensional memory mapping relationship model;

[0011] The multi-source heterogeneous mobile terminal data is stored based on a three-dimensional memory mapping relationship model, a three-dimensional memory mapping relationship table for storing non-relational data is obtained, and the field names of the multi-source heterogeneous mobile terminal data are redefined as field indexes corresponding to the non-relational mobile terminal data fields through the three-dimensional memory mapping table;

[0012] Perform vectorization processing on multi-source heterogeneous mobile terminal data obtained through field indexing, obtain vectorized data to store and obtain a mobile terminal database;

[0013] S3: Provides data retrieval services for mobile terminal databases to implement data retrieval operations for vectorized data in mobile terminal databases;

[0014] S4: constructing a virtual entity mapping space based on the vectorized data obtained by the data retrieval operation;

[0015] Define the application agent applied to the virtual entity mapping space, and obtain the mobile terminal digital twin model for realizing real-time perception processing of user environment and executing RPA based on the virtual entity mapping space.

[0016] Furthermore, the S2 specifically includes the following steps:

[0017] S21: constructing a three-dimensional memory mapping relationship model, which includes a data table setting module, a field set acquisition module, a data type specification module, and a three-dimensional memory mapping relationship module;

[0018] The data table setting module is used to set the data table for storing multi-source heterogeneous mobile terminal data according to the set SQLite database;

[0019] The field set acquisition module is used to add data fields to obtain the data field set F for the multi-source heterogeneous mobile terminal data stored in the data table;

[0020] And F={data_0,data_1,…,data_n}; where data_n represents the nth data field added;

[0021] The data type specification module is used to set the type label set of each data field type in the data field set;

[0022] The three-dimensional memory mapping relationship module is used to construct a three-dimensional memory mapping relationship expression according to the data field set and the type label set, and obtain a three-dimensional memory mapping table through the three-dimensional memory mapping relationship expression;

[0023] The three-dimensional memory mapping relationship is M:F×T type×N→V; where M represents the abbreviation of M(f, t, n); Ttype represents the type tag set; N represents the unique identifier of the data record; V represents the data value containing all multi-source heterogeneous mobile terminal data to be stored; f represents the data field and f∈F; t represents the data type and t∈T type; n represents the value corresponding to the record identifier and n∈N;

[0024] S22: Redefine the field names of the multi-source heterogeneous mobile terminal data into field indexes corresponding to the non-relational mobile terminal data fields through the obtained three-dimensional memory mapping table;

[0025] S23: Based on the pre-trained continuous bag-of-words model (CBOW), vectorize the multi-source heterogeneous mobile terminal data obtained through field indexing, obtain the vectorized data to store and obtain the mobile terminal database.

[0026] Furthermore, the data retrieval service in S3 includes a data retrieval service based on semantic retrieval and a data retrieval service based on SQL retrieval;

[0027] The retrieval method of the data retrieval service based on semantic retrieval:

[0028] Based on the mobile terminal database, several data vector index navigation layer nodes for data retrieval are constructed through the HNSW graph algorithm;

[0029] And based on the heuristic search method, the edge weights of the data vector index navigation layer nodes are obtained;

[0030] According to the edge weight and data vector index navigation layer nodes, obtain the index navigation graph structure for data retrieval;

[0031] J Vector, an embedded vector search engine based on SIMD, implements data indexing in mobile terminal databases using an index navigation graph structure.

[0032] The retrieval method of the data retrieval service based on SQL retrieval:

[0033] The data in the mobile terminal database is semantically parsed using the Large Language Model (LLM) to obtain structured query intent representations.

[0034] Based on the pre-trained Text2SQL model, SQL statements are obtained according to the structured query intent representation;

[0035] Through local data storage technology, SQL query statements are obtained based on the storage format of the local database and the field index characteristics of vectorized data, so as to implement data indexing of the mobile terminal database according to the SQL query statements.

[0036] Furthermore, the S4 specifically includes the following steps:

[0037] S41: Customize the entity model and entity events of any mobile terminal;

[0038] Using natural language technology, we customize prompt text for obtaining a structured semantic description framework based on the entity model and entity events of the mobile terminal.

[0039] S42: calling the AIGC model of the Wensheng image / Wensheng video to obtain the virtual digital entity model representation according to the prompt text;

[0040] S43: converting the prompt text into a query vector, and based on the data retrieval service provided by the mobile terminal database, retrieving and acquiring multi-source heterogeneous mobile terminal data related to the representation of the virtual digital entity model through the query vector;

[0041] Combined with the pre-trained large language model (LLM), an overview report of entity events is obtained based on the retrieved and acquired multi-source heterogeneous mobile terminal data. The overview report of entity events is the virtual entity mapping space where the user is located.

[0042] S44: defining an application agent applied to the virtual entity mapping space, and determining an execution action instruction for a certain scenario in the current user environment based on an overview report of entity events retrieved and obtained from the virtual entity mapping space;

[0043] S45: Executing an action on the mobile terminal according to the execution action instruction to implement the function of real-time perception and processing of the user environment and executing RPA, thereby realizing the construction of a digital twin model of the mobile terminal;

[0044] The execution action instructions at least include operation instructions for the user mobile terminal interface and access instructions for the remote server terminal interface.

[0045] Furthermore, the method for collecting and acquiring application data of the mobile terminal in S1 includes an active application data collection method based on accessibility services and a passive application data collection method;

[0046] The active application data collection method:

[0047] S001: Enable the accessibility service permission of Android system:

[0048] This includes enabling the permission setting of allowing gesture operations to be performed for simulating click operations on mobile terminals;

[0049] Enable the permission setting of "Allow traversal of window contents" to obtain interface data;

[0050] S002: Open a floating window for collecting mobile terminal application data;

[0051] S003: calling the backend server to download the application data collection plug-in to collect and obtain the structured data of the application data of the mobile terminal according to the location of the floating window based on the simulated click operation;

[0052] The passive collection method of application data:

[0053] S100: Enable accessibility permissions for Android:

[0054] This includes enabling the permission setting of allowing gesture operations to be performed for simulating click operations on mobile terminals;

[0055] Enable the permission setting of "Allow traversal of window contents" to obtain interface data;

[0056] S101: monitoring changes in the mobile terminal page in real time by calling the accessibility service monitoring event tool, and when the page changes, collecting the structured data displayed in real time on the mobile terminal page by calling the interface active window node acquisition tool; otherwise, no collection action is performed;

[0057] Method for calling the accessibility service event monitoring tool to monitor changes in mobile terminal pages in real time:

[0058] S1011: Acquire the click moment when the mobile terminal receives the interface click event, and use it as the first click moment;

[0059] Acquire a second click moment when the mobile terminal receives the interface click event, where the second click moment is the click moment corresponding to the interface click event subsequent to the interface click event corresponding to the first click moment;

[0060] S1012: Obtaining the click time interval between the second click moment and the first click moment;

[0061] Determine the difference between the click time interval and the preset time interval threshold;

[0062] If the click time interval is greater than the preset time interval threshold, it is confirmed that the mobile terminal page has changed; if the click time interval is less than or equal to the preset time interval threshold, it is confirmed that the mobile terminal page has not changed.

[0063] Furthermore, S1 also includes a method for compressing multi-source heterogeneous mobile terminal data:

[0064] Encode the source symbols in the multi-source heterogeneous mobile terminal data based on Huffman coding to obtain source coding symbols;

[0065] Obtain the occurrence probability of each source coding symbol, and compress multi-source heterogeneous mobile terminal data according to the source coding symbols through a source-based probability statistical model;

[0066] And the expression of the probability statistical model based on the information source is

[0067]

[0068] Where: Indicates the expected length of data compression; represents the average code length; Indicates the encoding length; E indicates the expectation; Represents all possible source coding symbols The summation is described; Ξ represents the set of source coding symbols; Indicates source coding symbol Probability of occurrence; Indicates source coding symbol The corresponding encoding length.

[0069] A mobile terminal digital twin model construction system, including a data acquisition module, a data compression module, a three-dimensional memory mapping relationship model construction module, a data vectorization module, a data retrieval service module, an application intelligent agent setting module, and a mobile terminal digital twin model construction module;

[0070] The data acquisition module is used to collect and acquire non-relational multi-source heterogeneous mobile terminal data;

[0071] Furthermore, multi-source heterogeneous mobile terminal data includes mobile terminal application data, hardware sensor data, and user data;

[0072] The data compression module is used to compress multi-source heterogeneous mobile terminal data to obtain compressed data;

[0073] The three-dimensional memory mapping relationship model construction module is used to construct a three-dimensional memory mapping relationship model;

[0074] The multi-source heterogeneous mobile terminal data is stored based on a three-dimensional memory mapping relationship model, a three-dimensional memory mapping relationship table for storing non-relational data is obtained, and the field names of the multi-source heterogeneous mobile terminal data are redefined as field indexes corresponding to the non-relational mobile terminal data fields through the three-dimensional memory mapping table;

[0075] The data vectorization module is used to perform vectorization processing on multi-source heterogeneous mobile terminal data obtained through field indexing, obtain vectorized data to store and obtain a mobile terminal database;

[0076] The data retrieval service module is used to provide data retrieval services to the mobile terminal database to implement data retrieval operations on vectorized data in the mobile terminal database;

[0077] And the data retrieval service includes data retrieval service based on semantic retrieval and data retrieval service based on SQL retrieval;

[0078] The application agent setting module is used to set the application agent applied to the virtual entity mapping space, and the application agent includes at least an information security agent and a health management agent;

[0079] The mobile terminal digital twin model construction module is used to construct a virtual entity mapping space based on the vectorized data obtained by the data retrieval operation, and based on the application agent, obtain a mobile terminal digital twin model for realizing real-time perception processing of the user environment and executing RPA.

[0080] Beneficial effects: The present invention provides a method and system for constructing a digital twin model of a mobile terminal. By storing multi-source heterogeneous mobile terminal data based on a constructed three-dimensional memory mapping relationship model, a three-dimensional memory mapping relationship table for storing non-relational data is obtained, and the collection and integration of non-relational multi-source heterogeneous mobile terminal data fragment information is effectively realized, greatly improving the comprehensiveness of obtaining user mobile terminal digital data; the field name of the multi-source heterogeneous mobile terminal data is redefined as the field index corresponding to the non-relational mobile terminal data field through the three-dimensional memory mapping table, and the multi-source heterogeneous mobile terminal data obtained through the field index is vectorized, and vectorized data is obtained to store and obtain the mobile terminal database, effectively realizing the structured storage of multi-source heterogeneous mobile terminal data, laying the foundation for providing users with complete data export services; by defining an application intelligent body applied to a virtual entity mapping space, and obtaining the construction of a mobile terminal digital twin model for realizing real-time perception processing and execution of RPA of the user environment according to the constructed virtual entity mapping space, technical support is provided for users in managing their own data security, maintaining data rights, processing and using data, and understanding data status, thereby greatly improving the management and application of mobile terminal data. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0082] Figure 1 This is a flow chart of the method for constructing a digital twin model of a mobile terminal of the present invention;

[0083] Figure 2 This is a system block diagram of the mobile terminal digital twin model construction system in this embodiment;

[0084] Figure 3 Schematic diagram of non-relational multi-source heterogeneous mobile terminal data in this embodiment. DETAILED DESCRIPTION

[0085] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0086] This embodiment provides a method for constructing a digital twin model of a mobile terminal. Figure 1 As shown, the following steps are included:

[0087] S1: Collect and acquire non-relational multi-source heterogeneous mobile terminal data;

[0088] Furthermore, multi-source heterogeneous mobile terminal data includes mobile terminal application data, hardware sensor data, and user private data;

[0089] Specifically, as the functions of smartphones continue to increase, users' dependence on mobile phones continues to deepen, so they are used as personal daily data collection terminals. At the same time, in order to ensure the security of user data while using it, all collected user data needs to be stored locally on the mobile terminal. By combining existing technologies, non-relational multi-source heterogeneous data of personal life and work fragments can be collected and fully covered; in this embodiment, personal data, i.e., mobile terminal data, is divided into the following three categories, such as Figure 3 As shown in the figure: first, application data, which is the data generated by users using the application services provided by various application vendors; second, sensor data, which is data collected by sensors and communication modules inside and outside the smart terminal; third, user private data, which is the user's thoughts, records, behaviors, etc. generated at any time;

[0090] Application data: Since daily application data is distributed across various application vendors and lacks data export functionality, and each system application uses a different technical framework, it cannot be collected using traditional data network crawlers. This makes acquiring, collecting, and integrating application-related data extremely difficult.

[0091] This embodiment collects application data on mobile smart terminal devices in a WYSIWYG manner. The application data collection methods include active application data collection methods based on accessibility services and passive application data collection methods. Active collection is the collection of personal data of third-party applications, and the active application data collection methods specifically include:

[0092] S001: Enable accessibility services for the Android system:

[0093] This includes enabling the "can Perform Gestures" permission to simulate click operations on mobile devices.

[0094] Enable the "Can Retrieve Window Content" permission to retrieve interface data.

[0095] S002: Opening a floating window, i.e., an active window, for collecting mobile terminal application data;

[0096] S003: calling the backend server to download the application data collection plug-in to collect and obtain the structured data of the application data of the mobile terminal according to the location of the floating window based on the simulated click operation;

[0097] The specific steps include:

[0098] S0031: Start the third-party APP according to the set third-party APP package name;

[0099] S0032: Obtain the root node of the accessibility tree of the current active window through the accessibility service's interface active window node acquisition tool (get Root In ActiveWindow) to obtain the similarity between the node tree of the current active window and the node tree of the expected settings interface of the third-party app's final data interface, thereby determining whether the current active window interface already belongs to the expected settings interface of the third-party app's final data interface;

[0100] The expected setting interface of the final data interface of the third-party APP is an interface for providing an identification for the final data interface; wherein the method for obtaining the similarity between the node tree of the current active window and the node tree of the expected setting interface of the final data interface of the third-party APP is a well-known technical means, which will not be described in detail here;

[0101] S0033: If the current active window interface is the expected setting interface of the final data interface of the third-party APP of the mobile terminal, the next level function page of the current active window is the final data interface;

[0102] If the current active window interface does not belong to the expected setting interface of the final data interface of the third-party APP of the mobile terminal, then enter the next level function page of the current active window, and repeat S132 until the current active window interface belongs to the expected setting interface of the final data interface of the third-party APP of the mobile terminal, and obtain the final data interface;

[0103] S0034: If the data on the final data interface is displayed in a list or loaded in real time via the network, execute S0035;

[0104] If the final data interface is displayed in the form of a picture, execute S0036;

[0105] S0035: Scroll down and turn pages on the final data interface until there is no more new data or the data collection requirement is met; according to the interface active window node acquisition tool (get Root In Active Window) method of the barrier-free service, traverse all node elements of the final data interface to obtain a large string of all text data on the final data interface;

[0106] S0036: Storing the images locally or in a remote server, and concatenating the image paths to form a large character string of all image paths in the final data interface;

[0107] S0037: According to the large character string and based on a regular expression, obtain the final data interface structured data of the third-party APP of the mobile terminal.

[0108] In this embodiment, the passive collection of application data refers to some information seen by the user when using the APP, including data received through conversations, browsing news, online shopping, and browsing Moments. By collecting the data viewed by the user in real time, it is possible to effectively understand the user's latest knowledge and information received, identify the user's recent status, etc. The traditional method of collecting third-party application data is to perform OCR recognition after taking a screenshot. However, due to mobile phone performance issues, it is not possible to effectively and completely collect the information seen by the user in real time. Therefore, this application proposes a method for passive collection of application data based on accessibility service, and the method specifically includes:

[0109] S100: Enabling accessibility services for the Android system: the mobile terminal is specifically an Android phone;

[0110] This includes enabling the "can Perform Gestures" permission to simulate click operations on mobile devices.

[0111] Enable the "Can Retrieve Window Content" permission to retrieve interface data.

[0112] S101: monitoring changes in the mobile terminal page in real time by calling the accessibility service monitoring event tool (on Accessibility Event), and when the page changes, collecting the structured data displayed in real time on the mobile terminal page by calling the interface active window node acquisition tool (get RootIn Active Window); otherwise, no collection action is performed;

[0113] Method for calling the accessibility service monitoring event tool (on Accessibility Event) to monitor changes in mobile terminal pages in real time:

[0114] S1011: Acquire the click moment when the mobile terminal receives the interface click event, and use it as the first click moment;

[0115] Acquire a second click moment when the mobile terminal receives the interface click event, where the second click moment is the click moment corresponding to the interface click event subsequent to the interface click event corresponding to the first click moment;

[0116] S1012: Obtaining the click time interval between the second click moment and the first click moment;

[0117] Determine the difference between the click time interval and the preset time interval threshold;

[0118] If the click time interval is greater than the preset time interval threshold, it is confirmed that the mobile terminal page has changed; if the click time interval is less than or equal to the preset time interval threshold, it is confirmed that the mobile terminal page has not changed;

[0119] Hardware sensor data: Hardware data can accurately detect and identify the user's physical environment. Through NFC tags, the user's real-time situation can be finely transmitted to the set digital twin space. Through WIFI and Bluetooth connection data, the user's physical space location can be determined. Through data processing algorithms, GPS, gyroscope, gravity sensor, light sensor and other data can be used to identify user-defined scenarios. By combining with external smart hardware sensors, the status collection of the user's physical world can be enriched, so that the digital twin space can provide more intelligent assistance that is more in line with the user's physical state.

[0120] User private data: This allows users to record all private data in their lives through text, voice, photos, and videos. This convenient and fast way of recording information helps users digitize previously unconscious information. This data can then be visualized using AI big model technology to build a digital twin space, providing a solid and powerful foundation for personal digital twins.

[0121] In a specific embodiment, S1 also includes a method for compressing multi-source heterogeneous mobile terminal data:

[0122] Encode the source symbols in the multi-source heterogeneous mobile terminal data based on Huffman coding to obtain source coding symbols;

[0123] Obtain the occurrence probability of each source coding symbol, and compress multi-source heterogeneous mobile terminal data according to the source coding symbols through a source-based probability statistical model;

[0124] In the actual data collection process, the local data processing speed may occasionally fail to keep up with the collection speed. This solution performs vector data retrieval on the user's attention dimension data, that is, by matching the collected mobile terminal data with the corresponding application scenario according to the user's attention program, that is, the user's settings, through the back-end identification, and adopts data compression to transmit it to the large model for processing according to the priority corresponding to the data type. In this embodiment, the user's daily data frequently appears in the same source symbol content characteristics, where the data source of the user's daily data is a core concept in the fields of data science, information technology and communications, referring to the original source of data or the entity / system that provides data. It can be any medium, device, platform or environment that generates, stores or transmits data, and the quality, reliability and type of the data source directly affect the application value and analysis results of the data;

[0125] Therefore, this embodiment adopts Huffman coding and defines the probability according to the frequency of occurrence of the source symbol in the data, so as to achieve the purpose of data compression based on the probability statistical model based on the source;

[0126] And the expression of the probability statistical model based on the information source is

[0127]

[0128] Where: Indicates the expected length of data compression; represents the average code length; Indicates the encoding length; E indicates the expectation; Represents all possible source coding symbols The summation is described; Ξ represents the set of source coding symbols; Indicates source coding symbol Probability of occurrence; Indicates the source symbol The corresponding code length; the source code symbols with high probability are encoded with short codes, and the source code symbols with low probability are encoded with long codes;

[0129] S2: Construct a three-dimensional memory mapping relationship model;

[0130] The multi-source heterogeneous mobile terminal data is stored based on a three-dimensional memory mapping relationship model, a three-dimensional memory mapping relationship table for storing non-relational data is obtained, and the field names of the multi-source heterogeneous mobile terminal data are redefined as field indexes corresponding to the non-relational mobile terminal data fields through the three-dimensional memory mapping table;

[0131] Perform vectorization processing on multi-source heterogeneous mobile terminal data obtained through field indexing, obtain vectorized data to store and obtain a mobile terminal database;

[0132] The standard digital twin technology application in this embodiment mainly focuses on the use of high-precision three-dimensional models to accurately replicate the target objects in the physical world one-to-one. It involves extensive data collection work, that is, intensively collecting data from various key locations of the physical target entity to ensure that a highly consistent and complete entity mapping with the real object can be constructed in the virtual digital space. However, the complexity and diversity of the physical world make the model data present a multi-dimensional characteristic. In particular, for entities with complex structures, diverse functions, and data-driven, it is far from enough to only consider their basic form for modeling. Users' needs for digital twin applications are often highly personalized, and they need to flexibly customize the models in the virtual space according to their actual needs. In view of this, this embodiment is based on comprehensive data collection technology, and constructs a three-dimensional memory mapping relationship model to process and integrate various generalized data of users in daily use, so as to obtain comprehensive information about user preferences, behavioral habits and potential needs;

[0133] Since the data collected in this embodiment comes from a wide range of sources, including active and passive, real-time and historical, full and incremental dimensions, and has diverse data types, including structured, unstructured, and semi-structured data, and these data specifically include text, voice, images, and videos, in order to better build a digital twin space model based on these data, it is necessary to adopt an effective strategy to pre-process these data. The current conventional practice is to store data after standard structuring. However, due to the extremely complex data sources, some data integrity and scalability will be lost. User-related private data cannot be used in model construction due to dimensional uncertainty. At the same time, since the original data needs to be stored for subsequent modeling, display, analysis, use, and circulation, it is necessary to format and store multi-source heterogeneous data. Existing solutions use non-relational data to achieve dynamic addition and deletion of fields and data tables. However, due to the weak computing performance of mobile terminals, the choice of non-relational databases is relatively limited, and when using these databases, there may be difficulties in understanding the meaning of their field content. Therefore, a three-dimensional memory mapping relational model is constructed to achieve data processing and formatted storage of multi-source heterogeneous mobile terminal data.

[0134] In a specific embodiment, the S2 specifically includes the following steps:

[0135] S21: constructing a three-dimensional memory mapping relationship model, which includes a data table setting module, a field set acquisition module, a data type specification module, and a three-dimensional memory mapping relationship module;

[0136] The data table setting module is used to set the data table for storing multi-source heterogeneous mobile terminal data according to the set SQLite database;

[0137] The field set acquisition module is used to add data fields to obtain the data field set F for the multi-source heterogeneous mobile terminal data stored in the data table;

[0138] And F={data_0,data_1,…,data_n}; where data_n represents the nth data field added;

[0139] The data type specification module is used to set the type label set of each data field type in the data field set;

[0140] The three-dimensional memory mapping relationship module is used to construct a three-dimensional memory mapping relationship expression according to the data field set and the type label set, and obtain a three-dimensional memory mapping table through the three-dimensional memory mapping relationship expression;

[0141] The three-dimensional memory mapping relationship is M:F×T type×N→V; where M represents the abbreviation of M(f, t, n); Ttype represents a type tag set; N represents a unique identifier of a data record; V represents a data value containing all multi-source heterogeneous mobile terminal data to be stored; f represents a data field and f∈F; t represents a data type and t∈T type; n represents a value corresponding to a record identifier and n∈N; due to the flexibility of F and T type in this embodiment, the model can store data from different sources and these data can be heterogeneous, and can support standard SQL query data; through M mapping, this embodiment can easily query, update and maintain the relationship between fields of various data types, perform pre-processing when using standard SQL data query, and redefine the field name as the field index corresponding to the field through the three-dimensional memory mapping model, so that in the subsequent use of data, the potential value of the data can be more fully utilized;

[0142] S22: Redefine the field names of the multi-source heterogeneous mobile terminal data into field indexes corresponding to the non-relational mobile terminal data fields through the obtained three-dimensional memory mapping table;

[0143] S23: Based on the pre-trained continuous bag-of-words model (CBOW), vectorize the multi-source heterogeneous mobile terminal data obtained through field indexing, obtain the vectorized data for storage, and obtain the mobile terminal database;

[0144] Specifically, this embodiment uses word2vec in combination with a custom dictionary and word vectors to convert text data collected by mobile terminals and other unstructured or semi-structured data slices using the TIKA tool into vectors and map them to the original data for storage. When using word2vec, considering the current insufficient computing performance of mobile terminals, the Embedding-improved Word2vec technology is introduced to vectorize the multi-source heterogeneous mobile terminal data using the CBOW (Continuous Bag-of-Words Model) continuous bag-of-words model pre-trained with relevant data, and directly select specific rows from the weight matrix and pass them to the next layer, avoiding lengthy matrix multiplication operations. The role of the Embedding layer is to extract the rows (vectors) corresponding to the word ID from the weight parameters, thereby reducing the amount of calculation. At the same time, combined with the common domain dictionary in users' daily life and work, the availability and practicality of data in subsequent modeling are greatly improved.

[0145] This embodiment uses SQLite, a relational database more suitable for embedded systems in mobile terminals, to store non-relational data. By creating a data table and adding data fields data_0 to data_63, it supports up to 64 field data types. The data type is specified by adding a data_type field. A three-dimensional memory mapping relationship model is constructed to index the original field, achieving characteristics similar to a columnar database. It can better store multi-source heterogeneous data and better maintain the relationships between fields of various data types, making it more suitable for OLAP (online analytical processing) scenarios. This embodiment not only enhances the scalability of data types and fields, but also allows the user to customize the data format of any data type through a preset data system or to implement formatted storage.

[0146] S3: Provides data retrieval services for mobile terminal databases to implement data retrieval operations for vectorized data in mobile terminal databases;

[0147] In a specific embodiment, the data retrieval service provided to the mobile terminal database includes a data retrieval service based on semantic retrieval and a data retrieval service based on SQL retrieval;

[0148] Specifically, traditional keyword matching algorithms only consider lexical similarity, ignoring the relationship between sentence structure and semantics. Semantic search algorithms can better understand user query intent and provide more accurate search results. To speed up vector search, this embodiment uses graph algorithms and SIMD (Single Instruction, Multiple Data) accelerated embedded vector search engine J Vector to perform local searches on relevant processed mobile terminal data.

[0149] The retrieval method of the data retrieval service based on semantic retrieval:

[0150] Based on the mobile terminal database, several data vector index navigation layer nodes for data retrieval are constructed through the HNSW graph algorithm;

[0151] And based on the heuristic search method, the edge weights of the data vector index navigation layer nodes are obtained;

[0152] According to the edge weight and data vector index navigation layer nodes, obtain the index navigation graph structure for data retrieval;

[0153] J Vector, an embedded vector search engine based on SIMD, implements data indexing in mobile terminal databases using an index navigation graph structure.

[0154] Specifically, this embodiment uses a large language model and Text2SQL combined with local optimized data storage technology to automatically analyze user questions and convert them into corresponding SQL query statements. This simplifies the database query process, improves query accuracy and efficiency, and lowers the threshold for database operations, allowing more users to easily obtain and utilize structured data and complete complex database queries without the need for professional SQL knowledge.

[0155] The retrieval method of the data retrieval service based on SQL retrieval:

[0156] The data in the mobile terminal database is semantically parsed using the Large Language Model (LLM) to obtain structured query intent representations.

[0157] Based on the pre-trained Text2SQL model, SQL statements are obtained according to the structured query intent representation;

[0158] By using local data storage technology, an SQL query statement is obtained based on the storage format of the local database and the field index characteristics of the vectorized data, so as to implement data indexing of the mobile terminal database according to the SQL query statement;

[0159] S4: constructing a virtual entity mapping space based on the vectorized data obtained by the data retrieval operation;

[0160] Define application agents applied to the virtual entity mapping space and obtain a mobile terminal digital twin model based on the virtual entity mapping space for real-time perception and processing of the user environment and execution of RPA;

[0161] In this embodiment, in the process of constructing a virtual model of a physical entity for use in a digital twin space, the traditional method usually uses Unity 3D / Unreal Engine for pre-modeling, and through real-time collection of relevant data of the physical entity, achieves comprehensive synchronous mapping with the virtual entity. This method enables users to intuitively grasp the overall status of the physical entity through the virtual entity, and then perform effective control, prediction and other operations. Its advantage is that it provides a comprehensive understanding of the physical entity. However, when using a mobile terminal to model the digital twin space of entities related to the user's daily work and life, it does not have good scalability. The user can only choose the system to build a digital twin for the pre-modeled physical entity, and can only display the system's preset basic information. It is impossible to apply digital twin technology to all aspects of the user's life to meet the user's personalized needs.

[0162] The specific steps include:

[0163] S41: Customize the entity model and entity events of any mobile terminal;

[0164] Using natural language technology, we customize prompt text for obtaining a structured semantic description framework based on the entity model and entity events of the mobile terminal.

[0165] S42: calling the AIGC model of the Wensheng image / Wensheng video to obtain the virtual digital entity model representation according to the prompt text;

[0166] S43: converting the prompt text into a query vector, and based on the data retrieval service provided by the mobile terminal database, retrieving and acquiring multi-source heterogeneous mobile terminal data related to the representation of the virtual digital entity model through the query vector;

[0167] Combined with the pre-trained large language model (LLM), an overview report of entity events is obtained based on the retrieved and acquired multi-source heterogeneous mobile terminal data. The overview report of entity events is the virtual entity mapping space where the user is located.

[0168] S44: defining application agents applied to the virtual entity mapping space; for example, information security agents and health management agents, and determining execution action instructions for a certain scenario in the current user environment based on the overview report of entity events retrieved and obtained from the virtual entity mapping space;

[0169] S45: Execute the action of the mobile terminal according to the execution action instruction to realize the function of executing RPA by real-time perception and processing of the user environment, and further realize the construction of the digital twin model of the mobile terminal; and the execution action instruction at least includes the operation instruction of the user mobile terminal interface and the access instruction of the remote server interface. This embodiment can realize the RPA function by executing the operations such as clicking, sliding, and inputting on the interface through the Accessibility Service, and provide action recording for users to realize more convenient operations. Enabling the transparent floating window can record the user's click coordinate path, record user actions, and associate the actions with the application scenarios to realize linkage. For example: the digital twin space recognizes that the scenario is to go home and connect to the home WIFI automatically, and it senses that the current ambient temperature is not suitable. The action is to automatically find the Mijia application and then turn on the air conditioner. Through large-model scenario recognition, the traditional rule-based scenario automation problem is solved, breaking the linkage limitations between various applications.

[0170] This embodiment uses AIGC technology to allow users to customize the entity model or object dimension of the mobile terminal, customize prompts through user natural language, and use cultural images / videos to build virtual digital entities. At the same time, semantic retrieval is performed on the collected vectorized relevant data using the customized prompt natural language to obtain all relevant original data describing the entity. In combination with the large language model (LLM), the full RAG (Retrieval-Augmented Generation) function of the local data of the mobile terminal is implemented. The retrieved and matched data is processed using the large model to obtain an overview report of the digital entity. In addition, users can redefine prompts so that the large model further processes and returns the dimension information of interest to the user. Placeholders are provided for real-time dimension information to be embedded in the large model's answer results, enabling it to maintain high-speed information synchronization. At the same time, users can further view and export detailed structured data related to the use of the entity model through the provided data retrieval function.

[0171] This embodiment also includes a mobile terminal digital twin model construction system, such as Figure 2 As shown, it includes a data acquisition module, a data compression module, a three-dimensional memory mapping relationship model construction module, a data vectorization module, a data retrieval service module, an application intelligent agent setting module, and a mobile terminal digital twin model construction module;

[0172] The data acquisition module is used to collect and acquire non-relational multi-source heterogeneous mobile terminal data;

[0173] Furthermore, multi-source heterogeneous mobile terminal data includes mobile terminal application data, hardware sensor data, and user data;

[0174] The data compression module is used to compress multi-source heterogeneous mobile terminal data to obtain compressed data;

[0175] The three-dimensional memory mapping relationship model construction module is used to construct a three-dimensional memory mapping relationship model;

[0176] The multi-source heterogeneous mobile terminal data is stored based on a three-dimensional memory mapping relationship model, a three-dimensional memory mapping relationship table for storing non-relational data is obtained, and the field names of the multi-source heterogeneous mobile terminal data are redefined as field indexes corresponding to the non-relational mobile terminal data fields through the three-dimensional memory mapping table;

[0177] The data vectorization module is used to perform vectorization processing on multi-source heterogeneous mobile terminal data obtained through field indexing, obtain vectorized data to store and obtain a mobile terminal database;

[0178] The data retrieval service module is used to provide data retrieval services to the mobile terminal database to implement data retrieval operations on vectorized data in the mobile terminal database;

[0179] And the data retrieval service includes data retrieval service based on semantic retrieval and data retrieval service based on SQL retrieval;

[0180] The application agent setting module is used to set the application agent applied to the virtual entity mapping space, and the application agent includes at least an information security agent and a health management agent;

[0181] The mobile terminal digital twin model construction module is used to construct a virtual entity mapping space based on the vectorized data obtained by the data retrieval operation, and based on the application agent, obtain a mobile terminal digital twin model for realizing real-time perception processing of the user environment and executing RPA.

[0182] The model application of the mobile terminal digital twin model constructed in this embodiment: through the collection of various data from individual users' daily lives, users can customize the complete digital twin virtual mapping of various entities in their lives, and have detailed status viewing and control of the entity. At the same time, it provides users with the application of the entire twin space data. Based on the large language model, through pre-training on large-scale text data, it has mastered rich language knowledge and patterns. These models can understand complex language structures and generate coherent and logical text. In turn, they can enable the large language model to parse the queries or prompts input by the user, understand the intentions and needs behind them, and thus perform accurate analysis and decision-making. For example, if a user needs movie recommendations, the semantic analysis model built through the user's daily corpus can perform vector retrieval of the user's preferences, status, movies and other related data based on the user's questions, and submit them to the large language model for processing and response. At the same time, various data AI AGENTs are provided to provide deeper mining of user data. For example: the information security agent can provide security reports and repair suggestions by detecting all user data collected in real time. When it finds out whether the user's current behavior is abnormal, it can find out whether the user is being defrauded through the real-time collected interface text conversation content. The health management agent conducts a comprehensive analysis of the collected daily diet, exercise, physiology, sleep and other data of the user to provide an accurate analysis based on real and complete data.

[0183] Beneficial effects of the method described in this embodiment:

[0184] This embodiment uses a constructed three-dimensional memory mapping relational model to store multi-source heterogeneous mobile terminal data and obtain a three-dimensional memory mapping relational table for storing non-relational data. This effectively enables the collection and integration of fragmented information on non-relational multi-source heterogeneous mobile terminal data, significantly improving the comprehensiveness of obtaining user mobile terminal digital data. The three-dimensional memory mapping table redefines the field names of the multi-source heterogeneous mobile terminal data as field indexes corresponding to the non-relational mobile terminal data fields. The multi-source heterogeneous mobile terminal data obtained through the field indexes is then vectorized to obtain vectorized data for storage and access to a mobile terminal database. This effectively implements structured storage of multi-source heterogeneous mobile terminal data, laying the foundation for providing users with complete data export services. By defining an application agent for a virtual entity mapping space and constructing a mobile terminal digital twin model for real-time user environment perception, processing, and RPA execution based on the constructed virtual entity mapping space, this embodiment provides technical support for users in managing their own data security, maintaining data rights, processing and using data, and understanding data status, significantly improving the management and application of mobile terminal data. All user data collected by this application is collected with the user's consent and authorization, and the collection, use and processing of relevant user data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0185] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing a digital twin model of a mobile terminal, characterized in that: The following steps are involved: S1: Collect and acquire non-relational multi-source heterogeneous mobile terminal data; Furthermore, multi-source heterogeneous mobile terminal data includes mobile terminal application data, hardware sensor data, and user private data; S1 also includes a method for compressing multi-source heterogeneous mobile terminal data: Encode the source symbols in the multi-source heterogeneous mobile terminal data based on Huffman coding to obtain source coding symbols; Obtain the occurrence probability of each source coding symbol, and compress multi-source heterogeneous mobile terminal data according to the source coding symbols through a source-based probability statistical model; And the expression of the probability statistical model based on the information source is Where: Indicates the expected length of data compression; represents the average code length; Indicates the encoding length; E indicates the expectation; Represents all possible source coding symbols The summation is described; Ξ represents the set of source coding symbols; Indicates source coding symbol Probability of occurrence; Indicates source coding symbol The corresponding encoding length; S2: Construct a three-dimensional memory mapping relationship model; The multi-source heterogeneous mobile terminal data is stored based on a three-dimensional memory mapping relationship model, a three-dimensional memory mapping relationship table for storing non-relational data is obtained, and the field names of the multi-source heterogeneous mobile terminal data are redefined as field indexes corresponding to the non-relational mobile terminal data fields through the three-dimensional memory mapping table; the multi-source heterogeneous mobile terminal data obtained through the field index is vectorized, and the vectorized data is obtained to store and obtain a mobile terminal database; The specific steps include: S21: constructing a three-dimensional memory mapping relationship model, which includes a data table setting module, a field set acquisition module, a data type specification module, and a three-dimensional memory mapping relationship module; The data table setting module is used to set the data table for storing multi-source heterogeneous mobile terminal data according to the set SQLite database; The field set acquisition module is used to add data fields to obtain the data field set F for the multi-source heterogeneous mobile terminal data stored in the data table; And F={data_0,data_1,…,data_n}; where data_n represents the nth data field added; The data type specification module is used to set the type label set of each data field type in the data field set; The three-dimensional memory mapping relationship module is used to construct a three-dimensional memory mapping relationship expression according to the data field set and the type label set, and obtain a three-dimensional memory mapping table through the three-dimensional memory mapping relationship expression; The three-dimensional memory mapping relationship is M:F×T type×N→V; where M represents the abbreviation of M(f, t, n); Ttype represents the type tag set; N represents the unique identifier of the data record; V represents the data value containing all multi-source heterogeneous mobile terminal data to be stored; f represents the data field and f∈F; t represents the data type and t∈T type; n represents the value corresponding to the record identifier and n∈N; S22: Redefine the field names of the multi-source heterogeneous mobile terminal data into field indexes corresponding to the non-relational mobile terminal data fields through the obtained three-dimensional memory mapping table; S23: Based on the pre-trained continuous bag-of-words model (CBOW), vectorize the multi-source heterogeneous mobile terminal data obtained through field indexing, obtain the vectorized data for storage, and obtain the mobile terminal database; S3: Provides data retrieval services for mobile terminal databases to implement data retrieval operations for vectorized data in mobile terminal databases; The retrieval method of the data retrieval service based on semantic retrieval: Based on the mobile terminal database, several data vector index navigation layer nodes for data retrieval are constructed through the HNSW graph algorithm; And based on the heuristic search method, the edge weights of the data vector index navigation layer nodes are obtained; According to the edge weight and data vector index navigation layer nodes, obtain the index navigation graph structure for data retrieval; J Vector, an embedded vector search engine based on SIMD, implements data indexing in mobile terminal databases using an index navigation graph structure. The retrieval method of the data retrieval service based on SQL retrieval: The data in the mobile terminal database is semantically parsed using the Large Language Model (LLM) to obtain structured query intent representations. Based on the pre-trained Text2SQL model, SQL statements are obtained according to the structured query intent representation; By using local data storage technology, an SQL query statement is obtained based on the storage format of the local database and the field index characteristics of the vectorized data, so as to implement data indexing of the mobile terminal database according to the SQL query statement; S4: constructing a virtual entity mapping space based on the vectorized data obtained by the data retrieval operation; Define application agents applied to the virtual entity mapping space and obtain a mobile terminal digital twin model based on the virtual entity mapping space for real-time perception and processing of the user environment and execution of RPA; The specific steps include: S41: Customize the entity model and entity events of any mobile terminal; Using natural language technology, we customize prompt text for obtaining a structured semantic description framework based on the entity model and entity events of the mobile terminal. S42: calling the AIGC model of the Wensheng image / Wensheng video to obtain the virtual digital entity model representation according to the prompt text; S43: converting the prompt text into a query vector, and based on the data retrieval service provided by the mobile terminal database, retrieving and acquiring multi-source heterogeneous mobile terminal data related to the representation of the virtual digital entity model through the query vector; Combined with the pre-trained large language model (LLM), an overview report of entity events is obtained based on the retrieved and acquired multi-source heterogeneous mobile terminal data. The overview report of entity events is the virtual entity mapping space where the user is located. S44: defining an application agent applied to the virtual entity mapping space, and determining an execution action instruction for a certain scenario in the current user environment based on an overview report of entity events retrieved and obtained from the virtual entity mapping space; S45: Executing an action on the mobile terminal according to the execution action instruction to implement the function of real-time perception and processing of the user environment and executing RPA, thereby realizing the construction of a digital twin model of the mobile terminal; The execution action instructions at least include operation instructions for the user mobile terminal interface and access instructions for the remote server terminal interface.

2. A method for constructing a digital twin model of a mobile terminal according to claim 1, characterized in that: The method for collecting and acquiring application data of the mobile terminal in S1 includes an active application data collection method based on accessibility services and a passive application data collection method; The active application data collection method: S001: Enable the accessibility service permission of Android system: This includes enabling the permission setting of allowing gesture operations to be performed for simulating click operations on mobile terminals; Enable the permission setting of "Allow traversal of window contents" to obtain interface data; S002: Open a floating window for collecting mobile terminal application data; S003: calling the backend server to download the application data collection plug-in to collect and obtain the structured data of the application data of the mobile terminal according to the location of the floating window based on the simulated click operation; The passive collection method of application data: S100: Enable accessibility services on Android: This includes enabling the permission setting of allowing gesture operations to be performed for simulating click operations on mobile terminals; Enable the permission setting of "Allow traversal of window contents" to obtain interface data; S101: monitoring changes in the mobile terminal page in real time by calling the accessibility service monitoring event tool, and when the page changes, collecting the structured data displayed in real time on the mobile terminal page by calling the interface active window node acquisition tool; otherwise, no collection action is performed; Method for calling the accessibility service event monitoring tool to monitor changes in mobile terminal pages in real time: S1011: Acquire the click moment when the mobile terminal receives the interface click event, and use it as the first click moment; Acquire a second click moment when the mobile terminal receives the interface click event, where the second click moment is the click moment corresponding to the interface click event subsequent to the interface click event corresponding to the first click moment; S1012: Obtaining the click time interval between the second click moment and the first click moment; Determine the difference between the click time interval and the preset time interval threshold; If the click time interval is greater than the preset time interval threshold, it is confirmed that the mobile terminal page has changed; if the click time interval is less than or equal to the preset time interval threshold, it is confirmed that the mobile terminal page has not changed.

3. A system based on the method for constructing a mobile terminal digital twin model according to any one of claims 1 to 2, characterized in that: It includes data acquisition module, data compression module, three-dimensional memory mapping relationship model construction module, data vectorization module, data retrieval service module, application intelligent body setting module, and mobile terminal digital twin model construction module; The data acquisition module is used to collect and acquire non-relational multi-source heterogeneous mobile terminal data; Furthermore, multi-source heterogeneous mobile terminal data includes mobile terminal application data, hardware sensor data, and user data; The data compression module is used to compress multi-source heterogeneous mobile terminal data to obtain compressed data; The three-dimensional memory mapping relationship model construction module is used to construct a three-dimensional memory mapping relationship model; The multi-source heterogeneous mobile terminal data is stored based on a three-dimensional memory mapping relationship model, a three-dimensional memory mapping relationship table for storing non-relational data is obtained, and the field names of the multi-source heterogeneous mobile terminal data are redefined as field indexes corresponding to the non-relational mobile terminal data fields through the three-dimensional memory mapping table; The data vectorization module is used to perform vectorization processing on multi-source heterogeneous mobile terminal data obtained through field indexing, obtain vectorized data to store and obtain a mobile terminal database; The data retrieval service module is used to provide data retrieval services to the mobile terminal database to implement data retrieval operations on vectorized data in the mobile terminal database; And the data retrieval service includes data retrieval service based on semantic retrieval and data retrieval service based on SQL retrieval; The application agent setting module is used to set the application agent applied to the virtual entity mapping space, and the application agent includes at least an information security agent and a health management agent; The mobile terminal digital twin model construction module is used to construct a virtual entity mapping space based on the vectorized data obtained by the data retrieval operation, and based on the application agent, obtain a mobile terminal digital twin model for realizing real-time perception processing of the user environment and executing RPA.

Images