Non-intrusive data access method and system

Abstract

The application relates to a non-intrusive data access method and system, which is applied to data interaction, and the method comprises the following steps: acquiring operation interface data of a user terminal, and generating real-time feature data according to the operation interface data; performing scene calibration on the real-time feature data through a predetermined business scene classification model to determine a current business scene of the user; performing key feature information extraction on the current business scene and the real-time feature data through a predetermined image classifier to obtain key feature data; performing intention and positioning analysis on the operation interface data, the current business scene and the key feature data through a predetermined feature vector model to obtain recommendation data and vector data, and sending the recommendation data to a configuration main data system; after obtaining reply data fed back by the configuration main data system, determining a backfill position of the reply data according to the vector data, and displaying the reply data on an operation interface; and the application can realize non-intrusive access to the configuration main data system.

Description

Technical Field

[0001] This invention relates to the field of computer data communication technology, and in particular to a non-intrusive data access method and system. Background Technology

[0002] The application systems in nuclear power plants include hundreds of business systems such as configuration master data systems and ERP systems. Among them, the configuration master data system of nuclear power plants is used to store the master data of nuclear power plant equipment, while other systems usually need to call the configuration master data system to retrieve the required data when performing business operations.

[0003] With the advancement of enterprise data transformation, the governed master data has entered the configuration master data system. To improve the efficiency of information exchange between the master data system and other business systems and shorten the time for master data to be implemented in application systems, the traditional method is to establish a communication interface between the configuration master data system and other business systems to realize data retrieval and delivery. However, due to the large number of business systems, the workload and cost of transformation are significant. If the system is not modified, although manual queries can achieve operations such as copying and editing, users need to frequently switch between different work interfaces and organize query conditions. This back-and-forth operation not only reduces user work efficiency but also leads to a poor user experience. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a non-intrusive data access method and system.

[0005] The technical solution adopted by this invention to solve its technical problem is: to construct a non-intrusive data access method for data interaction, comprising the following steps:

[0006] S10. Obtain the user terminal's operation interface data, and generate real-time feature data based on the operation interface data;

[0007] S20. The real-time feature data is calibrated using a predetermined business scenario classification model to determine the user's current business scenario;

[0008] S30. Extract key feature information from the current business scenario and real-time feature data using a predetermined image classifier to obtain key feature data;

[0009] S40. Perform intent and location analysis on the operation interface data, the current business scenario, and the key feature data using a predetermined feature vector model to obtain recommended data and vector data, and send the recommended data to the configuration master data system.

[0010] S50. After obtaining the response data fed back by the configuration master data system, determine the backfill position of the response data according to the vector data, and then display the response data on the operation interface.

[0011] Preferably, S10 includes:

[0012] S101. Obtain user terminal interface data;

[0013] S102. The operation interface data is parsed using computer vision technology to obtain the real-time feature data; wherein, the real-time feature data includes one or more of the following: real-time program, active window, screen, name, and cursor data.

[0014] Preferably, S20 includes:

[0015] The business scenarios corresponding to the real-time feature data are labeled in the business scenario classification model, and the correlation degree of each business scenario is calculated. The user's current business scenario is determined based on the correlation degree.

[0016] Preferably, S30 includes:

[0017] The image classifier extracts key feature information based on the predetermined requirements and real-time feature data of the current business scenario; the key feature information includes window title and text data;

[0018] The image classifier also performs localization analysis on the key feature information using a predetermined YOLO model to obtain localization data, and calculates the confidence rate data of the key feature information based on the localization data.

[0019] Preferably, S30 includes:

[0020] The image classifier is built using a residual neural network.

[0021] Preferably, S40 includes:

[0022] The feature vector model generates slot filling to represent user intent based on the relevance of the current business scenario, the key feature information, and the confidence rate data, and then generates the recommendation data based on the slot filling.

[0023] Preferably, S40 includes:

[0024] The feature vector model performs OCR processing on the user interface data to obtain the position information of all box elements on each page, as well as the text content in each box element.

[0025] The location information and the text content are vectorized to obtain the vector data of each of the box elements.

[0026] Preferably, the non-intrusive data access method further includes:

[0027] S60. Provide operation services related to the response data through a pre-defined service engine based on the entered operation instructions.

[0028] Preferably, in S60, the operation service includes editing, copying, and working mode settings.

[0029] This invention also constructs a non-intrusive data access system for data interaction, comprising:

[0030] The first processing unit is used to acquire user terminal operation interface data and generate real-time feature data based on the operation interface data.

[0031] A business scenario classification model is used to perform scenario labeling on the real-time feature data to determine the user's current business scenario;

[0032] An image classifier is used to extract key feature information from the current business scenario and generate key feature data.

[0033] The feature vector model is used to perform intent and location analysis on the operation interface data, the current business scenario, and the key feature data, generate recommendation data and vector data, and send the recommendation data to the configuration master data system.

[0034] The second processing unit is used to determine the backfilling position of the response data based on the vector data after obtaining the response data fed back by the configuration master data system, and to display the response data on the operation interface.

[0035] The technical solution of this invention first generates real-time feature data based on the acquired operation interface data; then, it processes the real-time feature data through a business scenario classification model to determine the current business scenario; next, it extracts key feature information from the current business scenario and real-time feature data using an image classifier to obtain key feature data; then, it analyzes the operation interface data, the current business scenario, and key feature data through a feature vector model to obtain recommended data and vector data, and then sends the recommended data to the configuration master data system; after obtaining the response data from the configuration master data system, it determines the backfilling position of the response data based on the vector data to display the response data on the operation interface. This allows users to access the data of the configuration master data system without frequently switching work interfaces or establishing a communication interface between the business system and the configuration master data system in advance, thus achieving non-intrusive access to the configuration master data system. This approach has the advantages of simple user operation and low modification costs. Attached Figure Description

[0036] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:

[0037] Figure 1 This is a flowchart illustrating a non-intrusive data access method in some embodiments of the present invention;

[0038] Figure 2 This is a flowchart illustrating step S10 in a non-intrusive data access method in some embodiments of the present invention.

[0039] Figure 3 This is a schematic block diagram of the structure of a non-intrusive data access system in some embodiments of the present invention. Detailed Implementation

[0040] To provide a clearer understanding of the technical features, objectives, and effects of the present invention, specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0041] It should be noted that the flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0042] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0043] Figure 1 This is a flowchart illustrating a non-intrusive data access method in some embodiments of the present invention. This non-intrusive data access method is applied to data interaction. Additionally, the configuration master data system is used to store the equipment master data of a nuclear power plant. This equipment data includes, but is not limited to, information such as equipment category, function, location, fault mode code, fault description, attributes, operating status, and 3D and 2D drawings. This non-intrusive data access method is applied in the control module of the user terminal to automatically provide recommended operations based on user actions, simplifying the operations required for users to access the configuration master data system and improving the user experience. Figure 1 As shown, the non-intrusive data access method includes steps S10, S20, S30, S40 and S50.

[0044] Step S10 includes: acquiring user terminal operation interface data and generating real-time feature data based on the operation interface data;

[0045] Step S20 includes: performing scenario labeling on real-time feature data using a predetermined business scenario classification model to determine the user's current business scenario;

[0046] Step S30 includes: extracting key feature information from the current business scenario and real-time feature data using a predefined image classifier to obtain key feature data;

[0047] Step S40 includes: performing intent and location analysis on the operation interface data, current business scenario and key feature data through a predetermined feature vector model to obtain recommended data and vector data, and sending the recommended data to the configuration master data system;

[0048] Step S50 includes: after obtaining the response data fed back by the configuration master data system, determining the backfilling position of the response data based on the vector data, then displaying the response data on the operation interface and providing operation services related to the response data through a pre-defined service script.

[0049] The technical solution of this invention first generates real-time feature data based on the acquired operation interface data; then, it processes the real-time feature data through a business scenario classification model to determine the current business scenario the user is currently engaged in; next, it extracts key feature information from the current business scenario and real-time feature data using an image classifier to obtain key feature data for identifying the user's operational intent; then, it analyzes the operation interface data, the current business scenario, and the key feature data through a feature vector model to obtain recommended data for controlling the configuration master data system to access data and vector data for representing the location of the data to be backfilled; the recommended data is then sent to the configuration master data system to await the response data after the configuration master data system completes the search action; after obtaining the response data, the backfill location of the response data is determined based on the vector data to display the response data on the operation interface. This allows users to access the data of the configuration master data system without frequently switching work interfaces or establishing a communication interface between the business system and the configuration master data system in advance, thus achieving non-intrusive access to the configuration master data system. This approach has the advantages of simple user operation and low modification cost.

[0050] In a preferred embodiment, such as Figure 2 As shown, step S10 includes steps S101 and S102.

[0051] Step S101 includes: acquiring user terminal interface data. The interface data may be a screenshot of the user terminal's interface, and the user terminal may be a computer.

[0052] Step S102 includes: using computer vision technology to parse the operation interface data to obtain real-time feature data; wherein, the real-time feature data includes one or more of the following: real-time program, active window, screen, name and cursor data.

[0053] Specifically, the real-time program includes the names of all applications currently running on the user terminal; the active window refers to the application type of the current work window on the operation interface; the screen corresponds to the operation screen (including text information) within the active window; the name refers to the name of the work type being performed; and the cursor data refers to the text data within a set range centered on the current cursor position. For example, if a user has multiple applications open simultaneously and is using an ERP system to fill out an equipment fault work order, then the real-time program corresponds to the directory of these application names, the active window corresponds to the ERP system, the name corresponds to the name of the work order being filled out (such as an equipment fault work order, notification, etc.), and the cursor position corresponds to the information in the function position input box where the cursor is located. The function position input box information includes the input box type and the text content within the input box; taking an equipment fault work order as an example, its input box types include equipment name input box, fault description input box, assembly method input box, etc.

[0054] In a preferred embodiment, step S20 includes: labeling the business scenarios corresponding to real-time feature data in the business scenario classification model, calculating the correlation degree of each business scenario, and determining the user's current business scenario based on the correlation degree.

[0055] The business scenario classification model includes several pre-set business scenarios, such as filling out various work orders, browsing and editing various equipment drawings, querying various equipment information, etc. In addition, each business scenario contains its own corresponding feature dataset. The content of the feature dataset can be any combination of application type, function location, equipment information (including equipment name, category, function and location), component information (including component name, category, function and location), media type, component category accessories, time series measurement points, room location, entity information (including entity name, entity number and entity location), pipeline information (including pipeline name, pipeline number and pipeline location), and weld information (including weld number and weld location). The specific content of the feature dataset needs to be determined according to the characteristics of the business scenario itself.

[0056] In this embodiment, the business scenario classification model compares the real-time feature data with the content of the feature datasets of each business scenario. If the feature dataset of a certain business scenario contains content consistent with the real-time feature data, it indicates that the user's current operation on the interface has similar or identical parts to the operation required for that business scenario. Therefore, it can be determined that the business scenario corresponds to the real-time feature data. Further, the content corresponding to or consistent with the real-time feature data in the feature dataset of the business scenario is marked and counted to obtain the number of marks for the business scenario. Then, the correlation degree of the business scenario is calculated based on the number of marks. The correlation degree is equal to the number of marks divided by the total number of content types in the feature datasets of all business scenarios. The higher the correlation degree, the more accurately the type of business scenario the user is currently performing can be determined. Therefore, the business scenario with the highest correlation degree can be selected as the user's current business scenario. In addition, if the feature dataset of a certain business scenario does not contain content corresponding to the real-time feature data, then the number of marks for the business scenario is 0, and therefore, the correlation degree corresponding to the business scenario is also 0. In a specific embodiment, this non-intrusive data access method can also establish a business scenario classification model through step S01.

[0057] Step S01 includes: collecting the key input or click features of users in the past when performing various business operations, and training based on these features to obtain a business scenario classification model.

[0058] In a preferred embodiment, step S30 includes: an image classifier extracting key feature information based on predetermined requirement features and real-time feature data of the current business scenario; the key feature information includes window title and text data; the image classifier also performs location analysis on the key feature information using a predetermined YOLO model to obtain location data, and calculates the confidence rate data of the key feature information based on the location data. The key feature data includes key feature information and confidence rate data.

[0059] Furthermore, the pre-defined requirements can be set according to the specific operational content of each business scenario. Taking filling out an equipment failure work order as an example, the required information includes the equipment name, failure description, and assembly method. Therefore, the equipment name, component information, and assembly method can be used to form the pre-defined requirements for filling out an equipment failure work order.

[0060] In this embodiment, the image classifier extracts features from the text content of the real-time feature data image based on predetermined requirements of the current business scenario, thereby obtaining key feature information. This key feature information includes features of several different text contents. Then, a predetermined YOLO model is used to locate various features, and the number of labels corresponding to each located feature is counted. Dividing this number of labels by the total number of labels for all features yields the confidence rate for that feature. The confidence rates of various features constitute the confidence rate data. Taking filling out an equipment fault work order as an example, the image classifier extracts equipment information and component information. Assuming the extracted equipment name is a steam generator, and the components of a steam generator include a boiler, related valves, and pipes, feature extraction is also needed for the information of the boiler, related valves, and pipes that make up the steam generator (including equipment information, component information, and pipe information). That is, the YOLO model is used to locate and label these features, and the labeled features are counted to obtain the number of labels corresponding to each feature and the total number of labels for all features, thereby calculating the confidence rate data. Generally, a higher confidence rate more accurately identifies the features that the user is most concerned about.

[0061] The YOLO model (also known as the object detection model) is used to identify objects with certain features and their locations in images. Therefore, image classifiers can perform localization analysis by calling a trained YOLO model.

[0062] Furthermore, in one specific embodiment, an image classifier can be built using a residual neural network.

[0063] Residual neural networks (ResNet, also known as convolutional neural networks or residual networks) are used for image classification and object recognition. They are easy to optimize and can improve accuracy by increasing their depth. Building an image classifier using residual neural networks can improve the accuracy and reliability of extracting key feature data.

[0064] In a preferred embodiment, in step S40, recommendation data can be generated in the following way: the feature vector model generates slot filling to represent user intent based on the relevance of the current business scenario, key feature information and confidence rate data, and then generates recommendation data based on the slot filling.

[0065] It should be noted that while a higher correlation between the current business scenario and the user's intent to perform the operation is more certain, and a higher confidence rate is more certain of the features the user is most interested in, there are varying degrees of correlation between the current business scenario and various features. Therefore, when generating slot filling, the correlation between the current business scenario and the features needs to be comprehensively considered. In addition, the recommended data is used to control the configuration master data system to access or query data based on certain keywords, so as to automatically control the configuration master data system to search for data based on the user's intent in the background, so as to display relevant information to the user in subsequent steps. For example, when the user fills out an equipment fault work order, it automatically helps the user search for component information related to the equipment being filled in; when browsing and editing drawings, it automatically helps the user search for related drawing information of the drawings displayed in the current operation interface.

[0066] In a preferred embodiment, in step S40, vector data can be generated as follows: a feature vector model performs OCR processing on the interface data to obtain the position information of all box elements on each page, as well as the text content in each box element; the position information and text content are then vectorized to obtain vector data for each box element. The vector data includes the position vector of each box element and the relative position vector of each text content within the corresponding box element.

[0067] In one specific embodiment, vectorization can be achieved using the tokenizer in the Word2Vec model or the BERT model.

[0068] In a preferred embodiment, the feature vector model can be established by training the model with the current business scenario and relevance obtained from the business scenario classification model based on the actual operations of users in various business operations in the past, as well as the key feature data obtained from the image classifier.

[0069] In a preferred embodiment, the non-intrusive data access method further includes step S60: providing operation services related to the response data through a predetermined service engine based on the entered operation instructions; wherein the operation services include editing, copying, and working mode settings. The service engine can be implemented using existing editing scripts, copying scripts, and function setting scripts.

[0070] In this embodiment, the operating modes include an automatic operating mode and a manual operating mode.

[0071] The automatic operation mode refers to the service engine automatically filling the response data into the corresponding fill-in positions using existing PRA technology. Taking filling out an equipment failure work order as an example, when a user fills out an equipment failure work order in the ERP system's active window and enters a device name, such as a generator, in the equipment name input box, the operation interface will display the specific name or model of the generator for the user to select. Then, after the user enters some text content in the fault description input box, the control module will also display relevant information based on the entered content. For example, if the confidence rate of the entered text containing gears is high, then information such as the model and drawings of the relevant gears in the generator will be displayed first for the user to select. The manual operation mode, on the other hand, fills the response data into the corresponding fill-in positions only after receiving a trigger signal.

[0072] Figure 3 The diagram shows a schematic block representation of a non-intrusive data access system in some embodiments of the present invention. This non-intrusive data access system is applied to data interaction. Figure 3 The non-intrusive data access system includes a first processing unit 1, a business scenario classification model 2, an image classifier 3, a feature vector model 4, and a second processing unit 5.

[0073] The first processing unit 1 is used to acquire the user terminal's operation interface data and generate real-time feature data based on the operation interface data;

[0074] Business scenario classification model 2 is used to perform scenario labeling on real-time feature data to determine the user's current business scenario;

[0075] Image classifier 3 is used to extract key feature information from the current business scenario and generate key feature data;

[0076] Feature vector model 4 is used to perform intent and location analysis on user interface data, current business scenarios and key feature data, generate recommendation data and vector data, and send the recommendation data to the configuration master data system;

[0077] The second processing unit 5 is used to determine the backfilling position of the response data based on the vector data after obtaining the response data fed back from the configuration master data system, and to display the response data on the operation interface.

[0078] In a preferred embodiment, the non-intrusive data access system further includes a service unit 6 for providing operational services for response data; wherein the operational services include editing, copying, and setting working modes.

[0079] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0080] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0081] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0082] It is understood that the above embodiments only illustrate preferred embodiments of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can freely combine the above technical features without departing from the concept of the present invention, and can also make several modifications and improvements, all of which fall within the protection scope of the present invention. Therefore, all equivalent transformations and modifications made with respect to the scope of the claims of the present invention should fall within the scope of the claims of the present invention.

Claims

1. A non-intrusive data access method applied to data interaction, characterized in that, Includes the following steps: S10. Obtain the user terminal's operation interface data, and generate real-time feature data based on the operation interface data; S20. The real-time feature data is calibrated using a predetermined business scenario classification model to determine the user's current business scenario; S30. Extract key feature information from the current business scenario and real-time feature data using a predetermined image classifier to obtain key feature data; the key feature data includes key feature information and confidence rate data. S40. Perform intent and location analysis on the operation interface data, the current business scenario, and the key feature data using a predetermined feature vector model to obtain recommended data and vector data, and send the recommended data to the configuration master data system. S50. After obtaining the response data fed back by the configuration master data system, determine the backfilling position of the response data according to the vector data, and then display the response data on the operation interface. S20 includes: labeling the business scenarios in the business scenario classification model that correspond to the real-time feature data, and calculating the correlation degree of each business scenario, and determining the user's current business scenario based on the correlation degree; the process of calculating the correlation degree includes: labeling and counting the content in the feature dataset of the business scenario that corresponds to or is consistent with the real-time feature data to obtain the number of labels for the business scenario, and dividing the number of labels by the total number of content types in the feature datasets of all business scenarios to obtain the correlation degree of the business scenario; S30 includes: the image classifier extracting key feature information based on the predetermined requirement features and real-time feature data of the current business scenario; the key feature information includes window title and text data; the image classifier also performs positioning analysis on the key feature information through a predetermined YOLO model to obtain positioning data, and calculates the confidence rate data of the key feature information based on the positioning data; In step S40, the following steps are included: the feature vector model generates slot filling to represent user intent based on the relevance of the current business scenario, the key feature information, and the confidence rate data, and then generates the recommended data based on the slot filling; the recommended data is used to control the configuration configuration master data system to access data, and the vector data is used to represent the location of the data to be filled.

2. The non-intrusive data access method according to claim 1, characterized in that, S10 includes: S101. Obtain user terminal interface data; S102. The operation interface data is parsed using computer vision technology to obtain the real-time feature data; wherein, the real-time feature data includes one or more of the following: real-time program, active window, screen, name, and cursor data.

3. The non-intrusive data access method according to claim 1, characterized in that, S30 includes: The image classifier is built using a residual neural network.

4. The non-intrusive data access method according to claim 1, characterized in that, S40 includes: The feature vector model performs OCR processing on the user interface data to obtain the position information of all box elements on each page, as well as the text content in each box element. The location information and the text content are vectorized to obtain the vector data of each of the box elements.

5. The non-intrusive data access method according to any one of claims 1 to 4, characterized in that, Also includes: S60. Provide operation services related to the response data through a pre-defined service engine based on the entered operation instructions.

6. The non-intrusive data access method according to claim 5, characterized in that, In S60, the operation services include editing, copying, and working mode settings.

7. A non-intrusive data access system for data interaction, characterized in that, include: The first processing unit (1) is used to acquire the user terminal's operation interface data and generate real-time feature data based on the operation interface data. A business scenario classification model (2) is used to perform scenario labeling on the real-time feature data and determine the user's current business scenario. This includes: labeling the business scenarios corresponding to the real-time feature data in the business scenario classification model, and calculating the correlation degree of each business scenario, and determining the user's current business scenario based on the correlation degree. The process of calculating the correlation degree includes: labeling and counting the content corresponding to or consistent with the real-time feature data in the feature dataset of the business scenario, thereby obtaining the number of labels for the business scenario, and dividing the number of labels by the total number of content types in the feature datasets of all business scenarios to obtain the correlation degree of the business scenario. An image classifier (3) is used to extract key feature information from the current business scenario and generate key feature data, including: the image classifier extracts key feature information based on the predetermined requirement features and real-time feature data of the current business scenario, the key feature information including window title and text data; the image classifier also performs positioning analysis on the key feature information through a predetermined YOLO model to obtain positioning data, and calculates the confidence rate data of the key feature information based on the positioning data; the key feature data includes key feature information and confidence rate data; The feature vector model (4) is used to perform intent and location analysis on the operation interface data, the current business scenario, and the key feature data, generate recommendation data and vector data, and send the recommendation data to the configuration master data system; the step of generating recommendation data includes: the feature vector model generates slot filling to represent user intent based on the correlation of the current business scenario, the key feature information, and the confidence rate data, and then generates the recommendation data based on the slot filling; the recommendation data is used to control the configuration master data system to access data; The second processing unit (5) is used to determine the backfill position of the response data according to the vector data after obtaining the response data fed back by the configuration master data system, and to display the response data on the operation interface.

Images