A real-time interactive and visualization system

Abstract

The present invention proposes a real-time interaction and visualization system, which forms a real-time interactive and intuitive display visualization system based on the display and operation of a graphical user interface UI, and realizes efficient use and synchronous operation of GIS data. Through network data synchronization, GIS data is rendered in real time based on operation instructions, and the operation results and display images are displayed differentially using the GIS panel, providing convenience for real-time GIS data loading. At the same time, the present invention takes multi-dimensional visualization as the basis, provides multi-dimensional interactive operation responses based on convenient gesture operations, and combines efficient big data model structure and algorithm design for command recognition and operation clustering, and uses network communication to realize synchronization between the display end and the interactive end of the graphical user interface data, thereby obtaining better interactive feedback effects.

Description

Technical Field

[0001] The present invention belongs to the field of new generation information technology, and in particular relates to a real-time interaction and visualization system. Background Art

[0002] In the process of engineering mapping, for mapping operations such as geographic information systems that have large data volumes and complex processing, it is often necessary to comprehensively process a variety of mapping information, such as various types of buildings, terrains, etc. As information processing requirements increase, the interaction of massive information has also become one of the problems that need to be solved urgently.

[0003] The background support technologies of real-time interactive and visual GIS systems can be briefly divided into computer technology, communication network technology, earth observation technology and big data platform technology.

[0004] Computer technology provides powerful storage and computing capabilities for geographic information services. At present, the storage capacity of computers has exceeded TB level, and the computing speed has exceeded 100 trillion times per second. The storage capacity and computing power of computers are crucial to promoting the development and evolution from geographic information systems to geographic information services. Communication network technology promotes the development and evolution of geographic information system architecture. The development of communication network technology has played a very important role in the development from geographic information systems to geographic information services. Earth observation technology provides a powerful information source support for solving the development and evolution of geographic information systems. GNSS provides high-precision positioning data and image data covering the world, providing strong support for solving the problem of high-precision information sources in the development and evolution of geographic information systems. Big data technology provides advanced and practical technical systems and engineering technology implementation support. The big data platform is built on SOA based on network / grid services, and adopts technology integration, data integration and business integration to realize multi-node collaborative management and operation across levels, regions, systems and businesses, greatly promoting the practical application of geographic information services.

[0005] In the development of GIS real-time interaction and visualization system, the understanding of GIS is also diverse. One is System, namely Geographic Information System (GISystem), called "Geographic Information System", emphasizing the engineering technology aspect of "GIS", and is defined as "the branch of information system related to geographic information", that is, a computer information system that acquires, stores, manages, analyzes and visualizes geographic information with the support of computer hardware and software; the second is Science, and the third is Service, namely Geographic Information Service (GIService), called "Geographic Information Service", emphasizing the application service of GIS, that is, the industrial aspect.

[0006] Taking GIS as an example, compared with traditional flat maps, GIS, as a visual representation of quantifiable data, is often presented in a dynamic way, for example, it can be updated in real time. At the same time, GIS system also has certain interactive functions, such as highlighting features and displaying according to given attributes or some GIS components can be used for operation and setting.

[0007] There are still many problems to be solved in the current real-time interaction and visualization systems. For example, the system process from command operation to data rendering in some real-time interaction and visualization systems is still relatively long, and the processing resources and computing power required are heavy, resulting in poor rendering performance of the system. Some real-time interaction and visualization systems cannot achieve real-time network information synchronization, and there are deficiencies in support for multi-point collection and comprehensive visualization display. In addition, although some real-time interaction and visualization GIS systems support gesture recognition and gesture operation, the corresponding gesture recognition and gesture operation technology still remains at the level of simple image comparison or mechanical contact perception of existing technologies, and does not fully consider the continuity of gesture operation actions, nor does it fully reflect the use of big data and adaptive learning AI networks, thereby improving the self-learning performance of the system recognition model of the real-time interaction and visualization GIS system, and providing higher accuracy and greater convenience for the operation of the real-time interaction and visualization system.

[0008] In order to solve the above-mentioned problems in the prior art, the present invention proposes a real-time interaction and visualization system, which forms a real-time interactive and intuitive display visualization system based on the display and operation of the graphical user interface UI, and realizes the efficient use and synchronous operation of GIS data. Through network data synchronization, GIS data is rendered in real time based on operation instructions, which provides convenience for real-time GIS data loading. At the same time, based on multi-dimensional visualization display, the present invention provides multi-dimensional interactive operation responses based on convenient gesture operations, and combines efficient big data model structure and algorithm design for command recognition and operation clustering, and uses network communication to realize the synchronization of the display end and the interaction end of the graphical user interface data, so as to obtain better interactive feedback effects. Summary of the invention

[0009] The application of real-time interaction and visualization systems is in the ascendant. For example, Wen Guojun and others from China University of Geosciences proposed the "Real-time Interactive Ship-Wave Motion Visualization System Based on Unity". In order to solve the problem of poor visual experience caused by the separation of ship-wave motion in existing navigation simulators, a ship-wave real-time interactive ship motion visualization simulation system was developed. The sea surface wave motion scene was constructed through wave spectrum modeling technology; on the basis of establishing a ship force model, the ship's response to wave force was realized by detecting the collision between the ship and the water surface to calculate the ship's real-time response posture change to wave motion. The ship-wave interaction has good real-time performance and good simulation effect.

[0010] The present invention aims to provide a real-time interaction and visualization system that is superior to the prior art. In connection with the above, the real-time interaction and visualization system proposed by the present invention can be applied to specific fields, such as GIS geographic information system.

[0011] In order to achieve the above object, the technical solution of the present invention is as follows:

[0012] A real-time interaction and visualization system, the system comprising:

[0013] A graphical user interface UI is used to display multi-dimensional visualized GIS images and perform interactive operations on the multi-dimensional visualized GIS images;

[0014] The graphical user interface UI may be located in a mobile terminal, and is used to implement the real-time interaction and visualization system operation layer provided by the present invention. The system operator interacts with the graphical user interface UI through a mobile terminal such as a mobile phone or a tablet. On the one hand, the system information is read from the graphical user interface UI to determine the system's command operation results and steady-state maintenance parameters. On the other hand, the command operation interaction is specifically performed on the graphical user interface UI to achieve two-way input and output of data and information.

[0015] The network data synchronization module synchronizes the recognition results of the interactive instruction recognition module, and the system platform determines whether the recognition results are correct based on the synchronization;

[0016] The network data synchronization module is connected to the interactive command recognition module and the graphical user interface UI, receives the recognition result of the interactive command recognition module, and synchronizes it to the graphical user interface UI, that is, feeds back the command recognition result to the graphical user interface UI for user reference. The user determines whether the command recognition result is the command result that he wants to achieve through the gesture operation through the display of the graphical user interface UI.

[0017] The recognition result is correct or wrong. Based on the recognition result synchronized by the network data synchronization module, when the instruction recognition result of the interactive instruction recognition module is judged to be correct, the multi-dimensional visualized GIS image is interactively operated to display the multi-dimensional visualized GIS image; and the GIS panel is a three-dimensional electronic display screen.

[0018] GIS panel, the GIS panel is an object panel for gesture operation, used to receive gesture operation instructions and present gesture operation results on the GIS panel;

[0019] The real-time interaction and visualization system also includes an interaction instruction recognition module, a GIS instruction information interaction module, and a system update module.

[0020] The interactive instruction recognition module is mainly used to recognize gestures and identify corresponding system operation instructions. The interactive instruction recognition module is connected to the network data synchronization module and is used to feed back instruction recognition results to the network data synchronization module so that the network data synchronization module can send them to the graphical user interface UI for display.

[0021] The interactive instruction recognition module is also connected to the GIS instruction information interaction module, and sends the specific gesture instruction recognized by the interactive instruction recognition module to the GIS instruction information interaction module, so that the GIS instruction information interaction module can convert it into GIS instruction information.

[0022] Preferably, the graphical user interface UI is used to display a multi-dimensional visualized GIS image and perform interactive operations on the multi-dimensional visualized GIS image, specifically: the graphical user interface UI receives the instruction recognition result of the interactive instruction recognition module, based on the recognition result synchronized by the network data synchronization module, when it is judged that the instruction recognition result of the interactive instruction recognition module is correct, interactive operations are performed on the multi-dimensional visualized GIS image to display the multi-dimensional visualized GIS image; and the GIS panel is a three-dimensional electronic display screen.

[0023] Preferably, the network data synchronization module performs data synchronization on the recognition results, and the system platform determines whether the recognition results are correct based on the synchronized recognition results. Specifically, the network data synchronization module is used to perform data synchronization on the recognition results according to the instruction recognition results of the interactive instruction recognition module, and feed back the results to the system platform to provide real-time feedback on the instruction recognition results of the interactive instruction recognition module; the system platform determines whether the instruction recognition results of the interactive instruction recognition module are correct based on the synchronized recognition results and the feedback from the operator.

[0024] Preferably, the interactive instruction recognition module performs motion capture and instruction recognition on the gesture operation, and the interactive instruction recognition module at least includes:

[0025] A gesture image capturing submodule, wherein the gesture image capturing submodule uses a three-dimensional holographic camera to capture gesture actions;

[0026] The gesture recognition submodule uses a big data gesture recognition model based on gesture actions to identify gesture actions, and recognizes the gesture operation as a specific gesture instruction according to the action identification result.

[0027] Preferably, the GIS instruction information interaction module is used to receive the specific gesture instruction recognized by the interactive instruction recognition module, and convert it into GIS instruction information to perform instruction-based GIS interaction; the GIS instruction information interaction module at least includes an instruction conversion submodule, an instruction fitting submodule, and an instruction calling submodule, wherein:

[0028] The command conversion submodule converts the gesture command conversion mapping relationship set into the corresponding GIS operation command signal;

[0029] The instruction fitting submodule receives a plurality of specific gesture instructions within a first time period; performs instruction fitting based on a continuity algorithm between the specific gesture instructions to obtain a fitting instruction;

[0030] The instruction calling submodule receives the fitting instruction, and calls the corresponding operation function of the GIS panel based on the fitting instruction to execute the GIS operation and present it visually.

[0031] Preferably, the system update module is used to receive the operation feedback of the instruction call, and perform adaptive learning of the instruction based on the feedback information to generate gesture preference settings; the system update module at least includes:

[0032] The command feedback submodule is used to receive feedback and evaluation of the operator on the visualization of GIS operations and determine the operation fidelity, specifically: the whole process of command conversion and command fitting with correct feedback results is entered into the big data gesture recognition model AI adaptive learning positive training database; the whole process of command conversion and command fitting with incorrect feedback results is entered into the big data gesture recognition model AI adaptive learning negative training database;

[0033] Adaptive AI learning submodule, which performs adaptive AI training and learning of big data gesture recognition models based on positive training database and negative training database;

[0034] The gesture preference submodule directly memorizes gesture operations that are repeated a large number of times and evaluated as correct by feedback, binds them to the corresponding fitting instructions, and provides an out-of-band gesture recognition strategy to avoid repeated recognition of preferred gestures.

[0035] Preferably, the big data gesture recognition model is used to perform gesture recognition based on a gesture three-dimensional image recognition pre-training model, perform clustering recognition processing on gesture actions, and output a recognition result that is recognized as a specific gesture instruction based on the action clustering result of the clustering algorithm;

[0036] The big data gesture recognition model also performs adaptive AI training and learning based on a positive training database and a negative training database, thereby improving recognition accuracy.

[0037] Preferably, the gesture instruction conversion mapping relationship set records the correspondence between specific gesture instructions preset in the system and GIS operation instruction signals, and based on the correspondence, converts the specific gesture instructions into GIS operation instruction signals.

[0038] Preferably, in addition to recording the correspondence between the specific gesture commands preset by the system and the GIS operation command signals, the gesture command conversion mapping relationship set also stores a custom gesture command setting module, and the custom gesture command setting module is used to copy the conversion mapping relationship of the specific gesture commands to the custom GIS operation command signals.

[0039] At the same time, the present invention also proposes a computer program product, which is stored in a computer-readable storage medium, and is characterized in that when the computer program product is executed by a processor, it is used to execute the corresponding functions of the real-time interaction and visualization system as described in claims 1-9.

[0040] The present invention proposes a real-time interaction and visualization system, which forms a real-time interactive and intuitive display visualization system based on the display and operation of a graphical user interface UI, and realizes efficient use and synchronous operation of GIS data. Through network data synchronization, GIS data is rendered in real time based on operation instructions, providing convenience for real-time GIS data loading. At the same time, based on multi-dimensional visualization display, the present invention provides multi-dimensional interactive operation responses based on convenient gesture operations, and combines efficient big data model structure and algorithm design for command recognition and operation clustering, and uses network communication to realize synchronization between the display end and the interactive end of the graphical user interface data, thereby obtaining better interactive feedback effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a basic system structure diagram of the real-time interaction and visualization system shown in the present invention;

[0042] Figure 2 It is a schematic diagram of one embodiment of the real-time interaction and visualization system instruction interactive flow process shown in the present invention;

[0043] Figure 3 It is a schematic diagram of one embodiment of a real-time interaction and visualization system interaction instruction recognition module, a GIS instruction information interaction module and a system update module shown in the present invention;

[0044] Figure 4 It is one of the embodiments of the real-time interaction and visualization system GIS panel interacting with other modules claimed in the present invention;

[0045] Figure 5 It is a schematic diagram of a preferred embodiment of the interaction between the network data synchronization module of the real-time interaction and visualization system and other modules shown in the present invention. DETAILED DESCRIPTION

[0046] Several embodiments and beneficial effects of the real-time interaction and visualization system and method claimed in the present invention are described in detail below to facilitate a more detailed review and analysis of the present invention.

[0047] In order to better understand the technical solution of the present invention, the embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0048] It should be clear that the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0050] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0051] It should be understood that although the terms first, second, etc. may be used to describe the method and corresponding device in the embodiments of the present invention, these keywords should not be limited to these terms. These terms are only used to distinguish keywords from each other. For example, without departing from the scope of the embodiments of the present invention, a template, etc. may also be referred to as a target plate.

[0052] It should be understood that although the terms first, second, etc. may be used to describe the method and corresponding device in the embodiments of the present invention, these keywords should not be limited to these terms. These terms are only used to distinguish the keywords from each other. For example, without departing from the scope of the embodiments of the present invention, the first time period, etc. may also be referred to as the second time period.

[0053] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0054] As the instruction manual Figure 1-5As shown, it is one of the embodiments of the real-time interaction and visualization system and its specific included modules claimed by the present invention. The real-time interaction and visualization system claimed by the present invention includes:

[0055] A graphical user interface UI is used to display multi-dimensional visualized GIS images and perform interactive operations on the multi-dimensional visualized GIS images;

[0056] The graphical user interface UI may be located in a mobile terminal, and is used to implement the real-time interaction and visualization system operation layer provided by the present invention. The system operator interacts with the graphical user interface UI through a mobile terminal such as a mobile phone or a tablet. On the one hand, the system information is read from the graphical user interface UI to determine the system's command operation results and steady-state maintenance parameters. On the other hand, the command operation interaction is specifically performed on the graphical user interface UI to achieve two-way input and output of data and information.

[0057] The network data synchronization module synchronizes the instruction recognition results of the interactive instruction recognition module, and the system platform determines whether the synchronized recognition results are correct;

[0058] The network data synchronization module is connected to the interactive command recognition module and the graphical user interface UI, receives the recognition result of the interactive command recognition module, and synchronizes it to the graphical user interface UI, that is, feeds back the command recognition result to the graphical user interface UI for user reference. The user determines whether the command recognition result is the command result that he wants to achieve through the gesture operation through the display of the graphical user interface UI.

[0059] The recognition result is correct or wrong. Based on the recognition result synchronized by the network data synchronization module, when the instruction recognition result of the interactive instruction recognition module is judged to be correct, the multi-dimensional visualized GIS image is interactively operated to display the multi-dimensional visualized GIS image; and the GIS panel is a three-dimensional electronic display screen.

[0060] GIS panel, the GIS panel is an object panel for gesture operation, used to receive gesture operation instructions and present gesture operation results on the GIS panel;

[0061] The real-time interaction and visualization system also includes an interaction instruction recognition module, a GIS instruction information interaction module, and a system update module.

[0062] The interactive instruction recognition module is mainly used to recognize gestures and identify corresponding system operation instructions. The interactive instruction recognition module is connected to the network data synchronization module and is used to feed back instruction recognition results to the network data synchronization module so that the network data synchronization module can send them to the graphical user interface UI for display.

[0063] The interactive instruction recognition module is also connected to the GIS instruction information interaction module, and sends the specific gesture instruction recognized by the interactive instruction recognition module to the GIS instruction information interaction module, so that the GIS instruction information interaction module can convert it into GIS instruction information.

[0064] As a superimposable preferred embodiment, the graphical user interface UI is used to display a multi-dimensional visualized GIS image and perform interactive operations on the multi-dimensional visualized GIS image. Specifically, the graphical user interface UI receives the instruction recognition result of the interactive instruction recognition module, and based on the recognition result synchronized by the network data synchronization module, when it is judged that the instruction recognition result of the interactive instruction recognition module is correct, interactive operations are performed on the multi-dimensional visualized GIS image to display the multi-dimensional visualized GIS image; and the GIS panel is a three-dimensional electronic display screen.

[0065] As a superimposable preferred embodiment, the network data synchronization module performs data synchronization on the recognition results, and the system platform determines whether the recognition results are correct based on the synchronized recognition results. Specifically, the network data synchronization module is used to perform data synchronization on the recognition results according to the instruction recognition results of the interactive instruction recognition module, and feed back to the system platform to feed back the instruction recognition results of the interactive instruction recognition module in real time; the system platform determines whether the instruction recognition results of the interactive instruction recognition module are correct based on the synchronized recognition results and the feedback from the operator.

[0066] As a superimposable preferred embodiment, an interactive instruction recognition module performs motion capture and instruction recognition on gesture operations, and the interactive instruction recognition module at least includes:

[0067] A gesture image capturing submodule, wherein the gesture image capturing submodule uses a three-dimensional holographic camera to capture gesture actions;

[0068] The gesture recognition submodule uses a big data gesture recognition model based on gesture actions to identify gesture actions, and recognizes the gesture operation as a specific gesture instruction according to the action identification result.

[0069] As a superimposable preferred embodiment, the GIS instruction information interaction module is used to receive the specific gesture instruction recognized by the interactive instruction recognition module, and convert it into GIS instruction information to perform instruction-based GIS interaction; the GIS instruction information interaction module at least includes an instruction conversion submodule, an instruction fitting submodule, and an instruction calling submodule, wherein:

[0070] The command conversion submodule converts the gesture command conversion mapping relationship set into the corresponding GIS operation command signal;

[0071] As a superimposable preferred embodiment, the command conversion submodule converts the gesture command conversion mapping relationship set to the corresponding GIS operation command signal based on the gesture command conversion mapping relationship set, specifically: the gesture command conversion mapping relationship set records the correspondence between the specific gesture commands preset by the system and the GIS operation command signals, and based on the correspondence, converts the specific gesture commands into GIS operation command signals. For example, the gesture command conversion mapping relationship set records that the swing gesture - waved twice corresponds to the GIS operation command signal "remove GIStag point"; the tap gesture - 1 time corresponds to the GIS operation command signal "add GIStag point" and so on, which will not be repeated here. At the same time, in addition to recording the correspondence between the specific gesture commands preset by the system and the GIS operation command signals, the gesture command conversion mapping relationship set also stores a custom gesture command setting module, which is used to copy the conversion mapping relationship of the specific gesture command to the custom GIS operation command signal. For example, an operator copies the "add GIStag point" from the tap gesture-1 time to the click gesture-double-click, which provides an out-of-band gesture operation method for "add GIStag point", and during the operation of this specific person, a specific method of click gesture-double-click is provided for "add GIStag point".

[0072] The instruction fitting submodule receives a plurality of specific gesture instructions within a first time period; performs instruction fitting based on a continuity algorithm between the specific gesture instructions to obtain a fitting instruction;

[0073] As a superimposable preferred embodiment, the instruction fitting submodule receives multiple specific gesture instructions within a first time period; based on the continuity algorithm between specific gesture instructions, the instruction fitting is performed to obtain a fitting instruction, specifically: the gesture operation interval is set to be longer than a specific time length as the fitting gesture interruption point, and the previous and subsequent points are different fitting gestures. For example, the operator performs 5 different gesture operations in succession and interrupts for 3S. If the gesture operation interval is 2S, then the other gesture operations after the interruption are determined to be the next group of gestures to be fitted rather than the current group; based on the continuity of the gesture, the continuity algorithm is used to determine whether the gesture to be fitted is a continuous gesture for the same operation object on the GIS. If so, the fitting gesture is defined as a continuous operation for the same operation object on the GIS to obtain a fitting instruction; if not, only the first gesture in the gesture to be fitted is processed.

[0074] Based on the continuity of the gesture, a continuity algorithm is used to determine whether the fitting gesture is a continuous gesture for the same operation object on the GIS. Specifically, based on the pause time threshold between continuous gestures, it is determined whether it is a continuous gesture.

[0075] The instruction calling submodule receives the fitting instruction, and calls the corresponding operation function of the GIS panel based on the fitting instruction to execute the GIS operation and present it visually.

[0076] As the instruction manual Figure 2 As shown, Figure 2 It is a schematic diagram of one embodiment of the real-time interaction and visualization system instruction interactive flow process shown in the present invention.

[0077] The interactive command recognition module completes the analysis of the gesture recognition and forms the gesture command recognition result. Subsequently, the interactive command recognition module is connected to the network data synchronization module to feed back the command recognition result to the network data synchronization module so that the network data synchronization module can send it to the graphical user interface UI for display.

[0078] At the same time, the interactive instruction recognition module is also connected to the GIS instruction information interaction module, and sends the specific gesture instructions recognized by the interactive instruction recognition module to the GIS instruction information interaction module, so that the GIS instruction information interaction module can convert it into GIS instruction information.

[0079] The GIS instruction information interaction module at least includes an instruction conversion submodule, an instruction fitting submodule, and an instruction calling submodule. The GIS instruction information interaction module receives the specific gesture instruction recognized by the interactive instruction recognition module and converts it into GIS instruction information after being processed by the following submodules:

[0080] The command conversion submodule converts the gesture command conversion mapping relationship set into the corresponding GIS operation command signal;

[0081] The instruction fitting submodule receives a plurality of specific gesture instructions within a first time period; performs instruction fitting based on a continuity algorithm between the specific gesture instructions to obtain a fitting instruction;

[0082] The instruction calling submodule receives the fitting instruction, and calls the corresponding operation function of the GIS panel based on the fitting instruction, executes the GIS operation and presents it visually.

[0083] Therefore, the GIS panel will perform interactive operations on the system based on the instructions received, and finally display the system status, system parameters, system operation status and other information after the operation on the GIS panel through the UI.

[0084] As a superimposable preferred embodiment, a system update module is used to receive operation feedback of a command call, and perform adaptive learning of the command based on the feedback information to generate gesture preference settings; the system update module at least includes:

[0085] The command feedback submodule is used to receive feedback and evaluation of the operator on the visualization of GIS operations and determine the operation fidelity, specifically: the whole process of command conversion and command fitting with correct feedback results is entered into the big data gesture recognition model AI adaptive learning positive training database; the whole process of command conversion and command fitting with incorrect feedback results is entered into the big data gesture recognition model AI adaptive learning negative training database;

[0086] Adaptive AI learning submodule, which performs adaptive AI training and learning of big data gesture recognition models based on positive training database and negative training database;

[0087] The gesture preference submodule directly memorizes gesture operations that are repeated a large number of times and evaluated as correct by feedback, binds them to the corresponding fitting instructions, and provides an out-of-band gesture recognition strategy to avoid repeated recognition of preferred gestures.

[0088] As a superimposable preferred embodiment, the big data gesture recognition model is used to perform gesture recognition based on a gesture three-dimensional image recognition pre-training model, perform clustering recognition processing on gesture actions, and output a recognition result that is recognized as a specific gesture instruction based on the action clustering result of the clustering algorithm;

[0089] The big data gesture recognition model also performs adaptive AI training and learning based on a positive training database and a negative training database, thereby improving recognition accuracy.

[0090] As a superimposable preferred embodiment, the gesture command conversion mapping relationship set records the correspondence between specific gesture commands preset by the system and GIS operation command signals, and based on the correspondence, converts the specific gesture commands into GIS operation command signals.

[0091] As a superimposable preferred embodiment, the gesture command conversion mapping relationship set not only records the correspondence between the specific gesture commands preset by the system and the GIS operation command signals, but also stores a custom gesture command setting module, and the custom gesture command setting module is used to copy the conversion mapping relationship of the specific gesture commands to the custom GIS operation command signals.

[0092] At the same time, the present invention also proposes a computer program product, which is stored in a computer-readable storage medium, and is characterized in that when the computer program product is executed by a processor, it is used to execute the corresponding functions of the real-time interaction and visualization system as described in claims 1-9.

[0093] The present invention proposes a real-time interaction and visualization system, which forms a real-time interactive and intuitive display visualization system based on the display and operation of a graphical user interface UI, and realizes efficient use and synchronous operation of GIS data. Through network data synchronization, GIS data is rendered in real time based on operation instructions, providing convenience for real-time GIS data loading. At the same time, based on multi-dimensional visualization display, the present invention provides multi-dimensional interactive operation responses based on convenient gesture operations, and combines efficient big data model structure and algorithm design for command recognition and operation clustering, and uses network communication to realize synchronization between the display end and the interactive end of the graphical user interface data, thereby obtaining better interactive feedback effects.

[0094] In all the above embodiments, in order to realize some special data transmission and read / write function requirements, the above method operation process and its corresponding device can add devices, modules, components, hardware, pin connections or memory, processor differences to expand functions.

[0095] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the methods, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0096] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the method steps is only a logical or functional division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0097] The units described as the various steps of the method and the separated components of the device may or may not be logically or physically separated, and may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0098] In addition, the method steps and their implementations and functional units in the various embodiments of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0099] The above method and device can be implemented as an integrated unit in the form of a software functional unit, which can be stored in a computer-readable storage medium. The above software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), NVRAM, disk or optical disk, etc., which can store program code.

[0100] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

[0101] It should be noted that the above embodiments are only used to more clearly explain and illustrate the technical solutions of the present invention, rather than to limit them. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A real-time interaction and visualization system, the system comprising: A graphical user interface UI is used to display multi-dimensional visualized GIS images and perform interactive operations on the multi-dimensional visualized GIS images; The graphical user interface UI is used to display a multi-dimensional visualized GIS image and perform interactive operations on the multi-dimensional visualized GIS image, specifically: the graphical user interface UI receives the instruction recognition result of the interactive instruction recognition module, based on the recognition result synchronized by the network data synchronization module, when it is judged that the instruction recognition result of the interactive instruction recognition module is correct, interactive operations are performed on the multi-dimensional visualized GIS image to display the multi-dimensional visualized GIS image; and the GIS panel is a three-dimensional electronic display screen; The network data synchronization module synchronizes the recognition results, and the system platform determines whether the recognition results are correct based on the synchronized recognition results; The real-time interaction and visualization system also includes an interaction instruction recognition module, a GIS instruction information interaction module, and a system update module; The network data synchronization module performs data synchronization on the recognition result, and the system platform determines whether it is correct based on the synchronized recognition result. Specifically, the network data synchronization module is used to synchronize the recognition result according to the instruction recognition result of the interactive instruction recognition module, and feed it back to the system platform, and feed back the instruction recognition result of the interactive instruction recognition module in real time; the system platform determines whether the instruction recognition result of the interactive instruction recognition module is correct based on the synchronized recognition result and the feedback of the operator; The interactive instruction recognition module performs motion capture and instruction recognition on the gesture operation, and the interactive instruction recognition module at least includes: A gesture image capturing submodule, wherein the gesture image capturing submodule uses a three-dimensional holographic camera to capture gesture actions; A gesture recognition submodule, which uses a big data gesture recognition model based on gesture actions to identify gesture actions, and recognizes the gesture operation as a specific gesture instruction according to the action identification result; GIS panel, which is an object panel for gesture operation, and is used to receive gesture operation instructions and present gesture operation results on the GIS panel; The GIS instruction information interaction module receives the specific gesture instruction recognized by the interactive instruction recognition module, converts it into GIS instruction information, and performs instruction-based GIS interaction; the GIS instruction information interaction module at least includes an instruction conversion submodule, an instruction fitting submodule, and an instruction calling submodule, wherein: The command conversion submodule converts the gesture command conversion mapping relationship set into the corresponding GIS operation command signal; The instruction fitting submodule receives a plurality of specific gesture instructions within a first time period; performs instruction fitting based on a continuity algorithm between the specific gesture instructions to obtain a fitting instruction; The instruction calling submodule receives the fitting instruction, and calls the corresponding operation function of the GIS panel based on the fitting instruction to perform the GIS operation and present it visually; The system update module receives the operation feedback of the instruction call, and performs adaptive learning of the instruction based on the feedback information to generate gesture preference settings; the system update module at least includes: The command feedback submodule is used to receive feedback and evaluation of the operator on the visualization of GIS operations and determine the operation fidelity, specifically: the whole process of command conversion and command fitting with correct feedback results is entered into the big data gesture recognition model AI adaptive learning positive training database; the whole process of command conversion and command fitting with incorrect feedback results is entered into the big data gesture recognition model AI adaptive learning negative training database; Adaptive AI learning submodule, which performs adaptive AI training and learning of big data gesture recognition models based on positive training database and negative training database; The gesture preference submodule directly memorizes gesture operations that are repeated a large number of times and evaluated as correct by feedback, binds them to the corresponding fitting instructions, and provides an out-of-band gesture recognition strategy to avoid repeated recognition of preferred gestures.

2. The real-time interaction and visualization system according to claim 1, characterized in that: The big data gesture recognition model is used to perform gesture recognition based on a gesture three-dimensional image recognition pre-training model, perform clustering recognition processing on gesture actions, and output a recognition result that is recognized as a specific gesture instruction according to the action clustering result of the clustering algorithm; The big data gesture recognition model also performs adaptive AI training and learning based on a positive training database and a negative training database, thereby improving recognition accuracy.

3. The real-time interaction and visualization system according to claim 2, characterized in that: The gesture instruction conversion mapping relationship set records the correspondence between specific gesture instructions preset by the system and GIS operation instruction signals, and based on the correspondence, converts the specific gesture instructions into GIS operation instruction signals.

4. The real-time interaction and visualization system according to claim 3, characterized in that: In addition to recording the correspondence between the specific gesture commands preset by the system and the GIS operation command signals, the gesture command conversion mapping relationship set also stores a custom gesture command setting module, which is used to copy the conversion mapping relationship of the specific gesture commands to the custom GIS operation command signals.

5. A computer program product, the computer program product being stored in a computer-readable storage medium, characterized in that: When the computer program product is executed by a processor, it is used to perform the corresponding functions of the real-time interaction and visualization system as described in any one of claims 1 to 4.

Images