Multi-screen linkage sand table display method and device

Abstract

The present application relates to the technical field of multimedia, and provides a multi-screen linkage sand table display method and device. The present application sets a multi-screen linkage sand table display device in an electronic sand table exhibition area, an operator displays designated city planning data by controlling multiple display screens of the multi-screen linkage sand table display device, acquires facial images of all visitors in the electronic sand table exhibition area to obtain job level tags of the visitors, preloads city planning data to be displayed according to the job level tags to obtain preloaded data, and displays the preloaded data on corresponding display screens when the operator selects to display the city planning data, so that the operator can display the data without feeling the data loading delay. The present application solves the problem that the content data volume to be displayed by a large screen in an electronic sand table is large, the loading speed is slow, the large screen picture is often disordered, and the visitor experience is affected.

Description

Technical Field

[0001] This invention relates to the field of multimedia technology, and in particular to a method and apparatus for multi-screen interactive sand table display. Background Technology

[0002] Electronic sand tables, as an emerging information display and decision support tool, are increasingly being widely used in various fields. In urban planning display scenarios, using electronic sand tables to display urban planning data can comprehensively showcase the geographical location, planning layout, supporting facilities, and other key information and unique advantages of a designated area to visitors in a two- or three-dimensional visualization format, maximizing display efficiency.

[0003] However, due to the large amount of data that needs to be displayed on the large screen in the electronic sand table and the slow loading speed, the screen often appears cluttered before the content is fully loaded, resulting in a poor experience for visitors and seriously affecting the efficiency of urban planning displays.

[0004] Therefore, overcoming the shortcomings of the existing technology is an urgent problem to be solved in this technical field. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a multi-screen interactive sand table display method and device. The purpose is to preload the urban planning data to be displayed according to the job level labels to obtain preloaded data; when the operator selects to display urban planning data, the preloaded data is displayed on the corresponding display screen, so that the visitors do not feel the data loading delay when the operator displays the data. This solves the problem that the large amount of content data to be displayed on the large screen of the electronic sand table is slow to load, which often results in the large screen screen being cluttered and affecting the visitor experience.

[0006] The present invention adopts the following technical solution:

[0007] In a first aspect, the present invention provides a multi-screen linkage sand table display method, wherein a multi-screen linkage sand table display device is set up in the electronic sand table exhibition area, and the operator displays specified urban planning data by controlling multiple displays of the multi-screen linkage sand table display device.

[0008] The multi-screen interactive sand table display method includes:

[0009] Obtain facial images of all visitors in the electronic sand table exhibition area to obtain the visitor's job title label;

[0010] Based on the job level labels, the urban planning data to be displayed is preloaded to obtain preloaded data;

[0011] When the operator selects to display the urban planning data, the pre-loaded data is displayed on the corresponding screen.

[0012] Furthermore, the preloading of the urban planning data to be displayed based on the job level tags, resulting in preloaded data, includes:

[0013] When a conversational voice source can be captured in the electronic sand table exhibition area, voice recording is performed to obtain voice information;

[0014] Determine the job title label of the visitor corresponding to the voice information;

[0015] When the voice information corresponds to multiple visitors, the validity of the pre-loading of the visitors' speech content in the voice information is determined according to the hierarchical order of the job level tags.

[0016] When the speech content of the visitors has pre-loading validity, the pre-loading priority of the visitors in the voice information is obtained according to the order of the job level tags.

[0017] According to the preloading priority, the urban planning data involved in the voice information is preloaded to obtain preloaded data.

[0018] Furthermore, when the voice information corresponds to multiple visitors, determining whether the pre-loading validity exists in the speech content of the visitors in the voice information based on the hierarchical order of the job level tags includes:

[0019] If the voice information is the first dialogue in this sand table display process, then when the visitor's job title tag belongs to management, the visitor's speech content has pre-load validity.

[0020] If the voice information is not the first dialogue in this sand table demonstration, the speech content of visitors whose job level tag belongs to management has pre-loading validity; and based on the contextual relationship between the speech content of each visitor in the voice information, the speech content of visitors whose job level tag does not belong to management has pre-loading validity.

[0021] Furthermore, based on the contextual relationships between the statements of each visitor in the voice information, the statements of visitors whose job level tags do not belong to management and who have pre-loaded validity are identified as follows:

[0022] The speech content of each visitor in the voice information is converted from voice format to text format to obtain text content;

[0023] Based on the recording time sequence of multiple text contents corresponding to the voice information, the semantic relationship between the multiple text contents is analyzed to obtain the contextual relationship between the speech contents of each visitor in the voice information;

[0024] Determine whether the statements made by visitors whose job titles are not in management positions represent a negation of the statements made by visitors whose job titles are in management positions.

[0025] When negating the speech of a visitor whose job title tag belongs to management, the corresponding recording time order is obtained based on the contextual relationship. It is determined that the speech content recorded earlier in the time order does not have pre-loading validity, and the speech content recorded later in the time order has pre-loading validity, so as to obtain the speech content with pre-loading validity among visitors whose job title tag does not belong to management.

[0026] Furthermore, before determining whether the pre-loading validity exists for the speech content of the visitor in the voice information based on the hierarchical order of the job title tags when the voice information corresponds to multiple visitors, the method further includes:

[0027] When voice recording begins, multiple facial images of each visitor in the electronic sand table exhibition area are continuously captured to obtain the time range of the capture of multiple facial images of each visitor and the time range of the recording of each visitor's speech.

[0028] When a collection time range matches a recording time range, the visitors corresponding to the collection time range are obtained, and the vocal information of the speech content corresponding to the recording time range is obtained.

[0029] A voice mapping relationship is established between the visitors and the voice information to extract the voice information. Based on the extracted voice information and the voice mapping relationship, the corresponding job title label of the visitors is obtained.

[0030] Furthermore, when the visitor's speech content has pre-loading validity, the pre-loading priority of the visitor in the voice information is obtained according to the hierarchical order of the job level tags, including:

[0031] The job title tags are determined in descending order to determine the preloading priority of the corresponding visitors in the voice information, so as to obtain the preloading priority.

[0032] Furthermore, the preloading of urban planning data related to the voice information according to the preloading priority, resulting in preloaded data including:

[0033] The speech content is converted from audio to text format to obtain text content; keywords are extracted from the text content.

[0034] In the urban planning data corresponding to the multiple display screens, the content is matched using the speaking keywords to obtain the data identifier of at least one urban planning data to be displayed;

[0035] Obtain data identifiers and cache at least one city planning data to be displayed in memory to obtain preloaded data.

[0036] Furthermore, a data database is pre-established, which includes the facial information and job title tags of visitors; cameras are installed in the electronic sand table exhibition area;

[0037] The step of obtaining facial images of all visitors in the electronic sand table exhibition area to obtain the visitor's job title label includes:

[0038] The camera captures facial images of the visitors; these facial images are then matched with facial information in the database to obtain the corresponding job title tags for the visitors.

[0039] Secondly, the present invention also provides a multi-screen interactive sand table display device, comprising:

[0040] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the processor for performing the multi-screen interactive sandbox display method described in the first aspect.

[0041] Thirdly, the present invention also provides a non-volatile computer storage medium storing computer-executable instructions, which are executed by one or more processors to perform the multi-screen interactive sand table display method described in the first aspect.

[0042] Fourthly, a computer program product containing instructions is provided, which, when executed on a computer or processor, causes the computer or processor to perform a multi-screen interactive sandbox display method as described in the first to third aspects and any one of them.

[0043] Fifthly, the present invention also provides a multi-screen interactive sand table display system, including a multi-screen interactive sand table display device as described in the second aspect, and using the multi-screen interactive sand table display method as described in the first aspect to complete the interaction of the multi-screen interactive sand table display device of the second aspect.

[0044] Unlike existing technologies, the present invention has at least the following beneficial effects:

[0045] This invention obtains the job title tags of all visitors in the electronic sand table exhibition area by analyzing their facial images. Based on these job title tags, the required urban planning data to be displayed is preloaded. When the operator selects to display urban planning data, the preloaded data is displayed on the corresponding screen. This ensures that visitors do not experience any data loading delay during the display, thus improving the visitor experience and increasing the efficiency of urban planning displays. It also solves the problem that the large amount of data to be displayed on the large screen of the electronic sand table and the slow loading speed often result in cluttered screen images that negatively impact the visitor experience. Attached Figure Description

[0046] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments of the present invention will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0047] Figure 1 This is a schematic diagram of an electronic sand table exhibition area provided in an embodiment of the present invention;

[0048] Figure 2 This is a flowchart illustrating a multi-screen interactive sand table display method provided in an embodiment of the present invention;

[0049] Figure 3 This is a flowchart illustrating step 20 provided in an embodiment of the present invention;

[0050] Figure 4 This is a flowchart illustrating another multi-screen interactive sand table display method provided in an embodiment of the present invention;

[0051] Figure 5 This is a flowchart illustrating step 103 provided in an embodiment of the present invention;

[0052] Figure 6 This is a flowchart illustrating step 202 provided in an embodiment of the present invention;

[0053] Figure 7 This is a flowchart illustrating step 105 provided in an embodiment of the present invention;

[0054] Figure 8 This is a schematic diagram of the large-screen structure of an electronic sand table according to an embodiment of the present invention;

[0055] Figure 9 This is an overall visual effect diagram of an electronic sand table provided in an embodiment of the present invention;

[0056] Figure 10 This is a schematic diagram of the data interaction process of a multi-screen interactive sand table display device provided in an embodiment of the present invention;

[0057] Figure 11 This is a schematic diagram of the architecture of a multi-screen interactive sand table display device provided in an embodiment of the present invention. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0059] Unless the context otherwise requires, throughout the specification and claims, the term "comprising" is interpreted as openly inclusive, meaning "including, but not limited to." In the description of the specification, terms such as "one embodiment," "some embodiments," "exemplary embodiment," "example," "specific example," or "some examples" are intended to indicate that a particular feature, structure, material, or characteristic associated with that embodiment or example is included in at least one embodiment or example of this disclosure. The illustrative representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics mentioned may be included in any suitable manner in any one or more embodiments or examples; that is, although they may be incorporated into embodiments or examples using the above terms due to reasons such as order and position, it does not limit them to be incorporated in combination by a single embodiment or example.

[0060] In the description of this invention, it should be understood that the terms "center", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this disclosure and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this disclosure.

[0061] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of this disclosure, unless otherwise stated, "a plurality of" means two or more. Furthermore, for example, the description may use the prefix "A" or "B" to describe the same type of nouns as two independent entities. In this case, the corresponding features defined with "A" and "B" are used only to distinguish between similar entities and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features.

[0062] In describing some embodiments, the terms "coupled," "coupled," and "connected," and their derivative expressions, may be used. For example, the term "connected" may be used in describing some embodiments to indicate that two or more components have direct physical or electrical contact with each other. Similarly, the term "coupled" may be used in describing some embodiments to indicate that two or more components have direct physical or electrical contact. However, the terms "connected" or "coupled" may also refer to two or more components that do not have direct contact with each other but still cooperate or interact with each other, such as "optical coupling," "wireless connection," etc. The embodiments disclosed herein are not necessarily limited to the scope of this invention.

[0063] In the description of this invention, the expression “A and / or B” (where A and B are used to formally represent specific features) will be used. The corresponding expression includes the following three combinations: only A, only B, and a combination of A and B.

[0064] As used in this invention, “about,” “approximately,” or “approximately” includes the stated value and the average value within an acceptable range of deviation from a particular value, wherein the acceptable range of deviation is determined by a person skilled in the art taking into account the measurement under discussion and the error associated with the measurement of the particular quantity (i.e., the limitations of the measurement system).

[0065] Example 1:

[0066] To address the aforementioned problems, this invention provides a multi-screen interactive sand table display method. The applicable scenarios for this invention are described below:

[0067] like Figure 1As shown, a multi-screen interactive sand table display device is set up in the electronic sand table exhibition area. Operators control the multiple displays of this device to show specified urban planning data. Visitors enter the electronic sand table exhibition area, stand in front of the multi-screen interactive sand table display device to view the electronic sand table, thus realizing the urban planning display, and inform the operators of the urban planning data they wish to display in the electronic sand table. The operators then select specific areas through clicks and other operations on the control panel, thereby changing the content displayed in the electronic sand table. Visitors include management leaders and ordinary staff.

[0068] like Figure 2 As shown, the multi-screen interactive sand table display method includes:

[0069] Step 10: Obtain facial images of all visitors in the electronic sand table exhibition area to obtain the visitor's job title label.

[0070] The process involves: establishing a database in advance, which includes the facial information and job title tags of visitors; setting up cameras in the electronic sand table exhibition area; capturing facial images of visitors through the cameras; and matching the facial images with the facial information in the database to obtain the corresponding job title tags of the visitors.

[0071] Step 20: Preload the urban planning data to be displayed according to the job level labels to obtain preloaded data.

[0072] In this embodiment of the invention, based on the recorded facial information, the rank tags of the visitors during the urban planning exhibition are matched in the data database. In this scenario, the visitors' speaking power is determined according to their rank, and the urban planning data that may be displayed is preloaded in order of speaking power from greatest to least.

[0073] Step 30: When the operator selects to display the urban planning data, the preloaded data is displayed on the corresponding screen.

[0074] In one embodiment, operators can be treated as visitors, so that their comments may serve as a basis for preloading urban planning data.

[0075] It should be noted that during the preloading process in this embodiment of the invention, the content currently displayed on the electronic sand table screen remains unchanged. The preloading is only for caching the corresponding urban planning data. The content currently displayed on the screen only changes when the operator selects to display the urban planning data on the screen. By preloading the urban planning data that may be displayed in advance, the perceived delay of the screen image for visitors when the operator selects to display a certain urban planning data can be minimized as much as possible.

[0076] This invention obtains the job title tags of all visitors in the electronic sand table exhibition area by analyzing their facial images. Based on these job title tags, the required urban planning data to be displayed is preloaded. When the operator selects to display urban planning data, the preloaded data is displayed on the corresponding screen. This ensures that visitors do not experience any data loading delay during the display, thus improving the visitor experience and increasing the efficiency of urban planning displays. It also solves the problem that the large amount of data to be displayed on the large screen of the electronic sand table and the slow loading speed often result in cluttered screen images that negatively impact the visitor experience.

[0077] The following is a detailed explanation of the preloading process based on job level tags:

[0078] like Figure 3 As shown, in step 20, the preloading of the urban planning data to be displayed based on the job level label, resulting in preloaded data, includes:

[0079] Step 101: When a dialogue source can be captured in the electronic sand table exhibition area, record the voice to obtain the voice information.

[0080] Step 102: Determine the job title label of the visitor corresponding to the voice information.

[0081] Step 103: When the voice information corresponds to multiple visitors, determine whether the content spoken by the visitors in the voice information has pre-loading validity based on the hierarchical order of the job level tags.

[0082] Among them, preloading validity refers to the possibility that the city regions mentioned in the speech content may need to be loaded.

[0083] Step 104: When the visitor's speech content has pre-loading validity, the pre-loading priority of the visitor in the voice information is obtained according to the rank order of the rank tags.

[0084] Specifically, the order of job title tags, from highest to lowest, is determined as the preloading priority order of the corresponding visitors in the voice information, in order to obtain the preloading priority.

[0085] The higher the job title tag, the higher the preloading priority. For example, when visitor 1's job title tag is a manager-level leader and visitor 2's job title tag is a regular staff member, visitor 1's preloading priority is higher than visitor 2's preloading priority.

[0086] Step 105: According to the preloading priority, preload the urban planning data involved in the voice information to obtain preloaded data.

[0087] The electronic sand table can display various city planning data with corresponding data identifiers. When the voice message mentions the data identifier or keywords of a certain city planning data, it means that the voice message involves that city planning data.

[0088] In one embodiment, the city planning data is cached in memory to complete the preloading process and obtain preloaded data.

[0089] To determine whether the pre-loading validity of the visitors' speech in the voice information is valid, such as... Figure 4 As shown, before determining whether the pre-loading validity exists for the speech content of the visitor in the voice information based on the hierarchical order of the job title tags when the voice information corresponds to multiple visitors, the method further includes:

[0090] Step 201: When the voice recording begins, continuously capture multiple facial images of each visitor in the electronic sand table exhibition area to obtain the time range of the capture of multiple facial images of each visitor and the time range of the recording of each visitor's speech.

[0091] Step 202: When a collection time range matches a recording time range, obtain the visitors corresponding to the collection time range and the vocal information of the speech content corresponding to the recording time range.

[0092] This invention, through matching the collection time range with the recording time range, associates the speech content with job title tags, thereby determining the preloading priority of the urban planning data involved in the speech content. In one embodiment, a voice recognition model can be used to extract the vocal information of each visitor from the speech content.

[0093] Step 203: Establish a voice mapping relationship between the visitors and the voice information to extract the voice information. Based on the extracted voice information and the voice mapping relationship, obtain the corresponding job title label of the visitors.

[0094] After obtaining the visitor's job title tags, it is also necessary to determine the validity of the pre-loading of the speech content based on the job title tags, such as... Figure 5 As shown, in step 103, when the voice information corresponds to multiple visitors, determining whether the visitor's speech content in the voice information has pre-load validity based on the hierarchical order of the job title tags includes:

[0095] Step 301: If the voice information is the first dialogue in this sand table display process, then when the visitor's job title tag belongs to management, the visitor's speech content has pre-loading validity.

[0096] Since the applicable scenario of this invention is that the visitors include both management leaders and ordinary visitors, and during the urban planning presentation, those with management-level titles have more say, this invention focuses on the leaders' speeches and determines the validity of preloading based on their speeches. For the first dialogue, the validity of preloading is determined only by recognizing the speeches of those with management-level titles, ignoring the speeches of other visitors before the leaders' speeches. The urban planning data that the leaders need to view is preloaded so that when the operator receives the leader's instruction to select the urban planning data, the large screen can display the urban planning data without any perceived delay.

[0097] In one optional embodiment, sound information can be continuously collected from the electronic sand table exhibition area. When a dialogue source can be captured from the sound information, the current time is recorded as the start time of the dialogue. When the dialogue source cannot be captured, and it remains uncaptured for a certain period of time, the start time of the inability to capture the dialogue source is determined as the end time of the dialogue. Based on the start time and end time of the dialogue, the first segment of dialogue in the current sand table display process is determined. The preset time is determined by those skilled in the art based on specific usage scenarios and experience. In another optional embodiment, the time interval between the end time of each sand table display process and the start time of the next sand table display process can be statistically analyzed to construct a time dataset. An intelligent model is then used to perform big data learning on the time dataset to obtain the preset time. Furthermore, the start time when the camera in the electronic sand table exhibition area can continuously capture the same batch of facial information can be determined as the start time of the dialogue; the start time when the inability to continuously capture the same batch of facial information is determined as the end time of the dialogue.

[0098] Step 302: If the voice information is not the first dialogue in this sand table display process, the speech content of the visitors whose job level tag belongs to the management level has pre-loading validity; and based on the contextual relationship between the speech content of each visitor in the voice information, determine the speech content of visitors whose job level tag does not belong to the management level that has pre-loading validity.

[0099] In this embodiment, the statements of visitors whose job titles do not belong to the management level may or may not have pre-loading validity, depending on the relationship between the statements of each visitor.

[0100] When the voice information is not the first dialogue in this sand table demonstration, this embodiment of the invention determines the validity of the pre-loading of the speech content of the management leaders; and based on the contextual relationship, it determines whether the speech content of visitors whose job level tags do not belong to management has pre-loading validity, specifically, such as... Figure 6 As shown, in step 202, the determination of pre-loaded valid speech content among visitors whose job level tags do not belong to management, based on the contextual relationship between the speech content of each visitor in the voice information, includes:

[0101] Step 401: Convert the speech content of each visitor in the voice information from voice format to text format to obtain text content.

[0102] Step 402: Based on the recording time sequence of the multiple text contents corresponding to the voice information, analyze the semantic relationship between the multiple text contents to obtain the contextual relationship between the speech content of each visitor in the voice information.

[0103] In one embodiment, a natural language model can be used to extract semantic relationships between multiple text contents to obtain contextual relationships.

[0104] Step 403: Determine whether the statements made by visitors whose job titles are not in management positions negate the statements made by visitors whose job titles are in management positions.

[0105] In one embodiment, a natural language model can be used to determine whether the meanings expressed by the statements of the two visitors are consistent, thereby improving the accuracy of the judgment.

[0106] Step 404: When negating the speech content of visitors whose job level tag belongs to management, the corresponding recording time order is obtained based on the context relationship. It is determined that the speech content recorded earlier in the time order does not have pre-loading validity, and the speech content recorded later in the time order has pre-loading validity, so as to obtain the speech content with pre-loading validity among visitors whose job level tag does not belong to management.

[0107] In this embodiment of the invention, the determination of the pre-loading validity of the speech content revolves around the speech content of the management leader. For example, when the voice information is not the first dialogue in this sand table display process, the speech content of the management leader is "enlarge the park facilities on screen 3", which is chronologically earlier. The speech content of the ordinary visitor is "there are no park facilities on screen 3, only on screen 1, enlarge the park facilities on screen 1", which is chronologically later. The latter speech content expresses a negation of the former speech content. In this case, the speech content "enlarge the park facilities on screen 3" is deemed not to have pre-loading validity, while the speech content "enlarge the park facilities on screen 1" is deemed to have pre-loading validity.

[0108] It should be noted that the statements made by visitors whose job titles are not in management positions may or may not have pre-loaded validity. For example, if a management leader says, "Enlarge the area highlighted in red on screen number 1," while a regular visitor says, "The park facilities in the highlighted area are well-planned and constructed," the latter statement does not negate the former's statement. Therefore, the statement "Enlarge the area highlighted in red on screen number 1" is considered to have pre-loaded validity, while the statement "The park facilities in the highlighted area are well-planned and constructed" is not.

[0109] To illustrate the preloading process when preloading validity exists, such as... Figure 7 As shown, in step 105, the urban planning data involved in the voice information is preloaded according to the preloading priority, and the preloaded data includes:

[0110] Step 501: Convert the speech content from voice format to text format to obtain text content; extract the speech keywords from the text content.

[0111] In one embodiment, a natural language model that performs named entity recognition tasks can be used to extract spoken keywords from text content.

[0112] Step 502: In the urban planning data corresponding to the multiple display screens, content matching is performed using speech keywords to obtain the data identifier of at least one urban planning data to be displayed.

[0113] Step 503: Obtain data identifiers and cache at least one city planning data to be displayed in memory to obtain preloaded data.

[0114] To ensure the normal operation of the server of the multi-screen interactive sand table display device, in one optional embodiment, a loading ratio threshold can be set to limit the amount of preloaded urban planning data. For example, when the loading ratio threshold is 60%, due to the limited storage space of the server, after preloading the content of the three large screens in the electronic sand table, the memory resource occupancy rate of the preloaded data reaches 60%, so preloading is temporarily stopped, and only the urban planning data in the three large screens is preloaded. Subsequently, after the operator selects and displays the urban planning data corresponding to each preloaded data in sequence, or after a preset time interval, voice recording is performed again, and new preloaded data is generated based on the re-recorded voice information. In one optional embodiment, since visitors generally do not need to frequently view areas with large location differences in a short period of time, the preset time interval can be 1 minute.

[0115] In this embodiment of the invention, since the time taken for visitors to speak is often on the order of seconds, and even in the applicable scenarios of this embodiment, visitors may observe the content displayed on the large screen and discuss the content of multiple large screens, this process is time-consuming. Therefore, in this process, the multi-screen linkage sand table display method of this embodiment of the invention is followed in advance to perform voice recording, voice information to text content, and preloading according to job level tags. When the operator selects the urban planning data to be displayed, the corresponding urban planning data is displayed immediately, so as to minimize the perceived delay for visitors through preloading.

[0116] Example 2:

[0117] like Figure 8 As shown, the electronic sand table in this embodiment of the invention consists of a bottom screen (such as...) Figure 8 The large screen with the number 1), a facade screen (such as...) Figure 8 The large screen marked with number 2), and two side screens (such as...) Figure 8 It consists of two large screens (numbered 3 and 4), where screens 1, 3, and 4 are two-dimensional screens, and screen 2 is a three-dimensional screen. The overall visual effect is as follows: Figure 9 As shown. In the application scenario of urban planning display, specifically, the bottom screen displays global information in the form of two-dimensional data through a map; the elevation screen displays the actual status of each area and related annotations through real-scene three-dimensional data, as shown in the annotations. Figure 9 As indicated by the label and text above the central screen, the corresponding data loading volume reaches 100GB; the two side screens provide supplementary displays using graphics and text, for example, as... Figure 9 The image is displayed at the top of the left-side side screen, and the corresponding text description is displayed at the bottom; for example... Figure 9The right-side side screen displays a two-dimensional map with corresponding text labels. The content displayed on the four large screens is controlled via a control terminal; in one optional embodiment, the control terminal can be a mobile phone or tablet. It should be noted that the multi-screen interactive sand table display method of this invention is applicable to electronic sand tables that simultaneously have two-dimensional and three-dimensional large screens, such as... Figure 8 and Figure 9 The electronic sand table shown is a specific example of an embodiment of the present invention, such as... Figure 8 The large screen of the electronic sand table shown is composed of various elements, and as such... Figure 9 The settings for the content displayed on each large screen shown are for illustrative purposes only and are not intended to limit the scope of the invention. For ease of description, the following will use... Figure 8 The labels for medium and large screens refer to the corresponding large screens.

[0118] The content displayed on the 3D and 2D screens under the same theme varies in loading time depending on the amount of data. Screens 3 and 4 typically load first, while screens 1 and 2 load at varying times depending on the amount of data involved in their respective themes. Screen 2, in particular, loads more than 100GB of 3D data, resulting in slower loading. In practice, this led to significant loading intervals between screens 1 and 2, resulting in a poor user experience.

[0119] It should be noted that the scenario addressed in this embodiment of the invention is as follows: the electronic sand table has been activated, meaning that urban planning data is being displayed on both the 3D and 2D screens. The center coordinates of the designated area displayed in real time on the 2D screen are the current 2D center coordinates, and the center coordinates of the designated area displayed in real time on the 3D screen are the current 3D center coordinates. The user selects the theme to be displayed on the corresponding screen and the designated area within that theme at the control terminal of the electronic sand table. At this time, the current 2D or 3D center coordinates of the selected screen are updated to the center coordinates of the designated area selected by the user, thereby switching the display focus of the corresponding screen to the designated area selected by the user. Specifically, when the electronic sand table is first activated, the current 2D or 3D center coordinates of the corresponding screen can be defaulted to the center coordinates of the complete area under a specific theme that the screen can display.

[0120] To solve the above problems, such as Figure 10As shown, in this embodiment, a two-dimensional page site controls a two-dimensional large screen, and a three-dimensional page site controls a three-dimensional large screen; the current two-dimensional center coordinates are obtained from the two-dimensional page site. The server in this embodiment stores all urban planning data, and data transmission is achieved through interaction between the server and the client. Here, a two-dimensional page site can be understood as the client for the corresponding two-dimensional large screen, a three-dimensional page site as the client for the corresponding three-dimensional large screen, and a control terminal page site as the client for the corresponding control terminal screen (e.g., a tablet screen); one two-dimensional page site can control at least one two-dimensional large screen, one three-dimensional page site can control at least one three-dimensional large screen, and one control terminal page site can interact with at least one control terminal screen. The server determines the current key display area of ​​the two-dimensional and three-dimensional page sites in real time by obtaining the current two-dimensional center coordinates from the two-dimensional page site and the current three-dimensional center coordinates from the three-dimensional page site.

[0121] The applicable scenarios of the embodiments of the present invention will be described below:

[0122] Users access the control terminal page through the control terminal application installed on the control terminal. They select a desired topic on the control terminal screen, and the control terminal page transmits the selected topic information to the server in the form of a display update command. Upon receiving the topic information, the server, after confirming that the user has permission to access the urban planning data resources within that topic, broadcasts the content update command to all sites. Each site, upon receiving the content update command, displays the required content according to the command, presenting the selected topic to the user. The user can be a visitor or a specific operator of the electronic sand table; this is not limited to any particular user. "All sites" refers to all 2D page sites, 3D page sites, and control terminal page sites controlled by the server of the electronic sand table.

[0123] When the 2D page site determines that the coordinates of the center to be displayed in 2D are different from the current 2D center coordinates, it generates control information based on the coordinates of the center to be displayed in 2D. The coordinates of the center to be displayed in 2D are the center coordinates of the designated area to be displayed. The control information is broadcast so that the 2D page site can display the designated area on the 2D large screen according to the coordinates of the center to be displayed in 2D, and the 3D page site can display the corresponding designated area on the 3D large screen according to the control information. Under the same theme, the 2D large screen displays urban planning data for the same designated area in 2D data format, and the 3D large screen displays urban planning data for the same designated area in 3D data format. For example... Figure 10As shown, after the user selects the desired area to be displayed via the control terminal, the 2D page site transmits the spatial information of the center point of the designated area (i.e., the coordinates of the 2D center to be displayed) to the server. The server then broadcasts the control information to all 2D and 3D page sites. Upon receiving the 2D center coordinates containing the control information, the 2D page site displays the designated area on the 2D large screen based on these coordinates. Similarly, upon receiving the 2D center coordinates containing the control information, the 3D page site displays the designated area on the 3D large screen based on these coordinates.

[0124] This invention controls a 2D large screen via a 2D page site and a 3D large screen via a 3D page site. When the 2D page site determines that the coordinates of the 2D center to be displayed are different from the current 2D center coordinates of the 2D large screen, the server generates control information based on the coordinates of the 2D center to be displayed and broadcasts it. This causes the 3D page site to display the corresponding designated area according to the control information. After obtaining the coordinates of the 2D center to be displayed on the 2D large screen, the server transmits the control information containing the coordinates of the 2D center to the 3D page site. Through the broadcast mechanism, all sites start loading data simultaneously, minimizing the time interval between the loading completion of the 2D and 3D large screens. This ensures that all sites display the corresponding images based on the coordinates of the 2D center to be displayed, achieving the effect that the 3D large screen follows the 2D large screen to adjust the display center. This enables control over the loading completion time and order of the displayed images, greatly improving the user experience. It solves the problem that due to the large difference in the amount of content data displayed by the 2D and 3D large screens, it is impossible to control the loading completion time and order of the 2D and 3D large screens, resulting in cluttered images and a poor experience for visitors.

[0125] The following section explains the specific process of displaying a designated area on a 2D or 3D large screen:

[0126] On the premise that all sites have been started and the theme selected by the user already exists, when the user does not switch the theme and selects the specified area to be displayed, the linkage between the 2D large screen and the 3D large screen needs to be carried out to display the detailed content of the specified area respectively. At this time, there are two situations: One situation is that when the user views the 2D large screen by dragging up, down, left, or right or zooming in or out through the control terminal, the 2D page site transmits the 2D center coordinates to be displayed to the server; the server generates control information according to the 2D center coordinates to be displayed, broadcasts the control information; the 2D page site receives the control information and displays the specified area on the 2D large screen according to the control information; the 3D page site receives the control information and displays the specified area on the 3D large screen according to the control information. At this time, the 2D center coordinates to be displayed are transmitted to the server, and the server encapsulates the 2D center coordinates to be displayed and the relevant parameters for loading into control information and broadcasts it. After the 2D page site receives it, it displays the 2D details of the specified area on the corresponding 2D large screen. After the 3D page site receives it, it displays the 3D details of the specified area on the corresponding 3D large screen, thereby ensuring that the 3D display content always follows the 2D large screen for switching and improving the user experience.

[0127] Another situation is that the user selects the specified area to be displayed on the 2D large screen through the control terminal page, so that the 2D page site obtains the 2D center coordinates to be displayed. The user clicks on the control terminal screen to select the specified area for display; in one embodiment, the position information of the specified area can be highlighted on the 2D large screen to facilitate the user to confirm whether to change the selection. Obtain the current 3D center coordinates from the 3D page site, and obtain the current 3D performance parameters from the 3D page site. In an optional embodiment, the current 3D performance parameters include the number of frames per second of the picture, and the number of frames per second (Frames Per Second, abbreviated as FPS) of the current browser picture; the 3D loading parameters include the viewport to be loaded and the animation loading duration; where the viewport to be loaded is used to adjust the actual amount of data loaded in the 3D scene; in an optional embodiment, the animation loading duration can be tested multiple times by setting different FPS values to obtain the corresponding empirical values, and the corresponding empirical value is selected as the animation loading duration for this loading according to the number of frames per second of the picture. The 2D page site determines whether the 2D center coordinates to be displayed are the same as the current 2D center coordinates; when the 2D center coordinates to be displayed are different from the current 2D center coordinates, the 2D page site transmits the 2D center coordinates to be displayed to the server. According to the current 3D center coordinates and the current three-dimensional performance parameters, three-dimensional loading parameters are obtained. The server encapsulates the 2D center coordinates to be displayed and the 3D loading parameters into control information and broadcasts the control information to all 2D page sites and 3D page sites.

[0128] like Figure 10 As shown, the two-dimensional page site, the three-dimensional page site, and the control page site log in to the server through a browser. In an optional embodiment, the WebSocket protocol is used to establish bidirectional connections between the control page site, the two-dimensional page site, and the three-dimensional page site and the server, so as to quickly transmit multiple messages on the same connection path.

[0129] Following the broadcast of the control information, the process further includes: after receiving the control information from their respective sites, the control terminal page controlled by the control terminal page site, the 2D large screen controlled by the 2D page site, and the 3D large screen controlled by the 3D page site synchronously displaying the designated area according to the control information. Specifically, one control terminal page site controls at least one control terminal page, one 2D page site controls at least one 2D large screen, and one 3D page site controls at least one 3D large screen. After receiving the broadcast control information from the 2D page site, the 2D large screen synchronously displays the designated area according to the control information; after receiving the broadcast control information from the 3D page site, the 3D large screen synchronously displays the designated area according to the control information; and after receiving the broadcast control information from the control terminal page site, the control terminal page synchronously displays the designated area according to the control information. In one embodiment, by broadcasting the control information to all sites, the coordinated control of the 2D large screen, 3D large screen, and control terminal screen can be achieved, allowing a single display to be adaptively displayed on each large screen and control terminal screen, thus improving the user experience.

[0130] The method of displaying images on the large screen is the same for both 2D and 3D page sites according to control information. The following explanation uses a 3D page site as an example. The process of displaying a designated area on the 3D large screen according to the control information includes: setting the current 3D center coordinates to the 2D center coordinates to be displayed, thereby completing the repositioning process of the current 3D center coordinates of the 3D large screen; obtaining the urban planning data to be displayed on the 3D large screen according to the repositioned current 3D center coordinates; and loading the urban planning data according to the 3D loading parameters, so that the 3D large screen displays the designated area. Since all sites in this embodiment of the invention perform repositioning of center coordinates according to the received corresponding spatial information (i.e., the 2D center coordinates to be displayed in the control information), it can ensure that the content displayed on the 3D large screen can switch with the 2D large screen at all times, improving the user experience.

[0131] In one optional embodiment, after the broadcast is detected on both the 2D and 3D page sites, the 2D and 3D scenes and the urban planning introduction animation can be loaded synchronously. The urban planning introduction animation is played first, and after the playback ends, the loaded data is displayed on the corresponding large screen. This further reduces the time interval between the loading of the content displayed on the 2D and 3D large screens, and avoids visitors seeing cluttered images during the data loading process, which would result in a poor viewing experience.

[0132] Furthermore, when a theme switch is required, the required urban planning data is determined based on the theme information; a content update instruction is generated; wherein, the content update instruction carries the required urban planning data and corresponding loading parameters. The corresponding loading parameters include: the viewport to be loaded and the animation loading duration for each 2D and 3D screen. The content update instruction is broadcast so that when both the 2D and 3D page sites receive the content update instruction, the urban planning data is synchronously loaded based on the received loading parameters and displayed on the corresponding screen.

[0133] In one embodiment, the urban planning sand table display method using the two-dimensional and three-dimensional multi-screen linkage of the present invention can use the two-dimensional and three-dimensional large screens in the electronic sand table to synchronously and completely present the picture of the designated area in front of the visitors, so as to control the loading time difference of the two-dimensional and three-dimensional complete picture and the picture switching time difference within 1 to 2 seconds.

[0134] Example 3:

[0135] like Figure 11 The diagram shown is a schematic representation of the architecture of a multi-screen interactive sand table display device according to an embodiment of the present invention. This embodiment of the multi-screen interactive sand table display device includes one or more processors 21 and a memory 22. Figure 11 Take a processor 21 as an example.

[0136] Processor 21 and memory 22 can be connected via a bus or other means. Figure 11 Taking the example of a connection between China and Israel via a bus.

[0137] The memory 22, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs and non-volatile computer-executable programs, such as the multi-screen interactive sand table display method in this embodiment. The processor 21 executes the multi-screen interactive sand table display method by running the non-volatile software programs and instructions stored in the memory 22.

[0138] Memory 22 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 22 may optionally include memory remotely located relative to processor 21, which can be connected to processor 21 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0139] The program instructions / modules are stored in the memory 22. When executed by one or more processors 21, they execute the multi-screen linkage sand table display method in the above embodiments, for example, executing each step of the multi-screen linkage sand table display method of the present invention described above.

[0140] This invention also provides a non-volatile computer storage medium storing computer-executable instructions that are executed by one or more processors, for example... Figure 11 A processor 21 enables one or more processors to execute the multi-screen interactive sand table display method of the present invention in specific embodiments, for example, to execute the various steps of the multi-screen interactive sand table display method of the present invention described above; it can also implement... Figure 11 The various modules and units described above; or the multi-screen interactive sand table display method implemented in the specific embodiments of the present invention, for example, executing the various steps of the multi-screen interactive sand table display method described above in the embodiments of the present invention; can also achieve Figure 11 The various modules and units mentioned above.

[0141] It is worth noting that the information interaction and execution process between the modules and units in the above-mentioned device and system are based on the same concept as the processing method embodiment of the present invention. For details, please refer to the description in the method embodiment of the present invention, and will not be repeated here.

[0142] Those skilled in the art will understand that all or part of the steps in the various methods of the embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc.

[0143] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for displaying a sand table using multiple screens in a coordinated manner, characterized in that, A multi-screen interactive sand table display device is set up in the electronic sand table exhibition area. Operators can display specified urban planning data by controlling the multiple displays of the multi-screen interactive sand table display device. A database is pre-established, which includes visitors' facial information and job title tags; Cameras were installed in the electronic sand table exhibition area; The multi-screen interactive sand table display method includes: The system acquires facial images of all visitors in the electronic sand table exhibition area to obtain the visitor's job title label; wherein, the facial images of the visitors are captured by the camera; the facial images are matched with facial information in the data database to obtain the corresponding visitor's job title label; The required urban planning data is preloaded based on the job level tags to obtain preloaded data. Specifically, when a dialogue source can be captured in the electronic sand table exhibition area, voice recording is performed to obtain voice information. The visitor corresponding to the voice information is identified, and their job level tag is obtained. When the voice information corresponds to multiple visitors, the validity of the visitor's speech in the voice information is determined based on the job level order of the job level tags. If the visitor's speech is valid for preloading, the preloading priority of the visitor in the voice information is obtained based on the job level order of the job level tags. The urban planning data involved in the voice information is preloaded according to the preloading priority to obtain preloaded data. When the operator selects to display the urban planning data, the preloaded data is displayed on the corresponding screen.

2. The multi-screen interactive sand table display method according to claim 1, characterized in that, When the voice information corresponds to multiple visitors, determining whether the pre-loading validity of the visitors' speech content in the voice information exists based on the hierarchical order of the job level tags includes: If the voice information is the first dialogue in this sand table display process, then when the visitor's job title tag belongs to management, the visitor's speech content has pre-load validity. If the voice information is not the first dialogue in this sand table demonstration, the speech content of visitors whose job level tag belongs to management has pre-loading validity; and based on the contextual relationship between the speech content of each visitor in the voice information, the speech content of visitors whose job level tag does not belong to management has pre-loading validity.

3. The multi-screen interactive sand table display method according to claim 2, characterized in that, Based on the contextual relationships between the statements of each visitor in the voice information, the statements of visitors whose job level tags do not belong to management and whose statements have pre-loaded validity are determined to include: The speech content of each visitor in the voice information is converted from voice format to text format to obtain text content; Based on the recording time sequence of multiple text contents corresponding to the voice information, the semantic relationship between the multiple text contents is analyzed to obtain the contextual relationship between the speech contents of each visitor in the voice information; Determine whether the statements made by visitors whose job titles are not in management positions constitute a negation of the statements made by visitors whose job titles are in management positions. When negating the speech of a visitor whose job title tag belongs to management, the corresponding recording time order is obtained based on the contextual relationship. It is determined that the speech content recorded earlier in the time order does not have pre-loading validity, and the speech content recorded later in the time order has pre-loading validity, so as to obtain the speech content with pre-loading validity among visitors whose job title tag does not belong to management.

4. The multi-screen interactive sand table display method according to claim 1, characterized in that, Before determining whether the pre-loading validity exists for the speech content of the visitors in the voice information based on the hierarchical order of the job level tags when the voice information corresponds to multiple visitors, the method further includes: When voice recording begins, multiple facial images of each visitor in the electronic sand table exhibition area are continuously captured to obtain the time range of the capture of multiple facial images of each visitor and the time range of the recording of each visitor's speech. When a collection time range matches a recording time range, the visitors corresponding to the collection time range are obtained, and the vocal information of the speech content corresponding to the recording time range is obtained. A voice mapping relationship is established between the visitors and the voice information to extract the voice information. Based on the extracted voice information and the voice mapping relationship, the corresponding job title label of the visitors is obtained.

5. The multi-screen interactive sand table display method according to claim 1, characterized in that, When the visitor's speech content has pre-loading validity, the pre-loading priority of the visitor in the voice information is obtained according to the hierarchical order of the job level tags, including: The job title tags are determined in descending order to determine the preloading priority of the corresponding visitors in the voice information, so as to obtain the preloading priority.

6. The multi-screen interactive sand table display method according to claim 1, characterized in that, The process of preloading urban planning data related to the voice information according to the preloading priority, resulting in preloaded data including: The speech content is converted from audio to text format to obtain text content; keywords are extracted from the text content. In the urban planning data corresponding to the multiple display screens, the content is matched using the speaking keywords to obtain the data identifier of at least one urban planning data to be displayed; Obtain data identifiers and cache at least one city planning data to be displayed in memory to obtain preloaded data.

7. A multi-screen interactive sand table display device, characterized in that, The multi-screen interactive sand table display device includes at least one processor and a memory, which are connected via a data bus. The memory stores instructions that can be executed by the at least one processor. After being executed by the processor, the instructions are used to implement the multi-screen interactive sand table display method according to any one of claims 1-6.

8. A non-volatile computer storage medium, characterized in that, The computer storage medium stores computer-executable instructions, which are executed by one or more processors to perform the multi-screen interactive sand table display method as described in any one of claims 1-6.

Images