A method and system for controlling smart city equipment based on digital twins

Abstract

The present invention belongs to the field of smart city technology. A method and system for controlling smart city devices based on digital twins are provided. The method includes: receiving first monitoring data of each smart city device in a region, generating a first virtual monitoring interface using digital twin technology; obtaining interface attribute information of the first virtual monitoring interface, and predicting a user's first operation behavior on the first virtual monitoring interface based on the interface attribute information; controlling each smart city device in the region to detect and upload second monitoring data according to the first operation behavior, and generating a second virtual monitoring interface according to the second monitoring data; outputting a second virtual monitoring interface when receiving the user's first operation behavior; and the user controls each smart city device in the region in the first virtual monitoring interface or the second virtual monitoring interface. The present invention can reduce the probability of buffer jamming when the user performs an operation on the virtual operation interface.

Description

Technical Field

[0001] The present invention relates to the field of smart city technology, and in particular to a method and system for controlling smart city equipment based on digital twins. Background Art

[0002] Smart city devices are a series of high-tech products and solutions that use advanced information technology, including the Internet of Things (IoT), big data, artificial intelligence (AI) and communication technology, to improve the efficiency of urban management and the quality of life of residents. In order to improve the display effect of the monitoring data of smart city devices, some existing technologies combine digital twin technology with smart city devices, that is, the monitoring data of smart city devices is visualized and output using digital twin technology. However, the digital twin technology has a large amount of data processing when simulating, and the number of smart city devices and the amount of data generated are large. The above reasons make it difficult to ensure the real-time performance of the visual simulation. This results in the state switching of the virtual model on the digital twin interface not being timely when the user zooms in or out of the simulated digital twin interface, affecting the user experience. Summary of the invention

[0003] In this regard, the present invention provides a method, system, electronic device, computer storage medium and computer program product for controlling smart city equipment based on digital twins to solve the above-mentioned technical problems.

[0004] The present invention discloses a method for controlling smart city equipment based on digital twins, the method comprising the following steps:

[0005] receiving first monitoring data of each smart city device in the area, and generating a first virtual monitoring interface using digital twin technology;

[0006] Acquire interface attribute information of the first virtual monitoring interface, and predict a first operation behavior of a user on the first virtual monitoring interface based on the interface attribute information, where the first operation behavior includes a zoom-in operation and a zoom-out operation;

[0007] According to the first operation behavior, each smart city device in the control area detects and uploads second monitoring data, and generates a second virtual monitoring interface according to the second monitoring data;

[0008] When receiving the first operation behavior of the user, outputting the second virtual monitoring interface;

[0009] The user manages and controls each smart city device in the area in the first virtual monitoring interface or the second virtual monitoring interface.

[0010] In some embodiments, the acquiring interface attribute information of the first virtual monitoring interface and predicting a first operation behavior of the user on the first virtual monitoring interface based on the interface attribute information includes:

[0011] Extracting image features from the first virtual monitoring interface, and performing matching analysis between the image features and preset events;

[0012] If the extracted image feature matches a preset event, predicting a first operation behavior of the user on the first virtual monitoring interface based on the image feature;

[0013] If the extracted image feature does not match the preset event, the interface attribute information of the first virtual monitoring interface is obtained, and a first operation behavior of the user on the first virtual monitoring interface is predicted based on the interface attribute information.

[0014] In some embodiments, the matching analysis of the image features with the preset events includes:

[0015] Acquire the identity information of the user, and determine a number of preset events that the user is interested in based on the identity information;

[0016] The image features are matched and analyzed with each of the preset events.

[0017] In some embodiments, controlling each smart city device in the area to detect and upload second monitoring data according to the first operation behavior includes:

[0018] Analyze and obtain an estimated operation time from the first operation behavior, and determine the detection range of each smart city device according to the estimated operation time;

[0019] Each smart city device in the control area detects the situation in the area according to the detection range, obtains the second monitoring data and uploads it.

[0020] In some embodiments, the estimated operation time is determined by:

[0021] Acquire the identity information of the user, and retrieve the operation history data of the user on the virtual monitoring interface and several preset events that the user is interested in according to the identity information;

[0022] An average operation interval duration corresponding to each preset event that the user is interested in is obtained based on the operation history data statistics, and the estimated operation time is determined based on the average operation interval duration.

[0023] In some embodiments, after outputting the second virtual monitoring interface, the method further includes:

[0024] Receiving third monitoring data detected and uploaded by each smart city device in the area, and generating a third virtual monitoring interface according to the third monitoring data;

[0025] When a second operation behavior of the user on the second virtual monitoring interface is received, the third virtual monitoring interface is output; wherein the second operation behavior includes a zoom-in operation and a zoom-out operation.

[0026] The present invention also discloses a system for managing and controlling smart city equipment based on digital twins, the system comprising a processing module and a storage module; a computer program is stored in the storage module; the processing module calls and executes the computer program stored in the storage module to implement any of the aforementioned methods.

[0027] The present invention also discloses an electronic device, comprising: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor executes the computer program to implement any of the above methods.

[0028] The present invention also discloses a computer storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement any of the above methods.

[0029] The present invention also discloses a computer program product, which implements any of the above methods when the computer program product is run on a terminal.

[0030] The beneficial effect of the present invention is that the present invention predicts the user's operating behavior and generates a virtual monitoring interface corresponding to the operating behavior in advance according to the prediction result, thereby reducing the probability of buffer jamming when the user performs operating behavior on the virtual operating interface. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0032] Figure 1 It is a flowchart of a method for controlling smart city equipment based on digital twins disclosed in an embodiment of the present invention;

[0033] Figure 2 is a schematic diagram of a second virtual monitoring interface disclosed in an embodiment of the present invention;

[0034] Figure 3 It is a structural schematic diagram of a system for controlling smart city equipment based on digital twins disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The following is a description of the implementation of the present application by specific specific embodiments. People familiar with the technology can easily understand other advantages and effects of the present application from the contents disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0036] In addition, the technical features involved in the different embodiments of the present application described below can be combined with each other as long as they do not conflict with each other.

[0037] like Figure 1 As shown, an embodiment of the present invention discloses a method for controlling smart city equipment based on digital twins, and the method comprises the following steps:

[0038] receiving first monitoring data of each smart city device in the area, and generating a first virtual monitoring interface using digital twin technology;

[0039] Acquire interface attribute information of the first virtual monitoring interface, and predict a first operation behavior of a user on the first virtual monitoring interface based on the interface attribute information, where the first operation behavior includes a zoom-in operation and a zoom-out operation;

[0040] According to the first operation behavior, each smart city device in the control area detects and uploads second monitoring data, and generates a second virtual monitoring interface according to the second monitoring data;

[0041] When the first operation behavior of the user is received, the second virtual monitoring interface is output.

[0042] The user manages and controls each smart city device in the area in the first virtual monitoring interface or the second virtual monitoring interface.

[0043] The present invention deploys multiple smart city devices in a designated area, including intelligent street lights, traffic monitoring equipment, intelligent environmental monitoring equipment, water monitoring equipment, etc. These smart city devices upload the data obtained from their respective monitoring in real time. The background system uses digital twin technology to generate a first virtual monitoring interface. The first virtual monitoring interface contains multiple realistic dynamic and static models. The dynamic models in these models can change their own motion states according to the data detected by the smart city devices. Under special circumstances, the static models may also change their states, such as when buildings are demolished by blasting.

[0044] At the same time, since digital twin technology consumes a lot of computing power, it is difficult to ensure the timeliness and accuracy of the response of the virtual monitoring interface after the user implements the zoom-in and zoom-out operations, and buffer jamming is prone to occur. In this regard, the present invention predicts the first operation behavior that the user may implement in the monitoring interface, namely the zoom-in and zoom-out operations, based on the interface attribute information of the first virtual monitoring interface, controls the smart city equipment to detect and upload the second monitoring data in advance, and generates the second virtual monitoring interface accordingly. Figure 2 As shown (corresponding to the zoom-in operation). Therefore, when the user performs the above-mentioned first operation, the background system can output the second virtual monitoring interface to the user, thereby saving the calculation time of the second virtual monitoring interface and effectively reducing the probability of buffer jamming.

[0045] Therefore, the present invention predicts the user's operating behavior and generates a virtual monitoring interface corresponding to the operating behavior in advance according to the prediction result, thereby reducing the probability of buffer jamming when the user performs operating behaviors on the virtual operating interface.

[0046] Users can remotely control various smart city devices in the first virtual monitoring interface and the second virtual monitoring interface, including calling up corresponding monitoring screens, setting working parameters, etc.

[0047] In some embodiments, the acquiring interface attribute information of the first virtual monitoring interface and predicting a first operation behavior of the user on the first virtual monitoring interface based on the interface attribute information includes:

[0048] Extracting image features from the first virtual monitoring interface, and performing matching analysis between the image features and preset events;

[0049] If the extracted image feature matches a preset event, predicting a first operation behavior of the user on the first virtual monitoring interface based on the image feature;

[0050] If the extracted image feature does not match the preset event, the interface attribute information of the first virtual monitoring interface is obtained, and a first operation behavior of the user on the first virtual monitoring interface is predicted based on the interface attribute information.

[0051] In this embodiment, the first virtual monitoring interface will include some real-time situation information, such as traffic congestion, traffic incidents, road control and other information. These information are all potential concerns of users, and these information can be directly displayed in the form of graphics, colors, prompt labels, etc.

[0052] Therefore, when these image features contained in the first virtual monitoring interface correspond to preset events, it can be directly predicted that the user will most likely perform the first operation on the first virtual monitoring interface, and there is no need to obtain the interface attribute information of the first virtual monitoring interface that cannot be directly displayed. For example, if the user finds that there is a red mark on a road in the first virtual monitoring interface from a macro perspective, that is, there are road users, it is predicted that there is a need for a zoom-in operation on the first virtual monitoring interface, that is, to view the detailed situation of the area through a zoom-in operation; for another example, there is a prompt arrow at the edge of the first virtual monitoring interface, and the meaning of the prompt arrow is that there is a preset event (such as a traffic accident) in the adjacent area (not in the first virtual monitoring interface), then it is predicted that there is a need for a zoom-out operation on the first virtual monitoring interface, that is, to switch to a more macro perspective through a zoom-out operation to view the detailed situation of the adjacent area where the preset event exists.

[0053] When the image features included in the first virtual monitoring interface do not correspond to the preset events, it is necessary to obtain the interface attribute information of the first virtual monitoring interface that cannot be directly displayed. The interface attribute information may be, for example, the historical accident rate of roads in the area, the historical congestion rate of roads, etc. The probability of attention to the area can be analyzed based on the interface attribute information. For example, when the historical accident rate of roads in the area is high, it is predicted that the user has a need to zoom in on the first virtual monitoring interface, that is, to view the detailed situation of the area through zooming in.

[0054] In some embodiments, the matching analysis of the image features with the preset events includes:

[0055] Acquire the identity information of the user, and determine a number of preset events that the user is interested in based on the identity information;

[0056] The image features are matched and analyzed with each of the preset events.

[0057] In this embodiment, the virtual monitoring interface can have different application scenarios. For example, the staff of the conventional city management department can grasp the real-time situation in the area and remotely control the smart city devices in the control area through the virtual monitoring interface. Ordinary citizens can grasp the real-time situation in the area through the virtual monitoring interface to plan their own itinerary. Therefore, different types of users have different interests in the virtual monitoring interface.

[0058] In view of the above actual situation, the present invention first obtains the identity information of the user, and then retrieves and determines several preset events that the user is interested in. For example, the preset events that the staff of the urban management department are interested in include traffic congestion, traffic accidents, road maintenance, etc., and the preset events that ordinary citizens are interested in include traffic conditions, live videos in specific venues (such as venues where concerts or exhibitions are being held), etc. After determining the corresponding preset events, it is possible to analyze whether the image features contain the preset events that the user is interested in, so as to decide what method to use to predict the user's first operation behavior on the first virtual monitoring interface.

[0059] In some embodiments, controlling each smart city device in the area to detect and upload second monitoring data according to the first operation behavior includes:

[0060] Analyze and obtain an estimated operation time from the first operation behavior, and determine the detection range of each smart city device according to the estimated operation time;

[0061] Each smart city device in the control area detects the situation in the area according to the detection range, obtains the second monitoring data and uploads it.

[0062] In this embodiment, the first operation behavior determined above also includes the estimated time when the user performs the operation corresponding to the first operation behavior, that is, it is predicted that the user will perform a zoom-in operation or a zoom-out operation on the first virtual monitoring interface at the estimated time of the operation. Therefore, it is necessary to control each smart city device in the area to obtain the second monitoring data corresponding to the estimated time of the operation, and then the background system generates the second virtual monitoring interface corresponding to the estimated time of the operation.

[0063] Specifically, the detection range of each smart city device is determined according to the estimated operation time. The detection range refers to the physical detection range, that is, the smart city device is controlled to expand its own detection range. For example, the smart city device is a road monitoring device, and the road monitoring device adjusts the monitoring angle toward the upstream direction of the road, thereby obtaining the vehicle driving situation in a larger road range, the vehicle speed, driving direction, lane, etc. Then, the situation information corresponding to the estimated operation time is predicted based on the situation information of the larger detection range (i.e., the second monitoring data), and the situation information is very close to the actual situation when the estimated operation time arrives, and can be used to replace the actual situation when the estimated operation time arrives.

[0064] Among them, the farther the estimated operation time is from the current time, the larger the detection range of the corresponding smart city equipment is adjusted, and more data is used to predict the situation information of the estimated operation time, and the accuracy is higher; conversely, the smaller it is, the less data is used to predict the situation information of the estimated operation time, and the prediction is faster.

[0065] In some embodiments, the estimated operation time is determined by:

[0066] Acquire the identity information of the user, and retrieve the operation history data of the user on the virtual monitoring interface and several preset events that the user is interested in according to the identity information;

[0067] An average operation interval duration corresponding to each preset event that the user is interested in is obtained based on the operation history data statistics, and the estimated operation time is determined based on the average operation interval duration.

[0068] In this embodiment, the average operation interval duration of the user for each preset event appearing in the virtual monitoring interface can be statistically obtained through the user's operation history data. The operation interval duration refers to the interval duration from the appearance of a specific preset event in the virtual monitoring interface or the user performing a zoom-in or zoom-out operation (a preset event already exists in the virtual monitoring interface after the zoom-in or zoom-out operation) to the user performing the zoom-in or zoom-out operation. The estimated operation time is obtained by adding the interval duration to the current time.

[0069] In some embodiments, after outputting the second virtual monitoring interface, the method further includes:

[0070] Receiving third monitoring data detected and uploaded by each smart city device in the area, and generating a third virtual monitoring interface according to the third monitoring data;

[0071] When a second operation behavior of the user on the second virtual monitoring interface is received, the third virtual monitoring interface is output; wherein the second operation behavior includes a zoom-in operation and a zoom-out operation.

[0072] In this embodiment, since the second virtual monitoring interface outputted previously is actually based on predicted data rather than real-time data, after a certain period of time, the second virtual monitoring interface cannot accurately represent the real-time situation in the area and needs to be updated. However, the user is viewing the content in the second virtual monitoring interface, so it is not appropriate to implement automatic switching. In this regard, the present invention allows the user to switch the second virtual monitoring interface to a third virtual monitoring interface based on real-time data by continuously performing zoom-in and zoom-out operations. For example, after the user has viewed the content in the second virtual monitoring interface, the user manually continuously performs zoom-in and zoom-out operations on the second virtual monitoring interface, or continuously performs zoom-in and zoom-out operations. At this time, the background system obtains the user's need to switch to the real-time third virtual monitoring interface based on real-time data, and automatically switches the second virtual monitoring interface to the third virtual monitoring interface.

[0073] In order to improve the operability of the second virtual monitoring interface for users, while outputting the third virtual monitoring interface, the original second virtual monitoring interface can also be copied and saved in a specified path. Users can switch to the specified path in the third virtual monitoring interface to view the second virtual monitoring interface, perform management and control operations on smart city devices, etc. This usage method is particularly suitable for user A to copy the second virtual monitoring interface to his own computer terminal, and user B to view, manage and control the newly output third virtual monitoring interface, etc., which can improve the management and control efficiency of smart city devices.

[0074] In addition, when the second virtual monitoring interface outputs the preset duration and does not receive the second operation behavior of the user, the third virtual monitoring interface can be controlled to output automatically. The preset duration can be determined according to the severity of the preset event corresponding to the second virtual monitoring interface. The higher the severity of the preset event, the longer the preset duration, and vice versa. The corresponding severity is set in advance for different preset events, for example, the severity of traffic slowdown is lower than that of traffic congestion.

[0075] like Figure 3 As shown, an embodiment of the present invention also discloses a system for controlling smart city equipment based on digital twins, the system comprising a processing module and a storage module; a computer program is stored in the storage module; the processing module calls and executes the computer program stored in the storage module to implement the method described in the aforementioned embodiment.

[0076] An embodiment of the present invention further discloses an electronic device, comprising: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor executes the computer program to implement the method described in the above embodiment.

[0077] An embodiment of the present invention further discloses a computer storage medium, wherein the computer storage medium stores a computer program, and the computer program is executed by a processor to implement the method described in the above embodiment.

[0078] The embodiment of the present invention further discloses a computer program product. When the computer program product is run on a terminal, the method described in the above embodiment is implemented.

[0079] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0080] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0081] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or can be electrical, mechanical or other forms of connection.

[0082] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present invention.

[0083] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0084] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for controlling smart city equipment based on digital twins, characterized in that: The method comprises the following steps: receiving first monitoring data of each smart city device in the area, and generating a first virtual monitoring interface using digital twin technology; Acquire interface attribute information of the first virtual monitoring interface, and predict a first operation behavior of a user on the first virtual monitoring interface based on the interface attribute information, where the first operation behavior includes a zoom-in operation and a zoom-out operation; According to the first operation behavior, each smart city device in the control area detects and uploads second monitoring data, and generates a second virtual monitoring interface according to the second monitoring data; When receiving the first operation behavior of the user, outputting the second virtual monitoring interface; The user controls and manages each smart city device in the area in the first virtual monitoring interface or the second virtual monitoring interface; The controlling each smart city device in the area to detect and upload second monitoring data according to the first operation behavior includes: Analyze and obtain an estimated operation time from the first operation behavior, and determine the detection range of each smart city device according to the estimated operation time; Controlling each smart city device in the area to detect the situation in the area according to the detection range, obtain the second monitoring data and upload it; Among them, the farther the expected operation time is from the current time, the larger the detection range of the smart city device is set, that is, more data is used to predict the situation information of the expected operation time; the closer the expected operation time is from the current time, the smaller the detection range of the smart city device is set, that is, less data is used to predict the situation information of the above-mentioned expected operation time.

2. According to claim 1, a method for controlling smart city equipment based on digital twins is characterized in that: The acquiring interface attribute information of the first virtual monitoring interface and predicting a first operation behavior of the user on the first virtual monitoring interface based on the interface attribute information includes: Extracting image features from the first virtual monitoring interface, and performing matching analysis between the image features and preset events; If the extracted image feature matches a preset event, predicting a first operation behavior of the user on the first virtual monitoring interface based on the image feature; If the extracted image feature does not match the preset event, the interface attribute information of the first virtual monitoring interface is obtained, and a first operation behavior of the user on the first virtual monitoring interface is predicted based on the interface attribute information.

3. According to claim 2, a method for controlling smart city equipment based on digital twins is characterized in that: The matching analysis of the image features with the preset events includes: Acquire the identity information of the user, and determine a number of preset events that the user is interested in based on the identity information; The image features are matched and analyzed with each of the preset events.

4. According to claim 1, a method for controlling smart city equipment based on digital twins is characterized in that: The estimated operation time is determined by: Acquire the identity information of the user, and retrieve the operation history data of the user on the virtual monitoring interface and several preset events that the user is interested in according to the identity information; An average operation interval duration corresponding to each preset event that the user is interested in is obtained based on the operation history data statistics, and the estimated operation time is determined based on the average operation interval duration.

5. A method for controlling smart city equipment based on digital twins according to any one of claims 1 to 4, characterized in that: After outputting the second virtual monitoring interface, the method further includes: Receiving third monitoring data detected and uploaded by each smart city device in the area, and generating a third virtual monitoring interface according to the third monitoring data; When a second operation behavior of the user on the second virtual monitoring interface is received, the third virtual monitoring interface is output; wherein the second operation behavior includes a zoom-in operation and a zoom-out operation.

6. A system for controlling smart city equipment based on digital twins, the system comprising a processing module and a storage module; the storage module stores a computer program; characterized in that: The processing module calls and executes the computer program stored in the storage module to implement the method according to any one of claims 1 to 5.

7. An electronic device comprising: At least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor executes the computer program to implement the method according to any one of claims 1 to 5.

8. A computer storage medium storing a computer program, characterized in that: The computer program is executed by a processor to implement the method according to any one of claims 1 to 5.

9. A computer program product, characterized in that: When the computer program product runs on a terminal, the method according to any one of claims 1 to 5 is implemented.

Images