Internet of Things data fusion method, device, equipment and medium

Abstract

The invention discloses an Internet of Things data fusion method, apparatus and device, and a medium. The Internet of Things data fusion method comprises the following steps: determining a target identification equipment image and target identification equipment position data according to video inspection data and a target equipment identification model; according to the target identification equipment image and the target identification equipment position data, creating a static Internet of Things twinborn model; and according to the real-time sensing data of the Internet of Things, adjusting the static twinborn model of the Internet of Things to obtain a dynamic twinborn model of the Internet of Things. According to the technical scheme provided by the embodiment of the invention, the efficiency of accurate twin modeling can be improved, the labor cost of position calibration of the twin model is reduced, and the display effect of the twin model is improved.

Description

Technical Field

[0001] The present invention relates to the field of digital twin technology, and in particular to an Internet of Things data fusion method, device, equipment and medium. Background Art

[0002] With the development of artificial intelligence, big data, the Internet of Things, cloud computing and other technologies, digital twin technology, as a powerful tool for the future development of industrial intelligence, has a milestone significance in the field of intelligent manufacturing. Applying digital twin technology to industrial production will greatly reduce the workload of workers.

[0003] However, the twins of IoT factories currently rely mainly on manual modeling, resulting in low twin modeling efficiency. After the twins are modeled, they need to be manually calibrated using CAD (Computer Aided Design) to mark the points. The large number of calibration positions makes the position calibration workload of the twin model large. Summary of the invention

[0004] The present invention provides an Internet of Things data fusion method, device, equipment and medium to solve the problems of low efficiency of current manual twin modeling and large workload of position calibration after modeling.

[0005] According to one aspect of the present invention, a method for fusion of Internet of Things data is provided, comprising:

[0006] Determine the target recognition device image and the target recognition device location data according to the video inspection data and the target device recognition model;

[0007] Create a static IoT twin model based on the target recognition device image and target recognition device location data;

[0008] According to the real-time perception data of the Internet of Things, the static Internet of Things twin model is adjusted to obtain the dynamic Internet of Things twin model.

[0009] According to another aspect of the present invention, there is provided an Internet of Things data fusion device, comprising:

[0010] A data preparation module, used to determine the target recognition device image and the target recognition device location data according to the video inspection data and the target device recognition model;

[0011] A static twin model creation module is used to create a static IoT twin model based on the target recognition device image and the target recognition device location data;

[0012] The dynamic twin model creation module is used to adjust the static IoT twin model according to the real-time perception data of the IoT to obtain a dynamic IoT twin model.

[0013] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0014] at least one processor; and

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the Internet of Things data fusion method described in any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the Internet of Things data fusion method described in any embodiment of the present invention when executed.

[0018] The technical solution of the embodiment of the present invention determines the target recognition device image and the target recognition device location data according to the video inspection data and the target device recognition model, thereby creating a static IoT twin model according to the target recognition device image and the target recognition device location data, and then adjusting the static IoT twin model according to the real-time perception data of the IoT to obtain a dynamic IoT twin model. In this solution, the twin modeling of the concerned equipment in the IoT factory can be automatically performed, and the spatial position of the twin model actually mapped by the target recognition device location data can also be used to mark the spatial position of the equipment. The modeling of the twin individuals in the entire static IoT twin model and the position marking can be automatically realized without manual work. Under the premise of ensuring the accuracy of the twin modeling and position marking, not only the labor cost is greatly reduced, but also the modeling efficiency is improved. In addition, the real-time perception data of the IoT is used to dynamically adjust the static IoT twin model, and the changes in the device status are reflected in real time and vividly, which solves the problems of low efficiency of the current manual twin modeling and large workload of position calibration after modeling, and can improve the efficiency of accurate twin modeling, reduce the labor cost of twin model position calibration, and improve the display effect of the twin model.

[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 A flowchart of an Internet of Things data fusion method provided in Embodiment 1 of the present invention;

[0022] Figure 2 A flowchart of an Internet of Things data fusion method provided in Embodiment 2 of the present invention;

[0023] Figure 3 A schematic diagram of the structure of an Internet of Things data fusion device provided in Embodiment 3 of the present invention;

[0024] Figure 4 A schematic diagram of the structure of an electronic device that can be used to implement an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0026] It should be noted that the term "target" and the like in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] Embodiment 1

[0028] Figure 1This is a flowchart of an Internet of Things data fusion method provided in the first embodiment of the present invention. This embodiment is applicable to the case of automatically performing twin modeling. The method can be executed by an Internet of Things data fusion device. The Internet of Things data fusion device can be implemented in the form of hardware and / or software. The Internet of Things data fusion device can be configured in an electronic device. The electronic device can be a computer, a server, or a patrol robot, etc. Figure 1 As shown, the method includes:

[0029] Step 110: Determine the target recognition device image and the target recognition device location data according to the video inspection data and the target device recognition model.

[0030] Among them, the video inspection data may be patrol video data of the IoT factory. Exemplarily, the video inspection data may be collected by the camera of the inspection robot. The embodiment of the present invention does not limit the specific processing and manufacturing scenarios of the IoT factory. The target device recognition model may be a model for identifying the devices of interest in the IoT factory. Exemplarily, the devices of interest in the IoT factory may include but are not limited to fans, air conditioners, and fire extinguishers, etc., which can be set and adjusted according to the actual production environment. The target recognition device image may be an image frame in which the device of interest exists in the video inspection data. The target recognition device location data may be data describing the spatial location of the device corresponding to the target recognition device image in the IoT factory.

[0031] In an embodiment of the present invention, based on the target device recognition model, images of devices that need attention in video inspection data can be identified, and the identified images can be preprocessed to obtain target recognition device images. The spatial positions of devices that need attention can be determined based on the video inspection data to obtain target recognition device position data, thereby realizing the identification and positioning of only devices that require twin modeling, so as to improve the accuracy of subsequent twin modeling and effectively reduce the complexity of data processing.

[0032] Step 120: Create a static IoT twin model based on the target recognition device image and the target recognition device location data.

[0033] Among them, the static IoT twin model can be a model of a twin factory created based on the target recognition device image and the target recognition device location data.

[0034] In an embodiment of the present invention, three-dimensional modeling and related simulation software can be used to create a twin of the device corresponding to the target recognition device image, and the position of the corresponding twin is marked according to the target recognition device position data to obtain a static Internet of Things twin model.

[0035] Step 130: Adjust the static IoT twin model according to the real-time IoT perception data to obtain a dynamic IoT twin model.

[0036] The IoT real-time perception data may be data describing the state changes of devices in the IoT factory. The IoT real-time perception data may include but is not limited to the real-time temperature of the device, the current shape of the workpiece, and the current size of the workpiece. The dynamic IoT twin model may be a model obtained by adjusting the static IoT twin model according to the changes in the device state.

[0037] In an embodiment of the present invention, real-time perception data of the Internet of Things can be analyzed to identify state changes of devices of interest in an Internet of Things factory, thereby making follow-up adjustments to the twins with state changes in a static Internet of Things twin model. For example, the state changes of devices can be expressed through shape, size, color, and special effects to obtain a dynamic Internet of Things twin model.

[0038] The technical solution of the embodiment of the present invention determines the target recognition device image and the target recognition device location data according to the video inspection data and the target device recognition model, thereby creating a static IoT twin model according to the target recognition device image and the target recognition device location data, and then adjusting the static IoT twin model according to the real-time perception data of the IoT to obtain a dynamic IoT twin model. In this solution, the twin modeling of the concerned equipment in the IoT factory can be automatically performed, and the spatial position of the twin model actually mapped by the target recognition device location data can also be used to mark the spatial position of the equipment. The modeling of the twin individuals in the entire static IoT twin model and the position marking can be automatically realized without manual work. Under the premise of ensuring the accuracy of the twin modeling and position marking, not only the labor cost is greatly reduced, but also the modeling efficiency is improved. In addition, the real-time perception data of the IoT is used to dynamically adjust the static IoT twin model, and the changes in the device status are reflected in real time and vividly, which solves the problems of low efficiency of the current manual twin modeling and large workload of position calibration after modeling, and can improve the efficiency of accurate twin modeling, reduce the labor cost of twin model position calibration, and improve the display effect of the twin model.

[0039] Embodiment 2

[0040] Figure 2 This is a flowchart of an IoT data fusion method provided in Embodiment 2 of the present invention. This embodiment is specific based on the above embodiment and provides a specific optional implementation method for determining the target recognition device image and the target recognition device location data based on the video inspection data and the target device recognition model. Figure 2 As shown, the method includes:

[0041] Step 210: Input the video inspection data into the target device recognition model to obtain the image to be processed, and determine the scanning position calibration data of the image to be processed based on the video inspection data.

[0042] The image to be processed may be an image frame in which the device of interest exists in the video inspection data. The scan position calibration data may be used to indicate the spatial position of the device in the image to be processed in the IoT factory.

[0043] In an embodiment of the present invention, video inspection data can be input into a target device recognition model, and the target device recognition model can be used to identify the device of interest for the image frames in the video inspection data to obtain an image to be processed including the device of interest. Based on the video inspection data, the spatial position of the device in the image to be processed in the Internet of Things factory can be determined as the scanning position calibration data of the image to be processed.

[0044] In an optional embodiment of the present invention, the target device recognition model can be used to identify the target recognition device in the video inspection data; the target device recognition model can be a model obtained by training the target IoT device library based on AI technology.

[0045] The target IoT device library may be an image library of devices in the IoT factory that need to establish a twin model. For example, the images in the target IoT device library may be obtained by manually taking photos, or by searching the Internet based on device specifications. The target identification device may be a device in the IoT factory that needs to establish a twin model, that is, a device of interest in the IoT factory.

[0046] In an embodiment of the present invention, a target IoT device library may be first obtained, and then AI (Artificial Intelligence) technology may be used to train the target IoT device library to obtain a target device recognition model, so as to identify the target recognition device in the video inspection data through the target device recognition model.

[0047] Among them, AI technology may include but is not limited to machine learning technology and neural network technology.

[0048] Step 220: perform image clustering and fusion on the image to be processed according to the scan position calibration data of the image to be processed, obtain a target recognition device image, and determine the target recognition device position data corresponding to the target recognition device image.

[0049] In an embodiment of the present invention, the images to be processed can be divided according to the scanning position calibration data, so that the images with the same scanning position calibration data can be clustered and spliced ​​to obtain a target recognition device image, and based on the correspondence between the scanning position calibration data and the target recognition device image, the target recognition device position data corresponding to the target recognition device image can be determined.

[0050] In an optional embodiment of the present invention, the image to be processed is clustered and fused according to the scanning position calibration data of the image to be processed to obtain a target recognition device image, which may include: clustering and cutting out the image to be processed according to the scanning position calibration data of the image to be processed to obtain an image to be spliced; splicing the image to be spliced ​​according to the scanning position calibration data of the image to be processed to obtain a target recognition device image.

[0051] The image to be spliced ​​may be a device picture obtained by clustering and cutting out the image to be processed.

[0052] In an embodiment of the present invention, images with the same scanning position calibration data in the images to be processed are clustered, and the devices of interest in the images belonging to the same cluster are cut out to obtain images to be spliced, and then the images to be spliced ​​of the same cluster are spliced ​​based on the scanning position calibration data of the images to be processed to obtain the target recognition device image.

[0053] Step 230: Create a static IoT twin model based on the target recognition device image and the target recognition device location data.

[0054] In an optional embodiment of the present invention, creating a static IoT twin model based on a target recognition device image and target recognition device location data may include: using computer-aided design tools and the target recognition device image to draw a twin model of the location to be identified; marking the target recognition device location data on the twin model of the location to be identified to obtain a static IoT twin model.

[0055] Among them, the twin model whose location is to be identified can be an IoT factory twin model in which the location of the device twin has not been marked.

[0056] In an embodiment of the present invention, the target recognition device image can be input into a computer-aided design tool to automatically draw the device twin and obtain a twin model of the location to be identified, thereby marking the location data of the target recognition device within the preset range of the corresponding twin of the twin model of the location to be identified to obtain a static Internet of Things twin model.

[0057] Step 240: Adjust the static IoT twin model according to the real-time IoT perception data to obtain a dynamic IoT twin model.

[0058] In an optional embodiment of the present invention, adjusting the static IoT twin model according to the real-time perception data of the IoT to obtain a dynamic IoT twin model may include: parsing the real-time perception data of the IoT to obtain the twin parameters to be adjusted; adjusting the static IoT twin model based on the twin parameters to be adjusted to obtain the dynamic IoT twin model.

[0059] Among them, the twin parameters to be adjusted can be the twin modeling adjustment parameters of the device of interest whose state has changed.

[0060] In an embodiment of the present invention, real-time perception data of the Internet of Things can be analyzed to obtain devices whose state changes have occurred, and the attribute update parameters of the twin model of the focus device whose state has changed can be determined, that is, the twin parameters to be adjusted are obtained, and then the static Internet of Things twin model is adjusted based on the twin parameters to be adjusted to obtain a dynamic Internet of Things twin model.

[0061] Optionally, twin parameter adjustment rules may be pre-configured, and then the adjustment rules may be matched based on the state change of the device of interest to obtain the twin parameters to be adjusted.

[0062] In an optional embodiment of the present invention, the twin parameters to be adjusted may include: at least one of twin shape, twin size, twin color and twin special effects.

[0063] In this solution, video inspection data can be collected through a video inspection robot, and the target device recognition model can be enabled to the video inspection robot so that the video inspection robot can identify which IoT devices are of concern, thereby automatically creating a static IoT twin model, solving the problem of the large amount of tasks required to mark the spatial geographic location after twin modeling. There is no need to mark according to CAD drawings, which saves money and effort, and the twin modeling can be achieved in a short time. In addition, this solution does not require writing any code to change the appearance of the twin, but only requires defining the twin parameters to be adjusted.

[0064] The technical solution of the embodiment of the present invention obtains an image to be processed by inputting video inspection data into a target device recognition model, and determines the scanning position calibration data of the image to be processed based on the video inspection data, thereby performing image clustering and fusion on the image to be processed based on the scanning position calibration data of the image to be processed to obtain a target recognition device image, and determines the target recognition device position data corresponding to the target recognition device image, and then creates a static Internet of Things twin model based on the target recognition device image and the target recognition device position data, and further adjusts the static Internet of Things twin model based on real-time perception data of the Internet of Things to obtain a dynamic Internet of Things twin model. In this solution, the twin modeling of the devices of interest in the IoT factory can be automatically performed, and the target identification device location data can be used to mark the spatial location of the device actually mapped by the twin model. The modeling of the twin individuals in the entire static IoT twin model and the location marking can be automatically achieved without manual intervention. While ensuring the accuracy of twin modeling and location marking, it not only greatly reduces the labor cost, but also improves the modeling efficiency. In addition, the real-time perception data of the IoT is used to dynamically adjust the static IoT twin model to reflect the changes in the equipment status in real time and vividly, solving the current problems of low efficiency of manual twin modeling and large workload of position calibration after modeling. It can improve the efficiency of accurate twin modeling, reduce the labor cost of twin model position calibration, and improve the display effect of the twin model.

[0065] Embodiment 3

[0066] Figure 3 This is a schematic diagram of the structure of an Internet of Things data fusion device provided by Embodiment 3 of the present invention. Figure 3 As shown, the device comprises:

[0067] The data preparation module 310 is used to determine the target recognition device image and the target recognition device location data according to the video inspection data and the target device recognition model;

[0068] A static twin model creation module 320, for creating a static IoT twin model based on the target recognition device image and the target recognition device location data;

[0069] The dynamic twin model creation module 330 is used to adjust the static IoT twin model according to the real-time IoT perception data to obtain a dynamic IoT twin model.

[0070] The technical solution of the embodiment of the present invention determines the target recognition device image and the target recognition device location data according to the video inspection data and the target device recognition model, thereby creating a static IoT twin model according to the target recognition device image and the target recognition device location data, and then adjusting the static IoT twin model according to the real-time perception data of the IoT to obtain a dynamic IoT twin model. In this solution, the twin modeling of the concerned equipment in the IoT factory can be automatically performed, and the spatial position of the twin model actually mapped by the target recognition device location data can also be used to mark the spatial position of the equipment. The modeling of the twin individuals in the entire static IoT twin model and the position marking can be automatically realized without manual work. Under the premise of ensuring the accuracy of the twin modeling and position marking, not only the labor cost is greatly reduced, but also the modeling efficiency is improved. In addition, the real-time perception data of the IoT is used to dynamically adjust the static IoT twin model, and the changes in the device status are reflected in real time and vividly, which solves the problems of low efficiency of the current manual twin modeling and large workload of position calibration after modeling, and can improve the efficiency of accurate twin modeling, reduce the labor cost of twin model position calibration, and improve the display effect of the twin model.

[0071] Optionally, a target device recognition model is used to identify the target device in the video inspection data; the target device recognition model is a model obtained by training a target IoT device library based on artificial intelligence (AI) technology.

[0072] Optionally, the data preparation module 310 is specifically used to input the video inspection data into the target device recognition model to obtain the image to be processed, and determine the scanning position calibration data of the image to be processed based on the video inspection data; perform image clustering and fusion on the image to be processed based on the scanning position calibration data of the image to be processed to obtain the target recognition device image, and determine the target recognition device position data corresponding to the target recognition device image.

[0073] Optionally, the data preparation module 310 is specifically used to cluster and cut out the image to be processed according to the scanning position calibration data of the image to be processed to obtain the image to be spliced; and to splice the image to be spliced ​​according to the scanning position calibration data of the image to be processed to obtain the target recognition device image.

[0074] Optionally, the static twin model creation module 320 is specifically used to use computer-aided design tools and the target recognition device image to draw a twin model of the location to be identified; mark the location data of the target recognition device on the twin model of the location to be identified to obtain the static Internet of Things twin model.

[0075] Optionally, the dynamic twin model creation module 330 is specifically used to parse the real-time perception data of the Internet of Things to obtain the twin parameters to be adjusted; based on the twin parameters to be adjusted, adjust the static Internet of Things twin model to obtain the dynamic Internet of Things twin model.

[0076] Optionally, the twin parameters to be adjusted may include: at least one of: twin shape, twin size, twin color and twin special effects.

[0077] The Internet of Things data fusion device provided in the embodiment of the present invention can execute the Internet of Things data fusion method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0078] Embodiment 4

[0079] Figure 4 A schematic diagram of an electronic device that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0080] like Figure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0081] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0082] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the IoT data fusion method.

[0083] In some embodiments, the IoT data fusion method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the IoT data fusion method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the IoT data fusion method in any other appropriate manner (e.g., by means of firmware).

[0084] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0085] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0086] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0087] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0088] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0089] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of traditional physical hosts and VPS servers, which are difficult to manage and have weak business scalability.

[0090] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0091] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for fusion of Internet of Things data, characterized in that: include: Determine the target recognition device image and the target recognition device location data according to the video inspection data and the target device recognition model; Creating a static IoT twin model based on the target recognition device image and the target recognition device location data; According to the real-time perception data of the Internet of Things, the static Internet of Things twin model is adjusted to obtain a dynamic Internet of Things twin model.

2. The method according to claim 1, characterized in that The target device recognition model is used to identify the target recognition device in the video inspection data; the target device recognition model is a model obtained by training the target Internet of Things device library based on artificial intelligence AI technology.

3. The method according to claim 1, characterized in that Determining the target recognition device image and the target recognition device location data according to the video inspection data and the target device recognition model includes: Inputting the video inspection data into the target device recognition model to obtain an image to be processed, and determining scanning position calibration data of the image to be processed based on the video inspection data; According to the scan position calibration data of the image to be processed, image clustering and fusion are performed on the image to be processed to obtain the target recognition device image, and the target recognition device position data corresponding to the target recognition device image is determined.

4. The method according to claim 3, characterized in that The step of performing image clustering and fusion on the image to be processed according to the scan position calibration data of the image to be processed to obtain the target recognition device image includes: According to the scan position calibration data of the image to be processed, clustering and cutting out the image to be processed to obtain an image to be spliced; The images to be spliced ​​are spliced ​​according to the scan position calibration data of the images to be processed to obtain the target recognition device image.

5. The method according to claim 1, characterized in that The step of creating a static IoT twin model according to the target recognition device image and the target recognition device location data includes: Using computer-aided design tools and the target recognition device image, draw a twin model of the location to be identified; The target identification device location data is marked on the location to be identified twin model to obtain the static Internet of Things twin model.

6. The method according to claim 1, characterized in that The step of adjusting the static IoT twin model according to the real-time IoT perception data to obtain the dynamic IoT twin model includes: Analyze the real-time perception data of the Internet of Things to obtain the twin parameters to be adjusted; Based on the twin parameters to be adjusted, the static Internet of Things twin model is adjusted to obtain a dynamic Internet of Things twin model.

7. The method according to claim 6, characterized in that The twin parameters to be adjusted include: at least one of twin shape, twin size, twin color and twin special effects.

8. An Internet of Things data fusion device, characterized in that: include: A data preparation module, used to determine the target recognition device image and the target recognition device location data according to the video inspection data and the target device recognition model; A static twin model creation module, used to create a static IoT twin model based on the target recognition device image and the target recognition device location data; The dynamic twin model creation module is used to adjust the static IoT twin model according to the real-time perception data of the IoT to obtain a dynamic IoT twin model.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the Internet of Things data fusion method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the Internet of Things data fusion method according to any one of claims 1 to 7 when executed.

Images