A self-service teller machine monitoring method, device and equipment based on digital twinning

Abstract

The embodiment of the specification relates to the computer technical field, and in particular to a self-service teller machine monitoring method, device and equipment based on digital twinning. The embodiment comprises the following steps: analyzing the structure of a self-service teller machine to obtain the basic information and assembly relationship of a plurality of components; generating an assembly structure tree of the self-service teller machine according to the assembly relationship; traversing each layer of nodes in the assembly structure tree, and for each node, generating a pose matrix of the node according to the position information of the component corresponding to the node and the position information of the component corresponding to the parent node of the previous layer of the node; calculating the world matrix of each component node according to the assembly structure tree and the pose matrix of each node, and generating a digital twinning model of the self-service teller machine according to the world matrix of the component node and the geometric information of the component. Through the embodiment of the specification, the problem that the traditional self-service teller machine monitoring method based on the client mode seriously affects the operation and maintenance monitoring efficiency is solved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present specification relate to the technical field of computer technology, and in particular to a self-service teller machine monitoring method, device and equipment based on digital twinning. BACKGROUND

[0002] With the development of digital transformation of banking business, part of the business is completed through self-service teller machines in the future. How to better monitor the running state of the teller machine to avoid the failure to timely solve the problem affecting user experience is a problem worthy of attention.

[0003] Self-service teller machines are important basic equipment for modern bank informatization construction. These products are assembled by multiple layers of nested parts, have a complex structure, and at the same time, such products have a long service time, the equipment runs under high pressure, and the safe and healthy operation requirement is extremely high. If the running state of the self-service teller machine cannot be accurately monitored in time, it may cause the customer to lose money due to transaction errors and other adverse events. As a typical mechanical complex equipment, the design of the model of the self-service teller machine is mostly based on professional modeling software such as CATIA, SolidWorks, ArchiCAD, and PROE. The design delivery results are three-dimensional models and two-dimensional engineering drawings exported, and the product attributes are implied in the model, which brings information barriers to data sharing in the subsequent operation and maintenance stage. At the same time, these software running in the client mode are large in size and complicated in operation process, and the installation and operation are limited by hardware devices and professional drawing knowledge. However, the equipment operation and maintenance work usually involves the cooperation of different departments and different professional personnel. Therefore, the traditional operation mode based on the client mode seriously affects the efficiency of operation and maintenance monitoring.

[0004] There is an urgent need for a self-service teller machine monitoring method based on digital twinning to solve the problem that the traditional self-service teller machine monitoring method based on the client mode seriously affects the efficiency of operation and maintenance monitoring. SUMMARY

[0005] To solve the problem that the traditional self-service teller machine monitoring method based on the client mode seriously affects the efficiency of operation and maintenance monitoring, the embodiments of the present specification provide a self-service teller machine monitoring method, device and equipment based on digital twinning. The API of the professional modeling software extracts the design attribute information of the parts, separates the geometric data of the assembly file from the attribute information, and completes the reconstruction of the exported three-dimensional geometric file on the Web side.

[0006] In order to solve any one of the above technical problems, the specific technical solutions of the embodiments of the present specification are as follows:

[0007] On the one hand, the embodiments of the present specification provide a self-service teller machine monitoring method based on digital twinning, comprising,

[0008] Analyzing a structure of a self-service teller machine, obtaining basic information of a plurality of components in the structure and an assembly relationship between the components, the components being a part or a component assembled by a plurality of parts, the basic information including position information and geometric information of the part;

[0009] Generating an assembly structure tree of the self-service teller machine according to the assembly relationship, one node in the assembly structure tree corresponding to one component, one or more child nodes under a parent node corresponding to one or more components assembled on the component of the parent node;

[0010] Traversing nodes of each layer in the assembly structure tree, for each node, generating a pose matrix of the node according to position information of a component corresponding to the node and position information of a component corresponding to a parent node of a previous layer of the node;

[0011] Calculating a world matrix of each part node according to the assembly structure tree and the pose matrix of each node, the part node representing a node in which the component is a part, the world matrix representing coordinates of the part in the structure;

[0012] Generating a digital twin model of the self-service teller machine according to the world matrix of the part node and the geometric information of the part, so as to monitor the self-service teller machine by using the digital twin model.

[0013] Further, the pose matrix is a 4-row 3-column matrix, wherein the first three rows of elements are respectively a direction vector of one coordinate axis in an absolute coordinate system of a component corresponding to a node relative to each coordinate axis in an absolute coordinate system of a component corresponding to a parent node of the node, and an element in the last row is a three-dimensional space coordinate of an origin of the absolute coordinate system of the component corresponding to the node in the absolute coordinate system of the component corresponding to the parent node.

[0014] Further, a formula of the pose matrix is:

[0015]

[0016] wherein M(0)~M(11) represent elements in the pose matrix, Ux, Uy and Uz are respectively a direction vector of an x-axis, a y-axis and a z-axis in an absolute coordinate system of a component corresponding to a node relative to the x-axis, the y-axis and the z-axis in an absolute coordinate system of a component corresponding to a parent node of the node, Vx, Vy and Vz are respectively a direction vector of an x-axis, a y-axis and a z-axis in an absolute coordinate system of the component corresponding to the node relative to the x-axis, the y-axis and the z-axis in the absolute coordinate system of the component corresponding to the parent node, Wx, Wy and Wz are respectively a direction vector of an x-axis, a y-axis and a z-axis in an absolute coordinate system of the component corresponding to the node relative to the x-axis, the y-axis and the z-axis in the absolute coordinate system of the component corresponding to the parent node, and Tx, Ty and Tz are respectively an x-coordinate, a y-coordinate and a z-coordinate of an origin T of the absolute coordinate system of the component corresponding to the node in the absolute coordinate system of the component corresponding to the parent node.

[0017] Further, calculating the world matrix of the part node according to the assembly structure tree and the pose matrix of each node further comprises:

[0018] homogenizing the pose matrix of the node to obtain a homogeneous matrix of the node;

[0019] determining an upper parent node of the part node in the assembly structure tree to obtain a parent node list;

[0020] calculating the world matrix of the part node according to a formula C_world A =C A ×C n ×···×C3×C2×C1×C root , wherein C_world A represents the world matrix of the part node A, C A represents the homogeneous matrix of the part node A, C root , C1, C2, C3, …, C n respectively represent the homogeneous matrix of the root node, the homogeneous matrix of the first layer parent node, the homogeneous matrix of the second layer parent node, the homogeneous matrix of the third layer parent node, …, the homogeneous matrix of the nth layer parent node in the parent node list of the part node A.

[0021] Further, the homogeneous matrix is a 4*4 matrix, and the first three columns of elements in the homogeneous matrix are the same as the 3 columns of elements of the pose matrix.

[0022] homogenizing the pose matrix of the node to obtain a homogeneous matrix of the node further comprises:

[0023] if the coordinates of a row element of the pose matrix are vector coordinates, the fourth column element of the row is 0.

[0024] if the coordinates of a row element of the pose matrix are point coordinates, the fourth column element of the row is 1.

[0025] On the other hand, the embodiments of the present specification also provide a self-service teller machine monitoring device based on digital twinning, comprising:

[0026] an assembly structure analysis unit configured to analyze the structure of the self-service teller machine to obtain basic information of a plurality of components in the structure and an assembly relationship between the components, the components being parts or components assembled by a plurality of parts, and the basic information including position information and geometric information of the parts;

[0027] The assembly structure tree generation unit is configured to generate an assembly structure tree of the self-service teller machine according to the assembly relationship, wherein one node in the assembly structure tree corresponds to one component, and one or more child nodes under a parent node correspond to one or more components assembled on the component of the parent node.

[0028] The pose matrix calculation unit is configured to traverse each node of each layer in the assembly structure tree, and for each node, generate a pose matrix of the node according to position information of a component corresponding to the node and position information of a component corresponding to a parent node of a previous layer of the node.

[0029] The world matrix calculation unit is configured to calculate a world matrix of each component node according to the assembly structure tree and the pose matrix of each node, wherein the component node represents a node in which a component is a component, and the world matrix represents coordinates of the component in the structure.

[0030] The digital twin model construction unit is configured to generate a digital twin model of the self-service teller machine according to the world matrix of the component node and geometric information of the component, so as to monitor the self-service teller machine by using the digital twin model.

[0031] Further, the world matrix calculation unit further includes:

[0032] The homogeneous matrix transformation module is configured to perform homogeneous processing on the pose matrix of the node to obtain a homogeneous matrix of the node.

[0033] The parent node list determination module is configured to determine upper layer parent nodes of the component node in the assembly structure tree to obtain a parent node list.

[0034] The world matrix calculation module is configured to calculate the world matrix of the component node according to a formula C_world A =C A ×C n ×···×C3×C2×C1×C root , wherein C_world A represents the world matrix of the component node A, C A represents the homogeneous matrix of the component node A, C root , C1, C2, C3,..., C n respectively represent the homogeneous matrix of a root node, the homogeneous matrix of a first layer parent node, the homogeneous matrix of a second layer parent node, the homogeneous matrix of a third layer parent node,..., and the homogeneous matrix of an n-th layer parent node in the parent node list of the component node A.

[0035] Further, the homogeneous matrix is a 4-row 4-column matrix, and elements in the first three columns of the homogeneous matrix are the same as 3-column elements of the pose matrix.

[0036] The homogeneous matrix transformation module is further used for:

[0037] If the coordinates of the elements of a row of the pose matrix are vector coordinates, the fourth column element of the row is 0;

[0038] If the coordinates of the elements of a row of the pose matrix are point coordinates, the fourth column element of the row is 1.

[0039] In another aspect, the embodiments of the present specification also provide a computer device, comprising a memory, a processor, and a computer program stored in the memory, and the processor implements the above method when executing the computer program.

[0040] In another aspect, the embodiments of the present specification also provide a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the above method.

[0041] Finally, the embodiments of the present specification also provide a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the above method.

[0042] With the embodiments of the present specification, the existing technology is abandoned by extracting constraint relationships to obtain assembly information, and the determination of coaxial, coplanar, concentric and other positional relationships is required, which is difficult to assemble parts into a digital twin model on the web. The embodiments of the present specification obtain the positional information of the parts by calculating the pose matrix of each layer of parts, and then obtain the world matrix of the parts through coordinate transformation of the parts, so as to obtain the final assembly position of each part, without considering the complex constraint relationship judgment, reducing the calculation amount. And in the embodiments of the present specification, for the same parts, a copy of geometric information is stored, so that the model file is more lightweight, facilitating the transmission of the digital twin model file in the network and the display on the browser side, reducing the rendering pressure of the browser, thereby improving the efficiency of monitoring using the digital twin model. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present specification or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present specification, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0044] Figure 1 The structure schematic diagram of an implementation system of a self-service teller machine monitoring method based on digital twin in the embodiments of the present specification is shown;

[0045] Figure 2 Fig. 1 shows a flowchart of a self-service teller machine monitoring method based on digital twinning according to an embodiment of the present specification;

[0046] Figure 3 Fig. 3 shows an absolute coordinate system corresponding to a parent node and a child node according to an embodiment of the present specification;

[0047] Figure 4 Fig. 5 shows a flowchart of a web server analyzing an uploaded file according to an embodiment of the present specification;

[0048] Figure 5 Fig. 6 shows a structural diagram of a self-service teller machine monitoring device based on digital twinning according to an embodiment of the present specification;

[0049] Figure 6 Fig. 7 shows a detailed structural diagram of a self-service teller machine monitoring device based on digital twinning according to an embodiment of the present specification;

[0050] Figure 7 Fig. 8 shows a structural diagram of a computer device according to an embodiment of the present specification.

[0051]

BRIEF DESCRIPTION OF THE DRAWINGS

[0052] 101, terminal;

[0053] 102, server;

[0054] 103, monitoring web;

[0055] 501, structure analysis unit;

[0056] 502, assembly structure tree generation unit;

[0057] 503, pose matrix calculation unit;

[0058] 504, world matrix calculation unit;

[0059] 5041, homogeneous matrix transformation module;

[0060] 5042, parent node list determination module;

[0061] 5043, world matrix calculation module;

[0062] 505, digital twinning model construction unit;

[0063] 702, computer device;

[0064] 704, processing device;

[0065] 706, storage resource;

[0066] 708、drive mechanism;

[0067] 710、input / output module;

[0068] 712、input device;

[0069] 714、output device;

[0070] 716、presentation device;

[0071] 718、graphical user interface;

[0072] 720、network interface;

[0073] 722、communication link;

[0074] 724、communication bus. DETAILED DESCRIPTION

[0075] The technical solutions in the embodiments of the present specification will be described clearly and completely below in combination with the drawings in the embodiments of the present specification. Obviously, the described embodiments are only part of the embodiments of the present specification, rather than all the embodiments. Based on the embodiments in the embodiments of the present specification, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the embodiments of the present specification.

[0076] It should be noted that the terms "first", "second" and the like in the specification and claims of the embodiments of the present specification and the above-described drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the embodiments of the present specification described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, device, product or equipment including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or equipment.

[0077] It should be noted that the acquisition, storage, use, processing and the like of data in the technical solutions of the present application comply with the relevant provisions of relevant laws and regulations.

[0078] As Figure 1As shown in the embodiment of the present specification, an implementation system schematic diagram of a self-service teller machine monitoring method based on digital twinning is shown, which can include a terminal 101, a server 102 and a monitoring web end 103. The terminal 101 and the server 102, and the server 102 and the browser 103 can communicate through a network, which can include a local area network (LAN), a wide area network (WAN), the Internet or a combination thereof, and be connected to a website, a user device (such as a computing device) and a backend system. The user inputs the structural information of the self-service teller machine to be monitored to the server 102 through the terminal 101, which can be a design document of a digital teller machine, etc. The server 102 analyzes the structure of the self-service teller machine input by the terminal 101, obtains the assembly position and geometric information of each component part of the self-service teller machine, etc., constructs a model file of a digital twinning model, and then sends the model file to the monitoring web end 103 through the network. The monitoring web end 103 parses and renders the model file to obtain a digital twinning model and display it to the monitoring personnel, so that the monitoring personnel can monitor the corresponding self-service teller machine according to the digital twinning model displayed on the monitoring web end 103.

[0079] When monitoring the corresponding self-service teller machine by using the digital twinning model of the self-service teller machine displayed on the monitoring web end 103, a sensor is deployed on the self-service teller machine to obtain the operation data of the self-service teller machine, which is sent to the monitoring web end 103. The monitoring web end 103 adjusts the digital twinning model according to the received operation data, thereby achieving the purpose of monitoring.

[0080] Optionally, the server 102 can be a node of a cloud computing system (not shown in the figure), or each processor can be a separate cloud computing system, including multiple computers interconnected by a network and working as a distributed processing system.

[0081] In addition, it should be noted that, Figure 1 The application environment shown is only one application environment provided by the present disclosure, and other application scenarios can also be included in actual application, which is not limited by the present specification.

[0082] In order to solve the problem that the traditional self-service teller machine monitoring method based on the client mode seriously affects the operation and maintenance monitoring efficiency, the embodiment of the present specification provides a self-service teller machine monitoring method based on digital twinning. The API of professional modeling software extracts the design attribute information of the parts, separates the geometric data of the assembly file from the attribute information, and completes the reconstruction of the exported three-dimensional geometric file on the Web end. Figure 2Fig. 1 shows a flowchart of a self-service teller machine monitoring method based on digital twinning according to an embodiment of the present specification. In this figure, the process of constructing a digital twin model of a self-service teller machine and monitoring the self-service teller machine is described. The order of steps listed in the embodiment is only one of the many possible execution orders, and does not represent the only execution order. In actual system or device product execution, the method order shown in the embodiment or the accompanying drawings can be executed in sequence or in parallel. Specifically, as Figure 2 As shown, the method can be executed by the server 102, and can include:

[0083] Step 201: Analyzing the structure of the self-service teller machine to obtain the basic information of a plurality of components in the structure and the assembly relationship between the components, the components being parts or components assembled from a plurality of parts, the basic information including position information and geometric information of the parts;

[0084] Step 202: Generating an assembly structure tree of the self-service teller machine according to the assembly relationship, one node in the assembly structure tree corresponding to one component, one or more child nodes under a parent node corresponding to one or more components assembled on the component of the parent node;

[0085] Step 203: Traversing the nodes of each layer in the assembly structure tree, for each node, generating a pose matrix of the node according to the position information of the component corresponding to the node and the position information of the component corresponding to the parent node of the last layer of the node;

[0086] Step 204: Calculating the world matrix of each part node according to the assembly structure tree and the pose matrix of each node, the part node representing the node where the component is a part, the world matrix representing the coordinates of the part in the structure;

[0087] Step 205: Generating a digital twin model of the self-service teller machine according to the world matrix of the part node and the geometric information of the part, so as to monitor the self-service teller machine using the digital twin model.

[0088] With the embodiment of the present specification, the prior art method of obtaining assembly information by extracting constraint relationships is abandoned, which requires determination of coaxial, coplanar, concentric and other positional relationships, and it is difficult to assemble parts into a digital twin model on the web. The embodiment of the present specification obtains the position information of the parts by calculating the pose matrix of each layer of components, and then obtains the world matrix of the parts through coordinate transformation of the parts, thereby obtaining the final assembly position of each part, without considering complex constraint relationship judgment, reducing the amount of calculation.

[0089] In the embodiments of the present application, a component is a part or a component assembled by multiple parts, a part is the smallest unit of an assembly type three-dimensional model and cannot be further decomposed, and a component is assembled by multiple parts according to position information of the parts. The digital teller machine is composed of multiple components and / or parts, and therefore the embodiments of the present application need to record position information of the parts and components and geometric information of the parts.

[0090] In the embodiments of the present application, the basic information of each component in the self-service teller machine and the assembly relationship between the components can be obtained by using the existing API interface. The basic information includes position information of the parts or components, geometric information of the parts, part or component number, part or component instance name, part or component mass, part or component center of inertia, part or component inertia matrix, and the like.

[0091] The position information of the parts or components includes coordinates of the parts or components in their own three-dimensional coordinate system, and the geometric information of the parts includes geometric dimensions of the parts. The part or component number represents the unique identification of the current part or component in the entire assembly, and the same part or component number can be the same. When the same part or component is located at multiple different assembly positions of the assembly, different instance names are used for differentiation. The mass of the part or component is used to construct the digital twin model from the weight dimension. The center of inertia of the part or component, the inertia matrix of the part or component, and the like can realize more monitoring information of the digital twin model.

[0092] Then, the embodiments of the present application generate an assembly structure tree of the self-service teller machine according to the assembly relationship. One node in the assembly structure tree corresponds to one component, and one or more child nodes under a parent node correspond to one or more components assembled on the parent node. For example, the Forma wheel of the self-service teller machine is composed of a wheel, a bearing, a wheel seat, and a bolt. The wheel, the bearing, the wheel seat, and the bolt are all components of the self-service teller machine. Since the Forma wheel is composed of the wheel, the bearing, the wheel seat, and the bolt, in the assembly structure tree, the Forma wheel is a parent node, and the wheel, the bearing, the wheel seat, and the bolt are child nodes of the parent node.

[0093] In addition, the base of the self-service teller machine is composed of multiple Forma wheels, a base body, and bolts. Then, the node corresponding to the base body and the nodes corresponding to the bolts are at the same level as the node corresponding to the Forma wheel, and the parent node is the base of the self-service teller machine.

[0094] Then the embodiment of the present specification traverses the nodes of each layer in the assembly structure tree, and the nodes of multiple parts directly assembled on the same part are located in the same layer. The embodiment of the present specification traverses the assembly structure tree from top to bottom, and extracts the corresponding pose matrix information of each layer part node. Each part of the model has an absolute three-dimensional coordinate system relative to itself, and the position information of the current object relative to its parent component is obtained in a 12-length array.

[0095] Specifically, the pose matrix is a 4-row 3-column matrix, wherein the first three rows of elements are respectively the direction vectors of each coordinate axis in the absolute coordinate system of the node corresponding part relative to the coordinate axis in the absolute coordinate system of the parent node corresponding part, which describes the rotation angle information of the child node corresponding part relative to the parent node corresponding part. The element of the last row is the three-dimensional space coordinates of the origin of the absolute coordinate system of the node corresponding part in the absolute coordinate system of the parent node corresponding part.

[0096] As shown in Figure 3 , the origin of the absolute coordinate system of the parent node corresponding part is O, and the coordinate system includes three coordinate axes X, Y and Z. The origin of the absolute coordinate system of the child node corresponding part is T, and the coordinate system includes three coordinate axes U, V and W. The formula of the pose matrix is:

[0097]

[0098] Wherein, M(0)~M(11) represent the elements in the pose matrix, Ux, Uy, Uz are respectively the direction vectors of U axis in the absolute coordinate system of the node corresponding part relative to x, y, z axes in the absolute coordinate system of the parent node corresponding part, Vx, Vy, Vz are respectively the direction vectors of V axis in the absolute coordinate system of the node corresponding part relative to x, y, z axes in the absolute coordinate system of the parent node corresponding part, Wx, Wy, Wz are respectively the direction vectors of W axis in the absolute coordinate system of the node corresponding part relative to x, y, z axes in the absolute coordinate system of the parent node corresponding part, Tx, Ty, Tz are respectively the x, y, z coordinates of the origin T of the absolute coordinate system of the node corresponding part in the absolute coordinate system of the parent node corresponding part.

[0099] In order to more completely represent the feature information of the three-dimensional model, some important attributes need to be extracted, so as to avoid that the model reconstructed on the web exists only as an isolated geometric entity. Through the API interface opened by the modeling software, part of the design information of the part is obtained, and the dependence of the engineering personnel on the two-dimensional engineering drawing is reduced.

[0100] The embodiment of the specification extracts the part number and instance name of the component by traversing all objects under the assembly, the part number is the unique identifier of the component to which the current object belongs in the entire assembly, different instances of the same type of component are distinguished by the instance name of the component, and various parameter data such as volume, mass, curved surface, center of inertia and inertia matrix also need to be extracted. The instance name of the direct parent node at the current component node is obtained to save the assembly information, and the order of output attribute data and the order of assembly structure tree are consistent. The part extraction of the multi-layer nested assembly needs to traverse the assembly structure tree from top to bottom, judge whether the current obtained node object type is a part, the part is the smallest unit of the assembly type three-dimensional model and cannot be further decomposed, only the object of the part type is exported as an STL file, if the traversed object is a sub-assembly type, the current object is skipped. At the same time, the redundant parts are removed, different part instances with the same geometric data only keep one copy of the geometric file, reducing the storage space occupied by the model file. In the subsequent reconstruction stage, translation, rotation and other operations are realized by the operation of the homogeneous matrix, realizing the cloning of the model parts. In the program, the folder for storing the exported STL file is specified, and the part document object name corresponding to each layer node is saved as the name of the exported geometric file, and the part number information and the storage path of the corresponding part are written into the assembly metadata file. The three-dimensional model of assembly design is usually very complex, only by obtaining accurate part identification information and the position relationship between parts, can the parts information be used for assembly.

[0101] The geometric data extracted by digital-analog separation is a set of STL files (i.e. the geometric information of the parts), and the attribute data is output to the assembly metadata file (the metadata file includes the pose matrix of the component, the part number, the instance name of the component, the mass of the component, the center of inertia of the component, the inertia matrix of the component, etc.). Before model reconstruction (i.e. generating a digital twin model), the extracted STL file and the assembly metadata file can be uploaded to the web server, and the web server parses the uploaded file as shown in Figure 4

[0102] ​First, the uploaded file is selected, and then a class inheriting AnalysisEventListener is created as a listener to monitor the processing of the file for subsequent backtracking. Then the file header and file data rows are read, and a corresponding node object is created for each read data row. An id field is added to each node object as a unique identifier for the part, and a pid field is added to save the hierarchical relationship of the parts. Then the object of the current row's owner (i.e., the part corresponding to the parent node) is obtained. If the read data is a top-level assembly, there is no owner object (i.e., there is no corresponding part), and the pid of the node object created for this row of data is set to 0. If the read data is a sub-assembly (i.e., a child node), the owner object of the current row is obtained (i.e., the parent node corresponding to the child node), and the id value of the owner object is assigned to the pid of the node object of the current read row for subsequent parsing. After saving the node hierarchical relationship, the parent component object (i.e., obtaining the information of the part corresponding to the parent node) is obtained according to the parent component id. If the parent component object is not empty, the parts belonging to the same parent component are assigned the same color. Otherwise, a self-generated hexadecimal string is used as the color information of the part to facilitate differentiation. After saving the current node object, the next row of the assembly metadata file is read for processing until all rows of data are parsed and saved to the database.

[0103] In the embodiments of the present specification, a digital twin model is created on a web server using the Three.js engine. The objects in Three.js include three matrix objects: local matrix, world matrix, and camera projection matrix. The local matrix is the local model transformation matrix of a child node relative to a parent node, the world matrix is the global model transformation matrix of a node, and the camera projection matrix represents the transformation of a node relative to a camera coordinate system.

[0104] Therefore, the embodiments of the present specification need to calculate the world matrix of a part node, where the part node represents a node of a part, and the world matrix represents the coordinates of the part in the structure.

[0105] Specifically, calculating the world matrix of a part node according to the assembly structure tree and the pose matrix of each node further includes:

[0106] Homogenizing the pose matrix of the node to obtain a homogeneous matrix of the node;

[0107] In the embodiments of the present specification, the homogeneous matrix is a 4x4 matrix, and the first three columns of elements in the homogeneous matrix are the same as the 3-column elements of the pose matrix.

[0108] Homogenizing the pose matrix of the node to obtain a homogeneous matrix of the node further includes:

[0109] If the coordinates of the elements of a row of the pose matrix are vector coordinates, the fourth column element of the row is 0;

[0110] If the coordinates of the elements of a row of the pose matrix are point coordinates, the fourth column element of the row is 1.

[0111] A one-dimensional array is usually used to describe a matrix in a Three.js program. There are two ways to store a matrix, row-major order and column-major order, and Three.js follows the OpenGL specification to store a matrix in column-major order. The extracted part pose matrix is composed of three three-dimensional vector coordinates and a point coordinate, and the translation, rotation, and scaling information of an Object3D object can be described by linear transformation of a homogeneous matrix in Three.js, so the original pose matrix needs to be converted into a homogeneous coordinate matrix. Homogeneous coordinates are represented by N+1 dimensions to represent N-dimensional coordinates. When converting ordinary coordinates to homogeneous coordinates, if the coordinates are point coordinates, the N+1 dimension is set to 1, and if the coordinates are vector coordinates, the N+1 dimension is set to 0. The following formula converts the part pose matrix into a 4x4 matrix.

[0112]

[0113] After the part pose matrix is homogenized, it is stored in column-major order, and the following formula is the local matrix of each part relative to the parent component, as shown in the following formula:

[0114]

[0115] Then traverse the list of parent nodes (parent node list) of the part node, take out the pose matrix of each parent node, and homogenize it according to the above steps to obtain the homogeneous matrix of each parent node in the parent node list. Specifically, a request can be sent to obtain all component nodes with multiple nested levels, and these nodes are traversed to determine whether the current node is a component or a part. If the node is a component, it means that the current object does not have a corresponding geometry file, and it does not need to be sent to the node list to be rendered, and the recursive traversal continues downward until the object type of the traversed node is a part, at which point the list of all upper parent nodes of the current node is calculated.

[0116] During the reconstruction process, the part is automatically moved to the correct assembly position by setting the coordinate transformation matrix of the part. According to the formula C_world A =C A ×C n ×···×C3×C2×C1×C root Calculate the world matrix of the part node, where C_world A represents the world matrix of part node A, C A represents the homogeneous matrix of part node A, Croot C1, C2, C3, ···, C n respectively represent homogeneous matrices of root nodes in a parent node list of a part node A, a first layer parent node, a second layer parent node, a third layer parent node, ···, and an n layer parent node.

[0117] After the world matrix of the part is calculated, the accurate assembly position of the part in the scene is obtained, then a model loader of a corresponding type is created, and the material of the part is taken out in the callback function to create a material object, a Mesh object is generated from the geometry object and the material object, and finally the Mesh object corresponding to the part is added to the group object for rendering by the Three.js rendering pipeline. That is, the visualization process of the assembly type three-dimensional model on the Web end is completed, which provides a basis for subsequent monitoring of the running state of the self-service teller machine.

[0118] Based on the same inventive concept, the embodiments of the present specification also provide a self-service teller machine monitoring device based on digital twinning, as shown in the accompanying drawings, comprising: Figure 5

[0119] The structure analysis unit 501 is configured to analyze the structure of the self-service teller machine to obtain the basic information of a plurality of components in the structure and the assembly relationship between the components, the components being parts or components assembled by a plurality of parts, and the basic information including position information and geometry information of the parts.

[0120] The assembly structure tree generation unit 502 is configured to generate an assembly structure tree of the self-service teller machine according to the assembly relationship, one node in the assembly structure tree corresponding to one component, and one or more child nodes under a parent node corresponding to one or more components assembled on the component of the parent node.

[0121] The pose matrix calculation unit 503 is configured to traverse each layer of nodes in the assembly structure tree, and for each node, generate a pose matrix of the node according to the position information of the component corresponding to the node and the position information of the component corresponding to the last layer parent node of the node.

[0122] The world matrix calculation unit 504 is configured to calculate a world matrix of each part node according to the assembly structure tree and the pose matrix of each node, the part node representing a node in which the component is a part, and the world matrix representing the coordinates of the part in the structure.

[0123] The digital twinning model construction unit 505 is configured to generate a digital twinning model of the self-service teller machine according to the world matrix of the part node and the geometry information of the part, so as to monitor the self-service teller machine by using the digital twinning model. ​

[0124] Further, as shown in Figure 6 The world matrix calculation unit 504 further comprises:

[0125] A homogeneous matrix transformation module 5041, configured to perform homogeneous processing on the pose matrix of the node to obtain a homogeneous matrix of the node; the homogeneous matrix is a 4*4 matrix, and the first three columns of the homogeneous matrix are the same as the 3 columns of the pose matrix; the homogeneous matrix transformation module 5041 is further configured to: if the coordinates of the elements of a row of the pose matrix are vector coordinates, the fourth column element of the row is 0; if the coordinates of the elements of a row of the pose matrix are point coordinates, the fourth column element of the row is 1.

[0126] A parent node list determination module 5042, configured to determine the upper parent nodes of the part node in the assembly structure tree to obtain a parent node list.

[0127] A world matrix calculation module 5043, configured to calculate the world matrix of the part node according to the formula C_world A =C A ×C n ×···×C3×C2×C1×C root , wherein C_world A represents the world matrix of the part node A, C A represents the homogeneous matrix of the part node A, C root , C1, C2, C3, …, C n respectively represent the homogeneous matrix of the root node, the homogeneous matrix of the first layer parent node, the homogeneous matrix of the second layer parent node, the homogeneous matrix of the third layer parent node, …, the homogeneous matrix of the n-th layer parent node in the parent node list of the part node A.

[0128] Since the principle of solving the problem of the above device is similar to that of the above method, the implementation of the above device can be referred to the implementation of the above method, and the repeated parts will not be described here.

[0129] As shown in Figure 7A structural diagram of a computer device of an embodiment of the present specification is shown. The apparatus herein can be a computer device in the present embodiment, which executes the method herein described. The computer device 702 can include one or more processing devices 704, such as one or more central processing units (CPUs), each of which can implement one or more hardware threads. The computer device 702 can also include any storage resources 706 for storing any kind of information, such as code, settings, data, etc. Without limitation, for example, the storage resources 706 can include any one or combination of: any type of RAM, any type of ROM, flash memory devices, hard disks, optical disks, etc. More generally, any storage resource can store information using any technology. Further, any storage resource can provide volatile or non-volatile retention of information. Further, any storage resource can represent a fixed or removable component of the computer device 702. In one case, the computer device 702 can perform any operation of the associated instructions when the processing device 704 executes the associated instructions stored in any storage resource or combination of storage resources. The computer device 702 also includes one or more drive mechanisms 708, such as a hard disk drive mechanism, an optical disk drive mechanism, etc., for interacting with any storage resources.

[0130] The computer device 702 can also include an input / output module 710 (I / O) for receiving various inputs (via input devices 712) and for providing various outputs (via output devices 714). One particular output mechanism can include a presentation device 716 and an associated graphical user interface (GUI) 718. In other embodiments, the input / output module 710 (I / O), the input devices 712, and the output devices 714 can also not be included, just as a computer device in a network. The computer device 702 can also include one or more network interfaces 720 for exchanging data with other devices via one or more communication links 722. One or more communication buses 724 couple the above-described components together.

[0131] The communication links 722 can be implemented in any manner, for example, through a local area network, a wide area network (e.g., the Internet), a point-to-point connection, etc., or any combination thereof. The communication links 722 can include any combination of hardwired links, wireless links, routers, gateway functionality, name servers, etc., governed by any protocol or combination of protocols.

[0132] The present specification also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method described above.

[0133] The embodiment of the present specification further provides a computer readable instruction, wherein when the processor executes the instruction, the program therein causes the processor to execute the method.

[0134] It should be understood that the magnitude of the sequence number of each process described above does not mean the order of execution in various embodiments of the embodiment of the present specification, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiment of the present specification.

[0135] It should also be understood that in the embodiment of the present specification, the term "and / or" is only a description of the association relationship of the associated objects, which means that there can be three relationships. For example, A and / or B can represent three cases of A alone, A and B together, and B alone. In addition, the character " / " in the embodiment of the present specification generally represents that the front and rear associated objects are in an "or" relationship.

[0136] Those skilled in the art can realize that the units and algorithm steps of each example described in combination with the disclosed embodiments in the embodiment of the present specification can be realized by electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been described in the above description. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiment of the present specification.

[0137] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

[0138] In several embodiments provided by the embodiment of the present specification, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displays or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can also be electrical, mechanical or other forms of connection.

[0139] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, i.e., may be located in one place, or may be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment of the present specification.

[0140] In addition, each functional unit in each embodiment of the present specification can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0141] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present specification essentially or say the part that contributes to the prior art, or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in each embodiment of the present specification. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.

[0142] The principles and implementation manners of the embodiments of the present specification are described in the specific embodiments in the present specification, and the above embodiment description is only used to help understand the method and its core idea of the present specification; meanwhile, for those skilled in the art, according to the idea of the present specification, the specific implementation manner and application range will be changed, and the above description should not be understood as a limitation of the present specification.

Claims

1. A self-service kiosk monitoring method based on digital twinning, characterized in that, The method comprises: analyzing the structure of the self-service teller machine to obtain basic information of a plurality of components in the structure and an assembly relationship between the components, the components being parts or components assembled by a plurality of parts, the basic information including position information and geometric information of the parts; generating an assembly structure tree of the self-service teller machine according to the assembly relationship, one node in the assembly structure tree corresponding to one component, one or more child nodes under a parent node corresponding to one or more components assembled on the component of the parent node; traversing nodes of each layer in the assembly structure tree, and for each node, generating a pose matrix of the node according to position information of a component corresponding to the node and position information of a component corresponding to a parent node of a previous layer of the node; calculating a world matrix of a part node of each part according to the assembly structure tree and the pose matrix of each node, the part node representing a node in which a component is a part, the world matrix representing coordinates of the part in the structure; generating a digital twin model of the self-service teller machine according to the world matrix of the part node and the geometric information of the part, so as to monitor the self-service teller machine by using the digital twin model.

2. The method of claim 1, wherein, The pose matrix is a 4-row 3-column matrix, wherein the first three rows of elements are respectively direction vectors of one coordinate axis in an absolute coordinate system of a component corresponding to a node relative to each coordinate axis in an absolute coordinate system of a component corresponding to a parent node of the node, and the last row of elements is a three-dimensional space coordinate of an origin of the absolute coordinate system of the component corresponding to the node in the absolute coordinate system of the component corresponding to the parent node.

3. The method of claim 2, wherein, The formula of the pose matrix is: wherein M(0)~M(11) represent elements in the pose matrix, Ux, Uy and Uz are respectively direction vectors of x, y and z axes in an absolute coordinate system of a component corresponding to a node relative to x, y and z axes in an absolute coordinate system of a component corresponding to a parent node of the node, Vx, Vy and Vz are respectively direction vectors of x, y and z axes in the absolute coordinate system of the component corresponding to the node relative to x, y and z axes in the absolute coordinate system of the component corresponding to the parent node, Wx, Wy and Wz are respectively direction vectors of x, y and z axes in the absolute coordinate system of the component corresponding to the node relative to x, y and z axes in the absolute coordinate system of the component corresponding to the parent node, and Tx, Ty and Tz are respectively x, y and z coordinates of an origin T of the absolute coordinate system of the component corresponding to the node in the absolute coordinate system of the component corresponding to the parent node.

4. The method of claim 1, wherein, The calculation of the world matrix of the part node according to the assembly structure tree and the pose matrix of each node further comprises: homogenizing the pose matrix of the node to obtain a homogeneous matrix of the node; determining an upper parent node of the part node in the assembly structure tree to obtain a parent node list; According to the formula C_world A = C A × C n × ··· × C3× C2× C1× C root Calculate the world matrix of the part node, wherein C_world A represents the world matrix of the part node A, C A represents the homogeneous matrix of the part node A, C root , C1, C2, C3, ···, C n respectively represent the homogeneous matrix of the root node in the parent node list of the part node A, the homogeneous matrix of the first layer parent node, the homogeneous matrix of the second layer parent node, the homogeneous matrix of the third layer parent node, ···, the homogeneous matrix of the n layer parent node.

5. The method of claim 4, wherein, the homogeneous matrix is a 4-row 4-column matrix, and the first three columns of elements in the homogeneous matrix are the same as the 3-column elements of the pose matrix; the homogenization of the pose matrix of the node to obtain the homogeneous matrix of the node further comprises: if a coordinate of an element of a row of the pose matrix is a vector coordinate, then a fourth column element of the row is 0. If the coordinate of an element of a row of the pose matrix is a point coordinate, the fourth column element of the row is 1.

6. A digital-twin-based self-service kiosk monitoring device, characterized by, The method comprises the following steps: The structural analysis unit is configured to analyze the structure of the self-service teller machine to obtain basic information of a plurality of components in the structure and an assembly relationship between the components, the components being parts or components assembled by a plurality of parts, the basic information including position information and geometric information of the parts; The assembly structure tree generation unit is configured to generate an assembly structure tree of the self-service teller machine according to the assembly relationship, one node in the assembly structure tree corresponding to one component, one or more child nodes under a parent node corresponding to one or more components assembled on the component of the parent node; The pose matrix calculation unit is configured to traverse nodes at each level of the assembly structure tree, and for each node, generate a pose matrix of the node according to position information of a component corresponding to the node and position information of a component corresponding to a parent node at a previous level of the node; The world matrix calculation unit is configured to calculate a world matrix of a part node according to the assembly structure tree and the pose matrix of each node, the part node representing a node in which a component is a part, the world matrix representing coordinates of the part in the structure; The digital twin model construction unit is configured to generate a digital twin model of the self-service teller machine according to the world matrix of the part node and the geometric information of the part, so as to facilitate monitoring of the self-service teller machine by using the digital twin model.

7. The apparatus of claim 6, wherein, The world matrix calculation unit further comprises: The homogeneous matrix transformation module is configured to perform homogeneous processing on the pose matrix of the node to obtain a homogeneous matrix of the node; The parent node list determination module is configured to determine upper parent nodes of the part node in the assembly structure tree to obtain a parent node list; a world matrix calculation module configured to calculate a world matrix of the part node according to a formula C_world A = C A × C n × ··· × C3× C2× C1× C root , wherein C_world A represents the world matrix of the part node A, C A represents a homogeneous matrix of the part node A, C root , C1, C2, C3, ···, C n respectively represent a homogeneous matrix of a root node in a parent node list of the part node A, a homogeneous matrix of a first layer parent node, a homogeneous matrix of a second layer parent node, a homogeneous matrix of a third layer parent node, ···, a homogeneous matrix of an n-th layer parent node.

8. The apparatus of claim 7, wherein, The homogeneous matrix is a 4x4 matrix, and the first three columns of the homogeneous matrix have the same elements as the 3 columns of the pose matrix; If the coordinate of an element of a row of the pose matrix is a vector coordinate, the fourth column element of the row is 0; If the coordinate of an element of a row of the pose matrix is a point coordinate, the fourth column element of the row is 1. The processor executes the computer program to implement the method of any one of claims 1 to 5.

9. A computer device comprising a memory, a processor, and a computer program stored on the memory, wherein, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the method of any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that, The computer program product comprises a computer program, and the computer program is executed by the processor to implement the method of any one of claims 1 to 5.

11. A computer program product, characterised in that, ​

Images